How to split CSV File into multiple Files with specific records

Sometime you should have lots of record in one file , you might need it to split in multiple files so that you can import in software easily.

Here , we have implement small script using python to do this.

Python Script

import pandas as pd
import os
import shutil

#give the csv file to read
input_csv = 'input.csv'

#checking number of lines in csv
number_lines = sum(1 for row in (open(input_csv)))

#number of record, should in each csv
rowsize = 1000

#creating split folder to store new csv and deleting old csv before storing.
try:
    # Remove folder (if exists) with all files
    if os.path.isdir(str("split")):
        shutil.rmtree("split", ignore_errors=True)
    # Create new folder
    os.mkdir("split")
except IOError:
    print("Error upon either deleting or creating the directory or files.")
else:
    print("Actions if file access was succesfull")

#start looping through data writing it to a new file for each set
for i in range(1,number_lines,rowsize):
    df = pd.read_csv(in_csv,
          header=None,
          nrows = rowsize, #number of rows to read at each loop
          skiprows = i) #skip rows that have been read
    #csv to write data to a new file with indexed name. input_1.csv etc.
    out_csv = str('split/input' + str(i) + '.csv')
    if os.path.exists(out_csv):
	     os.remove(out_csv)
    df.to_csv(out_csv,
          index=False,
          header=False) #size of data to append for each loop
	
print("your files are in [split] directory")

How to use

We have save above file with name “split-csv.py”. You should have “input.csv” file in same folder where script has been stored.

so we will use it like below.

python split-csv.py

View in command prompt