How to split CSV File into multiple Files with specific records

You are here:
< All Topics

Sometime you should have lots of record in one file , you might need it to split in multiple files so that you can import in software easily.

Here , we have implement small script using python to do this.

Python Script

import pandas as pd
import os
import shutil

#give the csv file to read
input_csv = 'input.csv'

#checking number of lines in csv
number_lines = sum(1 for row in (open(input_csv)))

#number of record, should in each csv
rowsize = 1000

#creating split folder to store new csv and deleting old csv before storing.
    # Remove folder (if exists) with all files
    if os.path.isdir(str("split")):
        shutil.rmtree("split", ignore_errors=True)
    # Create new folder
except IOError:
    print("Error upon either deleting or creating the directory or files.")
    print("Actions if file access was succesfull")

#start looping through data writing it to a new file for each set
for i in range(1,number_lines,rowsize):
    df = pd.read_csv(in_csv,
          nrows = rowsize, #number of rows to read at each loop
          skiprows = i) #skip rows that have been read
    #csv to write data to a new file with indexed name. input_1.csv etc.
    out_csv = str('split/input' + str(i) + '.csv')
    if os.path.exists(out_csv):
          header=False) #size of data to append for each loop
print("your files are in [split] directory")

How to use

We have save above file with name “”. You should have “input.csv” file in same folder where script has been stored.

so we will use it like below.


View in command prompt

Table of Contents
Send a Message