A BASH shell script to import a large number of CSV files into MySQL
In this article I’ll share a bash shell script I use periodically to import a directory of CSV files into MySQL tables. This script is most helpful when you need to process a large number of CSV files, and the tables have not yet been created. Of course you could use a GUI tool to accomplish this, but what’s the fun in that?
The following script will get a list of CSV files, loop through them, add each table, add each column to the table (based on the first row), and then use the mysqlimport command to load all the CSV records. There are a few caveats though: 1. the first row of each CSV file must contain the column names; 2. it works best when your column names are simple text; and 3. your MySQL user must have permission to process files (see: File_priv).
After creating my shell script file, and making it executable, I executed it. Since I added the line “set -x” to the script, a lot of helpful info is shown to debug.
Lastly I executed some SQL on the command line to verify the results. nice.