Effortlessly Convert Large CSV Files to JSON Using Python
Written on
Chapter 1: Introduction to CSV to JSON Conversion
In my exploration of Python and the Google Ads API for automating keyword planning, I encountered a sizable CSV file that I needed to transform into a structured JSON format. To facilitate this conversion, I crafted a quick Python script that I’m excited to share with you.
While working with CSV files in Python is straightforward, my goal was to convert the data into a keyed JSON file instead. Coming from a background in PHP and JavaScript, I find JSON files to be more visually appealing and more practical in various applications compared to CSV files. JSON is particularly useful for tasks like database storage (though I wouldn’t recommend it for that), embedding in HTML documents, or integrating with frameworks like VueJS or React. You get the idea.
Initially, I tried several online CSV to JSON converters, but they all crashed due to the file size. Realizing that I needed a different approach, I thought, "If the mountain won't come to Muhammad, then Muhammad must go to the mountain." Consequently, I developed a Python script that you can run from the command line to convert a CSV file into a keyed JSON file.
Here's the Python script:
import csv
import json
import sys
import os
def csv_to_json(csv_file, json_file):
"""
Convert a CSV file to a JSON file
"""
# Open the CSV
with open(csv_file, 'r') as csv_file:
reader = csv.DictReader(csv_file)
data = list(reader)
# Convert to JSON
json_data = json.dumps(data, indent=4)
# Create parent directories if they don't exist
if not os.path.exists(os.path.dirname(json_file)):
os.makedirs(os.path.dirname(json_file))
# Write JSON to a file
with open(json_file, 'w') as json_file:
json_file.write(json_data)
def main():
files = sys.argv[1].split(' ')
csv_to_json(files[0], files[1])
if __name__ == "__main__":
main()
Section 1.1: How to Use the Script
To use this script, save the code in a file named csv_to_json.py. Then, execute the script by running:
python csv_to_json.py input_csv_file_path_and_name.csv output_file_path_and_name.json
Feel free to use any folder structure; the script will automatically create the necessary directories.
Chapter 2: Visual Demonstrations
The first video, Convert CSV to JSON with Python, provides a step-by-step guide on using Python for this conversion process.
The second video, How to Convert CSV to JSON in Python, offers additional tips and tricks for efficient file conversion.