Handling CSV Uploads in Django

I spent a bit of time today trying to parse a CSV file uploaded to a Django view.

On the surface, this doesn’t seem too bad. After all, Django has facilities for file upload, and Python handles CSV files quite well. I had a bit of code that a coworker had written to use, although opening the file is really the only interesting part:

file = request.FILES['fileUpload']
data = [row for row in csv.reader(file)]

Unfortunately, when I tested everything out with my first test file, I got a fairly nasty error:

new-line character seen in unquoted field – do you need to open the file in universal-newline mode?

As soon as I see newline issues, I immediately check to see what newline characters are actually being used by the file. Having been created by the Mac version of Excel, this particular CSV had ‘\r’ ending each line, with no ‘\n’, which the Python CSV reader doesn’t interpret as a row-delimiting character.

To top it all off, it’s impossible to open the file in “universal-newline mode” because I’m reading the file out of memory, not off of the disk.

Luckily, after a decent amount of poking around, I found a fairly simple solution:

file = request.FILES['fileUpload']
data = [row for row in csv.reader(file.read().splitlines())]

It seems “splitlines()” is much better at handling newline characters than the Python CSV reader is. This seems to handle all of the different newline combinations that I’ve tried quite nicely.

Eric Jeney is a Java aficionado with a number of projects under his belt. He enjoys programming for the web, and writing about his experiences. He currently studies Computer Science at the University of Maryland.

View all posts by Eric →