Computer Science · Class 12 · CSV Files

CSV filesnewline=''⏱️ 14 min read

The Extra Blank Line

You have been typing newline='' for five lessons on trust. Here is what it is for. Leave it out on a Windows machine and your CSV comes back with a blank row between every record — and the cause is genuinely interesting, because it is two different pieces of software both being helpful at once.

1The symptom

Here is the program from the writing lesson with one thing removed: the newline=''. Run on a Windows machine, it produces this file:

no_newline.py

import csv

f = open('marks.csv', 'w')      # <- newline='' is missing
w = csv.writer(f)

w.writerow(['Roll', 'Name', 'Marks'])
w.writerow([1, 'Ravi', 78])
w.writerow([2, 'Meera', 91])
w.writerow([3, 'Amit', 65])

f.close()

marks.csv — opened in Notepad

Roll,Name,Marks

1,Ravi,78

2,Meera,91

3,Amit,65

Open it in Excel and you get the same thing: a grid with an empty row between every filled one. And read it back in Python and the damage is plainer still:

reading_it_back.py

import csv

f = open('marks.csv', 'r', newline='')
for row in csv.reader(f):
    print(row)
f.close()

Output

['Roll', 'Name', 'Marks']
[]
['1', 'Ravi', '78']
[]
['2', 'Meera', '91']
[]
['3', 'Amit', '65']
[]

Four rows written, eight rows read

Those [] entries are real rows as far as your program is concerned. A record count reports 8. A loop that does int(row[2]) crashes on the first blank one with IndexError: list index out of range, because an empty list has no position 2.

2Where the extra line comes from

Two separate pieces of software each add a line ending, and neither knows about the other.

The csv writer ends every row with \r\n

Not '\n'. The CSV standard says a row ends with a carriage return followed by a line feed, and the writer follows it — on every operating system, including this one.

Windows text mode turns every \n into \r\n

This is what open() in text mode does on Windows, and it is normally a kindness: you type '\n' and the file gets the ending Windows programs expect. But the writer has just put a '\n' there, and open() has no way of knowing it was already part of a \r\n.

So the file gets \r\r\n

One carriage return from the writer, then the \r\n that Windows made out of the writer's \n. That is one line ending followed by another — which is a blank line.

You can see all of it in the bytes. Opening the file in 'rb' shows the raw characters with nothing translated:

the_bytes.py

print(open('marks.csv', 'rb').read())

Output

b'Roll,Name,Marks\r\r\n1,Ravi,78\r\r\n2,Meera,91\r\r\n3,Amit,65\r\r\n'

\r\r\n — two carriage returns where there should be one. With newline='' in the open(), the same program writes:

the_bytes_fixed.py

print(open('marks.csv', 'rb').read())

Output

b'Roll,Name,Marks\r\n1,Ravi,78\r\n2,Meera,91\r\n3,Amit,65\r\n'

3Flip the switch and watch

🩹 The same program, on a Windows machine

Four writerow() calls, written to marks.csv. The only thing that changes is the switch.

f = open('marks.csv', 'w', newline='')

the bytes on the disk

Roll,Name,Marks\r\n

1,Ravi,78\r\n

2,Meera,91\r\n

3,Amit,65\r\n

One \r\n at the end of each row — the ending the csv writer chose, written down untouched.

opened in Notepad

Roll,Name,Marks

1,Ravi,78

2,Meera,91

3,Amit,65

for row in csv.reader(f)

['Roll', 'Name', 'Marks']

['1', 'Ravi', '78']

['2', 'Meera', '91']

['3', 'Amit', '65']

4 rows read

Four rows written, four rows read. Every program that counts records now gets the right answer.

4What newline='' actually does

open('marks.csv', 'w', newline='')

“Do not translate line endings. Write down exactly the characters you are given.”

It does not add anything and it does not remove anything. It switches off a translation that open() would otherwise perform. Step 2 of the three above simply never happens, so the writer's \r\n reaches the disk unchanged.

Key Takeaway

The two quote marks have nothing between them. It is an empty string, not a space and not '\n'. newline=' ' with a space in it is a different setting entirely, and newline='\n' is another one again.

5Reading as well as writing

The blank-line problem is a writing problem, so newline='' on a file you are only reading fixes nothing. Put it there anyway, for a different reason: a CSV value is allowed to contain a line break inside its quotes, like an address spread over two lines. Without newline='' the file translates those breaks before the reader ever sees them, so the value you get back is not quite the value that was written:

embedded_break.py

import csv

f = open('addr.csv', 'w', newline='')
csv.writer(f).writerow(['Ravi', 'House 12\r\nSector 4'])
f.close()

f = open('addr.csv', 'r', newline='')
print("with newline='' :", list(csv.reader(f)))
f.close()

f = open('addr.csv', 'r')
print("without         :", list(csv.reader(f)))
f.close()

Output

with newline='' : [['Ravi', 'House 12\r\nSector 4']]
without         : [['Ravi', 'House 12\nSector 4']]

Two values either way, so nothing crashes — but the address itself came back altered. This is a corner you are unlikely to meet in a board question, and it is the reason the habit is worth having anyway.

One habit, both directions

Put newline='' in every open() that touches a CSV file, reading or writing. It never causes a problem and it prevents two, so there is nothing to weigh up each time.

6Why you may not see the bug on your own computer

Step 2 is a Windows behaviour. On Linux and on a Mac, text mode writes \n as \n and translates nothing, so a program with no newline='' produces a perfectly good file. Two students can run identical code and only one of them has the bug.

Let's Recap!

	Windows	Linux / macOS
Text mode translates \n to…	\r\n	\n — nothing changes
CSV written without newline=''	\r\r\n — a blank line per row	\r\n — correct
CSV written with newline=''	\r\n — correct	\r\n — correct
So is newline='' needed?	yes	no, but it changes nothing

Write it even if your computer does not need it

School machines and exam labs run Windows, and a program that only works on your laptop is not a working program. The bottom row of that table is the argument: with newline='' the result is correct everywhere, so there is never a reason to leave it out.

7Two other repairs you will see — and why this one is better

if row == []: continue

Skips the blank rows while reading. It works, but the file is still wrong — Excel still shows the gaps, and every other program that reads it still has to know about the workaround.

open('marks.csv', 'wb')

Binary mode does no translation, so this was the Python 2 answer. In Python 3 the csv writer produces strings, so it fails with TypeError: a bytes-like object is required, not 'str'.

newline='' fixes the file rather than working around it, which is why it is the answer the documentation gives and the one to write in an exam.

8See the bytes for yourself

This playground runs on Linux, so it shows the correct side of the story: the writer's own \r\n, with or without the argument. It is worth running to fix in your mind what a healthy CSV looks like underneath.

see_the_bytes.py

import csv

f = open('marks.csv', 'w', newline='')
w = csv.writer(f)
w.writerow(['Roll', 'Name', 'Marks'])
w.writerow([1, 'Ravi', 78])
f.close()

print(open('marks.csv', 'rb').read())

# On Windows, removing newline='' above turns every
# \r\n in that output into \r\r\n.

import csv

f = open('marks.csv', 'w', newline='')
w = csv.writer(f)
w.writerow(['Roll', 'Name', 'Marks'])
w.writerow([1, 'Ravi', 78])
f.close()

print(open('marks.csv', 'rb').read())

# On Windows, removing newline='' above turns every
# \r\n in that output into \r\r\n.

open('marks.csv', 'rb').read()

'rb' is reading, binary — no translation at all, so you see the characters that are genuinely on the disk.

b'...'

The b in front means these are bytes rather than a string. It is the same b you saw on a pickled file.

\r\n

The healthy ending: one carriage return, one line feed. \r\r\n is the broken one.

9Recap

The symptom

A blank line between every record in Notepad and Excel, and an empty list [] between every row when you read it back.

The csv writer ends rows with \r\n

That is the CSV standard, and it does it on every operating system.

Windows text mode adds another \r

It turns the writer's \n into \r\n, making \r\r\n — two line endings, so a blank line.

newline='' switches translation off

open() then writes exactly the characters it is given. It adds nothing.

Two empty quotes

newline='' — nothing between them. Not a space, not '\n'.

Put it in every CSV open()

Reading and writing, on every machine. It is never wrong, and leaving it out is sometimes badly wrong.

✍️ Now write these yourself

1
Write a CSV and print open('marks.csv', 'rb').read(). Find the \r\n at the end of each row.
Hint · The b in front of the quotes tells you these are bytes.
2
On a Windows machine, run the same program without newline='' and print the bytes again.
Hint · \r\r\n. On Linux or a Mac nothing changes — that is the point of section 6.
3
Count the rows a reader gives back from a file that has the blank-line problem.
Hint · Twice as many as you wrote. Half of them are [].
4
Try int(row[2]) on a file with blank rows and read the error.
Hint · IndexError — an empty list has no position 2.

Quick Check

What does newline='' do?

Quick Check

Why does a blank line appear between the rows on Windows?

Quick Check

Reading a CSV that has the blank-line problem, what does csv.reader() give you between the records?

Quick Check

Your program runs correctly on your Mac without newline=''. Should you add it?

← Previous

📖 reader()

Choosing the Separator 🔀