Manipulating Data in a Text File
You can open a file, read it and write it. This lesson puts those together into the four jobs every board question asks for — count something, find something, change something, remove something — and they all turn out to be the same three steps.
1The file these programs read
Every program on this page runs on the same file, story.txt. Five lines:
India is a land of many festivals.
The lights of Diwali are famous.
A festival brings the people together.
Every state has its own colours.
People travel home to celebrate them.2Counting characters, words and lines
Read the file once, then use the string methods you already know. split() with no argument breaks text at every run of blank space, so it counts words. splitlines() breaks at every newline.
f = open('story.txt', 'r')
data = f.read()
f.close()
words = data.split()
lines = data.splitlines()
print('Characters:', len(data))
print('Words:', len(words))
print('Lines:', len(lines))Characters: 178 Words: 31 Lines: 5
\n. If a question asks for the number of characters excluding spaces, you have to take them out yourself — usually with a loop and if ch != ' '.For counting per line, walk the file instead. This is the shape most exam answers take:
f = open('story.txt', 'r')
count = 0
for line in f:
if line[0] in 'AEIOUaeiou':
count = count + 1
f.close()
print('Lines starting with a vowel:', count)Lines starting with a vowel: 3
for line in f:One line at a time, in order. The line still has its \n on the end, which does not matter here because we only look at line[0].
line[0]The first character of the line. Indexing a string, exactly as in Class 11.
in 'AEIOUaeiou'Membership on a string: is this character one of these ten? Both cases, because India and Every start with capitals.
3Finding and counting a word
To count a word, split each line into words and compare whole words. Counting with data.count('the') instead answers a different question, and the difference is the trap:
f = open('story.txt', 'r')
data = f.read()
f.close()
whole = 0
for word in data.split():
if word.strip('.,').lower() == 'the':
whole = whole + 1
print("'the' as a whole word:", whole)
print("count('the') on the whole text:", data.count('the'))'the' as a whole word: 2
count('the') on the whole text: 3count() found three: the word the, and the the hiding inside together and them. The loop found two: The and the, as whole words, once .lower() had made the capital match. Read the question carefully — “how many times does the word appear” is the loop, not count().split() breaks at spaces only, so the last word of a sentence arrives as 'together.' with the full stop attached. strip('.,') takes those punctuation marks off both ends before the comparison.4Changing a file: the three steps
A text file cannot be edited where it lies. The lines are different lengths, so replacing a short line with a longer one would land on top of the next one — you saw that with 'r+' in the modes lesson. Every change therefore goes the same way round:
lines = f.readlines()The whole file into a list (or read() into one string), then close it.
lines, data = …Ordinary list and string work. The file is not open and nothing has happened to it yet.
f.writelines(lines)Open the same file in 'w' — which empties it — and write the new version.
5Updating — changing a word everywhere
f = open('story.txt', 'r')
data = f.read()
f.close()
data = data.replace('festival', 'mela')
f = open('story.txt', 'w')
f.write(data)
f.close()
print(open('story.txt').read())India is a land of many melas. The lights of Diwali are famous. A mela brings the people together. Every state has its own colours. People travel home to celebrate them.
festivals and now says melas — replace() found festival inside it and swapped that part out. Sometimes that is what you want. When it is not, split the line into words and compare whole words instead.6Deleting — removing the lines you do not want
You never delete a line from a file. You write a new file that does not have it in:
f = open('story.txt', 'r')
lines = f.readlines()
f.close()
kept = []
for line in lines:
if 'Diwali' not in line:
kept.append(line)
f = open('story.txt', 'w')
f.writelines(kept)
f.close()
print(open('story.txt').read())India is a land of many festivals. A festival brings the people together. Every state has its own colours. People travel home to celebrate them.
kept avoids that and reads better anyway.7Inserting a line in the middle
Same three steps, and the middle one is insert() from the Class 11 list methods. Remember the \n:
f = open('story.txt', 'r')
lines = f.readlines()
f.close()
lines.insert(2, 'Holi fills the streets with colour.\n')
f = open('story.txt', 'w')
f.writelines(lines)
f.close()
print(open('story.txt').read())India is a land of many festivals. The lights of Diwali are famous. Holi fills the streets with colour. A festival brings the people together. Every state has its own colours. People travel home to celebrate them.
8The safer way: write to a second file
Every program above holds the whole file in memory between the read and the write. If the program crashed in between, the file would already be empty. The professional version writes a new file and only then replaces the old one:
import os
source = open('story.txt', 'r')
temp = open('temp.txt', 'w')
for line in source:
if 'Diwali' not in line:
temp.write(line)
source.close()
temp.close()
os.remove('story.txt')
os.rename('temp.txt', 'story.txt')
print(open('story.txt').read())India is a land of many festivals. A festival brings the people together. Every state has its own colours. People travel home to celebrate them.
temp = open('temp.txt', 'w')A second file, opened at the same time. The original is only ever read.
os.remove('story.txt')Deletes the old file. Nothing has been lost yet — temp.txt holds the new version, already written and closed.
os.rename('temp.txt', 'story.txt')Renames the new file into the old one's place. Now story.txt is the edited file.
9Try it
Change the word being searched for, or the word it is replaced with, and run it again:
10Recap
The three steps behind every update, delete and insert. A text file cannot be edited where it lies.
With no argument it breaks at every run of blank space. splitlines() breaks at every newline.
It counts a piece of text wherever it appears, including inside other words.
split() leaves the full stop stuck to the last word of a sentence.
Build a list of the lines you want to keep and write that list back.
Write a temp file, delete the original, rename the temp into its place. The version that cannot lose your data.
- 1
Count how many lines in a file are longer than 30 characters.
Hint · len(line.strip()) — otherwise the \n is counted too.
- 2
Replace every occurrence of one name with another, throughout a file.
Hint · read(), replace(), then write it back in mode 'w'.
- 3
Remove every blank line from a file.
Hint · A blank line is '\n', so line.strip() is empty for it.
- 4
Add a line at the top of a file without losing what is already there.
Hint · readlines(), lines.insert(0, ...), writelines().
Why can a program not simply edit line 3 of a text file?
data.count('the') gives 3 but the word 'the' appears twice. Why?
Which mode do you open the file in for step 3, writing the changed version back?
What is the point of writing to temp.txt and then renaming it?