Python Data Persistence - File API

Python uses built-in input() and print() functions to perform standard input/output operations. The input() function reads bytes from a standard input stream device, i.e. keyboard.

The print() function on the other hand, sends the data towards standard output stream device i.e. the display monitor. Python program interacts with these IO devices through standard stream objects stdin and stdout defined in sys module.

The input() function is actually a wrapper around readline() method of sys.stdin object. All keystrokes from the input stream are received till ‘Enter’ key is pressed.

>>> import sys
>>> x=sys.stdin.readline()
Welcome to Howcodex
>>> x
'Welcome to Howcodex\n'

Note that, readline() function leave a trailing ‘\n’ character. There is also a read() method which reads data from standard input stream till it is terminated by Ctrl+D character.

>>> x=sys.stdin.read()
Hello
Welcome to Howcodex
>>> x
'Hello\nWelcome to Howcodex\n'

Similarly, print() is a convenience function emulating write() method of stdout object.

>>> x='Welcome to Howcodex\n'
>>> sys.stdout.write(x)
Welcome to Howcodex
26

Just as stdin and stdout predefined stream objects, a Python program can read data from and send data to a disk file or a network socket. They are also streams. Any object that has read() method is an input stream. Any object that has write() method is an output stream. The communication with the stream is established by obtaining reference to the stream object with built-in open() function.

open() function

This built-in function uses following arguments −

f=open(name, mode, buffering)

The name parameter, is name of disk file or byte string, mode is optional one-character string to specify the type of operation to be performed (read, write, append etc.) and buffering parameter is either 0, 1 or -1 indicating buffering is off, on or system default.

File opening mode is enumerated as per table below. Default mode is ‘r’

In order to save data to file it must be opened with ‘w’ mode.

f=open('test.txt','w')

This file object acts as an output stream, and has access to write() method. The write() method sends a string to this object, and is stored in the file underlying it.

string="Hello Howcodex\n"
f.write(string)

It is important to close the stream, to ensure that any data remaining in buffer is completely transferred to it.

file.close()

Try and open ‘test.txt’ using any test editor (such as notepad) to confirm successful creation of file.

To read contents of ‘test.txt’ programmatically, it must be opened in ‘r’ mode.

f=open('test.txt','r')

This object behaves as an input stream. Python can fetch data from the stream using read() method.

string=f.read()
print (string)

Contents of the file are displayed on Python console. The File object also supports readline() method which is able to read string till it encounters EOF character.

However, if same file is opened in ‘w’ mode to store additional text in it, earlier contents are erased. Whenever, a file is opened with write permission, it is treated as if it is a new file. To add data to an existing file, use ‘a’ for append mode.

f=open('test.txt','a')
f.write('Python Tutorials\n')

The file now, has earlier as well as newly added string. The file object also supports writelines() method to write each string in a list object to the file.

f=open('test.txt','a')
lines=['Java Tutorials\n', 'DBMS tutorials\n', 'Mobile development tutorials\n']
f.writelines(lines)
f.close()

Example

The readlines() method returns a list of strings, each representing a line in the file. It is also possible to read the file line by line until end of file is reached.

f=open('test.txt','r')
while True:
   line=f.readline()
   if line=='' : break
   print (line, end='')
f.close()

Output

Hello Howcodex
Python Tutorials
Java Tutorials
DBMS tutorials
Mobile development tutorials

Binary mode

By default, read/write operation on a file object are performed on text string data. If we want to handle files of different other types such as media (mp3), executables (exe), pictures (jpg) etc., we need to add ‘b’ prefix to read/write mode.

Following statement will convert a string to bytes and write in a file.

f=open('test.bin', 'wb')
data=b"Hello World"
f.write(data)
f.close()

Conversion of text string to bytes is also possible using encode() function.

data="Hello World".encode('utf-8')

We need to use ‘rb’ mode to read binary file. Returned value of read() method is first decoded before printing.

f=open('test.bin', 'rb')
data=f.read()
print (data.decode(encoding='utf-8'))

In order to write integer data in a binary file, the integer object should be converted to bytes by to_bytes() method.

n=25
n.to_bytes(8,'big')
f=open('test.bin', 'wb')
data=n.to_bytes(8,'big')
f.write(data)

To read back from a binary file, convert output of read() function to integer by from_bytes() function.

f=open('test.bin', 'rb')
data=f.read()
n=int.from_bytes(data, 'big')
print (n)

For floating point data, we need to use struct module from Python’s standard library.

import struct
x=23.50
data=struct.pack('f',x)
f=open('test.bin', 'wb')
f.write(data)

Unpacking the string from read() function, to retrieve the float data from binary file.

f=open('test.bin', 'rb')
data=f.read()
x=struct.unpack('f', data)
print (x)

Simultaneous read/write

When a file is opened for writing (with ‘w’ or ‘a’), it is not possible, to read from it and vice versa. Doing so throws UnSupportedOperation error. We need to close the file before doing other operation.

In order to perform both operations simultaneously, we have to add ‘+’ character in the mode parameter. Hence, ‘w+’ or ‘r+’ mode enables using write() as well as read() methods without closing a file. The File object also supports seek() function to rewind the stream to any desired byte position.

f=open('test.txt','w+')
f.write('Hello world')
f.seek(0,0)
data=f.read()
print (data)
f.close()

Following table summarizes all the methods available to a file like object.