Home Code How to read a large text file character by character in Python?

How to read a large text file character by character in Python?

Every python programmer knows how to read a text file line by line without loading the entire file into memory using the below simple script.

with open('text_file.txt') as fl:
    for line in fl:
        do_some_processing(line)

But what if you want to load a text file character by character? Possible use case is when the text file is a huge file with all its contents in a single line without any newline or line breaking character.

So what if the text file is in a single large line, and the size of the file does not fit into your system’s RAM or memory? In that case you will have to read the text file one character at a time using the Python script below.

with open('text_file_with_single_large_line.txt') as fl:
    while True:
        #read a huge chunk of characters. 
        #Increase this if you want to read more characters in every single read
        chars=infile.read(1000000)

        #no more pending characters in file?
        if not chars:break

        #process the data
        do_some_processing(chars)
Content Protection by DMCA.com
Gurudevhttps://www.hitxp.com
Gurudev is the developer of Gurunudi AI Platform. This is his official website where he pens his thoughts on a wide range of topics, answers queries, shares resources and tools developed by him.

LEAVE A REPLY

Please enter your comment!
Please enter your name here

4,258FansLike
7FollowersFollow
133FollowersFollow
380FollowersFollow

Subscribe to HitXP Articles

To be updated with all the latest articles, offers and special announcements.

Latest Articles

Corona Virus, SARS – Why are new viruses increasingly originating from China?

The reason why new strains of deadly viruses are emerging from China. Viruses like coronavirus and SARS should be stopped from evolving and spreading to prevent another global pandemic.

Evolution of life – primitive cells, complex organisms, intelligence, what next after humans?

Evolution of life from primitive life forms to complex multi cellular organisms to intelligent human species - the advent of AI raises a natural question - What next after human?

The individual human is smart, but the human species is dumb

The smart individual human has to clean up the mess created by his dumb species. Species polluted the planet, individuals are cleaning it up.

What if Dinosaurs did not go extinct 65 million years ago?

Imagine a world where dinosaurs did not go extinct, because that asteroid missed colliding with Earth. How would have life evolved then?
Content Protection by DMCA.com