Parsing a text file

This is the place for queries that don't fit in any of the other categories.

Parsing a text file

Postby macpy » Mon Jul 01, 2013 5:59 pm

Hi,

I am a beginner with python and I want to do this: Read in a text file, manipulate the text in some way and then write out another text file. Now, so far, to get the framework of the code going, I have this.

Code: Select all
filename = "....../inputParser.txt"
f_in = open(filename, 'r')
f_out = open("outputParser.txt",'w')
text = f_in.read()
:
:
:
:
:
f_out.write(text)
f_in.close()
f_out.close()


Now, do you have any recommendations as to how I should go about replacing dotted lines above with something more complicated that manipulates the text from an input file? For instance, 1. replace every occurrence of 'abc' by 'cba' and keep doing so if another abc is formed during this replacement; 2. Access some nth word in a comma separated line, 3. Delete any line starting with a # etc; just any kind of text modification. How do I do all such stuff using python? Something in particular that I should read?

Thank you for your help.
Last edited by Yoriz on Mon Jul 01, 2013 6:22 pm, edited 1 time in total.
Reason: Got rid of the colour tags and put the proper code tags in place
macpy
 
Posts: 4
Joined: Mon Jul 01, 2013 5:42 pm

Re: Parsing a text file

Postby stranac » Mon Jul 01, 2013 7:18 pm

Take a look at http://docs.python.org/2/library/stdtyp ... ng-methods and http://docs.python.org/2/library/csv.html

Also, use the with statement when opening files.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 888
Joined: Thu Feb 07, 2013 3:42 pm

Re: Parsing a text file

Postby ochichinyezaboombwa » Mon Jul 01, 2013 11:08 pm

read() will give you the whole file. You likely want to work with your text line by line. Typically it's done like this:
Code: Select all
for line in open(flnm_in): # I didn't create a f_in object; I often don't need it
    # process this line ... and put it to the f_out.
You may also want to look at the Python's with statement (although I personally avoid it).
ochichinyezaboombwa
 
Posts: 200
Joined: Tue Jun 04, 2013 7:53 pm

Re: Parsing a text file

Postby snippsat » Tue Jul 02, 2013 6:39 pm

Python's with statement (although I personally avoid it)

Why avoid it?
As you probably know with statement is doing some error handling for you to in the background.
Raymond Hettinger mean with it`s a clear and way of doing stuff,and in newer versions of Python with statement will be implemented to do more stuff.

example:
Code: Select all
try:
    os.remove('somefile.tmp')
except OSError:
    pass

# Use with
with ignored(OSError):
    os.remove('somefile.tmp')


Code: Select all
old_context = getcontext().copy()
getcontext().prec = 50
print Decimal(355) / Decimal(113)
setcontext(old_context)

# Use with
with localcontext(Context(prec=50)):
    print Decimal(355) / Decimal(113)


Code: Select all
# Old-way to use a lock
lock.acquire()
try:
    print 'Critical section 1'
    print 'Critical section 2'
finally:
    lock.release()

# New-way to use a lock
with lock:
    print 'Critical section 1'
    print 'Critical section 2'
User avatar
snippsat
 
Posts: 85
Joined: Thu Feb 21, 2013 12:04 am

Re: Parsing a text file

Postby Yoriz » Tue Jul 02, 2013 6:42 pm

I'm with with on this one.
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 565
Joined: Fri Feb 08, 2013 1:35 am
Location: UK

Re: Parsing a text file

Postby ochichinyezaboombwa » Tue Jul 02, 2013 9:18 pm

Guys, I was only talking about the
Code: Select all
with open (flnm) as f:...

There are four reasons I avoid it:
  • I don't like the syntax; (IMO it's harder to read compared to for ln in open(flnm) )
  • it creates an explicit file object (same as the loop for i in range uses i which is not used;
  • it is 1 line longer;
  • it wouldn't have improved anything in what i do: dayly open 10s of 1000s of files :-)
But I emphasized that that's my personal p.o.v. and actually recommended looking at with to the OP.

Finally,
Code: Select all
with ignored(OSError):
    os.remove('somefile.tmp')
is a really bad example. I always try to handle and never ignore bad situations.
ochichinyezaboombwa
 
Posts: 200
Joined: Tue Jun 04, 2013 7:53 pm

Re: Parsing a text file

Postby Mekire » Wed Jul 03, 2013 3:09 am

Code: Select all
for line in open(flnm_in): # I didn't create a f_in object; I often don't need it
    # process this line ... and put it to the f_out.

How do you close the file? There is no name referring to it if you open it in the for loop header. Not closing at all is really bad practice.

I resisted with for a while when I started but now use it exclusively.
-Mek
User avatar
Mekire
 
Posts: 816
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: Parsing a text file

Postby snippsat » Wed Jul 03, 2013 7:26 am

How do you close the file? There is no name referring to it if you open it in the for loop header. Not closing at all is really bad practice.

There is no need to close file when you use with statement,this is one of the advantages of using with.

with open dos.
* open a file
* process its contents, and
* make sure to close it.

If you should write the same code in old style format.
Code: Select all
afile = open("/tmp/foo.txt")
try:
    data = afile.read()
finally:
    afile.close()


In newer version of python 2.7---> can also combine multiple expressions in one with statement.
Code: Select all
with open("input") as f, open("output", "w") as f_out:
    f_out.write(f.read())


is a really bad example. I always try to handle and never ignore bad situations.

Yes i agree with this,code is more an example of more ways of using with.
Raymond Hettinger used code in his excellent talk at PyCon 2013.
Transforming Code into Beautiful, Idiomatic Python
User avatar
snippsat
 
Posts: 85
Joined: Thu Feb 21, 2013 12:04 am

Re: Parsing a text file

Postby Mekire » Wed Jul 03, 2013 8:09 am

snippsat wrote:There is no need to close file when you use with statement,this is one of the advantages of using with.

That was my point. He didn't use with.

He used:
Code: Select all
for line in open(flnm_in):
which doesn't give you a way to close the file because there is no name referencing it.

-Mek
User avatar
Mekire
 
Posts: 816
Joined: Thu Feb 07, 2013 11:33 pm
Location: Amakusa, Japan

Re: Parsing a text file

Postby snippsat » Wed Jul 03, 2013 8:26 am

That was my point. He didn't use with.

Ok i misunderstand it,
but anyway showing some of the underlying stuff with dos and linking to Raymond Hettinger excellent talk may make the post useful :oops:
User avatar
snippsat
 
Posts: 85
Joined: Thu Feb 21, 2013 12:04 am

Re: Parsing a text file

Postby ochichinyezaboombwa » Mon Jul 08, 2013 6:48 pm

Guys,
I perfectly understand that files need to be closed and I perfectly understand that not closing them is considered bad practice. However let me emphasize the word "practice" here. In my practice (since Python, because in C it's a totally different picture) I don't close any files, they simply get out of scope quickly and get deleted. I don't say that's good practice but I didn't ever have a practical problem with this practice so I don't give a damn.
ochichinyezaboombwa
 
Posts: 200
Joined: Tue Jun 04, 2013 7:53 pm


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 4 guests