File Processing - help (noob)

This is the place for queries that don't fit in any of the other categories.

File Processing - help (noob)

Postby ylafont » Mon Apr 07, 2014 6:20 am

I have various xlm files that need processing with the code below

Code: Select all
#!/usr/bin/env python
import sys,os,re,fnmatch
import subprocess
import xml.etree.ElementTree as ET
# tree = ET.parse('xmltv.xml')
# root = tree.getroot()
Dir ='/home/xbmc/tmp/xmltv/'
#FilesToProcess = ('*.xml')
FilesToProcess = ('xmltv.xml')

def ProcessFiles(self, *args):
   TotalChannels = []
   ChannelCount = 0
   LinesProcessed = 0
   MinChannels = 0
   MaxChannels = 0
   Count = 0
   for Channel in root.iter('channel'):
      ChannelCount += 1
      ChannelID = '.'.join(Channel.attrib.get('id').split('.')[0:2])
      
      if Count > 0:
         TotalChannels.append (Count)
         MinChannels = min(TotalChannels)
         MaxChannels = max(TotalChannels)
         
      Count = 0   
      for ChannelInfo in Channel:   
         if ChannelInfo.tag == 'display-name':
            Count += 1   
            LinesProcessed += 1   

      # print (MinChannels, MaxChannels,'\t', ChannelID )
   # print ('')
   print ('Processed: ', FilesToProcess,'\t', MinChannels, MaxChannels,'\t',ChannelCount,'\t', LinesProcessed)   
   # print ('Total Channels Processed ' , ChannelCount)
   # print ('Total Lines Processed ', LinesProcessed)
   return
      
for file in os.listdir(Dir):
   if fnmatch.fnmatch(file, FilesToProcess):
      tree = ET.parse(FilesToProcess)
      root = tree.getroot()
      ProcessFiles(FilesToProcess)


which returns the following output
Code: Select all
Processed:  xmltv.xml  4, 5,  83,  411


However I cannot seem to get it process all *.xml files in a directory. when i use

FilesToProcess = ('*.xml')

Gives me the following:
Code: Select all
Traceback (most recent call last):
  File "./fs.py", line 55, in <module>
    tree = ET.parse(FilesToProcess)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 1183, in parse
    tree.parse(source, parser)
  File "/usr/lib/python2.7/xml/etree/ElementTree.py", line 647, in parse
    source = open(source, "rb")
IOError: [Errno 2] No such file or directory: '*.xml'


I under that the error, However i don't understand why the files names are not being passed. should i be using a different method?

thanks in advance.
ylafont
 
Posts: 23
Joined: Tue Apr 01, 2014 10:29 pm

Re: File Processing - help (noob)

Postby stranac » Mon Apr 07, 2014 9:37 am

Code: Select all
      tree = ET.parse(FilesToProcess)

You're trying to parse a file named *.xml. There is of course no such file.
I'm assuming you meant to pass file(which is probably a bad name, since it's a name of a built-in in python 2)

This is not really a good idea:
Code: Select all
def ProcessFiles(filename):
    # do stuff with tree and root globals

#...
      tree = ET.parse(FilesToProcess)
      root = tree.getroot()
      ProcessFiles(FilesToProcess)

Better would be:
Code: Select all
def ProcessFiles(filename):
    tree = ET.parse(filename)
    root = tree.getroot()
    # do stuff


Also, you might want to take a look at glob.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1246
Joined: Thu Feb 07, 2013 3:42 pm

Re: File Processing - help (noob)

Postby ylafont » Mon Apr 07, 2014 3:58 pm

Code: Select all
for file in os.listdir(Dir):
   if fnmatch.fnmatch(file, FilesToProcess):
      tree = ET.parse(FilesToProcess)
      root = tree.getroot()
      ProcessFiles(FilesToProcess)


I want to pass the results if fnmatch which should be all xml files in the directory, Sine the command works , i would assume that the list of files would be passed one at a time. You are correct, I did want to process file, not sure how I missed that.

Code: Select all
Processed:  3.xml        4 5     90      423
Processed:  allguide.xml         5 6     103     583
Processed:  2.xml        4 5     560     2755
Processed:  6.xml        5 6     103     583
Processed:  4.xml        4 5     34      150
Processed:  10.xml       4 5     893     4399
Processed:  12.xml       4 5     1395    6823
Processed:  xmltv.xml    4 5     83      411
Processed:  11.xml       4 5     1033    5021
Processed:  0.xml        0 0     1       5
Processed:  7.xml        4 5     736     3628
Processed:  8.xml        4 5     92      433
Processed:  13.xml       4 5     737     3633
Processed:  18.xml       4 5     83      411
Processed:  5.xml        4 5     897     4420
Processed:  16.xml       4 5     1347    6607
Processed:  9.xml        4 5     61      285
Processed:  1.xml        0 0     1       5
Processed:  17.xml       5 5     126     630
Processed:  15.xml       4 5     981     4793
Processed:  14.xml       4 5     13      64


Will try glob next. to see what the difference is.
ylafont
 
Posts: 23
Joined: Tue Apr 01, 2014 10:29 pm


Return to General Coding Help

Who is online

Users browsing this forum: wichita12 and 5 guests

cron