For Loop not Iterating over Files in Directory

This is the place for queries that don't fit in any of the other categories.

For Loop not Iterating over Files in Directory

Postby 5tanczak » Sun Sep 01, 2013 3:55 pm

I am trying to get the first for loop below "for fn in dirs:" to iterate over a list of files in a directory and merge the ones with similar filenames. The loop seems to only execute once, instead of for all files listed in "dirs = os.listdir(path)". I have tried several things mentioned on other posts but nothing has worked.

Can you help me understand why this for loop is not iterating?

Code: Select all
    import pandas as pd
    import os
    import glob
   
    path = r"C:\Users\St\Documents\House\m2"
    dirs = os.listdir(path)  #list of .csv files
   
    for fn in dirs:
        for fn in glob.glob(os.path.join(path, '*2.csv')):
            W = fn     
            if len(W) == 48:   # some filenames are one character shorter
                root2 = W[:43]
            else:
                root2 = W[:42]
        for fn in glob.glob(os.path.join(path, '*J.csv')):
            J = fn
            if len(J) == 48:
                jroot = J[:43]
            else:
                jroot = J[:42]
        for fn in glob.glob(os.path.join(path, '*T.csv')):
            T = fn     
            if len(T) == 48:
                troot = T[:43]
            else:
                troot = T[:42]         
        for fn in glob.glob(os.path.join(path, '*S.csv')):
            S = fn     
            if len(S) == 48:
                sroot = S[:43]
            else:
                sroot = S[:42]             
        for fn in glob.glob(os.path.join(path, '*D.csv')):
            D = fn     
            if len(D) == 48:
                droot = D[:43]
            else:
                droot = D[:42]
        for fn in glob.glob(os.path.join(path, '*R.csv')):
            R = fn       
            if len(R) == 48:               
                rroot = R[:43]
            else:
                rroot = R[:42]       
               
        WW = pd.read_csv(W) #read csv so I can use pd.merge
        JJ = pd.read_csv(J)   
        TT = pd.read_csv(T)
        SS = pd.read_csv(S)
        DD = pd.read_csv(D)
        RR = pd.read_csv(R)
               
        if jroot == root2:  #merge files with names that start with same characters
            JW = pd.merge(WW, JJ, on="JN")
        if jroot == troot:
            TW = pd.merge(JW, TT, on="TN")
        if jroot == sroot:
            SW = pd.merge(TW, SS, on="SN")   
        if jroot == droot:
            DW = pd.merge(SW, DD, on="DN")
        if jroot == rroot:
            RW = pd.merge(RR, DW, on="HN")
            data = RW
            data.to_csv(jroot + "x.csv")  #export final merged dataframe to .csv
5tanczak
 
Posts: 3
Joined: Fri Aug 30, 2013 9:46 pm

Re: For Loop not Iterating over Files in Directory

Postby Yoriz » Sun Sep 01, 2013 4:23 pm

The first thing that jumps out at me is that your first loop sets each dirs to the name fn and doesn't get used anywhere in the rest of the code as the following loops then also set fn to something else inside inner loops.
New Users, Read This
Join the #python-forum IRC channel on irc.freenode.net!
Spam topic disapproval technician
Windows7, Python 2.7.4., WxPython 2.9.5.0., some Python 3.3
User avatar
Yoriz
 
Posts: 782
Joined: Fri Feb 08, 2013 1:35 am
Location: UK


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 5 guests