ftplib - download all files from directory and the newer one

This is the place for queries that don't fit in any of the other categories.

ftplib - download all files from directory and the newer one

Postby precyoz » Sun Aug 25, 2013 3:47 pm

Does anyone here know any scripts that downloads all the files from an ftp directory? And if every time the ftp server/site updates(adds a newer data/file), and that when you'll run the script the newer file will only be downloaded and not the existing file.

I really don't have any idea and I really need help about this.
Last edited by Mekire on Wed Nov 06, 2013 7:56 am, edited 1 time in total.
Reason: post_title.lower()
precyoz
 
Posts: 18
Joined: Wed Jul 03, 2013 6:23 am

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby metulburr » Sun Aug 25, 2013 4:16 pm

if server file's last modified date > local directory file's download date, then download, else file is up to date
New Users, Read This
version Python 3.3.2 and 2.7.5, tkinter 8.5, pyqt 4.8.4, pygame 1.9.2 pre
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
User avatar
metulburr
 
Posts: 1122
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby precyoz » Mon Aug 26, 2013 2:58 am

I really didn't get what you mean by that. Can you give me an example script instead?
precyoz
 
Posts: 18
Joined: Wed Jul 03, 2013 6:23 am

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby metulburr » Mon Aug 26, 2013 4:50 am

Code: Select all
import os
import time

filename = "test123.txt"

if not os.path.exists(filename):
    open(filename, 'w')


print("file was modified on {}".format(time.ctime(os.stat(filename).st_mtime)))
New Users, Read This
version Python 3.3.2 and 2.7.5, tkinter 8.5, pyqt 4.8.4, pygame 1.9.2 pre
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
User avatar
metulburr
 
Posts: 1122
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby precyoz » Mon Aug 26, 2013 6:44 am

Along with that script you gave, what should be the script to add so that I can download all the files from the directory? Thanks in advance!
precyoz
 
Posts: 18
Joined: Wed Jul 03, 2013 6:23 am

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby tnknepp » Mon Aug 26, 2013 7:07 pm

From your title I assume you want to download from ftp, correct? Nearly everything I do with ftp involves sftp, but I do have one example of open ftp available (albeit a bit of a kludge). FYI: I was transferring data TO an ftp server for several data sources (hence the dictionaries), but the same general concept should hold.

Code: Select all
import ftplib, os, time, glob

ftp = ftplib.FTP('site','uname','password')
       
# Local/Remote storage directories.
locald = {0:'dir'}

remoted = {0:'dest'}

file_type = {0:'r''}

for d in [0,1]:
    # Transfer level0 data, but only if file is not already on ftp server. 
    transfer = False
    tries = 0

    while transfer == False and tries != 10:
        try:
            ftp.cwd('/root/'+remoted[d]+'/'+'L0') # Change to proper ftp directory. 
            files = glob.glob(locald[d] + '\\*.*')

            for f in files:
               # Here, you should be able to do a modtime = f.sendcmd("MDTM file.name") to get the mod date (I THINK!!)
                if f.split('\\')[-1] not in ftp.nlst():
                    t = open(f,file_type[d])
                    ftp.storlines('STOR ' + f.split('\\')[-1],t)
                    t.close()
        except: transfer = False; tries += 1
        else: transfer = True
    if transfer == True:
        print remoted[d] + ' lev0 files transfered.  '
    else: print remoted[d] + ' lev0 transfer FAILURE.'
Python: 2.7 via Anaconda
Numpy: 1.7
Pandas: 0.11
OS: Windows 7
IDE: Spyder/IPython
User avatar
tnknepp
 
Posts: 117
Joined: Mon Mar 11, 2013 7:41 pm

Re: FTPLIB - DOWNLOAD ALL FILES FROM DIRECTORY AND THE NEWER

Postby precyoz » Wed Nov 06, 2013 7:05 am

I've tried the script but there's an error.
And where should I place my ftp directory and destination folder?


Code: Select all
dest lev0 transfer FAILURE.

Traceback (most recent call last):
  File "C:/Users/turtle/Desktop/Jesse Palmer FTP Downloader/FTP DOWNLOAD/Modified.py", line 32, in <module>
    else: print remoted[d] + ' lev0 transfer FAILURE.'
KeyError: 1
precyoz
 
Posts: 18
Joined: Wed Jul 03, 2013 6:23 am


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 3 guests