Webpage & Downloading Links

This is the place for queries that don't fit in any of the other categories.

Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 9:50 pm

Hi,

I've created two scripts and they both run but my second script, to download/parse links doesn't return any links. Can anyone tell see why? I'm relatively new to Python and not completely sure what the problem could be.

Webpage script;
Code: Select all
import sys, urllib
def getWebpage(url):
    print '[*] getWebpage()'
    url_file = urllib.urlopen(url)
    page = url_file.read()
    return page
def main():
    sys.argv.append('http://www.funeralformyfat.tumblr.com)
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_get URL'
        return
    else:
        print getWebpage(sys.argv[1])

if __name__ == '__main__':
    main()


Links script;
Code: Select all
import sys, urllib
def print_links(page):
    print '[*] print_links()'
    links = re.findall(r'\<a.*href\=.*http\:.+', page)
    links.sort()
    print '[+]', str(len(links)), 'HyperLinks Found:'

    for link in links:
        print link
   
def main():
    sys.argv.append('http://www.funeralformyfat.tumblr.com')
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)

       
if __name__ == '__main__':
    main()
Last edited by Yoriz on Fri Nov 22, 2013 10:23 pm, edited 1 time in total.
Reason: First post lock
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby stranac » Fri Nov 22, 2013 10:39 pm

RandomPast wrote:
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)


The print_links() call is inside the if block, after the return statement. It can never be executed.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1092
Joined: Thu Feb 07, 2013 3:42 pm

Re: Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 11:08 pm

Hi, thanks for answering. Where do you think I should move it to? Because I've tried moving it to a number of places and still nothing happens.
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby micseydel » Fri Nov 22, 2013 11:13 pm

Do you know what return does? You should really try to understand exactly what your code is doing.
Join the #python-forum IRC channel on irc.freenode.net!
User avatar
micseydel
 
Posts: 1113
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 11:16 pm

micseydel wrote:Do you know what return does? You should really try to understand exactly what your code is doing.

Yes I know what return does, but I'm not 100% sure about the layout at this moment. I've only been using Python for a few weeks.
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby micseydel » Sat Nov 23, 2013 12:29 am

So that you realize your code quoted above,
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)

is equivalent to
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return

then? In face, in-context, the return isn't even necessary. The function ends implicitly right there.
Join the #python-forum IRC channel on irc.freenode.net!
User avatar
micseydel
 
Posts: 1113
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA


Return to General Coding Help

Who is online

Users browsing this forum: mighty and 2 guests