Webpage & Downloading Links

This is the place for queries that don't fit in any of the other categories.

Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 9:50 pm

Hi,

I've created two scripts and they both run but my second script, to download/parse links doesn't return any links. Can anyone tell see why? I'm relatively new to Python and not completely sure what the problem could be.

Webpage script;
Code: Select all
import sys, urllib
def getWebpage(url):
    print '[*] getWebpage()'
    url_file = urllib.urlopen(url)
    page = url_file.read()
    return page
def main():
    sys.argv.append('http://www.funeralformyfat.tumblr.com)
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_get URL'
        return
    else:
        print getWebpage(sys.argv[1])

if __name__ == '__main__':
    main()


Links script;
Code: Select all
import sys, urllib
def print_links(page):
    print '[*] print_links()'
    links = re.findall(r'\<a.*href\=.*http\:.+', page)
    links.sort()
    print '[+]', str(len(links)), 'HyperLinks Found:'

    for link in links:
        print link
   
def main():
    sys.argv.append('http://www.funeralformyfat.tumblr.com')
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)

       
if __name__ == '__main__':
    main()
Last edited by Yoriz on Fri Nov 22, 2013 10:23 pm, edited 1 time in total.
Reason: First post lock
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby stranac » Fri Nov 22, 2013 10:39 pm

RandomPast wrote:
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)


The print_links() call is inside the if block, after the return statement. It can never be executed.
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1246
Joined: Thu Feb 07, 2013 3:42 pm

Re: Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 11:08 pm

Hi, thanks for answering. Where do you think I should move it to? Because I've tried moving it to a number of places and still nothing happens.
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby micseydel » Fri Nov 22, 2013 11:13 pm

Do you know what return does? You should really try to understand exactly what your code is doing.
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1507
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Webpage & Downloading Links

Postby RandomPast » Fri Nov 22, 2013 11:16 pm

micseydel wrote:Do you know what return does? You should really try to understand exactly what your code is doing.

Yes I know what return does, but I'm not 100% sure about the layout at this moment. I've only been using Python for a few weeks.
RandomPast
 
Posts: 3
Joined: Fri Nov 22, 2013 9:33 pm

Re: Webpage & Downloading Links

Postby micseydel » Sat Nov 23, 2013 12:29 am

So that you realize your code quoted above,
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return
        page = webpage_get.getWebpage(sys.argv[1])
        print_links(page)

is equivalent to
Code: Select all
    if len(sys.argv) != 2:
        print '[-] Usage: webpage_links URL'
        return

then? In face, in-context, the return isn't even necessary. The function ends implicitly right there.
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1507
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA


Return to General Coding Help

Who is online

Users browsing this forum: Bing [Bot], Mekire and 10 guests