Need help running a script

This is the place for queries that don't fit in any of the other categories.

Need help running a script

Postby mightyplate » Wed Jan 01, 2014 11:33 pm

I need help running a simple web spider python script. I put about 4 hours trying to get this darn code to work and turns out I need beautiful soup,
I just want to avoid installing beautiful soup and all I want is for one of you guys to run the code for me and reply the outcome. I'm hoping it will be faster this way than installing beautifulsoup/ all the other things I need for this code.



Code: Select all
import BeautifulSoup
import urllib2
def union(p,q):
    for e in q:
        if e not in p:
            p.append(e)

def crawler(SeedUrl):
    tocrawl=[SeedUrl]
    crawled=[]
    while tocrawl:
        page=tocrawl.pop()
        pagesource=urllib2.urlopen(page)
        s=pagesource.read()
        soup=BeautifulSoup.BeautifulSoup(s)
        links=soup('a')       
        if page not in crawled:
            union(tocrawl,links)
            crawled.append(page)

    return crawled
crawler('https://www.toontownrewritten.com/')
Last edited by metulburr on Wed Jan 01, 2014 11:51 pm, edited 1 time in total.
Reason: locked post and added code tags
mightyplate
 
Posts: 2
Joined: Wed Jan 01, 2014 11:30 pm

Re: Need help running a script

Postby metulburr » Wed Jan 01, 2014 11:56 pm

1) beautifulsoup is not hard to install
2) how did you spend 4 hours writing this code when you dont even have beautifulsoup installed?
3) i am not going to respond with teh output because it sounds fishy. It sounds like a professor is ensuring that you can install a 3rd party library to python. And instead of doing the work, you post to forums trying to get a freebie. I could be wrong though.
4) http://www.crummy.com/software/BeautifulSoup/
you can install it with pip, or download the source, and
Code: Select all
sudo python setup.py install


if your using Beautifulsoup 4 its:
Code: Select all
from bs4 import BeautifulSoup


I'm hoping it will be faster this way than installing beautifulsoup/ all the other things I need for this code.

It would be faster for you jsut to download/install beautifulsoup and see the outcome yourself.

If your trying to avoid 3rd party libs altogether, you can use lxml instead
we will be moving to python-forum.io on October 1 2016
more details here
User avatar
metulburr
 
Posts: 2231
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: Need help running a script

Postby mightyplate » Thu Jan 02, 2014 11:50 pm

should have known none of you would help me.
I didn't write the code
Last edited by Mekire on Fri Jan 03, 2014 1:44 pm, edited 1 time in total.
Reason: Yep, lock.
mightyplate
 
Posts: 2
Joined: Wed Jan 01, 2014 11:30 pm

Re: Need help running a script

Postby Mike_S » Fri Jan 03, 2014 8:32 am

Very strange and suspicious post indeed.

If you don't care to install a package why bother with running code whose first import is said package you would rather not install - kinda dumb.

You might have asked, "can someone help me rewrite this code using only native Python packages - no BeautifulSoup."
* Portable Python * 2.7.6.1 * 3.2.5.1 *
Mike_S
 
Posts: 13
Joined: Sat Feb 09, 2013 4:59 am
Location: >!<


Return to General Coding Help

Who is online

Users browsing this forum: No registered users and 10 guests