best way for scraping and crawling for beginners?

A forum for general discussion of the Python programming language.

best way for scraping and crawling for beginners?

Postby snake » Thu Dec 05, 2013 10:41 am

Hi all,

I want to write a program that reads and writes excel files, performs scraping and crawling and interacts with websites.

My programming skills are basic and I want to have it as soon and as easy as possible.

I have read about BeautifulSoup, lxml and Scrapy but I am confused.

So my question is if python in general is appropriate for my case in the first place considering that such programs can be written in perl, PHP etc.

If it is python then what from the above should I use?

Thanks in advance.

Ragrds
snake
 
Posts: 1
Joined: Thu Dec 05, 2013 10:25 am

Re: best way for scraping and crawling for beginners?

Postby stranac » Thu Dec 05, 2013 12:43 pm

Python is a good choice for doing this.
Scrapy would probably be the best option here, since it's a framework specifically developed for crawling and scraping websites. But it does have a bit of a learning curve...
Friendship is magic!

R.I.P. Tracy M. You will be missed.
User avatar
stranac
 
Posts: 1218
Joined: Thu Feb 07, 2013 3:42 pm

Re: best way for scraping and crawling for beginners?

Postby Kebap » Thu Dec 05, 2013 1:06 pm

snake wrote:My programming skills are basic and I want to have it as soon and as easy as possible.

Sounds like an interesting place to start from. Maybe put an offer in the "jobs" forum here?
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 400
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe


Return to General Discussions

Who is online

Users browsing this forum: CasualCoding.org and 2 guests