Help! a Script that will fetch the Google News

This is the place for queries that don't fit in any of the other categories.

Help! a Script that will fetch the Google News

Postby kia » Wed Dec 18, 2013 10:32 am

Hi Dears

I hope all are having a Great time!
im a new learner of Python and im trying to create a Script that will fetch the Google News from Google News Updates and put in some Websites. my searches leaded me to Feedparser.
Does anyOne have an Idea about how to start this?

Thank you very much in advance
Last edited by Mekire on Wed Dec 18, 2013 1:39 pm, edited 1 time in total.
Reason: First post lock.
kia
 
Posts: 5
Joined: Wed Dec 18, 2013 10:13 am

Re: Help! a Script that will fetch the Google News

Postby metulburr » Wed Dec 18, 2013 11:38 am

you can use lxml library or 3rd party library BeautifulSoup
New Users, Read This
OS Ubuntu 14.04, Arch Linux, Gentoo, Windows 7/8
https://github.com/metulburr
steam
User avatar
metulburr
 
Posts: 1470
Joined: Thu Feb 07, 2013 4:47 pm
Location: Elmira, NY

Re: Help! a Script that will fetch the Google News

Postby snippsat » Wed Dec 18, 2013 12:46 pm

As poster over two very good parser for Python,that deal with all kind of parsing.
Feedparser is a specialized parser that deal with Rss feed,that would work fine for this task.

Does anyOne have an Idea about how to start this?

Yes can show a quick demo with BeautifulSoup and lxml.
Here take out headlines(<title>).
Code: Select all
from bs4 import BeautifulSoup
from urllib2 import urlopen

url = urlopen('http://news.google.com/?output=rss')
soup = BeautifulSoup(url)
tag_title = soup.find_all('title')
for news in tag_title:
    print news.text

The same with lxml using XPath,just take out first headline.
Can of course iterate over all headline like in code over.
Code: Select all
from lxml.html import parse

page = parse('http://news.google.com/?output=rss').getroot()
print page.xpath('//item//title//text()')[0]

#--> Golden pride: Gays get big role in opening and closing ceremonies at Sochi ... - New York Daily News

Some info Incorporate Google RSS feeds onto your site
User avatar
snippsat
 
Posts: 218
Joined: Thu Feb 21, 2013 12:04 am

Re: Help! a Script that will fetch the Google News

Postby kia » Wed Dec 18, 2013 7:31 pm

Dear metulburr & snippsat,

I really would like to thank you very much indeed! I got new thing to learn and benefit from "beautifulsoup an lxml"
according to my limited knowledge i came across this

Code: Select all
import feedparser
feed = feedparser.parse('http://news.google.com/?output=rss')
 post = []
 for i in range (0, len(feed['entries'])):
    post.append({
        'title' : feed['entries'][i].title,
        'description' : feed['entries'][i].summary,
        'url' : feed['entries'][1].link,
        })

as we got many feeds so we can just run a for loop & can append the title, description & url of each feed using a list. what do you kindly think!?
snippsat you have mentioned that the quick demo will only take the headlines, is that mean that there should be other code that will also read the content and display them right.?
Dears would kindly recommend a link or a tutorial where i can learn from A to Z the whole process in order to get a complete recipe.

Again thanx indeed!
kia
 
Posts: 5
Joined: Wed Dec 18, 2013 10:13 am

Re: Help! a Script that will fetch the Google News

Postby snippsat » Thu Dec 19, 2013 11:13 pm

as we got many feeds so we can just run a for loop & can append the title, description & url of each feed using a list. what do you kindly think!?

It's up to you,and append stuff you need to a list is of course ok.
Do you understand the code and an how to use it?
snippsat you have mentioned that the quick demo will only take the headlines, is that mean that there should be other code that will also read the content and display them right.?

Yes you parse what's needed.
Dears would kindly recommend a link or a tutorial where i can learn from A to Z the whole process in order to get a complete recipe.

There is no tutorial that you can learn all from A to Z,just read tutorial/books to borden your Pyhon knowles then solving topic like will be a lot easier.
User avatar
snippsat
 
Posts: 218
Joined: Thu Feb 21, 2013 12:04 am

Re: Help! a Script that will fetch the Google News

Postby Kebap » Fri Dec 20, 2013 10:19 am

Hi kia, welcome to python and the forum :D

Just a quick note on something I noticed. The following code you wrote, does not seem very much like python, maybe you try to copy other language features to python?
Code: Select all
 for i in range (0, len(feed['entries'])):
    post.append({
        'title' : feed['entries'][i].title,
        'description' : feed['entries'][i].summary,
        'url' : feed['entries'][1].link,
        })


It can be written easier, then also be read easier:

Code: Select all
 for entry in feed['entries']:
    post.append({
        'title' : entry.title,
        'description' : entry.summary,
        'url' : entry.link,
        })
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 396
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe

Re: Help! a Script that will fetch the Google News

Postby kia » Sun Mar 30, 2014 5:41 pm

Hi all Dears
I hope you all doing Great!
Im sorry for not flowing my post up, my absence was due to my Health; i was sick lately.
SO here im back again! but before that i would like to thanx all of u!
kia
 
Posts: 5
Joined: Wed Dec 18, 2013 10:13 am

Re: Help! a Script that will fetch the Google News

Postby Kebap » Mon Mar 31, 2014 2:35 pm

Hello kia, glad you're back! :mrgreen:
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 396
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe

Re: Help! a Script that will fetch the Google News

Postby kia » Wed Apr 02, 2014 5:15 pm

Kebap wrote:Hello kia, glad you're back! :mrgreen:


Thnx u so very much indeed Kebap :)
kia
 
Posts: 5
Joined: Wed Dec 18, 2013 10:13 am

Re: Help! a Script that will fetch the Google News

Postby kia » Wed Apr 02, 2014 5:18 pm

Hello Again
As my research contuni on how to fetch web news using python scripts, i was driven to some scripts...
My question here is that how will i test these scripts to see whether they function correctly?

Thnx in advance
kia
 
Posts: 5
Joined: Wed Dec 18, 2013 10:13 am

Re: Help! a Script that will fetch the Google News

Postby Kebap » Thu Apr 03, 2014 12:07 pm

kia wrote: how will i test these scripts to see whether they function correctly?

It really depends on the script. Unfortunately, you gave very little detail, let alone show a script.

For short scripts, you can just save it in a file, then run it. Even shorter scripts, you can test directly in the interactive python shell, which is what I recommend.

It gets more complicated, if the scripts try and import other packages, which you may not yet have installed. You will see the error message then.
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 396
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe


Return to General Coding Help

Who is online

Users browsing this forum: Bing [Bot] and 2 guests