Convert html response to json

A forum for general discussion of the Python programming language.

Convert html response to json

Postby bhishan » Tue Jul 08, 2014 1:09 am

Hi,
Is there a way in python to convert the html response to a json(non-api websites). By html response I mean the response from any websites.Is there some library functions/modules for doing that?
Thanks
bhishan
 
Posts: 18
Joined: Sun May 18, 2014 7:57 am
Location: Kathmandu, Nepal

Re: Convert html response to json

Postby micseydel » Tue Jul 08, 2014 1:09 am

Can you give a hard example of what you're looking for? Such as something in Python to represent an "html response" (such as a string, or something from httplib) and what you want the json to look like?
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1354
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Convert html response to json

Postby bhishan » Tue Jul 08, 2014 1:14 am

Code: Select all
from  urllib2 import urlopen
r = urlopen('http://www.domain.com')


By domain I mean any websites. I don't if this is a crazy question. Just wanted to know if there exists some modules that does this conversion. I want to convert response(r in the above code) to a json
bhishan
 
Posts: 18
Joined: Sun May 18, 2014 7:57 am
Location: Kathmandu, Nepal

Re: Convert html response to json

Postby micseydel » Tue Jul 08, 2014 1:30 am

Can you give a real website and the JSON you'd want from it?
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1354
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Convert html response to json

Postby bhishan » Tue Jul 08, 2014 1:59 am

Sure, here's the link to it. I want json from different categories of this website. Not specifically the home page.
http://dwitnews.com/
bhishan
 
Posts: 18
Joined: Sun May 18, 2014 7:57 am
Location: Kathmandu, Nepal

Re: Convert html response to json

Postby micseydel » Tue Jul 08, 2014 7:31 pm

There is no way to get the exact thing you want from an arbitrary website. Assuming they don't have an API, you'll have to scrape their web page (which may be illegal and/or against their terms of use), which requires examining the DOM or HTML of their page and crossing your fingers that it doesn't change during the lifetime of the code you're writing.

For future reference, when we ask for "real website and the JSON you'd want from it" you should provide real example JSON that corresponds to the website, and not just the website without any JSON.
Join the #python-forum IRC channel on irc.freenode.net!

Please do not PM members regarding questions which are meant to be discussed publicly. The point of the forum is so that others can benefit from it. We don't want to help you over PMs or emails.
User avatar
micseydel
 
Posts: 1354
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Re: Convert html response to json

Postby bhishan » Wed Jul 09, 2014 9:07 am

Thanks for the response. I plan to do this with BeautifulSoup module although the primary goal was to get a json and process that.
bhishan
 
Posts: 18
Joined: Sun May 18, 2014 7:57 am
Location: Kathmandu, Nepal

Re: Convert html response to json

Postby Kebap » Wed Jul 09, 2014 10:12 am

bhishan wrote:I plan to do this

I have no idea, what "this" you are planning, because you never give details.

Please show some example input (=website) and output (=Json), so we can give you ideas how to get there via python.

Or describe in general terms, what result you would be expecting. How do people use your code, what do they see there, once it is done?
Learn: How To Ask Questions The Smart Way
Join the #python-forum IRC channel on irc.freenode.net and chat with uns directly!
Kebap
 
Posts: 396
Joined: Thu Apr 04, 2013 1:17 pm
Location: Germany, Europe


Return to General Discussions

Who is online

Users browsing this forum: No registered users and 2 guests