Why json?

A forum for general discussion of the Python programming language.

Why json?

Postby Mekire » Sun Dec 22, 2013 3:04 pm

I used to use pickle for all my object serialization needs. Besides the obvious disadvantage that the files created are not human readable I encountered other issues. Pickled files created with python 2 and 3 used different protocols (I believe 3 could read pickled files made with python 2 but the reverse wasn't true). People also constantly talk about how unpickling something is almost as bad as using eval or exec if the source is unknown.

I decided this project to try some alternatives and I'm constantly seeing people talk about json so I gave that a try. I quickly realized, and confirmed, that you can't use nonstring keys when serializing an object in json. This was a pretty big deal breaker as the data I'm trying to serialize are map coordinates. Keys are all tuples.

I ended up going with YAML for the moment and it seems to work pretty well (according to documentation all the same security risks associated with pickle remain).

I am however left wondering why json is seen as useful in python at all. Keys are all coerced to strings when possible (and flat out fail if not); certain objects like tuples can be used as the value for a key, but on converting to json will become lists; certain other objects are simply unserializable, like sets.

So... why json?

New Users, Read This
  • Use code tags when posting code.
  • Include any errors with your post (in code tags).
  • Describe your problem; not your chosen solution.
  • Make examples the minimum length to demonstrate your issue.
User avatar
Posts: 1711
Joined: Thu Feb 07, 2013 11:33 pm
Location: Tucson, Arizona

Re: Why json?

Postby micseydel » Mon Dec 23, 2013 10:38 am

JSON as far as my purposes has had these large advantages: (1) human readable, as you mentioned, and of course therefor modifiable, (2) very interoperable with other languages, and (3) very safe, as you also mentioned. (2) is the reason it's heavily used in RESTful interfaces, of which Python is very commonly on both ends. Because of this, you also have the limitation that arbitrary Python objects aren't supported; you can however have simple wrappers that do coersion, and I believe things like pymongodb can do so for datatime objects, although I may be mistaken.

As for the security aspect you mentioned, you can store arbitrary Python code in pickled objects, so yes, strictly speaking they're "bad" in terms of security because if you unpickle from an arbitrary source then you're in trouble. It's the same as eval/exec, in that it's a tremendous effort to solve the security problem in technical terms, so people will usually choose the policy solution. You just decide to trust it. Often, this is fine. It's a terrible habit, because programs often change over time and if you open source it someone may use your code without realizing the implications as their project changes. That said, it's your code, and you can do as you wish, especially if you document such a limitation to the best of your ability.

JSON has been excellent for me though. I've used it a lot for REST-related things, and recently I've stored records which look like this
Code: Select all
{"tobed": [1, 44], "alarm": [12, 30], "awakemoments": [[1, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 4, 4, 4, 4, 4, 5, 5, 5, 6, 6, 6, 7, 7, 7, 8, 8, 9, 9, 10, 10, 10, 11], [44, 13, 22, 31, 41, 55, 5, 18, 27, 38, 55, 10, 25, 34, 43, 51, 26, 35, 44, 13, 37, 52, 6, 34, 48, 27, 57, 11, 29, 6, 25, 58, 2]], "month": 12, "window": 10, "year": 2013, "day": 22}

(data from my Sleeptracker watch). It is good for a very large amount of use cases.
Due to the reasons discussed here we will be moving to python-forum.io on October 1, 2016.

This forum will be locked down and no one will be able to post/edit/create threads, etc. here from thereafter. Please create an account at the new site to continue discussion.
User avatar
Posts: 2992
Joined: Tue Feb 12, 2013 2:18 am
Location: Mountain View, CA

Return to General Discussions

Who is online

Users browsing this forum: No registered users and 5 guests