JSON read and written by Python

Home

1 JSON

JavaScript Object Notation (JSON) is a lightweight data-interchange format based on the syntax of JavaScript objects. It is a text-based, human-readable, language-independent format for representing structured object data for easy transmission or saving.

JSON objects can also be stored in files — typically a text file with a .json extension and a application/json MIME type. Commonly, JSON is used for two way data transmission between a web-server and a client in a REST API.

1.1 JSON exists as a string

Being a string, means it is a sequence of bytes, which makes it easy to transmit over a network. That means that an object, typically a dictionary, needs to be converted into a string, (process called serialization) in .json format.

The reverse operation is deserialization which takes a json string and reads it into a data object or dictionary.

2 JSON "Key":Value,

Characteristics of JSON, JavaScript Object Notation are:

  • JSON contains either an array of values or an object
  • {} curly braces define an object of {"key": value, "key": value} pairs
  • , commas separate object entries of the same type, in a object
  • [] square brackets define arrays, or lists
  • , commas separate list entries of the same type, in a list
  • unordered set of name : value pairs
  • all keys are strings, so "string1": (need the colon :)
  • wrap strings in double quotation marks
  • whitespaces are ignored {key1":"Cleese"} = "{ "key1" : "Cleese" }
  • dictionaries can have sub dictionaries.
  • objects can contain other objects
  • values can be: - strings
    • integers
    • floats
    • null
    • booleans
    • lists
    • even other json objects
  • NO comments SUPPORTED
  • .json file ending
  • is JSON heirarchical? More accurate to say JSON can represent hierarchical data structures. List are not heirarchical, but an object can have sub- objects, and those are hierarchical.
  • JSON is parsed in python with
    • json.loads() for loading from a string.
    • json.load() for loading data from a file or file-like object.
  • NO round brackets. EVER!

This example has all the possible types. 

{
   "title" : "Monty Python and the Holy Grail",
   "release_year" : 1975,
   "is_silly": true,
   "box_offce_in_millions" : 5.1,
   "actors": ["John Cleese",  "Terry Jones",  "Eric Idle",
               "Terry Gilliam", "Michael Palin", "Graham Chapman"],
   "budget" : null,
   "credits" : {
           "directors" : ["Terry Gilliam", "Terry Jones"],
           "writers" : "Monty Python",
           "composer" : "Neil Innes",
           "cinematographer" : "Terry Bedford"}
}

2.1 Online json lint checker

Check your JSON using a lint website, such as https://jsonlint.com

These two examples are the same valid json file: 

[{
        "user": 12
}, {
        "role": 5
}]

The above is easier to read, i.e. with braces and brackets on separate lines, however the example next is the equivalent json data, just wihtout extra new lines: 

[{"user": 12}, {"role": 5}]

Invalid: 

{{
        "user": 12
}, {
        "role": 5
}}

which is the same as this invalid JSON: 

{{"user": 12}, {"role": 5}}

invalid-json.png

Figure 1: Valid and Invalid JSON
  {
    "Triple Pleasure" : {

        "ounces of alchohol" : {
            "Vodka" : 1,
            "Tequila" : 1,
            "Yukon Jack" : 1
        },
        "ounces of juice" :{
             "Orange" : 2,
             "Cranberry"  : 2,
            "Pineapple"  : 2
        }
    },
    "Northern Lights" : {
        "ounces of alchohol" : {
            "peach snapps" : "splash",
            "Yukon Jack" : 2
        },
        "ounces of juice" :{
             "Orange" : 2,
             "Cranberry"  : 2
        }
    },
    "Snake Bite" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2
        },
        "ounces of juice" :{
            "Lime" : 2
        }
    },
    "Teabag" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2,
            "Sloe gin" : 1,
            "Vodka" : 1
        },
        "ounces of juice" :{
            "Orange" : 2,
            "Lime" : 2
        }
    },
    "Yukon Cornelius" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2,
            "Goldschlager" : 2
        },
        "ice" : true
    },
    "Puff Ader" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2,
            "Creme de cacao" : 2,
            "Rum" : 2
        },
        "ice" : true
    },
    "Canadian Ice" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2,
            "Peppermint scnapps" : 2
        },
        "ice" : true
    },
    "Canadian 3 wise men" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2,
            "Jack Daniels" : 2,
            "Jim Beam" : 2
        },
        "ice" : true
    },
    "Agent Orange" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 1,
            "Rum" : 1,
            "Southern Comfort" : 1,
            "apple schnapps" : 1,
            "melon liqueur" : 1,
            "gin" : 1,
            "vodka" : 2
        },
        "ounces of juice" : {
            "orange" : 2,
            "Grenadine" : 2
        },
        "ice" : true
    },
    "Golden Nugget" : {
        "ounces of alchohol" : {
            "Yukon Jack" : 2
        },
        "ounces of juice" :{
            "Grapefruite" : 5
        }               
    }
}

2.2 Comparison to python dictionaries

Python  
dictionary JSON
none null
True true
False false

Note that python dictionary keys can be strings or any hashable type where as JSON keys must be strings

3 Using the json module in python

3.0.1 First load and investigate the json module

  1. Import json module

    import json

  2. Investigate the available methods in the json module

    dir(json)

    ['JSONDecodeError', 'JSONDecoder', 'JSONEncoder', 'all', 'author', 'builtins', 'cached', 'doc', 'file', 'loader', 'name', 'package', 'path', 'spec', 'version', 'defaultdecoder', 'defaultencoder', 'codecs', 'decoder', 'detectencoding', 'dump', 'dumps', 'encoder', 'load', 'loads', 'scanner']

    Notice that ther is a load method as well as a loads method. load takes data from a file or file-like object, where loads takes data from a string.

    Finally, notice dump and dumps same thing for writing json to a file, or writing to a string.

  3. dir(json.dump) same as dir(json.load)

    ['annotations', 'call', 'class', 'closure', 'code', 'defaults', 'delattr', 'dict', 'dir', 'doc', 'eq', 'format', 'ge', 'get', 'getattribute', 'globals', 'gt', 'hash', 'init', 'initsubclass', 'kwdefaults', 'le', 'lt', 'module', 'name', 'ne', 'new', 'qualname', 'reduce', 'reduceex', 'repr', 'setattr', 'sizeof', 'str', 'subclasshook']

3.1 Key methods used:

  1. json.load(f): (where f is a file of json data)

    read in json data from a file

  2. json.loads(s): (where s is a string of json data)

    notice the "s" in loads when you want to load a string.

  3. json.dump(j, f):

    writes JSON object to a file (or file-like object) j is the json object, f is the file handle

    Example: 

    import json
with open('saved-directory.json', 'w') as sj:
   json.dump(data, sj)

   # add extra arguments to make it pretty or sorted:
   # like sort_keys=True, indent=4
with open('saved-directory.json', 'w') as sj:
   json.dump(data, sj, sort_keys=True, indent=4)
  4. json.dumps(j):

    Output JSON object as a string

4 Examples

4.1 Load json from the file create above pertaining to the Holy Grail

import json

json_ingest = open("json_example.json").read()
py_dict = json.load(json_ingest)        # at this point the python dictionary is set up.
json_ingest.close()

# we can then assign variables straight out of the python dictionary by
# referencing the [key][value] pair.
int_name = py_dict["ietf-interfaces:interface"]["name"]

print (int_name)

# back to json for output:
out-file = json.dumps(json_python)

Better to use Context Manager "with" construct: 

with open("json_example.json", "r")as json_ingest:  
# optionally , "r", encoding="utf-8")as json_ingest"
    json_python = json.load(json_ingest)

    .
    .
    .

 with open("json_output.json", "w")as jdump:
    jdump = json.dump(json_pthon)

4.2 Now save a python dictionary as a json file (string later)

Say you have a python dictionary called covidstatsdict. To convert that to json use:

covid-stats-json = json.dump(covid_stats_dict)

5 pythony dict to json double quotes

If you need to create the dictionary with all names inside double quotes instead of default single quotes which Python uses then you can use the following code. 

# app.py

import json


class App(dict):
    def __str__(self):
    return json.dumps(self)


couples = [['eleven', 'Millie'],
       ['mike', 'Finn'],
       ['max', 'Sadie'],
       ['dustin', 'Gaten']]

pairs = App(couples)
print(pairs)

{"eleven": "Millie", "mike": "Finn", "max": "Sadie", "dustin": "Gaten"}

6 Example conversion of JSON into Python dictionary:

You will see that json looks very much like python dictionaries. Two minor differences:

JSON <c15> Python dictionary
true True
false False
null none

6.1 Sample json

building = """
    {"office": 
       {"medical": [
     { "room-number": 100,
       "use": "reception",
       "sq-ft": 50,
       "price": 75
     },
     { "room-number": 101,
       "use": "waiting",
       "sq-ft": 250,
       "price": 75
     },
     { "room-number": 102,
       "use": "examination",
       "sq-ft": 125,
       "price": 150
     },
     { "room-number": 103,
       "use": "examination",
       "sq-ft": 125,
       "price": 150
     },
     { "room-number": 104,
       "use": "office",
       "sq-ft": 150,
       "price": 100
     }
     ],
     "parking": {
      "location": "premium",
      "style": "covered",
      "price": 750
      }
     },
 "address" : "850 York Mills Rd",
 "year_built": 1981,
 "for_sale": true,
 "outstanding_bids": false,
 "past_listings": ["1997", "2010", "2019"]    
 }"""

 z = json.loads(build2)

 type(z) ==> dict
 # you will see True and False becuase it is ia python dictionary

 type(build2) ==> str 
 # but build2 will still have "true" and "false"

import json
json.dumps(building)

Where this: json.dumps(building) Results in this output: 

'{"office": {"medical": [{"room-number": 100, "use": "reception",
"sq-ft": 50, "price": 75}, {"room-number": 101, "use": "waiting", "sq-ft": 250,
 "price": 75}, {"room-number": 102, "use": "examination", "sq-ft": 125,
 "price": 150}, {"room-number": 103, "use": "examination", "sq-ft": 125,
 "price": 150}, {"room-number": 104, "use": "office", "sq-ft": 150,
 "price": 100}], "parking": {"location": "premium", "style": "covered",
 "price": 750}}}'

Where this: json.dumps(building, indent=4) Results in this output: 

{
    "office": {
        "medical": [
            {
                "room-number": 100,
                "use": "reception",
                "sq-ft": 50,
                "price": 75
            },
            {
                "room-number": 101,
                "use": "waiting",
                "sq-ft": 250,
                "price": 75
            },
            {
                "room-number": 102,
                "use": "examination",
                "sq-ft": 125,
                "price": 150
            },
            {
                "room-number": 103,
                "use": "examination",
                "sq-ft": 125,
                "price": 150
            },
            {
                "room-number": 104,
                "use": "office",
                "sq-ft": 150,
                "price": 100
            }
        ],
        "parking": {
            "location": "premium",
            "style": "covered",
            "price": 750
        }
    }
}

In [9]: :

7 nested dictionaries

From netprog-basics I saw references to nested dictionary items being recalled through this method dict["outerkey"]["level2-key"]["level3-key"] Confirm that that works. It does! 

building = { 
   "office": 
       {"medical": [ 
           { "room-number": 100, 
             "use": "reception", 
             "sq-ft": 50, 
             "price": 75 
           }, 
           { "room-number": 101, 
             "use": "waiting", 
             "sq-ft": 250, 
             "price": 75 
           }, 
           { "room-number": 102, 
             "use": "examination", 
             "sq-ft": 125, 
             "price": 150 
           }, 
           { "room-number": 103, 
             "use": "examination", 
             "sq-ft": 125, 
             "price": 150 
           }, 
           { "room-number": 104, 
             "use": "office", 
             "sq-ft": 150, 
             "price": 100 
           } 
       ], 
        "parking": { 
            "location": "premium", 
            "style": "covered", 
            "price": 750 
        } 
    } 
}

Then I can recall items like this:

  • building['office']['parking']['price']

which returns 750

or also: - building['office']['medical'][0]['room-number'] which returns 100 

for rdict in building['office']['medical']: 
    print (f"  Room number {rdict['room-number']} is going for {rdict['price']}" )

8 External Resources

The main resource is www.json.org

8.1 Home