Updated 2014-01-17 15:47:03 by RLE

Documentation for this Tcllib package can be found at

This package is a part of the Tcllib distribution.

The JSON site probably explains it best:
"JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on a subset of the JavaScript Programming Language, Standard ECMA-262 3rd Edition - December 1999. This feature can also be found in Python. JSON is a text format that is completely language independent but uses conventions that are familiar to programmers of the C-family of languages, including C, C++, C#, Java, JavaScript, Perl, TCL, and many others. These properties make JSON an ideal data-interchange language."

A Slashdot article about JSON from 24 Jan 2005.

The JSON format is specified in RFC 4627.

JH 20060817: I have made the JSON parser functional and moved it into tcllib (see http://tcllib.cvs.sourceforge.net/tcllib/tcllib/modules/json/). I started with what CMcC had here, but rewrote it for correctness and improved speed.

JMN 2009-01: The way I see it, a JSON generator is a little more useful than a JSON parser. The primary usecase I've had for JSON is pumping a chunk of structured data to the client where there is a JavaScript engine available to consume it.

Whilst in some scenarios JSON formatted structured data might also be sent to the server where another language such as TCL or PHP might need to parse it - it seems more common that the server-side language gets it's data as simple values already split into the fields of a query string or form POST data.

2009-03-08 TCV I've been solving this problem lately myself, the primary difficulty being how to encode the multiple types which are available in JSON in an ambiguously typed Tcl value. The solution for me is to make all values lists where the first element is the type name and the rest are the data for that type. Objects are tagged "obj" and then followed by alternating keys and values (the keys are strings so there's no reason to tag them, but values must be). Lists are tagged "list" and every subsequent element is an item in the list. Strings are "string", numbers are "num", and booleans are "bool". The special "null" when appearing alone is null. I have some preliminary code, not quite ready for release, on my website [1].

slebetman April 1 2009: The huddle package [2] in tcllib does the tagged data thing quite well. And huddle has a built-in jsondump subcommand. I've been working on a different approach though. Having tagged data means I either have to jump through hoops to access the data in tcl or rely on third party APIs (can't use regular list and dict commands for example). What I've done instead is to implement a JSON compiler. Given a specification of the data structure, and the tcl data, generate a JSON string:
  # data is plain old tcl values
  # spec is defined as follows:
  # {string} - data is simply a string, "quote" it if it's not a number
  # {list} - data is a tcl list of strings, convert to JSON arrays
  # {list list} - data is a tcl list of lists
  # {list dict} - data is a tcl list of dicts
  # {dict} - data is a tcl dict of strings
  # {dict xx list} - data is a tcl dict where the value of key xx is a tcl list
  # {dict * list} - data is a tcl dict of lists
  # etc..
  proc compile_json {spec data} {
    while [llength $spec] {
      set type [lindex $spec 0]
      set spec [lrange $spec 1 end]
      
      switch -- $type {
        dict {
          lappend spec * string
          
          set json {}
          foreach {key val} $data {
            foreach {keymatch valtype} $spec {
              if {[string match $keymatch $key]} {
                lappend json [subst {"$key":[compile_json $valtype $val]}]
                break
              }
            }
          }
          return "{[join $json ,]}"
        }
        list {
          if {![llength $spec]} {
            set spec string
          } else {
            set spec [lindex $spec 0]
          }
          set json {}
          foreach {val} $data {
            lappend json [compile_json $spec $val]
          }
          return "\[[join $json ,]\]"
        }
        string {
          if {[string is double -strict $data]} {
            return $data
          } else {
            return "\"$data\""
          }
        }
        default {error "Invalid type"}
      }
    }
  }

  # Usage:
  % compile_json {dict * list} {a {1 2 3} b {4 5}}
  {"a":[1,2,3],"b":[4,5]}

  # Data may be nested:
  % compile_json {dict * {list {dict d list}}} {a {{c 1} {d {2 2 2} e 3}} b {{f 4 g 5}}}
  {"a":[{"c":1},{"d":[2,2,2],"e":3}],"b":[{"f":4,"g":5}]}

For the last example, the specification reads as:
  {
    dict <---------------- a dict
      * {list     <------- where all (*) values are lists
          {dict     <----- of dicts
            d list  <----- where the value of "d" is a list
          }                and all other values are strings (default)
        }
  }

13 Oct 10 Steve Bennett I independently came up with something almost identical. My version correctly escapes string values using:
    return \"[string map [list \\ \\\\ \" \\" \n \\n / \\/ \b \\b \r \\r \t \\t] $value]\"

Also, rather than automatically deciding whether to encode a string or a number, I use 'num' as a type. This also allows the special values false, true and null to be encoded as bare values rather than being quoted.

2009-04-15 kanryu It's good way! I implement one for huddle as "huddle compile" :)
  % huddle jsondump [huddle compile {dict * {list {dict d list}}} {a {{c 1} {d {2 2 2} e 3}} b {{f 4 g 5}}}] "" ""
  {"a":[{"c":1},{"d":[2,2,2],"e":3}],"b":[{"f":4,"g":5}]}

slebetman April 19 2009. That's great kanryu. However I'd prefer this to be implemented as part of the json package rather than huddle. It would avoid having to do the conversion twice (dict->huddle->json). Could the maintainer of the json package in tcllib please steal my code and make a nice json::compile command? Pretty please..? ;-)

JSON is also used as the data format for a lightweight RPC protocol - see JSON-RPC.

Some tests of json package using examples given on the JSON page.
if {[info script] == $argv0} {
    set examples {
            {
                "glossary": {
                    "title": "example glossary",
                    "GlossDiv": {
                        "title": "S",
                        "GlossList": [{
                            "ID": "SGML",
                            "SortAs": "SGML",
                            "GlossTerm": "Standard Generalized Markup Language",
                            "Acronym": "SGML",
                            "Abbrev": "ISO 8879:1986",
                            "GlossDef": 
                            "A meta-markup language, used to create markup languages such as DocBook.",
                            "GlossSeeAlso": ["GML", "XML", "markup"]}]}}
            }
            
            {"menu": {
                "id": "file",
                "value": "File:",
                "popup": {
                    "menuitem": [
                                 {"value": "New", "onclick": "CreateNewDoc()"},
                                 {"value": "Open", "onclick": "OpenDoc()"},
                                 {"value": "Close", "onclick": "CloseDoc()"}
                                ]
                }
            }
            }
            
            {"widget": {
                "debug": "on",
                "window": {
                    "title": "Sample Konfabulator Widget",
                    "name": "main_window",
                    "width": 500,
                    "height": 500},
                "image": { 
                    "src": "Images/Sun.png",
                    "name": "sun1",
                    "hOffset": 250,
                    "vOffset": 250,
                    "alignment": "center"},
                "text": {
                    "data": "Click Here",
                    "size": 36,
                    "style": "bold",
                    "name": "text1",
                    "hOffset": 250,
                    "vOffset": 100,
                    "alignment": "center",
                    "onMouseUp": "sun1.opacity = (sun1.opacity / 100) * 90;"
                }
            }
            }            
    
            {"menu": {
                "header": "SVG Viewer",
                "items": [
                          {"id": "Open"},
                          {"id": "OpenNew", "label": "Open New"},
                          null,
                          {"id": "ZoomIn", "label": "Zoom In"},
                          {"id": "ZoomOut", "label": "Zoom Out"},
                          {"id": "OriginalView", "label": "Original View"},
                          null,
                          {"id": "Quality"},
                          {"id": "Pause"},
                          {"id": "Mute"},
                          null,
                          {"id": "Find", "label": "Find..."},
                          {"id": "FindAgain", "label": "Find Again"},
                          {"id": "Copy"},
                          {"id": "CopyAgain", "label": "Copy Again"},
                          {"id": "CopySVG", "label": "Copy SVG"},
                          {"id": "ViewSVG", "label": "View SVG"},
                          {"id": "ViewSource", "label": "View Source"},
                          {"id": "SaveAs", "label": "Save As"},
                          null,
                          {"id": "Help"},
                          {"id": "About", "label": "About Adobe CVG Viewer..."}]}
            }
    }
    
    foreach example $examples {
    puts "convert: $example"
    puts [json::json2dict "\{$example\}"]
    }
}

RLH I do a "package require JSON" on OSX and the library isn't there or my setup is broken?

JH The package name is "json". RLH Is there any kind of "rule" about package names? I should have tried the lowercase json but didn't think of it. Just wondering.

RLH I found it after doing a find. I am stupid sometimes.

HolgerJ 2008-08-14, when I do a "package require json", I get a message "can't find package dict". I checked the Tcl version (Kubuntu Hardy 8.04) and found out it was 8.4.16; after installing 8.5.0 and calling the interpreter with tclsh8.5, it seems to work.

glennj you can get the dict package for tcl 8.4: see the dict page

See also Tclon.


karll on the chat also points to another JSON Tcl binding at [3]: yajl-tcl.

"tom.rmadilo" writes on comp.lang.tcl:
I put up a JSON text to Tcl list converter for my own testing, if anyone wants to see what the Tcl format looks like:

http://www.rmadilo.com/json/

It also prints out debugging info on a char-by-char basis

There is also http://code.google.com/p/tcl-json/ , a package using Yeti.

An example of http and JSON doing PHP page queries IP-geolocation

[Antender] - 2010-12-25 13:51:59

I'll post here my 1 line JSON parser which has such advantages:

  • Parses JSON directly in dict structure
  • Handles all JSON datatypes: arrays, objects, strings, other values
  • Has O(N) difficulty, because of the use of strict replaces

Reformatted version by Antender and AMG. It's indented for readability, it combines many of the [regsub]s into a [string map], it uses a single [string range] instead of two [string replace]s.
proc json2dict {JSONtext} {
string range [
    string trim [
        string trimleft [
            string map {\t {} \n {} \r {} , { } : { } \[ \{ \] \}} $JSONtext
            ] {\uFEFF}
        ]
    ] 1 end-1
}

Antender: Added dict2json. Have no support of JSON arrays due to Tcl's dynamical typing.
proc dict2json {dictionary} {
        dict for {key value} $dictionary {
                if {[string match {\[*\]} $value]} {
                        lappend Result "\"$key\":$value"
                } elseif {![catch {dict size $value}]} {
                        lappend Result "\"$key\":\"[dict2json $value]\""
                } else {
                        lappend Result "\"$key\":\"$value\""
                }
        }
        return "\{[join $Result ",\n"]\}"
}

kanryu - 2011-04-12 17:12:59

Karl Lehenbauer expose a new library, yajl-tcl[4], that runs 150 times as fast as tcllib/json or tcllib/huddle!

Let's try :)

Code for translating Tcl data structures to json using 8.6's ::tcl::unsupported::representation feature: [5]

Tcllib json::write : [6]

ABU 18-apr-2012

Few weeks ago I posted a patch-proposal [7] for the json package.

I added a new proc named json2prettydict. Aim of this proc is to return a "pretty-printed" tcl-dictionary

I think this proc could be useful for documenting complex nested dictionaries.

Here is an example:
set jsonStr { \
  { "photos": { "page": 1, "pages": "726", "perpage": 3, "total": "7257", 
    "photo": [
      { "id": "6974156079", "owner": "74957296@N08", "secret": "005d743f82", "server": "7197", "farm": 8, "title": "Kenya Watamu \"Deep Sea Fishing\" \"Indian Ocean\" \"Blue Marlin\"", "ispublic": 1, "isfriend": 0, "isfamily": 0 },
      { "id": "6822988100", "owner": "52857411@N08", "secret": "56630c18e8", "server": "7183", "farm": 8, "title": "Gedi Ruins, Local Guide", "ispublic": 1, "isfriend": 0, "isfamily": 0 },
      { "id": "6822909640", "owner": "52857411@N08", "secret": "f4e392ea36", "server": "7063", "farm": 8, "title": "Local Fisherman, Mida Creek", "ispublic": 1, "isfriend": 0, "isfamily": 0 }
    ] }, "stat": "ok" }
}

First, just for comparision, call json2dict
set d1 [json::json2dict $jsonStr]

result is the following
photos {page 1 pages 726 perpage 3 total 7257 photo {{id 6974156079 owner 74957296@N08 secret 005d743f82 server 7197 farm 8 title {Kenya Watamu "Deep Sea Fishing" "Indian Ocean" "Blue Marlin"} ispublic 1 isfriend 0 isfamily 0} {id 6822988100 owner 52857411@N08 secret 56630c18e8 server 7183 farm 8 title {Gedi Ruins, Local Guide} ispublic 1 isfriend 0 isfamily 0} {id 6822909640 owner 52857411@N08 secret f4e392ea36 server 7063 farm 8 title {Local Fisherman, Mida Creek} ispublic 1 isfriend 0 isfamily 0}}} stat ok

then create a pretty printed dictionary
set d2 [json::json2prettydict $jsonStr]

result is still a valid tcl-dict, just more readable
photos {
        page 1
        pages 726
        perpage 3
        total 7257
        photo {
                {
                        id 6974156079
                        owner 74957296@N08
                        secret 005d743f82
                        server 7197
                        farm 8
                        title {Kenya Watamu "Deep Sea Fishing" "Indian Ocean" "Blue Marlin"}
                        ispublic 1
                        isfriend 0
                        isfamily 0
                }
                {
                        id 6822988100
                        owner 52857411@N08
                        secret 56630c18e8
                        server 7183
                        farm 8
                        title {Gedi Ruins, Local Guide}
                        ispublic 1
                        isfriend 0
                        isfamily 0
                }
                {
                        id 6822909640
                        owner 52857411@N08
                        secret f4e392ea36
                        server 7063
                        farm 8
                        title {Local Fisherman, Mida Creek}
                        ispublic 1
                        isfriend 0
                        isfamily 0
                }
        }

}
stat ok

OpenACS has its own take on handling JSON data, see [8]. From the intro:
   Utility ad_procs for Tcl <-> JSON conversion.

   This is based on the tcllib json package written by Andreas Kupries, and
   later rewritten to parse via regular expressions by Thomas Maeder.

   The tcllib version suffers from generating Tcl structures from JSON strings
   with no type (JSON array or object) information.  The resulting structures can't
   be converted back to JSON strings, you have to munge them with type information
   first.  And the code making use the Tcl structure also needs to know whether each
   field is an object or array.

   It also depends on the DICT package or Tcl 8.5.

   This rewrite doesn't depend on DICT, declares procs using ad_proc (so the API
   will be picked up by our API browser), and is symmetrical (you can convert from
   JSON to the Tcl representation and back again).

[gchung] 2012-11-06 Tcl's [string is double] does not conform to JSON specification. For testing valid numbers, you should use the following regexp:
proc is_valid_json_number {value} {
    regexp -- {-?(?:[1-9][[:digit:]]*|0)(?:\.[[:digit:]]+)?(?:[eE][+-]?[[:digit:]]+)?} $value
}

Using [string is double] allows non-valid numbers to slip through. E.g. "00", "0x0", "0.".

AMG: That regular expression may be fine for limiting the acceptable number forms, but also continue to use [string is double] to check for over/underflow.

A Tcl extension written in C for parsing JSON: http://aldan.algebra.com/~mi/tcljson.y

Announcement: https://groups.google.com/forum/#!topic/comp.lang.tcl/W0FyXCuPaBE