A Tour of Transit

This tour introduces the Transit format via transit-js, an implementation of the Transit format for JavaScript.

Transit implementations for other languages may provide slightly different APIs that reflect the idioms of the implementation language and language specific performance concerns. However, regardless of implementation, the semantics of the format must be preserved.

The guide is interactive. Many of the code snippets may be evaluated by pressing the Eval button underneath the snippet. These examples are CodeMirror editors so that you can edit and try out the transit-js API youself:

Creating a Reader

Most Transit implementations support encodings in JSON, msgpack, or both, but transit-js currently only supports JSON. That said, you can specify what type of reader you wish to construct:

var r = transit.reader("json");

Reading Data

Once a reader has been constructed you can read transit wire data. All JSON is valid wire data for transit JSON readers. Some Transit read implementations take an encoded input stream. In the transit-js case read simply takes a string of encoded data:

We can read JSON strings, numbers, and arrays as supported by JSON.parse:

var r = transit.reader("json"); r.read('[1,1.5,1e5,"A string!","\u03BB"]'); Eval

We can also read values not supported by JSON.parse. This is possible by encoding values as strings and using a simple tag convention. Transit JSON strings that represent a scalar value are prefixed with ~ followed by single character. For example the tag ~m denotes a non-verbose encoding of dates (human readable dates can also be read/written).

var r = transit.reader("json"); r.read('["~m0","~m1"]'); Eval

The example above shows that the ~m tag uses the number of milliseconds after the date January 1, 1970 at 00:00:00 to encode a date value. For a full list of the scalar tags please refer to the Transit specification.

We've already seen that JSON arrays can be read, but you can also read JSON objects. Note that transit-js read returns hash maps when reading objects because JavaScript objects cannot support scalar values beyond strings as keys.

var r = transit.reader("json"), m = r.read('{"foo":"bar"}'); m.get("foo") Eval

This means that you can't dereference properties using bracket notation, since they are not plain JavaScript objects:

var r = transit.reader("json"), m = r.read('{"foo":"bar"}'); m["foo"] Eval

While this may initially seem less convenient, there are benefits to opting into returning maps in transit-js.

Maps

transit-js hash maps support most of the proposed ECMAScript 6 Map interface, however look up is based on value instead of reference:

var m = transit.map([new Date(0), "Works!"]); m.get(new Date(0)); Eval

The above is impossible under the proposed ES6 Map specification.

transit-js supports reading and writing maps with richer keys including collection types like JavaScript objects, arrays, and transit-js maps. For example you can read a map with a date key:

var r = transit.reader("json"), m = r.read('{"~m0":"Works!"}'); m.get(new Date(0)); Eval

transit-js maps and sets may be used à la carte. They have been written with performance in mind - they are competitive with plain JavaScript objects, they outperform existing ES6 Map/Set shims and they are comparable in many cases to pending native implementations of Map and Set:

var m = transit.map([{first:"John",last:"McCarthy"}, new Date(-1335729600000)]); m.get({first:"John",last:"McCarthy"}); Eval

Tagged Values

Transit also supports non-scalar values beyond arrays and maps via tagged values. Tagged values are represented as a single key-value pair - either a JSON object or a two element array. The key is a tag - a string prefixed with ~#. The value is a Transit value that can be further interpreted by a user-supplied tagged value handler, after being decoded by Transit itself. Transit ships with several tagged values built in including ES6-like sets:

var r = transit.reader("json"), s = r.read('{"~#set":["cat","dog","bird"]}'); s.has("bird"); Eval

Tagged values that have no user installed handler will be handled by a default handler. This allows sending data intact through heterogenous independent systems that utilize Transit. For example in the following no handler exists for line. Yet we can read it and if we write it back out it remains intact.

var r = transit.reader("json"), l = r.read('{"~#line":[0,0,100,100]}'), w = transit.writer("json-verbose"); w.write(l); Eval

This brings us to the topic of encoding transit values.

Creating a Writer

Creating a Transit writer is similar to constructing a reader:

var w = transit.writer("json");

Writing Data

We can serialize simple JavaScript values as we would with JSON.stringify:

var w = transit.writer("json"); w.write([1,1.5,1e5,"A string!","\u03BB"]); Eval

We can also encode values not supported by JSON.stringify, for example dates and 64 bit integer values:

var w = transit.writer("json"), v0 = new Date(), v1 = transit.integer("9007199254740993"); // Math.pow(2,53)+1 w.write([v0,v1]); Eval

We can also encode map and map-like collection types:

var w = transit.writer("json"), m = {"foo": 1, "bar": 2}; w.write(m); Eval

Notice the map is written as a JavaScript array. While this may be more efficient it's also less readable. You can construct a verbose writer to get human readable output for debugging purposes:

var w = transit.writer("json-verbose"), m = {"foo": 1, "bar": 2}; w.write(m); Eval

Extensible Reading & Writing

One of the biggest drawbacks of JSON as a data format is the lack of extensibility. Transit allows users to customize both reading and writing.

Read handlers

Transit readers can be customized with user provided handlers. For example:

var Point = function(x, y) {this.x=x;this.y=y}, r = transit.reader("json", { "handlers": { "point": function(v) { return new Point(v[0], v[1]); } } }); r.read('[{"~#point":[0.5,1.5]}]'); Eval

Write handlers

Transit writers can similarly be extended to handle custom types:

var Point = function(x, y) {this.x=x;this.y=y}, ph = transit.makeWriteHandler({ tag: function(v) { return "point"; }, rep: function(v) { return [v.x, v.y]; }, stringRep: function(v) { return null; } }), w = transit.writer("json", { "handlers": transit.map([Point, ph]) }); w.write([new Point(0.5,1.5),new Point(1.5,2.5)]); Eval

You may be wondering where the ^0 came from, this is an artifact of caching.

Caching

Transit implementations cache map keys, transit symbols, transit keywords, and tagged value tags on read and write. This simple optimization allows Transit implementations to compete with traditional JSON encoder/decoder usage - transit-js read can be faster than JSON.parse under some JavaScript engines for the same logical data:

var w = transit.writer("json"), d = [{"userId": 0}, {"userId": 1}]; w.write(d); Eval

As the redundancy in the data grows, caching delivers increasingly compact results.

var w = transit.writer("json"), d = [{"userCommentId": 9, "userUpvotes": 10}, {"userCommentId": 0, "userUpvotes": 3}, {"userCommentId": 1, "userUpvotes": 4}, {"userCommentId": 4, "userUpvotes": 1}, {"userCommentId": 3, "userUpvotes": 100}], ret = {json: JSON.stringify(d).length, transit: w.write(d).length}; ret; Eval

By clicking Time you can compare JSON.parse to Transit read on typical database data. You can view the JSON here and the Transit JSON here. Under Node.js and Chrome 36 transit-js read slightly outperforms JSON.parse in this case. Opera 22, Internet Explorer 11, and 10 transit-js read is only slightly slower than JSON.parse. Note the transit-js read timings include JSON.parse in addition to hydration. The second row includes the cost of supplying a JSON.parse hydration (reviver) function that simply returns the value to give further context.

Method	Size	Avg. over 100 iters (ms)
JSON.parse	89K
JSON.parse w/ hydrate	89K
JSON.parse Transit JSON	53K
Transit read	53K

Time

These numbers demonstrate that in several browsers parsing a smaller amount of JSON data encoded as arrays is often significantly faster and in the saved time it is possible to hydrate keys and values and compare favorably to JSON parsing of plain JSON data. Even in the case of browsers like Internet Explorer 9, Safari and Firefox, transit-js read compares quite well to JSON.parse supplied with a hydration function (which of course does not permit key hydration).