jsonhilo.js

Fast lossless JSON parse event streaming, akin to SAX.

Minimal, modular, and dependency-free.

Provides two interfaces: a high-level one and a low-level one.

Written in runtime-independent JavaScript with Deno as the primary target.

A stand-alone part of the TAO-JSON project.

Part of the TAO-Deno project.

Rationale

Initially written to enable fast lossless translation between JSON and Jevko, as no suitable JSON parser in JavaScript exists.

JSON-Jevko translators are still to be implemented, but I decided to release this as a separate library, because I am also tinkering with Deno and found that there is no streaming JSON parser available at all for Deno.

So this might be especially useful in Deno land.

Status

Passes standards-compliance tests and performs well in benchmarks.

Ready for initial battle-testing.

Installation

Not necessary. Import modules directly from deno.land/x:

import {JsonHigh} from 'https://deno.land/x/jsonhilo@v0.1.0/mod.js'

Or from a CDN such as jsDelivr:

import {JsonHigh} from 'https://cdn.jsdelivr.net/gh/tree-annotation/jsonhilo@v0.1.0/mod.js'

This should work out of the box in Deno and the browser.

Quickstart

See a basic example in demo/basic.js, pasted below:

import {JsonHigh} from 'https://deno.land/x/jsonhilo@v0.1.0/mod.js'
const stream = JsonHigh({
  openArray: () => console.log('<array>'),
  openObject: () => console.log('<object>'),
  closeArray: () => console.log('</array>'),
  closeObject: () => console.log('</object>'),
  key: (key) => console.log(`<key>${key}</key>`),
  value: (value) => console.log(`<value type="${typeof value}">${value}</value>`),
})
stream.push('{"tuple": [null, true, false, 1.2e-3, "[demo]"]}')

This uses the simplified high-level interface built on top of the more powerful low-level core.

Features

Simple and minimal
Dependency-free
Runtime-independent
Lossless
Modular
Fast
Streaming-friendly
Optionally standards-compliant
Unicode-compatible

Runtime-independent

The library logic is written in modern JavaScript and relies upon some of its features, standard modules in particular.

Beyond that it does not use any runtime-specific features and should work in any modern JavaScript environment. It was tested in Deno, Node.js, and the browser.

That said, the primary target runtime is Deno, and tests depend on it.

Lossless

Unlike any other known streaming JSON parser, jsonhilo provides a low-level interface for lossless parsing, i.e. it is possible to recover the exact input, including whitespace and string escape sequences, from parser events.

This feature can be used to implement accurate translators from JSON to other representations (see Rationale), syntax highlighters (demo below), JSON scanners that search for substrings in strings on-the-fly, without first loading them into memory, and more.

Pictured above is the syntax highlighting demo: demo/highlight.js

Modular

The library is highly modular with a fully independent core, around which various adapters and extensions are built, including an easy-to-use high-level interface.

JsonLow

The core module is JsonLow.js. It has no dependencies, so it can be used on its own. It is very minimal and optimized for maximum performance and accuracy, as well as minimum memory footprint. It provides the most fine-grained control over the parsing process. The events generated by the parser carry enough information to losslessly recreate the input exactly, including whitespace.

See JsonLow.d.ts for type information and demo/highlight.js for usage example.

Detailed description to be written.

JsonHigh

JsonHigh.js is the high-level module which provides a more convenient interface. It is composed of auxiliary modules and adapters built around the core. It is optimized for convenience and provides similar functionality and granularity to other streaming parsers, such as clarinet or creationix/jsonparse.

See JsonHigh.d.ts for type information and Quickstart for usage example.

Parameters

JsonHigh is called with an object which contains named event handlers that are invoked during parsing. All handlers are optional and described below.

Return value

JsonHigh returns a stream object with two methods:

push which accepts a JSON chunk to parse. It returns the stream object for chaining.
end with no arguments which signals that parsing is completed. It calls the corresponding end event handler, passing its return value to the caller.

Events

There are 4 event handlers without arguments which indicate start and end of structures:

openArray: an array started ([)
closeArray: an array ended (])
openObject: an object started ({)
closeObject: an object ended (})

And 2 event handlers with one argument which capture primitives:

key: an object’s key ended. The argument of the handler contains the key as a JavaScript string.
value: a primitive JSON value ended. The argument of the event contains the corresponding JavaScript value: true, false, null, a number, or a string.

Finally, there is the argumentless end event handler which is called by the end method of the stream.

Fast

xtao-org/jsonhilo-benchmarks contains benchmarks used to compare the performance of jsonhilo with clarinet (the fastest streaming JSON parser in JavaScript I could find) and jq (a fast and versatile command-line JSON processor).

For validating JSON (just parsing without any further processing) jsonhilo is the fastest, before jq, which is in turn faster than clarinet.

Overall for comparable tasks the low-level jsonhilo interface is up to 2x faster than clarinet, whereas the high-level interface is on par.

Streaming-friendly

By default the parser is streaming-friendly by accepting the following:

Multiple consecutive top-level JSON values – it can read line-delimited JSON and concatenated JSON, e.g. JSON Lines, ndjson. Whitespace-separated primitives are also supported.
Trailing commas – a single trailing comma in an array or an object generates no errors.
Zero-length or whitespace-only input – generates no errors.

Standards-compliant

The streaming-friendly features can be supressed by Ecma404.js, an adapter module which provides full ECMA-404/RFC 8259 compliance.

This is confirmed by passing the JSON Parsing Test Suite by Nicolas Seriot, available under test/JSONTestSuite.

Tests can be run with Deno as follows:

deno test --allow-read

Unicode-compatible

The core logic operates on Unicode code points – in line with spec – rather than code units or characters.

Released under the MIT license.