Canonical JSON

The one byte form ODS hashes: RFC 8785, with shared test vectors every implementation must pass.

Status: in use. Reference implementation: canonicalJson in @opencycle/ods-package.

Canonical JSON gives one exact byte form for a JSON value, so that two implementations compute the same hash. ODS uses it for the package hash (package.md, §4) and for the hash of a run's inputs.

The words MUST, MUST NOT and SHOULD are used as in RFC 2119.

1. Rule

ODS canonical JSON is RFC 8785, the JSON Canonicalization Scheme:

  1. Keys. Object keys sort by their UTF-16 code units, at every level. Integer-like keys sort as text: "1000", then "125", then "63".
  2. Numbers. A number prints as ECMAScript's Number::toString prints it: -0 as 0, 1e21 as 1e+21, 1e-7 as 1e-7, 100.0 as 100.
  3. Strings. A string prints as ECMAScript's JSON.stringify prints it: \b, \t, \n, \f and \r as short escapes, other control characters as lowercase \u00xx, " and \ escaped. All other characters, the solidus and non-ASCII text included, are written as they are.
  4. Layout. No whitespace. Arrays keep their order. true, false and null print as they are.

The hash of a canonical form is the SHA-256 of its UTF-8 bytes, as 64 lowercase hex characters.

2. Input

  • The input MUST be a JSON value. NaN and Infinity are not JSON; an implementation MUST refuse them.
  • A producer that holds richer values (dates, record ids) MUST turn them into JSON values first: a date into its ISO 8601 UTC string, an id into its string.

3. Test vectors

vectors/canonical-json.json lists inputs, their canonical forms and the SHA-256 of each. Every implementation MUST pass all of them. The expected forms are worked out by hand from RFC 8785.

4. Common mistakes

These are not rules. They are the ways implementations most often fail the vectors.

  • JavaScript. Sorting the keys and putting them back into an object does not work: a JavaScript object lists integer-like keys first, in numeric order. Build the output string from the sorted keys.
  • Python. json.dumps(sort_keys=True, separators=(",", ":")) is not RFC 8785. It sorts keys by code point, not by UTF-16 code unit (this differs for characters outside the Basic Multilingual Plane, such as emoji). It prints some numbers differently (1e-07, 100.0). It escapes non-ASCII text unless ensure_ascii=False. Use an RFC 8785 library and run the vectors.
  • Any language. Hash the canonical text's UTF-8 bytes, not a pretty-printed copy, and not a copy read back from a file that an editor saved.

Test vectors

Download canonical-json.json. Each vector's input must turn into exactly this text, and the text's UTF-8 bytes must hash to this SHA-256.

  • rfc8785-3.2.3-key-order

    The RFC 8785 section 3.2.3 example: keys sort by UTF-16 code units.

    {"\r":"Carriage Return","1":"One","€":"Control","ö":"Latin Small Letter O With Diaeresis","€":"Euro Sign","😀":"Emoji: Grinning Face","דּ":"Hebrew Letter Dalet With Dagesh"}

    sha256 5e321556d22018a9656991a9e94f77ec175fa193e52a2429d312f8419ec8b08c

  • band-keyed-object

    Integer-like keys sort as text, not as numbers. JavaScript objects list integer-like keys first in numeric order, so an implementation that rebuilds an object gets this wrong.

    {"1000":3,"125":2,"63":1,"a":4}

    sha256 e1194a978fa8817e6191269b0d504384615b58b89d21aec3bcc5a25567004ad5

  • band-keyed-nested

    Octave-band sound power levels inside an array, as a PropagationRun's inputs hold them.

    {"absorptionAlpha":{"4000":0.6,"500":0.2},"pwl":[{"125":78,"250":75.5,"63":80.1}]}

    sha256 d0ce1d75aa80904181dc759aee170cb119f37b85c687a0e9afbb4764e62d1557

  • key-outside-bmp

    U+1F600 is the UTF-16 pair D83D DE00, so it sorts before U+FB33. Sorting by code point (Python's sorted()) gives the opposite order.

    {"😀":2,"דּ":1}

    sha256 ec4e7d8c2963caa38dccc3d42693719ac9c6ecd783891b583d333565620ac2be

  • numbers

    ECMAScript Number::toString (RFC 8785 section 3.2.2.3): -0 prints 0; exponent form from 1e21 up and below 1e-6.

    [0,0,1,-1,1.5,100,1e+21,1e-7,0.000001,123456789012345680000,0.30000000000000004,5e-324,1.7976931348623157e+308]

    sha256 1e44df68175c28de76e213e0cf2cb906b7960682407d5ce2d9ebd5bfa36abf91

  • strings

    JSON.stringify escapes (RFC 8785 section 3.2.2.2): \b \t \n \f \r as short escapes, other control characters as lowercase \u00xx, quote and backslash escaped; solidus, DEL, non-ASCII and U+2028 written as they are.

    {"s":"\u0000\b\t\n\u000b\f\r\u001f\"\\/é
😀"}

    sha256 d0049f6b3fb500377475a8701ead56a2907a2f95c3703ab4400f75b56559a3b6

  • literals-and-empty

    true, false, null, an empty object and an empty array.

    {"a":[],"e":{},"f":false,"n":null,"t":true}

    sha256 662637dc36893be2d6cf9a19094ffdb6ff407bfa66ef8f53d872017725010d69

  • nested-order

    Keys sort at every level; array order is kept.

    {"a":0,"b":{"a":[{"x":1,"y":2},3],"z":1}}

    sha256 60063233a4232cd0a62c2e269f901c755e275659c0470e3b4be60d760c1c4363