ECMAScript proposal: source text access for JSON.parse() and JSON.stringify()

[2022-11-18] dev, javascript, es proposal
(Ad, please don’t block)

In this blog post, we look at the ECMAScript proposal “JSON.parse source text access” by Richard Gibson and Mathias Bynens.

It gives access to source text to two kinds of callbacks:

  • Revivers, callbacks that are passed to JSON.parse() and post-process the data it parses.
  • Replacers, callbacks that are passed to JSON.stringify() and pre-process data before it is stringified.

We’ll examine how exactly that works and what you can do with this feature.

JSON parsing: reading the source text from a reviver  

JSON.parse() can be customized via a reviver, a callback that post-processes the data that is parsed:

JSON.parse(
  text: string,
  reviver?: (key: string, value: any, context: RContext) => any
): any;

type RContext = {
  /** Only provided if a value is primitive */
  source?: string,
};

The proposal gives revivers access to the source text via the new parameter context.

Example: faithfully parsing large integers  

JSON does not support bigints. But its syntax can represent arbitrarily large integers. The following interaction shows a large integer that can be stored as JSON, but when it is parsed as a number value, we lose precision and don’t get an accurate value:

> JSON.parse('1234567890123456789')
1234567890123456800

If we could parse that string as a bigint, we would not lose precision:

> BigInt('1234567890123456789')
1234567890123456789n

We can achieve that by accessing the source text from a reviver:

function bigintReviver(key, val, {source}) {
  if (key.endsWith('_bi')) {
    return BigInt(source);
  }
  return val;
}

The reviver assumes that properties whose names end with '_bi' contain bigints. Let’s use it to parse an object with a bigint property:

assert.deepEqual(
  JSON.parse('{"prop_bi": 1234567890123456789}', bigintReviver),
  { prop_bi: 1234567890123456789n }
);

JSON stringification: specifying the source text via replacers and JSON.rawJSON()  

JSON.stringify() can be customized via a replacer, a callback that pre-processes data before it is stringified.

JSON.stringify(
  value: any,
  replacer?: (key: string, value: any) => any,
  space?: string | number
): string;

Replacers can use JSON.rawJSON() to specify how a value should be stringified:

JSON.rawJSON: (jsonStr: string) => RawJSON;
type RawJSON = {
  rawJSON: string,
};

Notes:

  • JSON.rawJSON() coerces jsonStr to a string.
  • If jsonStr has leading or trailing whitespace, it throws an exception.
  • The function also enforces that jsonStr, when JSON-parsed, is a primitive value.

Example: stringifying bigints  

The following replacer stringifies bigints (for which JSON.stringify() normally throws exceptions) as integer numbers:

function bigintReplacer(_key, val) {
  if (typeof val === 'bigint') {
    return JSON.rawJSON(String(val)); // (A)
  }
  return val;
}

The manual type conversion via String() in line A is not strictly needed, but I like to be explicit about conversions.

We can use bigintReplacer to convert the object we have previously parsed back to JSON:

assert.equal(
  JSON.stringify({prop_bi: 10765432100123456789n}, bigintReplacer),
  '{"prop_bi":10765432100123456789}'
);

A generic approach for stringifying and parsing numeric values  

When stringifying, we can distinguish integer numbers and bigints by marking the former with a decimal point and decimal fraction of zero. That is:

  • The integer number 123 is stringified as '123.0'.
  • The bigint 123n is stringified as '123'.
const re_int = /^[0-9]+$/;

function numericReplacer(_key, val) {
  if (Number.isInteger(val)) {
    const str = String(val);
    if (re_int.test(str)) {
      // `str` has neither a decimal point nor an exponent:
      // Mark as number so that we can distinguish it from bigints
      return JSON.rawJSON(str + '.0');
    }
  } else if (typeof val === 'bigint') {
    return JSON.rawJSON(String(val));
  }
  return val;
}

assert.equal(
  JSON.stringify(123, numericReplacer),
  '123.0'
);
assert.equal(
  JSON.stringify(123n, numericReplacer),
  '123'
);

When parsing, integer literals are always parsed as bigints, all other number literals as numbers:

function numericReviver(key, val, {source}) {
  if (typeof val === 'number' && re_int.test(source)) {
    return BigInt(source);
  }
  return val;
}

assert.equal(
  JSON.parse('123.0', numericReviver),
  123
);
assert.equal(
  JSON.parse('123', numericReviver),
  123n
);

Real-world example: Twitter’s ID problem  

When Twitter switched to 64-bit IDs, these IDs couldn’t be (only) stored as numbers in JSON anymore. Quoting Twitter’s documentation:

Numbers as large as 64-bits can cause issues with programming languages that represent integers with fewer than 64-bits. An example of this is JavaScript, where integers are limited to 53-bits in size. In order to provide a workaround for this, in the original designs of the Twitter API (v1, v1.1), ID values were returned in two formats: both as integers, and as strings.

{
  "id": 10765432100123456789,
  "id_str": "10765432100123456789"
}

Let’s see what happens if we parse this integer as a number and as a bigint:

> Number('10765432100123456789')
10765432100123458000
> BigInt('10765432100123456789')
10765432100123456789n

Implementations of the proposal  

There is a GitHub issue that lists the bug tickets for implementing this feature in various JavaScript engines.

V8 has an implementation behind the --harmony-json-parse-with-source flag (V8 v10.9.1+).

Further reading