Horton

Re: WIP Incremental JSON Parser

Jacob Champion <jacob.champion@enterprisedb.com>

View thread View on postgr.es
From: Jacob Champion <jacob.champion@enterprisedb.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Andrew Dunstan <andrew@dunslane.net>, Tom Lane <tgl@sss.pgh.pa.us>, Nathan Bossart <nathandbossart@gmail.com>, Jelte Fennema-Nio <postgres@jeltef.nl>, Amit Langote <amitlangote09@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-04-09T16:41:44Z
Lists: pgsql-hackers 
On Mon, Apr 8, 2024 at 10:24 PM Michael Paquier <michael@paquier.xyz> wrote:
> At the end, having a way to generate JSON blobs randomly to test this
> stuff would be more appealing

For the record, I'm working on an LLVM fuzzer target for the JSON
parser. I think that would be a lot more useful than anything we can
hand-code.

But I want it to cover both the recursive and incremental code paths,
and we'd need to talk about where it would live. libfuzzer is seeded
with a bunch of huge incomprehensible blobs, which is something we're
now trying to avoid checking in. There's also the security aspect of
"what do we do when it finds something", and at that point maybe we
need to look into a service like oss-fuzz.

Thanks,
--Jacob

Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.