Horton

Re: WIP Incremental JSON Parser

Jacob Champion <jacob.champion@enterprisedb.com>

View thread View on postgr.es
From: Jacob Champion <jacob.champion@enterprisedb.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-25T23:21:40Z
Lists: pgsql-hackers 
On Mon, Mar 25, 2024 at 4:12 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Stack size should be pretty limited, at least on the platforms I'm
> familiar with. So yeah, the recursive descent will segfault pretty
> quickly, but it won't repalloc() an unbounded amount of heap space.
> The alternative would just be to go back to a hardcoded limit in the
> short term, I think.

And I should mention that there are other ways to consume a bunch of
memory, but I think they're bounded by the size of the JSON file.
Looks like the repalloc()s amplify the JSON size by a factor of ~20
(JS_MAX_PROD_LEN + sizeof(char*) + sizeof(bool)). That may or may not
be enough to be concerned about in the end, since I think it's still
linear, but I wanted to make sure it was known.

--Jacob

Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.