Horton

Re: WIP Incremental JSON Parser

Andrew Dunstan <andrew@dunslane.net>

View thread View on postgr.es
From: Andrew Dunstan <andrew@dunslane.net>
To: Jacob Champion <jacob.champion@enterprisedb.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-25T23:24:08Z
Lists: pgsql-hackers 
On Mon, Mar 25, 2024 at 7:12 PM Jacob Champion <
jacob.champion@enterprisedb.com> wrote:

> On Mon, Mar 25, 2024 at 4:02 PM Andrew Dunstan <andrew@dunslane.net>
> wrote:
> > Well, what's the alternative? The current parser doesn't check stack
> depth in frontend code. Presumably it too will eventually just run out of
> memory, possibly rather sooner as the stack frames could  be more expensive
> than the incremental parser stack extensions.
>
> Stack size should be pretty limited, at least on the platforms I'm
> familiar with. So yeah, the recursive descent will segfault pretty
> quickly, but it won't repalloc() an unbounded amount of heap space.
> The alternative would just be to go back to a hardcoded limit in the
> short term, I think.
>
>
>
OK, so we invent a new error code and have the parser  return that if the
stack depth gets too big?

cheers

andrew

Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.