Horton

Re: WIP Incremental JSON Parser

Andrew Dunstan <andrew@dunslane.net>

View thread View on postgr.es
From: Andrew Dunstan <andrew@dunslane.net>
To: Jacob Champion <jacob.champion@enterprisedb.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-21T06:56:00Z
Lists: pgsql-hackers

Attachments

On Wed, Mar 20, 2024 at 3:06 PM Jacob Champion <
jacob.champion@enterprisedb.com> wrote:

> This new return path...
>
> > +       if (!tok_done)
> > +       {
> > +           if (lex->inc_state->is_last_chunk)
> > +               return JSON_INVALID_TOKEN;
>
> ...also needs to set the token pointers. See one approach in the
> attached diff, which additionally asserts that we've consumed the
> entire chunk in this case, along with a handful of new tests.
>


Thanks, included that and attended to the other issues we discussed. I
think this is pretty close now.

cheers

andrew

Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.