Horton

Re: WIP Incremental JSON Parser

Jacob Champion <jacob.champion@enterprisedb.com>

View thread View on postgr.es
From: Jacob Champion <jacob.champion@enterprisedb.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-11T13:47:16Z
Lists: pgsql-hackers 
On Sun, Mar 10, 2024 at 11:43 PM Andrew Dunstan <andrew@dunslane.net> wrote:
> I haven't managed to reproduce this. But I'm including some tests for it.

If I remove the newline from the end of the new tests:

> @@ -25,7 +25,7 @@ for (my $size = 64; $size > 0; $size--)
>  foreach my $test_string (@inlinetests)
>  {
>         my ($fh, $fname) = tempfile();
> -       print $fh "$test_string\n";
> +       print $fh "$test_string";
>         close($fh);
>
>         foreach my $size (1..6, 10, 20, 30)

then I can reproduce the same result as my local tests. That extra
whitespace appears to help the partial token logic out somehow.

--Jacob

Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.