Re: WIP Incremental JSON Parser

Andrew Dunstan <andrew@dunslane.net>

From: Andrew Dunstan <andrew@dunslane.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Peter Smith <smithpb2250@gmail.com>, Jacob Champion <champion.p@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-02-20T04:41:07Z
Lists: pgsql-hackers
On 2024-01-26 Fr 12:15, Andrew Dunstan wrote:
>
> On 2024-01-24 We 13:08, Robert Haas wrote:
>>
>> Maybe you should adjust your patch to dump the manifests into the log
>> file with note(). Then when cfbot runs on it you can see exactly what
>> the raw file looks like. Although I wonder if it's possible that the
>> manifest itself is OK, but somehow it gets garbled when uploaded to
>> the server, either because the client code that sends it or the server
>> code that receives it does something that isn't safe in 32-bit mode.
>> If we hypothesize an off-by-one error or a buffer overrun, that could
>> possibly explain how one field got garbled while the rest of the file
>> is OK.
>
>
> Yeah, I thought earlier today I was on the track of an off by one 
> error, but I was apparently mistaken, so here's the same patch set 
> with an extra patch that logs a bunch of stuff, and might let us see 
> what's upsetting the cfbot.
>
>
>

Well, that didn't help a lot, but meanwhile the CFBot seems to have 
decided in the last few days that it's now happy, so full steam aead! ;-)


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com




Commits

  1. Post review fixes for test_json_parser test module

  2. Shrink test file for test_json_parser module

  3. Add support for incrementally parsing backup manifests

  4. Introduce a non-recursive JSON parser

  5. Use incremental parsing of backup manifests.