Re: WIP Incremental JSON Parser

Andrew Dunstan <andrew@dunslane.net>

From: Andrew Dunstan <andrew@dunslane.net>

To: Jacob Champion <jacob.champion@enterprisedb.com>

Cc: Robert Haas <robertmhaas@gmail.com>, Peter Smith <smithpb2250@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2024-03-11T06:43:17Z

Lists: pgsql-hackers

Attachments

v9-0001-Introduce-a-non-recursive-JSON-parser.patch (text/x-patch) patch v9-0001
v9-0002-Add-support-for-incrementally-parsing-backup-mani.patch (text/x-patch) patch v9-0002
v9-0003-Use-incremental-parsing-of-backup-manifests.patch (text/x-patch) patch v9-0003

On 2024-03-07 Th 22:42, Andrew Dunstan wrote:
>
>
>
>
>
>
>
>>
>> Top-level scalars like `false` or `12345` do not parse correctly if
>> the chunk size is too small; instead json_errdetail() reports 'Token
>> "" is invalid'. With small chunk sizes, json_errdetail() additionally
>> segfaults on constructions like `[tru]` or `12zz`.
>
>
> Ugh. Will investigate.


I haven't managed to reproduce this. But I'm including some tests for it.

>
>
>
>
> I'll have a fresh patch set soon which will also take care of the bitrot.
>
>
>

See attached.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Commits

Post review fixes for test_json_parser test module
- ba3e6e2bca97 17.0 landed
Shrink test file for test_json_parser module
- b8a7bfa33324 17.0 landed
Add support for incrementally parsing backup manifests
- ea7b4e9a2a7c 17.0 landed
Introduce a non-recursive JSON parser
- 3311ea86edc7 17.0 landed
Use incremental parsing of backup manifests.
- 222e11a10ae9 17.0 landed