Re: WIP Incremental JSON Parser

On Wed, Feb 21, 2024 at 6:50 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Tue, Feb 20, 2024 at 9:32 PM Andrew Dunstan <andrew@dunslane.net> wrote:
> > *sigh* That's weird. I wonder why you can reproduce it and I can't. Can
> > you give me details of the build? OS, compiler, path to source, build
> > setup etc.? Anything that might be remotely relevant.

This construction seems suspect, in json_lex_number():

>       if (lex->incremental && !lex->inc_state->is_last_chunk &&
>               len >= lex->input_length)
>       {
>               appendStringInfoString(&lex->inc_state->partial_token,
>                                                          lex->token_start);
>               return JSON_INCOMPLETE;
>       }

appendStringInfoString() isn't respecting the end of the chunk: if
there's extra data after the chunk boundary (as
AppendIncrementalManifestData() does) then all of that will be stuck
onto the end of the partial_token.

I'm about to context-switch off of this for the day, but I can work on
a patch tomorrow if that'd be helpful. It looks like this is not the
only call to appendStringInfoString().

--Jacob