Re: WIP Incremental JSON Parser

On 2024-03-25 Mo 19:02, Andrew Dunstan wrote:
>
>
> On Mon, Mar 25, 2024 at 6:15 PM Jacob Champion 
> <jacob.champion@enterprisedb.com> wrote:
>
>     On Wed, Mar 20, 2024 at 11:56 PM Andrew Dunstan
>     <andrew@dunslane.net> wrote:
>     > Thanks, included that and attended to the other issues we
>     discussed. I think this is pretty close now.
>
>     Okay, looking over the thread, there are the following open items:
>     - extend the incremental test in order to exercise the semantic
>     callbacks [1]
>
>
>
> Yeah, I'm on a super long plane trip later this week, so I might get 
> it done then :-)
>
>     - add Assert calls in impossible error cases [2]
>
>
> ok, will do
>
>     - error out if the non-incremental lex doesn't consume the entire
>     token [2]
>
>
> ok, will do
>
>     - double-check that out of memory is an appropriate failure mode for
>     the frontend [3]
>
>
>
> Well, what's the alternative? The current parser doesn't check stack 
> depth in frontend code. Presumably it too will eventually just run out 
> of memory, possibly rather sooner as the stack frames could  be more 
> expensive than the incremental parser stack extensions.
>
>
>     Just as a general style nit:
>
>     > +   if (lex->incremental)
>     > +   {
>     > +       lex->input = lex->token_terminator = lex->line_start = json;
>     > +       lex->input_length = len;
>     > +       lex->inc_state->is_last_chunk = is_last;
>     > +   }
>     > +   else
>     > +       return JSON_INVALID_LEXER_TYPE;
>
>     I think flipping this around would probably make it more readable;
>     something like:
>
>         if (!lex->incremental)
>             return JSON_INVALID_LEXER_TYPE;
>
>         lex->input = ...
>
>
>
> Noted. will do, Thanks.
>
>

Here's a new set of patches, with I think everything except the error 
case Asserts attended to. There's a change to add semantic processing to 
the test suite in patch 4, but I'd fold that into patch 1 when committing.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com