Re: WIP Incremental JSON Parser

On 2024-01-09 Tu 13:46, Jacob Champion wrote:
> On Tue, Dec 26, 2023 at 8:49 AM Andrew Dunstan <andrew@dunslane.net> wrote:
>> Quite a long time ago Robert asked me about the possibility of an
>> incremental JSON parser. I wrote one, and I've tweaked it a bit, but the
>> performance is significantly worse that that of the current Recursive
>> Descent parser.
> The prediction stack is neat. It seems like the main loop is hit so
> many thousands of times that micro-optimization would be necessary...
> I attached a sample diff to get rid of the strlen calls during
> push_prediction(), which speeds things up a bit (8-15%, depending on
> optimization level) on my machines.


Thanks for looking! I've been playing around with a similar idea, but 
yours might be better.



>
> Maybe it's possible to condense some of those productions down, and
> reduce the loop count? E.g. does every "scalar" production need to go
> three times through the loop/stack, or can the scalar semantic action
> just peek at the next token prediction and do all the callback work at
> once?


Also a good suggestion. Will look and see. IIRC I had trouble with this bit.


>
>> +               case JSON_SEM_SCALAR_CALL:
>> +                   {
>> +                       json_scalar_action sfunc = sem->scalar;
>> +
>> +                       if (sfunc != NULL)
>> +                           (*sfunc) (sem->semstate, scalar_val, scalar_tok);
>> +                   }
> Is it safe to store state (scalar_val/scalar_tok) on the stack, or
> does it disappear if the parser hits an incomplete token?


Good point. In fact it might be responsible for the error I'm currently 
trying to get to the bottom of.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com