Re: WIP Incremental JSON Parser

On Mon, Mar 25, 2024 at 3:14 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> - add Assert calls in impossible error cases [2]

To expand on this one, I think these parts of the code (marked with
`<- here`) are impossible to reach:

> switch (top)
> {
>     case JSON_TOKEN_STRING:
>         if (next_prediction(pstack) == JSON_TOKEN_COLON)
>             ctx = JSON_PARSE_STRING;
>         else
>             ctx = JSON_PARSE_VALUE;                    <- here
>         break;
>     case JSON_TOKEN_NUMBER:                            <- here
>     case JSON_TOKEN_TRUE:                              <- here
>     case JSON_TOKEN_FALSE:                             <- here
>     case JSON_TOKEN_NULL:                              <- here
>     case JSON_TOKEN_ARRAY_START:                       <- here
>     case JSON_TOKEN_OBJECT_START:                      <- here
>         ctx = JSON_PARSE_VALUE;
>         break;
>     case JSON_TOKEN_ARRAY_END:                         <- here
>         ctx = JSON_PARSE_ARRAY_NEXT;
>         break;
>     case JSON_TOKEN_OBJECT_END:                        <- here
>         ctx = JSON_PARSE_OBJECT_NEXT;
>         break;
>     case JSON_TOKEN_COMMA:                             <- here
>         if (next_prediction(pstack) == JSON_TOKEN_STRING)
>             ctx = JSON_PARSE_OBJECT_NEXT;
>         else
>             ctx = JSON_PARSE_ARRAY_NEXT;
>         break;

Since none of these cases are non-terminals, the only way to get to
this part of the code is if (top != tok). But inspecting the
productions and transitions that can put these tokens on the stack, it
looks like the only way for them to be at the top of the stack to
begin with is if (tok == top). (Otherwise, we would have chosen a
different production, or else errored out on a non-terminal.)

This case is possible...

>     case JSON_TOKEN_STRING:
>         if (next_prediction(pstack) == JSON_TOKEN_COLON)
>             ctx = JSON_PARSE_STRING;

...if we're in the middle of JSON_PROD_[MORE_]KEY_PAIRS. But the
corresponding else case is not, because if we're in the middle of a
_KEY_PAIRS production, the next_prediction() _must_ be
JSON_TOKEN_COLON.

Do you agree, or am I missing a way to get there?

Thanks,
--Jacob