Re: WIP Incremental JSON Parser

On 2024-03-26 Tu 17:52, Jacob Champion wrote:
> On Mon, Mar 25, 2024 at 3:14 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> - add Assert calls in impossible error cases [2]
> To expand on this one, I think these parts of the code (marked with
> `<- here`) are impossible to reach:
>
>> switch (top)
>> {
>>      case JSON_TOKEN_STRING:
>>          if (next_prediction(pstack) == JSON_TOKEN_COLON)
>>              ctx = JSON_PARSE_STRING;
>>          else
>>              ctx = JSON_PARSE_VALUE;                    <- here
>>          break;
>>      case JSON_TOKEN_NUMBER:                            <- here
>>      case JSON_TOKEN_TRUE:                              <- here
>>      case JSON_TOKEN_FALSE:                             <- here
>>      case JSON_TOKEN_NULL:                              <- here
>>      case JSON_TOKEN_ARRAY_START:                       <- here
>>      case JSON_TOKEN_OBJECT_START:                      <- here
>>          ctx = JSON_PARSE_VALUE;
>>          break;
>>      case JSON_TOKEN_ARRAY_END:                         <- here
>>          ctx = JSON_PARSE_ARRAY_NEXT;
>>          break;
>>      case JSON_TOKEN_OBJECT_END:                        <- here
>>          ctx = JSON_PARSE_OBJECT_NEXT;
>>          break;
>>      case JSON_TOKEN_COMMA:                             <- here
>>          if (next_prediction(pstack) == JSON_TOKEN_STRING)
>>              ctx = JSON_PARSE_OBJECT_NEXT;
>>          else
>>              ctx = JSON_PARSE_ARRAY_NEXT;
>>          break;
> Since none of these cases are non-terminals, the only way to get to
> this part of the code is if (top != tok). But inspecting the
> productions and transitions that can put these tokens on the stack, it
> looks like the only way for them to be at the top of the stack to
> begin with is if (tok == top). (Otherwise, we would have chosen a
> different production, or else errored out on a non-terminal.)
>
> This case is possible...
>
>>      case JSON_TOKEN_STRING:
>>          if (next_prediction(pstack) == JSON_TOKEN_COLON)
>>              ctx = JSON_PARSE_STRING;
> ...if we're in the middle of JSON_PROD_[MORE_]KEY_PAIRS. But the
> corresponding else case is not, because if we're in the middle of a
> _KEY_PAIRS production, the next_prediction() _must_ be
> JSON_TOKEN_COLON.
>
> Do you agree, or am I missing a way to get there?
>


One good way of testing would be to add the Asserts, build with 
-DFORCE_JSON_PSTACK, and run the standard regression suite, which has a 
fairly comprehensive set of JSON errors. I'll play with that.

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com