Horton

pure parsers and reentrant scanners

Peter Eisentraut <peter@eisentraut.org>

View thread
From: Peter Eisentraut <peter@eisentraut.org>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-12-02T09:46:00Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Return yyparse() result not via global variable

  2. Remove flex version checks

  3. Drop warning-free support for Flex 2.5.35

  4. plpgsql: pure parser and reentrant scanner

  5. flex code modernization: Replace YY_EXTRA_TYPE define with flex option

  6. guc: reentrant scanner

  7. jsonpath scanner: reentrant scanner

  8. syncrep parser: pure parser and reentrant scanner

  9. replication parser: pure parser and reentrant scanner

  10. bootstrap: pure parser and reentrant scanner

  11. Small whitespace improvement

  12. Prevent redeclaration of typedef yyscan_t

  13. seg: pure parser and reentrant scanner

  14. cube: pure parser and reentrant scanner

Attachments

This patch series changes several parsers in the backend and contrib 
modules to use bison pure parsers and flex reentrant scanners. This is 
ultimately toward thread-safety, but I think it's also just nicer in 
general, and it might also fix a few possible small memory leaks.

I organized this patch series into very small incremental changes so 
that it's easier to follow. The final commits should probably combined a 
bit more (e.g., one per module).

In this patch series I have so far dealt with

* contrib/cube/
* contrib/seg/
* src/backend/replication/repl_*
* src/backend/replication/syncrep_*

These four needed the whole treatment: pure parser, reentrant scanner, 
and updated memory handling.

Also:
* src/backend/utils/adt/jsonpath_scan.l

This one already had a pure parser and palloc-based memory handling, but 
not a reentrant scanner, so I just did that.

The above are all pretty similar, so it was relatively easy to work 
through them once I had the first one figured out.

A couple of things that are still missing in the above:

* For repl_scanner.l, I want to use yyextra to deal with the static 
variables marked /*FIXME*/, but somehow I made that buggy, I'll need to 
take another look later.
* For both the replication parser and the syncrep parser, get rid of the 
global variables to pass back the results. Again, here I need another 
look later. I confused either myself or the compiler on these.

cube, seg, and jsonpath are about as done as I would want them to be.

Not done yet:
* src/backend/utils/misc/guc-file.l
* src/pl/plpgsql/src/pl_gram.y

These have quite different structures and requirements, so I plan to 
deal with them separately.

Not relevant for backend thread-safety:
* src/backend/bootstrap/

It might make sense to eventually covert that one as well, just so that 
the APIs are kept similar. But that could be for later.

Note that the core scanner and parser are already reentrant+pure.

Also, there are various other scanners and parsers in frontends (psql, 
pgbench, ecpg) that are not relevant for this. (Again, it might make 
sense to convert some of them later, and some of them are already done.)

AFAICT, all the options and coding techniques used here are already in 
use elsewhere in the tree, so there shouldn't be any concerns about 
bison or flex version compatibility.