Re: pure parsers and reentrant scanners

Peter Eisentraut <peter@eisentraut.org>

From: Peter Eisentraut <peter@eisentraut.org>

To: pgsql-hackers <pgsql-hackers@postgresql.org>

Date: 2024-12-16T07:39:06Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Return yyparse() result not via global variable
- 473a575e0597 18.0 landed
Remove flex version checks
- 0869ea43e9c2 18.0 landed
Drop warning-free support for Flex 2.5.35
- 6fdd5d956343 18.0 landed
plpgsql: pure parser and reentrant scanner
- 7b27f5fd36cb 18.0 landed
flex code modernization: Replace YY_EXTRA_TYPE define with flex option
- b1ef48980ddd 18.0 landed
guc: reentrant scanner
- d663f150b5ed 18.0 landed
jsonpath scanner: reentrant scanner
- 2a7425d7eef9 18.0 landed
syncrep parser: pure parser and reentrant scanner
- db6856c9913f 18.0 landed
replication parser: pure parser and reentrant scanner
- e4a8fb8fefb9 18.0 landed
bootstrap: pure parser and reentrant scanner
- 3e4bacb17100 18.0 landed
Small whitespace improvement
- 399d0f1e11b5 18.0 landed
Prevent redeclaration of typedef yyscan_t
- 382092a0cd2c 18.0 landed
seg: pure parser and reentrant scanner
- 1f0de66ea2a5 18.0 landed
cube: pure parser and reentrant scanner
- 802fe923e3cd 18.0 landed

Attachments

v1-0001-cube-pure-parser-and-reentrant-scanner.patch (text/plain) patch v1-0001
v1-0002-cube-Use-palloc-instead-of-malloc-for-flex.patch (text/plain) patch v1-0002
v1-0003-cube-Simplify-flex-scan-buffer-management.patch (text/plain) patch v1-0003
v1-0004-seg-pure-parser-and-reentrant-scanner.patch (text/plain) patch v1-0004
v1-0005-seg-Use-palloc-instead-of-malloc-for-flex.patch (text/plain) patch v1-0005
v1-0006-seg-Simplify-flex-scan-buffer-management.patch (text/plain) patch v1-0006
v1-0007-replication-parser-pure-parser-and-reentrant-scan.patch (text/plain) patch v1-0007
v1-0008-replication-parser-Use-palloc-instead-of-malloc-f.patch (text/plain) patch v1-0008
v1-0009-replication-parser-Simplify-flex-scan-buffer-mana.patch (text/plain) patch v1-0009
v1-0010-syncrep-parser-pure-parser-and-reentrant-scanner.patch (text/plain) patch v1-0010
v1-0011-syncrep-parser-Use-palloc-instead-of-malloc-for-f.patch (text/plain) patch v1-0011
v1-0012-syncrep-parser-Simplify-flex-scan-buffer-manageme.patch (text/plain) patch v1-0012
v1-0013-syncrep-parser-Use-flex-yyextra.patch (text/plain) patch v1-0013
v1-0014-jsonpath-scanner-reentrant-scanner.patch (text/plain) patch v1-0014
v1-0015-jsonpath-scanner-Use-flex-yyextra.patch (text/plain) patch v1-0015
v1-0016-bootstrap-pure-parser-and-reentrant-scanner.patch (text/plain) patch v1-0016
v1-0017-bootstrap-Use-palloc-instead-of-malloc-for-flex.patch (text/plain) patch v1-0017
v1-0018-guc-reentrant-scanner.patch (text/plain) patch v1-0018
v1-0019-plpgsql-reentrant-scanner.patch (text/plain) patch v1-0019

On 02.12.24 10:46, Peter Eisentraut wrote:
> This patch series changes several parsers in the backend and contrib 
> modules to use bison pure parsers and flex reentrant scanners. This is 
> ultimately toward thread-safety, but I think it's also just nicer in 
> general, and it might also fix a few possible small memory leaks.

I did a bit more work on this, so here is an updated patch set.

> Not done yet:
> * src/backend/utils/misc/guc-file.l
> * src/pl/plpgsql/src/pl_gram.y

I converted both of these scanners to reentrant, but I haven't done the 
plpgsql parser yet.

> Not relevant for backend thread-safety:
> * src/backend/bootstrap/
> 
> It might make sense to eventually covert that one as well, just so that 
> the APIs are kept similar. But that could be for later.

I have done this one.

I'll leave it at this for now and wait for some reviews.