Re: pure parsers and reentrant scanners
Peter Eisentraut <peter@eisentraut.org>
From: Peter Eisentraut <peter@eisentraut.org>
To: Andreas Karlsson <andreas@proxel.se>,
pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2024-12-19T20:57:55Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Return yyparse() result not via global variable
- 473a575e0597 18.0 landed
-
Remove flex version checks
- 0869ea43e9c2 18.0 landed
-
Drop warning-free support for Flex 2.5.35
- 6fdd5d956343 18.0 landed
-
plpgsql: pure parser and reentrant scanner
- 7b27f5fd36cb 18.0 landed
-
flex code modernization: Replace YY_EXTRA_TYPE define with flex option
- b1ef48980ddd 18.0 landed
-
guc: reentrant scanner
- d663f150b5ed 18.0 landed
-
jsonpath scanner: reentrant scanner
- 2a7425d7eef9 18.0 landed
-
syncrep parser: pure parser and reentrant scanner
- db6856c9913f 18.0 landed
-
replication parser: pure parser and reentrant scanner
- e4a8fb8fefb9 18.0 landed
-
bootstrap: pure parser and reentrant scanner
- 3e4bacb17100 18.0 landed
-
Small whitespace improvement
- 399d0f1e11b5 18.0 landed
-
Prevent redeclaration of typedef yyscan_t
- 382092a0cd2c 18.0 landed
-
seg: pure parser and reentrant scanner
- 1f0de66ea2a5 18.0 landed
-
cube: pure parser and reentrant scanner
- 802fe923e3cd 18.0 landed
Attachments
- v2-0001-replication-parser-pure-parser-and-reentrant-scan.patch (text/plain) patch v2-0001
- v2-0002-replication-parser-Use-palloc-instead-of-malloc-f.patch (text/plain) patch v2-0002
- v2-0003-replication-parser-Simplify-flex-scan-buffer-mana.patch (text/plain) patch v2-0003
- v2-0004-replication-parser-Use-flex-yyextra.patch (text/plain) patch v2-0004
- v2-0005-syncrep-parser-pure-parser-and-reentrant-scanner.patch (text/plain) patch v2-0005
- v2-0006-syncrep-parser-Use-palloc-instead-of-malloc-for-f.patch (text/plain) patch v2-0006
- v2-0007-syncrep-parser-Simplify-flex-scan-buffer-manageme.patch (text/plain) patch v2-0007
- v2-0008-syncrep-parser-Use-flex-yyextra.patch (text/plain) patch v2-0008
- v2-0009-jsonpath-scanner-reentrant-scanner.patch (text/plain) patch v2-0009
- v2-0010-guc-reentrant-scanner.patch (text/plain) patch v2-0010
- v2-0011-plpgsql-reentrant-scanner.patch (text/plain) patch v2-0011
On 19.12.24 13:48, Peter Eisentraut wrote: > On 17.12.24 01:46, Andreas Karlsson wrote: >> On 12/16/24 8:39 AM, Peter Eisentraut wrote: >>> I'll leave it at this for now and wait for some reviews. >> >> I really like this work since it makes the code cleaner to read on top >> of paving the way for threading. >> >> Reviewed the patches and found a couple of issues. >> >> - Shouldn't yyext in syncrep_scanner_init() be allocated on the heap? >> Or at least on the stack but by the caller? > > I think it's correct the way it is. It's only a temporary space for the > scanner, so we can allocate it in the innermost scope. >> - Also fixed the static remaining variables in the replication parser >> in an attached patch. > > Thanks, I'll take a look at that. I see what was going on here. I was allocating yyext as a local variable in the init function and then it would go out of scope while the scanner is still in use. That's why this didn't work for me. I had written essentially the same patch as you for the replication scanner yyextra but with a local variable, and it was "mysteriously" failing the tests for me. Your solution is better. (For the jsonpath scanner, the local variable works because the scanner init and shutdown are called from the same function.) Here is an updated patch set on top of what has been committed so far, with all the issues you pointed out addressed.