Re: [PATCH] Optimize json_lex_string by batching character copying

Andres Freund <andres@anarazel.de>

From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: John Naylor <john.naylor@enterprisedb.com>, Jelte Fennema <Jelte.Fennema@microsoft.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Merlin Moncure <mmoncure@gmail.com>, Andrew Dunstan <andrew.dunstan@2ndquadrant.com>, Stephen Frost <sfrost@snowman.net>
Date: 2022-07-11T16:07:28Z
Lists: pgsql-hackers
Hi,

On 2022-07-11 11:53:26 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > I wonder if we can't abstract this at least a bit better. If we go that route
> > a bit further, then add another arch, this code will be pretty much
> > unreadable.
>
> IMO, it's pretty unreadable *now*, for lack of comments about what it's
> doing and why.

Yea, that could at least be addressed by adding comments. But even with a
bunch of comments, it'd still be pretty hard to read once the events above
have happened (and they seem kind of inevitable).

I wonder if we can add a somewhat more general function for scanning until
some characters are found using SIMD? There's plenty other places that could
be useful.

Greetings,

Andres Freund



Commits

  1. Speed up lexing of long JSON strings

  2. Add optimized functions for linear search within byte arrays

  3. Build de-escaped JSON strings in larger chunks during lexing

  4. Simplify json lexing state