Re: [PATCH] Optimize json_lex_string by batching character copying

Nathan Bossart <nathandbossart@gmail.com>

From: Nathan Bossart <nathandbossart@gmail.com>
To: John Naylor <john.naylor@enterprisedb.com>
Cc: Andres Freund <andres@anarazel.de>, Tom Lane <tgl@sss.pgh.pa.us>, Jelte Fennema <Jelte.Fennema@microsoft.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Merlin Moncure <mmoncure@gmail.com>, Andrew Dunstan <andrew.dunstan@2ndquadrant.com>, Stephen Frost <sfrost@snowman.net>
Date: 2022-08-26T03:14:37Z
Lists: pgsql-hackers
On Thu, Aug 25, 2022 at 01:35:45PM +0700, John Naylor wrote:
> - For the following comment, pgindent will put spaced operands on a
> separate line which is not great for readability. and our other
> reference to the Stanford bithacks page keeps the in-page link, and I
> see no reason to exclude it -- if it goes missing, the whole page will
> still load. So I put back those two details.
> 
> +        * To find bytes <= c, we can use bitwise operations to find
> bytes < c+1,
> +        * but it only works if c+1 <= 128 and if the highest bit in v
> is not set.
> +        * Adapted from
> +        * https://graphics.stanford.edu/~seander/bithacks.html#HasLessInWord

This was just unnecessary fiddling on my part, sorry about that.

> +test_lfind8_internal(uint8 key)
> +{
> +	uint8		charbuf[LEN_WITH_TAIL(Vector8)];
> +	const int	len_no_tail = LEN_NO_TAIL(Vector8);
> +	const int	len_with_tail = LEN_WITH_TAIL(Vector8);
> +
> +	memset(charbuf, 0xFF, len_with_tail);
> +	/* search tail to test one-byte-at-a-time path */
> +	charbuf[len_with_tail - 1] = key;
> +	if (key > 0x00 && pg_lfind8(key - 1, charbuf, len_with_tail))
> +		elog(ERROR, "pg_lfind8() found nonexistent element <= '0x%x'", key - 1);
> +	if (key < 0xFF && !pg_lfind8(key, charbuf, len_with_tail))
> +		elog(ERROR, "pg_lfind8() did not find existing element <= '0x%x'", key);
> +	if (key < 0xFE && pg_lfind8(key + 1, charbuf, len_with_tail))
> +		elog(ERROR, "pg_lfind8() found nonexistent element <= '0x%x'", key + 1);
> +
> +	memset(charbuf, 0xFF, len_with_tail);
> +	/* search with vector operations */
> +	charbuf[len_no_tail - 1] = key;
> +	if (key > 0x00 && pg_lfind8(key - 1, charbuf, len_no_tail))
> +		elog(ERROR, "pg_lfind8() found nonexistent element <= '0x%x'", key - 1);
> +	if (key < 0xFF && !pg_lfind8(key, charbuf, len_no_tail))
> +		elog(ERROR, "pg_lfind8() did not find existing element <= '0x%x'", key);
> +	if (key < 0xFE && pg_lfind8(key + 1, charbuf, len_no_tail))
> +		elog(ERROR, "pg_lfind8() found nonexistent element <= '0x%x'", key + 1);
> +}

nitpick: Shouldn't the elog() calls use "==" instead of "<=" for this one?

Otherwise, 0001 looks good to me.

-- 
Nathan Bossart
Amazon Web Services: https://aws.amazon.com



Commits

  1. Speed up lexing of long JSON strings

  2. Add optimized functions for linear search within byte arrays

  3. Build de-escaped JSON strings in larger chunks during lexing

  4. Simplify json lexing state