Horton

Re: BUG #18080: to_tsvector fails for long text input

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@alvh.no-ip.org>
Cc: uwe.binder@pass-consulting.com, pgsql-bugs@lists.postgresql.org
Date: 2023-09-22T17:48:59Z
Lists: pgsql-bugs

Attachments

I wrote:
> Yeah.  My thought about blocking the error had been to limit
> prs.lenwords to MaxAllocSize/sizeof(ParsedWord) in this code.

Concretely, as attached.  This allows the given test case to
complete, since it doesn't actually create very many distinct
words.  In other cases we could expect to fail when the array
has to get enlarged, but that's just a normal implementation
limitation.

I looked for other places that might initialize lenwords
to not-sane values, and didn't find any.

BTW, the field order in ParsedWord is such that there's a fair
amount of wasted pad space on 64-bit builds.  I doubt we can
get away with rearranging it in released branches; but maybe
it's worth doing something about that in HEAD, to push out
the point at which you hit the 1Gb limit.

			regards, tom lane

Commits

  1. Pack struct ParsedWord more tightly.

  2. Limit to_tsvector_byid's initial array allocation to something sane.