Re: Should pg 11 use a lot more memory building an spgist index?

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Amit Langote <Langote_Amit_f8@lab.ntt.co.jp>, Bruno Wolff III <bruno@wolff.to>, pgsql-general@lists.postgresql.org, pgsql-hackers@lists.postgresql.org
Date: 2018-10-26T14:20:02Z
Lists: pgsql-hackers
Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2018-Oct-26, Tom Lane wrote:
>> After a quick look around, I think that making systable_begin/endscan
>> do this is a nonstarter; there are just too many call sites that would
>> be affected.  Now, you could imagine specifying that indexes on system
>> catalogs (in practice, only btree) have to clean up at endscan time
>> but other index types don't, so that only operations that might be
>> scanning user indexes need to have suitable wrapping contexts.  Not sure
>> there's a lot of benefit to that, though.

> How about modifying SysScanDescData to have a memory context member,
> which is created by systable_beginscan and destroyed by endscan?

I think it would still have problems, in that it would affect in which
context index_getnext delivers its output.  We could reasonably make
these sorts of changes in places where the entire index_beginscan/
index_getnext/index_endscan call structure is in one place, but that
doesn't apply for the systable functions.

Also, going in that direction would imply an additional memory context
switch / switch-back per tuple processed (around the index_getnext call),
which would create questions about whether it has a negative performance
impact.

			regards, tom lane


Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Fix memory leak in repeated SPGIST index scans.

  2. Add support for nearest-neighbor (KNN) searches to SP-GiST