Re: suboverflowed subtransactions concurrency performance optimize

x4mmm@yandex-team.ru

From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Pengchengliu <pengchengliu@tju.edu.cn>
Cc: pgsql-hackers@postgresql.org
Date: 2021-08-30T10:24:57Z
Lists: pgsql-hackers
Hi Pengcheng!

You are solving important problem, thank you!

> 30 авг. 2021 г., в 13:43, Pengchengliu <pengchengliu@tju.edu.cn> написал(а):
> 
> To resolve this performance problem, we think about a solution which cache
> SubtransSLRU to local cache. 
> First we can query parent transaction id from SubtransSLRU, and copy the
> SLRU page to local cache page.
> After that if we need query parent transaction id again, we can query it
> from local cache directly.

A copy of SLRU in each backend's cache can consume a lot of memory. Why create a copy if we can optimise shared representation of SLRU?

JFYI There is a related patch to make SimpleLruReadPage_ReadOnly() faster for bigger SLRU buffers[0].
Also Nik Samokhvalov recently published interesting investigation on the topic, but for some reason his message did not pass the moderation. [1]

Also it's important to note that there was a community request to move SLRUs to shared_buffers [2].

Thanks!

Best regards, Andrey Borodin.

[0] https://commitfest.postgresql.org/34/2627/
[1] https://www.postgresql.org/message-id/flat/BE73A0BB-5929-40F4-BAF8-55323DE39561%40yandex-team.ru
[2] https://www.postgresql.org/message-id/flat/20180814213500.GA74618%4060f81dc409fc.ant.amazon.com


Commits

  1. Revert "Add single-item cache when looking at topmost XID of a subtrans XID"

  2. Add single-item cache when looking at topmost XID of a subtrans XID

  3. Fix handling of partitioned index in RelationGetNumberOfBlocksInFork()