Horton

Re: MultiXact\SLRU buffers configuration

x4mmm@yandex-team.ru

View thread View on postgr.es
From: Andrey Borodin <x4mmm@yandex-team.ru>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Gilles Darold <gilles@darold.net>, Tomas Vondra <tomas.vondra@enterprisedb.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alexander Korotkov <aekorotkov@gmail.com>, Anastasia Lubennikova <a.lubennikova@postgrespro.ru>, Daniel Gustafsson <daniel@yesql.se>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-04-11T18:37:21Z
Lists: pgsql-hackers

Attachments


> 8 апр. 2021 г., в 15:22, Thomas Munro <thomas.munro@gmail.com> написал(а):
> 
> On Thu, Apr 8, 2021 at 7:24 PM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
>> I agree that this version of eviction seems much more effective and less intrusive than RR. And it's still LRU, which is important for subsystem that is called SLRU.
>> shared->search_slotno is initialized implicitly with memset(). But this seems like a common practice.
>> Also comment above "max_search = Min(shared->num_slots, MAX_REPLACEMENT_SEARCH);" does not reflect changes.
>> 
>> Besides this patch looks good to me.
> 
> Thanks!  I chickened out of committing a buffer replacement algorithm
> patch written 11 hours before the feature freeze, but I also didn't
> really want to commit the GUC patch without that.  Ahh, if only we'd
> latched onto the real problems here just a little sooner, but there is
> always PostgreSQL 15, I heard it's going to be amazing.  Moved to next
> CF.

I have one more idea inspired by CPU caches.
Let's make SLRU n-associative, where n ~ 8.
We can divide buffers into "banks", number of banks must be power of 2.
All banks are of equal size. We choose bank size to approximately satisfy user's configured buffer size.
Each page can live only within one bank. We use same search and eviction algorithms as we used in SLRU, but we only need to search\evict over 8 elements.
All SLRU data of a single bank will be colocated within at most 2 cache line.

I did not come up with idea how to avoid multiplication of bank_number * bank_size in case when user configured 31337 buffers (any number that is radically not a power of 2).

PFA patch implementing this idea.

Best regards, Andrey Borodin.

Commits

  1. Rework new SLRU test with injection points

  2. injection_point: Add injection_points.stats

  3. injection_points: Add initialization of shmem state when loading module

  4. Add injection-point test for new multixact CV usage

  5. Use conditional variable to wait for next MultiXact offset

  6. Improve performance of subsystems on top of SLRU

  7. Adjust VACUUM hastup LP_REDIRECT comments.

  8. Test replay of regression tests, attempt II.