Re: Extensible Rmgr for Table AMs

Julien Rouhaud <rjuju123@gmail.com>

From: Julien Rouhaud <rjuju123@gmail.com>
To: Jeff Davis <pgsql@j-davis.com>
Cc: pgsql-hackers@postgresql.org, Andres Freund <andres@anarazel.de>, Amit Kapila <amit.kapila16@gmail.com>, Tom Lane <tgl@sss.pgh.pa.us>, Simon Riggs <simon@2ndquadrant.com>
Date: 2022-02-03T05:34:18Z
Lists: pgsql-hackers
Hi,

On Tue, Feb 01, 2022 at 03:38:32PM -0800, Jeff Davis wrote:
> On Tue, 2022-02-01 at 20:45 +0800, Julien Rouhaud wrote:
> > 
> > One last thing, did you do some benchmark with a couple custom rmgr
> > to see how
> > much the O(n) access is showing up in profiles?
> 
> What kind of a test case would be reasonable there? You mean having a
> lot of custom rmgrs?
> 
> I was expecting that few people would have more than one custom rmgr
> loaded anyway, so a sparse array or hashtable seemed wasteful. If
> custom rmgrs become popular we probably need to have a larger ID space
> anyway, but it seems like overengineering to do so now.

I agree that having dozen of custom rmgrs doesn't seem likely, but I also have
no idea of how much overhead you get by not doing a direct array access.  I
think it would be informative to benchmark something like simple OLTP write
workload on a fast storage (or a ramdisk, or with fsync off...), with the used
rmgr being the 1st and the 2nd custom rmgr.  Both scenario still seems
plausible and shouldn't degenerate on good hardware.



Commits

  1. Fix off-by-one error in pg_waldump, introduced in 5c279a6d350.

  2. Fix another buildfarm issue from commit 5c279a6d350.

  3. Fix warning introduced in 5c279a6d350.

  4. Custom WAL Resource Managers.

  5. Make logical decoding a part of the rmgr.