Re: [Patch] Optimize dropping of relation buffers using dlist

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: amit.kapila16@gmail.com
Cc: k.jamison@fujitsu.com, tsunakawa.takay@fujitsu.com, tgl@sss.pgh.pa.us, andres@anarazel.de, robertmhaas@gmail.com, tomas.vondra@2ndquadrant.com, pgsql-hackers@postgresql.org
Date: 2020-11-10T03:26:54Z
Lists: pgsql-hackers
At Tue, 10 Nov 2020 08:33:26 +0530, Amit Kapila <amit.kapila16@gmail.com> wrote in 
> On Tue, Nov 10, 2020 at 8:19 AM k.jamison@fujitsu.com
> <k.jamison@fujitsu.com> wrote:
> >
> > I repeated the recovery performance test for vacuum. (I made a mistake previously in NBuffers/128)
> > The 3 kinds of thresholds are almost equally performant. I chose NBuffers/256 for this patch.
> >
> > | s_b   | Master | NBuffers/512 | NBuffers/256 | NBuffers/128 |
> > |-------|--------|--------------|--------------|--------------|
> > | 128MB | 1.006  | 1.007        | 1.007        | 1.007        |
> > | 1GB   | 0.706  | 0.606        | 0.606        | 0.606        |
> > | 20GB  | 1.907  | 0.606        | 0.606        | 0.606        |
> > | 100GB | 7.013  | 0.706        | 0.606        | 0.606        |
> >
> 
> I think this data is not very clear. What is the unit of time? What is
> the size of the relation used for the test? Did the test use an
> optimized path for all cases? If at 128MB, there is no performance
> gain, can we consider the size of shared buffers as 256MB as well for
> the threshold?

In the previous testing, it was shown as:

Recovery Time (in seconds)
| s_b   | master | patched | %reg   | 
|-------|--------|---------|--------| 
| 128MB | 3.043  | 2.977   | -2.22% | 
| 1GB   | 3.417  | 3.41    | -0.21% | 
| 20GB  | 20.597 | 2.409   | -755%  | 
| 100GB | 66.862 | 2.409   | -2676% |


So... The numbers seems to be in seconds, but the master gets about 10
times faster than this result for uncertain reasons. It seems that the
result contains something different from the difference by this patch
as a larger part.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center



Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a