RE: [Patch] Optimize dropping of relation buffers using dlist

tsunakawa.takay@fujitsu.com <tsunakawa.takay@fujitsu.com>

From: "tsunakawa.takay@fujitsu.com" <tsunakawa.takay@fujitsu.com>
To: "k.jamison@fujitsu.com" <k.jamison@fujitsu.com>
Cc: 'Kyotaro Horiguchi' <horikyota.ntt@gmail.com>, "amit.kapila16@gmail.com" <amit.kapila16@gmail.com>, "tgl@sss.pgh.pa.us" <tgl@sss.pgh.pa.us>, "andres@anarazel.de" <andres@anarazel.de>, "robertmhaas@gmail.com" <robertmhaas@gmail.com>, "tomas.vondra@2ndquadrant.com" <tomas.vondra@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-11-12T04:13:35Z
Lists: pgsql-hackers
The patch looks OK.  I think as Thomas-san suggested, we can remove the modification to smgrnblocks() and don't care wheter the size is cached or not.  But I think the current patch is good too, so I'd like to leave it up to a committer to decide which to choose.

I measured performance in a different angle -- the time DropRelFileNodeBuffers() and DropRelFileNodeAllBuffers() took.  That reveals the direct improvement and degradation.

I used 1,000 tables, each of which is 1 MB.  I used shared_buffers = 128 MB for the case where the traditional full buffer scan is done, and shared_buffers = 100 GB for the case where the optimization path takes effect.

The results are almost good as follows:

A. UNPATCHED

128 MB shared_buffers
1. VACUUM = 0.04 seconds
2. TRUNCATE = 0.04 seconds

100 GB shared_buffers
3. VACUUM = 69.4 seconds
4. TRUNCATE = 69.1 seconds


B. PATCHED

128 MB shared_buffers (full scan)
5. VACUUM = 0.04 seconds
6. TRUNCATE = 0.07 seconds

100 GB shared_buffers (optimized path)
7. VACUUM = 0.02 seconds
8. TRUNCATE = 0.08 seconds


So, I'd like to mark this as ready for committer.


Regards
Takayuki Tsunakawa




Commits

  1. Fix size overflow in calculation introduced by commits d6ad34f3 and bea449c6.

  2. Optimize DropRelFileNodesAllBuffers() for recovery.

  3. Optimize DropRelFileNodeBuffers() for recovery.

  4. Cache smgrnblocks() results in recovery.

  5. Add a check to prevent overwriting valid data if smgrnblocks() gives a