Horton

Re: refactoring relation extension and BufferAlloc(), faster COPY

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: David Rowley <dgrowleyml@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, pgsql-hackers@postgresql.org, Thomas Munro <thomas.munro@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, Yura Sokolov <y.sokolov@postgrespro.ru>, Robert Haas <robertmhaas@gmail.com>
Date: 2023-02-11T22:25:06Z
Lists: pgsql-hackers 
Hi,

On 2023-01-20 13:40:55 +1300, David Rowley wrote:
> On Tue, 10 Jan 2023 at 15:08, Andres Freund <andres@anarazel.de> wrote:
> > Thanks for letting me now. Updated version attached.
> 
> I'm not too sure I've qualified for giving a meaningful design review
> here, but I have started looking at the patches and so far only made
> it as far as 0006.

Thanks!


> I noted down the following while reading:
> 
> v2-0001:
> 
> 1. BufferCheckOneLocalPin needs a header comment
> 
> v2-0002:
> 
> 2. The following comment and corresponding code to release the
> extension lock has been moved now.
> 
> /*
> * Release the file-extension lock; it's now OK for someone else to extend
> * the relation some more.
> */
> 
> I think it's worth detailing out why it's fine to release the
> extension lock in the new location. You've added detail to the commit
> message but I think you need to do the same in the comments too.

Will do.


> v2-0003
> 
> 3. FileFallocate() and FileZero() should likely document what they
> return, i.e zero on success and non-zero on failure.

I guess I just tried to fit in with the rest of the file :)


> 4. I'm not quite clear on why you've modified FileGetRawDesc() to call
> FileAccess() twice.

I do not have the faintest idea what happened there... Will fix.


> v2-0004:
> 
> 5. Is it worth having two versions of PinLocalBuffer() one to adjust
> the usage count and one that does not? Couldn't the version that does
> not adjust the count skip doing pg_atomic_read_u32()?

I think it'd be nicer to just move the read inside the if
(adjust_usagecount). That way the rest of the function doesn't have to be
duplicated.


Thanks,

Andres Freund

Commits

  1. lwlock: Fix quadratic behavior with very long wait lists

  2. Support RBM_ZERO_AND_CLEANUP_LOCK in ExtendBufferedRelTo(), add tests

  3. Use ExtendBufferedRelTo() in XLogReadBufferExtended()

  4. hio: Use ExtendBufferedRelBy() to extend tables more efficiently

  5. heapam: Pass number of required pages to RelationGetBufferForTuple()

  6. Convert many uses of ReadBuffer[Extended](P_NEW) to ExtendBufferedRel()

  7. Use ExtendBufferedRelTo() in {vm,fsm}_extend()

  8. bufmgr: Introduce infrastructure for faster relation extension

  9. bufmgr: Support multiple in-progress IOs by using resowner

  10. bufmgr: Acquire and clean victim buffer separately

  11. bufmgr: Add Pin/UnpinLocalBuffer()

  12. bufmgr: Add some more error checking [infrastructure] around pinning

  13. Add smgrzeroextend(), FileZero(), FileFallocate()

  14. Don't initialize page in {vm,fsm}_extend(), not needed

  15. bufmgr: Remove buffer-write-dirty tracepoints

  16. hio: Release extension lock before initializing page / pinning VM

  17. bufmgr: Fix undefined behaviour with, unrealistically, large temp_buffers

  18. Fix memory leak and inefficiency in CREATE DATABASE ... STRATEGY WAL_LOG

  19. pgstat: Track more detailed relation IO statistics

  20. Use unnamed POSIX semaphores on Cygwin.

  21. Include RelFileLocator fields individually in BufferTag.