Horton

Re: logtape.c stats don't account for unused "prefetched" block numbers

Jeff Davis <pgsql@j-davis.com>

View thread View on postgr.es
From: Jeff Davis <pgsql@j-davis.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-09-15T03:07:44Z
Lists: pgsql-hackers 
On Mon, 2020-09-14 at 19:29 -0700, Peter Geoghegan wrote:
> Let's assume that we'll teach LogicalTapeSetBlocks() to use
> nBlocksWritten in place of nBlocksAllocated in all cases, as now
> seems
> likely. Rather than asserting "nBlocksWritten == nBlocksAllocated"
> inside LogicalTapeSetBlocks() (as I suggested earlier at one point),
> we could instead teach LogicalTapeSetBlocks() to iterate through each
> tape from the tapeset and make sure each tape has no writes buffered
> (so everything must be flushed). We could add a loop that would only
> be used on assert-enabled builds.

Sounds reasonable.

> You suggested this yourself, Jeff (my suggestion about the assertion
> is just an expansion on your suggestion from earlier). This all seems
> like a good idea to me. Can you write a patch that adjusts
> LogicalTapeSetBlocks() along these lines? Hopefully the assertion
> loop
> thing won't reveal some other problem with this plan.

Sure. Will backporting either patch into REL_13_STABLE now interfere
with RC1 release in any way?

Regards,
	Jeff Davis

Commits

  1. Change LogicalTapeSetBlocks() to use nBlocksWritten.

  2. HashAgg: release write buffers sooner by rewinding tape.

  3. logtape.c: do not preallocate for tapes when sorting

  4. Fix bogus MaxAllocSize check in logtape.c.

  5. Avoid fragmentation of logical tapes when writing concurrently.

  6. Logical Tape Set: use min heap for freelist.

  7. Don't create "holes" in BufFiles, in the new logtape code.