Horton

Re: logtape.c stats don't account for unused "prefetched" block numbers

Peter Geoghegan <pg@bowt.ie>

View thread View on postgr.es
From: Peter Geoghegan <pg@bowt.ie>
To: Jeff Davis <pgsql@j-davis.com>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2020-09-15T02:09:17Z
Lists: pgsql-hackers 
On Mon, Sep 14, 2020 at 6:56 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I'm not sure what I was talking about earlier when I connected this
> with the main/instrumentation issue, since preallocation used by
> logtape.c to help HashAggs-that-spill necessarily reserves blocks
> without writing them out for a while (the fires in California have
> made it difficult to be productive). You might write blocks out as
> zero blocks first, and then only write the real data later
> (overwriting the zero blocks). But no matter how the writes among
> tapes are interlaced, the fact is that nBlocksAllocated can exceed
> nBlocksWritten by at least one block per active tape.

Oh, wait. Of course the point was that we wouldn't even have to use
nBlocksAllocated in LogicalTapeSetBlocks() anymore -- we would make
the assumption that nBlocksWritten could be used for all callers in
all cases. Which is a reasonable assumption once you enforce that
there are no active write buffers. Which is evidently a good idea
anyway, since it saves on temp file disk space in
HashAggs-that-spill/prealloc cases with very little work_mem.


-- 
Peter Geoghegan

Commits

  1. Change LogicalTapeSetBlocks() to use nBlocksWritten.

  2. HashAgg: release write buffers sooner by rewinding tape.

  3. logtape.c: do not preallocate for tapes when sorting

  4. Fix bogus MaxAllocSize check in logtape.c.

  5. Avoid fragmentation of logical tapes when writing concurrently.

  6. Logical Tape Set: use min heap for freelist.

  7. Don't create "holes" in BufFiles, in the new logtape code.