Re: Polyphase merge is obsolete

Heikki Linnakangas <hlinnaka@iki.fi>

From: Heikki Linnakangas <hlinnaka@iki.fi>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Peter Geoghegan <pg@bowt.ie>, Tomas Vondra <tomas.vondra@2ndquadrant.com>
Date: 2021-01-22T22:19:28Z
Lists: pgsql-hackers

Attachments

On 22/10/2020 14:48, Heikki Linnakangas wrote:
> On 11/09/2017 13:37, Tomas Vondra wrote:
>> I planned to do some benchmarking on this patch, but apparently the
>> patch no longer applies. Rebase please?
> 
> Here's a rebase of this. Sorry to keep you waiting :-).

Here's an updated version that fixes one bug:

The CFBot was reporting a failure on the FreeBSD system [1]. It turned 
out to be an out-of-memory issue caused by an underflow bug in the 
calculation of the size of the tape read buffer size. With a small 
work_mem size, the memory left for tape buffers was negative, and that 
wrapped around to a very large number. I believe that was not caught by 
the other systems, because the other ones had enough memory for the 
incorrectly-sized buffers anyway. That was the case on my laptop at 
least. It did cause a big slowdown in the 'tuplesort' regression test 
though, which I hadn't noticed.

The fix for that bug is here as a separate patch for easier review, but 
I'll squash it before committing.

[1] https://cirrus-ci.com/task/6699842091089920

- Heikki

Commits

  1. Fix and clarify function comment on LogicalTapeSetCreate.

  2. Refactor LogicalTapeSet/LogicalTape interface.

  3. Replace polyphase merge algorithm with a simple balanced k-way merge.

  4. logtape.c: do not preallocate for tapes when sorting