Re: Write Ahead Logging for Hash Indexes

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Stephen Frost <sfrost@snowman.net>
Cc: Robert Haas <robertmhaas@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Jesper Pedersen <jesper.pedersen@redhat.com>, Jeff Janes <jeff.janes@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-03-14T19:14:44Z
Lists: pgsql-hackers
Stephen Frost <sfrost@snowman.net> writes:
> * Tom Lane (tgl@sss.pgh.pa.us) wrote:
>> It's true that as soon as we need another overflow page, that's going to
>> get dropped beyond the 2^{N+1}-1 point, and the *apparent* size of the
>> index will grow quite a lot.  But any modern filesystem should handle
>> that without much difficulty by treating the index as a sparse file.

> Uh, last I heard we didn't allow or want sparse files in the backend
> because then we have to handle a possible out-of-disk-space failure on
> every write.

For a hash index, this would happen during a bucket split, which would
need to be resilient against out-of-disk-space anyway.

>> There may be some work to be done in places like pg_basebackup to
>> recognize and deal with sparse files, but it doesn't seem like a
>> reason to panic.

> Well, and every file-based backup tool out there..

Weren't you the one leading the charge to deprecate use of file-based
backup?

			regards, tom lane


Commits

  1. Add a regression test for snapshot too old with hash indexes.

  2. hash: Add write-ahead logging support.

  3. Improve coding in _hash_addovflpage.

  4. Remove _hash_wrtbuf() in favor of calling MarkBufferDirty().

  5. Improve hash index bucket split behavior.