Horton

Re: Relation bulk write facility

Heikki Linnakangas <hlinnaka@iki.fi>

View thread
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: Melanie Plageman <melanieplageman@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>, Andres Freund <andres@anarazel.de>
Date: 2023-12-14T13:02:01Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Relax fsyncing at end of a bulk load that was not WAL-logged

  2. Fix cross-version upgrade tests after f0827b443.

  3. Remove AIX support

  4. Fix compiler warning on typedef redeclaration

  5. Introduce a new smgr bulk loading facility.

Melanie just reminded about an older thread about this same thing:
https://www.postgresql.org/message-id/CAAKRu_ZQEpk6Q1WtNLgfXBdCmdU5xN_w0boVO6faO_Ax%2Bckjig%40mail.gmail.com. 
I had completely forgotten about that.

Melanie's patches in that thread implemented the same optimization of 
avoiding the fsync() if no checkpoint has happened during the index 
build. My patch here also implements batching the WAL records of 
multiple blocks, which was not part of those older patches. OTOH, those 
patches included an additional optimization of not bypassing the shared 
buffer cache if the index is small. That seems sensible too.

In this new patch, I subconsciously implemented an API close to what I 
suggested at the end of that old thread.

So I'd like to continue this effort based on this new patch. We can add 
the bypass-buffer-cache optimization later on top of this. With the new 
API that this introduces, it should be an isolated change to the 
implementation, with no changes required to the callers.

-- 
Heikki Linnakangas
Neon (https://neon.tech)