Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Julien Rouhaud <rjuju123@gmail.com>

View thread View on postgr.es
From: Julien Rouhaud <rjuju123@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-06-15T11:31:05Z
Lists: pgsql-hackers 
On Tue, Jun 15, 2021 at 04:50:24PM +0530, Dilip Kumar wrote:
> Currently, CREATE DATABASE forces a checkpoint, then copies all the
> files, then forces another checkpoint. The comments in the createdb()
> function explain the reasons for this. The attached patch fixes this
> problem by making CREATE DATABASE completely WAL-logged so that now we
> can avoid checkpoints.  The patch modifies both CREATE DATABASE and
> ALTER DATABASE..SET TABLESPACE to be fully WAL-logged.
> 
> One main advantage of this change is that it will be cheaper. Forcing
> checkpoints on an idle system is no big deal, but when the system is
> under heavy write load, it's very expensive. Another advantage is that
> it makes things better for features like TDE, which might want the
> pages in the source database to be encrypted using a different key or
> nonce than the pages in the target database.

I only had a quick look at the patch but AFAICS your patch makes the new
behavior mandatory.  Wouldn't it make sense to have a way to use the previous
approach?  People creating wanting to copy somewhat big database and with a
slow replication may prefer to pay 2 checkpoints rather than stream everything.
Same for people who have an otherwise idle system (I often use that to make
temporary backups and/or prepare multiple dataset and most of the time the
checkpoint is basically free).

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.