Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Julien Rouhaud <rjuju123@gmail.com>
Cc: Bruce Momjian <bruce@momjian.us>, Ashutosh Sharma <ashu.coek88@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-10T15:32:42Z
Lists: pgsql-hackers 
On Thu, Feb 10, 2022 at 2:52 AM Julien Rouhaud <rjuju123@gmail.com> wrote:
> Those extra WALs will also impact backups and replication.  You could have
> fancy hardware, a read-mostly workload and the need to replicate over a slow
> WAN, and in that case the 10GB could be much more problematic.

True, I guess, but how bad does your WAN have to be for that to be an
issue? On a 1 gigabit/second link, that's a little over 2 minutes of
transfer time. That's not nothing, but it's not extreme, either,
especially because there's no sense in querying an empty database.
You're going to have to put some stuff in that database before you can
do anything meaningful with it, and that's going to have to be
replicated with or without this feature.

I am not saying it couldn't be a problem, and that's why I'm endorsing
making the behavior optional. But I think that it's a niche scenario.
You need a bigger-than-normal template database, a slow WAN link, AND
you need the amount of data loaded into the databases you create from
the template to be small enough to make the cost of logging the
template pages material. If you create a 10GB database from a template
and then load 200GB of data into it, the WAL-logging overhead of
creating the template is only 5%.

I won't really be surprised if we hear that someone has a 10GB
template database and likes to make a ton of copies and only change
500 rows in each one while replicating the whole thing over a slow
WAN. That can definitely happen, and I'm sure whoever is doing that
has reasons for it which they consider good and sufficient. However, I
don't think there are likely to be a ton of people doing stuff like
that - just a few.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.