Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

From: Dilip Kumar <dilipbalaut@gmail.com>

To: Robert Haas <robertmhaas@gmail.com>

Cc: Ashutosh Sharma <ashu.coek88@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2022-03-14T11:51:32Z

Lists: pgsql-hackers

Attachments

v14-0002-Extend-relmap-interfaces.patch (text/x-patch) patch v14-0002
v14-0001-Refactor-relmap-load-and-relmap-write-functions.patch (text/x-patch) patch v14-0001
v14-0004-New-interface-to-lock-relation-id.patch (text/x-patch) patch v14-0004
v14-0003-Allow-ReadBufferWithoutRelcache-to-support-unlog.patch (text/x-patch) patch v14-0003
v14-0005-WAL-logged-CREATE-DATABASE.patch (text/x-patch) patch v14-0005
v14-0006-Support-create-database-strategy-in-createdb-too.patch (text/x-patch) patch v14-0006

On Sat, Mar 12, 2022 at 11:06 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> In fact while we are rewriting the relation during vacuum full that
> time also we are calling log_newpage() under RelationCopyStorage() and
> during standby if it gets promoted we will be having some buffers in
> the buffer pool with the new relfilenode.  So I think our case is also
> the same.
>
> So here my stand is that we need to drop database buffers and remove
> pending sync requests because we are deleting underlying files and if
> we do not do that in some extreme cases then there is no need to drop
> the buffers or remove the pending sync request and the worst
> consequences would be waste of disk space.

So other than this open point I have fixed other comments given by you
which includes,

- Avoid copying temp relfilenode
- Rename of functions CopyDatabase* -> CreateDatabaseUsing*
- GetDatabaseRelationList and friends to ScanSourceDatabasePgClass*
- Removed unused structure and macro because we are using the same WAL
for copying the database using the old method or creating the
directory and version files for the new method.  Do you think we
should introduce a new WAL for that instead of using the same?

Other than that, I have fixed some mistakes in comments and supported
tab completion for the new options.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

When using the WAL-logged CREATE DATABASE strategy, bulk extend.
- 3e63e8462f31 16.0 landed
- 576bb0fc9342 15.0 landed
Avoid using a fake relcache entry to own an SmgrRelation.
- 1b94f8f232f6 15.0 landed
- 76733b399c49 16.0 landed
Fix data-corruption hazard in WAL-logged CREATE DATABASE.
- 692df425b688 16.0 landed
- 811203d4aff5 15.0 landed
initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.
- ad43a413c4f7 15.0 landed
Simplify a needlessly-complicated regular expression.
- c6863b858291 15.0 landed
In 020_createdb.pl, change order of command-line arguments.
- 3d067c53b26d 15.0 landed
Add new block-by-block strategy for CREATE DATABASE.
- 9c08aea6a309 15.0 landed
Fix replay of create database records on standby
- 49d9cfc68bf4 15.0 cited
Refactor code for reading and writing relation map files.
- 39f0c4bd670c 15.0 landed
Replace RelationOpenSmgr() with RelationGetSmgr().
- f10f0ae420ee 15.0 cited
Refactor the fsync queue for wider use.
- 3eb77eba5a51 12.0 cited