Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

From: Dilip Kumar <dilipbalaut@gmail.com>

To: Greg Nancarrow <gregn4422@gmail.com>

Cc: Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2021-12-02T13:49:50Z

Lists: pgsql-hackers

Attachments

v7-0003-Refactor-index_copy_data.patch (text/x-patch) patch v7-0003
v7-0004-Extend-bufmgr-interfaces.patch (text/x-patch) patch v7-0004
v7-0001-Refactor-relmap-load-and-relmap-write-functions.patch (text/x-patch) patch v7-0001
v7-0002-Extend-relmap-interfaces.patch (text/x-patch) patch v7-0002
v7-0005-New-interface-to-lock-relation-id.patch (text/x-patch) patch v7-0005
v7-0006-Extend-ForgetDatabaseSyncRequests-interface.patch (text/x-patch) patch v7-0006
v7-0007-WAL-logged-CREATE-DATABASE.patch (text/x-patch) patch v7-0007

On Wed, Dec 1, 2021 at 6:04 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:

> Thanks a lot for testing this. From the error, it seems like some of
> the old buffer w.r.t. the previous tablespace is not dropped after the
> movedb.  Actually, we are calling DropDatabaseBuffers() after copying
> to a new tablespace and dropping all the buffers of this database
> w.r.t the old tablespace.  But seems something is missing, I will
> reproduce this and try to fix it by tomorrow.  I will also fix the
> other review comments raised by you in the previous mail.

Okay, I got the issue, basically we are dropping the database buffers
but not unregistering the existing sync request for database buffers
w.r.t old tablespace. Attached patch fixes that.  I also had to extend
ForgetDatabaseSyncRequests so that we can delete the sync request of
the database for the particular tablespace so added another patch for
the same (0006).

I will test the performance scenario next week, which is suggested by John.

--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

When using the WAL-logged CREATE DATABASE strategy, bulk extend.
- 3e63e8462f31 16.0 landed
- 576bb0fc9342 15.0 landed
Avoid using a fake relcache entry to own an SmgrRelation.
- 1b94f8f232f6 15.0 landed
- 76733b399c49 16.0 landed
Fix data-corruption hazard in WAL-logged CREATE DATABASE.
- 692df425b688 16.0 landed
- 811203d4aff5 15.0 landed
initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.
- ad43a413c4f7 15.0 landed
Simplify a needlessly-complicated regular expression.
- c6863b858291 15.0 landed
In 020_createdb.pl, change order of command-line arguments.
- 3d067c53b26d 15.0 landed
Add new block-by-block strategy for CREATE DATABASE.
- 9c08aea6a309 15.0 landed
Fix replay of create database records on standby
- 49d9cfc68bf4 15.0 cited
Refactor code for reading and writing relation map files.
- 39f0c4bd670c 15.0 landed
Replace RelationOpenSmgr() with RelationGetSmgr().
- f10f0ae420ee 15.0 cited
Refactor the fsync queue for wider use.
- 3eb77eba5a51 12.0 cited