Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

From: Dilip Kumar <dilipbalaut@gmail.com>

To: Ashutosh Sharma <ashu.coek88@gmail.com>

Cc: Neha Sharma <neha.sharma@enterprisedb.com>, Greg Nancarrow <gregn4422@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2021-12-12T08:09:37Z

Lists: pgsql-hackers

Attachments

v8-0001-Refactor-relmap-load-and-relmap-write-functions.patch (text/x-patch) patch v8-0001
v8-0002-Extend-relmap-interfaces.patch (text/x-patch) patch v8-0002
v8-0005-New-interface-to-lock-relation-id.patch (text/x-patch) patch v8-0005
v8-0004-Extend-bufmgr-interfaces.patch (text/x-patch) patch v8-0004
v8-0003-Refactor-index_copy_data.patch (text/x-patch) patch v8-0003
v8-0007-WAL-logged-CREATE-DATABASE.patch (text/x-patch) patch v8-0007
v8-0006-Extend-ForgetDatabaseSyncRequests-interface.patch (text/x-patch) patch v8-0006

On Fri, Dec 10, 2021 at 7:39 AM Ashutosh Sharma <ashu.coek88@gmail.com> wrote:
>>
>> Logfile Snippet:
>> 2021-12-09 17:49:18.110 +04 [18151] PANIC:  could not fsync file "base/116398/116400": No such file or directory
>> 2021-12-09 17:49:19.105 +04 [18150] LOG:  checkpointer process (PID 18151) was terminated by signal 6: Aborted
>> 2021-12-09 17:49:19.105 +04 [18150] LOG:  terminating any other active server processes
>
>
> This is different from the issue you raised earlier. As Dilip said, we need to unregister sync requests for files that got successfully copied to the target database, but the overall alter database statement failed. We are doing this when the database is created successfully, but not when it fails.
> Probably doing the same inside the cleanup function movedb_failure_callback() should fix the problem.

Correct, I have done this cleanup, apart from this we have dropped the
fsyc request in create database failure case as well and also need to
drop buffer in error case of creatdb as well as movedb.  I have also
fixed the other issue for which you gave the patch (a bit differently)
basically, in case of movedb the source and destination dboid are same
so we don't need an additional parameter and also readjusted the
conditions to avoid nested if.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

When using the WAL-logged CREATE DATABASE strategy, bulk extend.
- 3e63e8462f31 16.0 landed
- 576bb0fc9342 15.0 landed
Avoid using a fake relcache entry to own an SmgrRelation.
- 1b94f8f232f6 15.0 landed
- 76733b399c49 16.0 landed
Fix data-corruption hazard in WAL-logged CREATE DATABASE.
- 692df425b688 16.0 landed
- 811203d4aff5 15.0 landed
initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.
- ad43a413c4f7 15.0 landed
Simplify a needlessly-complicated regular expression.
- c6863b858291 15.0 landed
In 020_createdb.pl, change order of command-line arguments.
- 3d067c53b26d 15.0 landed
Add new block-by-block strategy for CREATE DATABASE.
- 9c08aea6a309 15.0 landed
Fix replay of create database records on standby
- 49d9cfc68bf4 15.0 cited
Refactor code for reading and writing relation map files.
- 39f0c4bd670c 15.0 landed
Replace RelationOpenSmgr() with RelationGetSmgr().
- f10f0ae420ee 15.0 cited
Refactor the fsync queue for wider use.
- 3eb77eba5a51 12.0 cited