Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints
Dilip Kumar <dilipbalaut@gmail.com>
Attachments
- v6-0005-New-interface-to-lock-relation-id.patch (application/octet-stream) patch v6-0005
- v6-0003-Refactor-index_copy_data.patch (application/octet-stream) patch v6-0003
- v6-0002-Extend-relmap-interfaces.patch (application/octet-stream) patch v6-0002
- v6-0004-Extend-bufmgr-interfaces.patch (application/octet-stream) patch v6-0004
- v6-0001-Refactor-relmap-load-and-relmap-write-functions.patch (application/octet-stream) patch v6-0001
- v6-0006-WAL-logged-CREATE-DATABASE.patch (application/octet-stream) patch v6-0006
On Thu, Nov 25, 2021 at 1:07 PM Greg Nancarrow <gregn4422@gmail.com> wrote:
>
> On Tue, Oct 5, 2021 at 7:07 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >
> > Patch details:
> > 0001 to 0006 implements an approach1
> > 0007 removes the code of pg_class scanning and adds the directory scan.
> >
>
> I had a scan through the patches, though have not yet actually run any
> tests to try to better gauge their benefit.
> I do have some initial review comments though:
>
> 0003
>
> src/backend/commands/tablecmds.c
> (1) RelationCopyAllFork()
> In the following comment:
>
> +/*
> + * Copy source smgr all fork's data to the destination smgr.
> + */
>
> Shouldn't it say "smgr relation"?
> Also, you could additionally say ", using a specified fork data
> copying function." or something like that, to account for the
> additional argument.
>
>
> 0006
>
> src/backend/commands/dbcommands.c
> (1) function prototype location
>
> The following prototype is currently located in the "non-export
> function prototypes" section of the source file, but it's not static -
> shouldn't it be in dbcommands.h?
>
> +void RelationCopyStorageUsingBuffer(SMgrRelation src, SMgrRelation dst,
> + ForkNumber forkNum, char relpersistence);
>
> (2) CreateDirAndVersionFile()
> Shouldn't the following code:
>
> + fd = OpenTransientFile(versionfile, O_RDWR | O_CREAT | O_EXCL | PG_BINARY);
> + if (fd < 0 && errno == EEXIST && isRedo)
> + fd = OpenTransientFile(versionfile, O_RDWR | PG_BINARY);
>
> actually be:
>
> + fd = OpenTransientFile(versionfile, O_WRONLY | O_CREAT | O_EXCL | PG_BINARY);
> + if (fd < 0 && errno == EEXIST && isRedo)
> + fd = OpenTransientFile(versionfile, O_WRONLY | O_TRUNC | PG_BINARY);
>
> since we're only writing to that file descriptor and we want to
> truncate the file if it already exists.
>
> The current comment says "... open it in the write mode.", but should
> say "... open it in write mode."
>
> Also, shouldn't you be writing a newline (\n) after the
> PG_MAJORVERSION ? (compare with code in initdb.c)
>
> (3) GetDatabaseRelationList()
> Shouldn't:
>
> + if (PageIsNew(page) || PageIsEmpty(page))
> + continue;
>
> be:
>
> + if (PageIsNew(page) || PageIsEmpty(page))
> + {
> + UnlockReleaseBuffer(buf);
> + continue;
> + }
>
> ?
>
> Also, in the following code:
>
> + if (rnodelist == NULL)
> + rnodelist = list_make1(relinfo);
> + else
> + rnodelist = lappend(rnodelist, relinfo);
>
> it should really be "== NIL" rather than "== NULL".
> But in any case, that code can just be:
>
> rnodelist = lappend(rnodelist, relinfo);
>
> because lappend() will create a list if the first arg is NIL.
>
> (4) RelationCopyStorageUsingBuffer()
>
> In the function comments, IMO it is better to use "APIs" instead of "apis".
>
> Also, better to use "get" instead of "got" in the following comment:
>
> + /* If we got a cancel signal during the copy of the data, quit */
Thanks for the review and many valuable comments, I have fixed all of
them except this comment (/* If we got a cancel signal during the copy
of the data, quit */) because this looks fine to me. 0007, I have
dropped from the patchset for now. I have also included fixes for
comments given by John.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Commits
-
When using the WAL-logged CREATE DATABASE strategy, bulk extend.
- 3e63e8462f31 16.0 landed
- 576bb0fc9342 15.0 landed
-
Avoid using a fake relcache entry to own an SmgrRelation.
- 1b94f8f232f6 15.0 landed
- 76733b399c49 16.0 landed
-
Fix data-corruption hazard in WAL-logged CREATE DATABASE.
- 692df425b688 16.0 landed
- 811203d4aff5 15.0 landed
-
initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.
- ad43a413c4f7 15.0 landed
-
Simplify a needlessly-complicated regular expression.
- c6863b858291 15.0 landed
-
In 020_createdb.pl, change order of command-line arguments.
- 3d067c53b26d 15.0 landed
-
Add new block-by-block strategy for CREATE DATABASE.
- 9c08aea6a309 15.0 landed
-
Fix replay of create database records on standby
- 49d9cfc68bf4 15.0 cited
-
Refactor code for reading and writing relation map files.
- 39f0c4bd670c 15.0 landed
-
Replace RelationOpenSmgr() with RelationGetSmgr().
- f10f0ae420ee 15.0 cited
-
Refactor the fsync queue for wider use.
- 3eb77eba5a51 12.0 cited