Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-03-18T04:39:39Z
Lists: pgsql-hackers 
On Fri, Mar 18, 2022 at 1:44 AM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Wed, Mar 16, 2022 at 12:53 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > Thanks Ashutosh and Robert.  Other pacthes cleanly applied on this
> > patch still generated a new version so that we can find all patches
> > together.  There are no other changes.
>
> I committed my v3 of my refactoring patch, here 0001.
>
> I'm working over the comments in the rest of the patch series and will
> post an updated version when I get done. I think I will likely merge
> all the remaining patches together just to make it simpler to manage;
> we can split things out again if we need to do that.

Thanks for the effort.

> One question that occurred to me when looking this over is whether, or
> why, it's safe against concurrent smgr invalidations.

We are only accessing the smgr of the source database and the
destination database.  And there is no one else that can be connected
to the source db and the destination db is not visible to anyone.  So
do we really need to worry about the concurrent smgr invalidation?
What am I missing?

It seems to me
> that every loop in the new CREATE DATABASE code needs to
> CHECK_FOR_INTERRUPTS() -- some do already -- and when they do that,

Yes, the pg_class reading code is missing this check so we need to put
it.  But copying code like
CreateDatabaseUsingWalLog() have it inside the deepest loop in
RelationCopyStorageUsingBuffer() and similarly
CreateDatabaseUsingFileCopy() have it in copydir().  Maybe we should
put it in all loop so that we do not skip checking due to some
condition.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.