Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Justin Pryzby <pryzby@telsasoft.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-08-07T04:17:09Z
Lists: pgsql-hackers 
Hi,

On 2022-08-07 09:24:40 +0530, Dilip Kumar wrote:
> On Sat, Aug 6, 2022 at 9:36 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Dilip Kumar <dilipbalaut@gmail.com> writes:
> > > On Fri, Aug 5, 2022 at 10:43 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> > >> Yeah maybe it is not necessary to close as these unowned smgr will
> > >> automatically get closed on the transaction end.
> >
> > I do not think this is a great idea for the per-relation smgrs created
> > during RelationCopyStorageUsingBuffer.  Yeah, they'll be mopped up at
> > transaction end, but that doesn't mean that creating possibly tens of
> > thousands of transient smgrs isn't going to cause performance issues.

I was assuming that the files would get reopened at the end of the transaction
anyway, but it looks like that's not the case, unless wal_level=minimal.

Hm. CreateAndCopyRelationData() calls RelationCreateStorage() with
register_delete = false, which is ok because createdb_failure_callback will
clean things up. But that's another thing that's not great for a routine with
a general name...


> Okay, so for that we can simply call smgrcloserellocator(rlocator);
> before exiting the RelationCopyStorageUsingBuffer() right?

Yea, I think so.


> > I think RelationCopyStorageUsingBuffer needs to open and then close
> > the smgrs it uses, which means that ReadBufferWithoutRelcache is not the
> > appropriate API for it to use, either; need to go down another level.
> 
> Not sure how going down another level would help, the whole point is
> that we don't want to keep the reference of the smgr for a long time
> especially in the loop which is interruptible.

Yea, I'm not following either.


Greetings,

Andres Freund

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.