Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Andres Freund <andres@anarazel.de>, Justin Pryzby <pryzby@telsasoft.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-08-07T03:54:40Z
Lists: pgsql-hackers 
On Sat, Aug 6, 2022 at 9:36 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Dilip Kumar <dilipbalaut@gmail.com> writes:
> > On Fri, Aug 5, 2022 at 10:43 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
> >> Yeah maybe it is not necessary to close as these unowned smgr will
> >> automatically get closed on the transaction end.
>
> I do not think this is a great idea for the per-relation smgrs created
> during RelationCopyStorageUsingBuffer.  Yeah, they'll be mopped up at
> transaction end, but that doesn't mean that creating possibly tens of
> thousands of transient smgrs isn't going to cause performance issues.

Okay, so for that we can simply call smgrcloserellocator(rlocator);
before exiting the RelationCopyStorageUsingBuffer() right?

> I think RelationCopyStorageUsingBuffer needs to open and then close
> the smgrs it uses, which means that ReadBufferWithoutRelcache is not the
> appropriate API for it to use, either; need to go down another level.

Not sure how going down another level would help, the whole point is
that we don't want to keep the reference of the smgr for a long time
especially in the loop which is interruptible.  So everytime we need
smgr we can call smgropen and if it is already in the smgr cache then
we will get it from there.  So I think it makes sense that when we are
exiting the function that time we can just call smgrcloserellocator()
so that if it is opened it will be closed and otherwise it will do
nothing.

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.