Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Dilip Kumar <dilipbalaut@gmail.com>, Justin Pryzby <pryzby@telsasoft.com>, Robert Haas <robertmhaas@gmail.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, pgsql-hackers@lists.postgresql.org
Date: 2022-08-05T20:41:47Z
Lists: pgsql-hackers 
Hi,

On 2022-08-04 19:14:08 -0400, Tom Lane wrote:
> Andres Freund <andres@anarazel.de> writes:
> > On 2022-08-04 18:05:25 -0400, Tom Lane wrote:
> >> In any case, DROP DATABASE is far from the only place with a problem.
> 
> > What other place has a database corrupting potential of this magnitude just
> > because interrupts are accepted?  We throw valid s_b contents away and then
> > accept interrupts before committing - with predictable results. We also accept
> > interrupts as part of deleting the db data dir (due to catalog access).
> 
> Those things would be better handled by moving the data-discarding
> steps to post-commit.  Maybe that argues for having an internal
> commit halfway through DROP DATABASE: remove pg_database row,
> commit, start new transaction, clean up.

That'd still require holding interrupts, I think. We shouldn't accept
interrupts until the on-disk data is actually deleted.


In theory I think we should have a pg_database column indicating whether the
database is valid or not. For database creation, insert the pg_database row
with valid=false, commit, then do the filesystem operation, then mark as
valid, commit. For database drop, mark as invalid, commit, remove filesystem
stuff, delete row, commit.  With dropdb allowed against an invalid database,
but obviously nothing else.  But clearly this isn't a short term /
backpatchable thing.


Greetings,

Andres Freund

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.