Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, Dilip Kumar <dilipbalaut@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-06-17T18:17:15Z
Lists: pgsql-hackers 
Hi,

On 2021-06-17 13:53:38 -0400, Robert Haas wrote:
> On Thu, Jun 17, 2021 at 5:20 AM Heikki Linnakangas <hlinnaka@iki.fi> wrote:
> > You only need relpersistence if you want to use the buffer cache, right?
> > I think that's a good argument for not using it.

Do we really need pg_class to figure this out? Can't we just check if
the relation has an init fork?


> I can also think of at least one significant advantage of driving this
> off the remote database's pg_class rather than the filesystem
> contents. It's a known defect of PostgreSQL that if you create a table
> and then crash, you leave behind a dead file that never gets removed.
> If you now copy the database that contains that orphaned file, you
> would ideally prefer not to copy that file, but if you do a copy based
> on the filesystem contents, then you will. If you drive the copy off
> of pg_class, you won't.

I'm very unconvinced this is the place to tackle the issue of orphan
relfilenodes. It'd be one thing if it were doable by existing code,
e.g. because we supported cross-database relation accesses fully, but we
don't.

Adding a hacky special case implementation for cross-database relation
accesses that violates all kinds of assumptions (like holding a lock on
a relation when accessing it / pinning pages, processing relcache
invals, ...) doesn't seem like a good plan.

I don't think this is an academic concern: You need to read from shared
buffers to read the "remote" pg_class, otherwise you'll potentially miss
changes. But it's not correct to read in pages or to pin pages without
holding a lock, and there's code that relies on that (see
e.g. InvalidateBuffer()).

Greetings,

Andres Freund

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.