Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Dilip Kumar <dilipbalaut@gmail.com>

View thread View on postgr.es
From: Dilip Kumar <dilipbalaut@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Maciek Sakrejda <m.sakrejda@gmail.com>, Bruce Momjian <bruce@momjian.us>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andrew Dunstan <andrew@dunslane.net>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-18T04:27:09Z
Lists: pgsql-hackers 
On Fri, Feb 18, 2022 at 4:30 AM Robert Haas <robertmhaas@gmail.com> wrote:
>
>
> > This thread is long. Could you summarize what lead you to consider other
> > approaches (e.g. looking in the filesystem for relfilenodes) as not feasible /
> > too ugly / ...?
>
> I don't think it's infeasible to look at the filesystem for files and
> just copy whatever files we find. It's a plausible alternate design. I
> just don't like it as well. I think that relying on the filesystem
> contents to tell us what's going on is kind of hacky. The only
> technical issue I see there is that the WAL logging might require more
> kludgery, since that mechanism is kind of intertwined with
> shared_buffers. You'd have to get the right block references into the
> WAL record, and you have to make sure that checkpoints don't move the
> redo pointer at an inopportune moment.


Actually based on the previous discussion, I also tried to write the
POC with the file system scanning approach to identify the relation to
be copied seet patch 0007 in this thread [1].  And later we identified
one issue [2], i.e. while scanning directly the disk file we will only
know the relfilenode but we can not identify the relation oid that
means we can not lock the relation.  Now, I am not saying that there
is no way to work around that issue but that was also one of the
reasons for not pursuing that approach.

[1] https://www.postgresql.org/message-id/CAFiTN-v1KYsVAhq_fOWFa27LZiw9uK4n4cz5XmQJxJpsVcfq1w%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAFiTN-v%3DU58by_BeiZruNhykxk1q9XUxF%2BqLzD2LZAsEn2EBkg%40mail.gmail.com

-- 
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.