Horton

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

Ashutosh Sharma <ashu.coek88@gmail.com>

View thread View on postgr.es
From: Ashutosh Sharma <ashu.coek88@gmail.com>
To: Dilip Kumar <dilipbalaut@gmail.com>
Cc: Neha Sharma <neha.sharma@enterprisedb.com>, Bruce Momjian <bruce@momjian.us>, Greg Nancarrow <gregn4422@gmail.com>, Robert Haas <robertmhaas@gmail.com>, Andres Freund <andres@anarazel.de>, Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-12-22T14:32:38Z
Lists: pgsql-hackers 
On Wed, Dec 22, 2021 at 5:06 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
>
> On Wed, Dec 22, 2021 at 4:26 PM Ashutosh Sharma <ashu.coek88@gmail.com> wrote:
>
> >> Basically, ALTER TABLE SET TABLESPACE, will register the
> >> SYNC_UNLINK_REQUEST for the table files w.r.t the old tablespace, but
> >> those will get unlinked during the next checkpoint.  Although the
> >> files must be truncated during commit itself but unlink might not have
> >> been processed until the next checkpoint.  This is the explanation for
> >> the behavior you found during your investigation, but I haven't looked
> >> into the issue so I will do it latest by tomorrow and send my
> >> analysis.
> >>
> >> Thanks for working on this.
> >
> >
> > Yeah the problem here is that the old rel file that needs to be unlinked still exists in the old tablespace. Earlier, without your changes we were doing force checkpoint before starting with the actual work for the alter database which unlinked/deleted the rel file from the old tablespace, but that is not the case here.  Now we have removed the force checkpoint from movedb() which means until the auto checkpoint happens the old rel file will remain in the old tablespace thereby creating this problem.
>
> One solution to this problem could be that, similar to mdpostckpt(),
> we invent one more function which takes dboid and dsttblspc oid as
> input and it will unlink all the requests which are w.r.t. the dboid
> and tablespaceoid, and before doing it we should also do
> ForgetDatabaseSyncRequests(), so that next checkpoint does not flush
> some old request.

I couldn't find the mdpostchkpt() function. Are you talking about
SyncPostCheckpoint() ? Anyway, as you have rightly said, we need to
unlink all the files available inside the dst_tablespaceoid/dst_dboid/
directory by scanning the pendingUnlinks list. And finally we don't
want the next checkpoint to unlink this file again and PANIC so for
that we have to update the entry for this unlinked rel file in the
hash table i.e. cancel the sync request for it.

--
With Regards,
Ashutosh Sharma.

Commits

  1. When using the WAL-logged CREATE DATABASE strategy, bulk extend.

  2. Avoid using a fake relcache entry to own an SmgrRelation.

  3. Fix data-corruption hazard in WAL-logged CREATE DATABASE.

  4. initdb: When running CREATE DATABASE, use STRATEGY = WAL_COPY.

  5. Simplify a needlessly-complicated regular expression.

  6. In 020_createdb.pl, change order of command-line arguments.

  7. Add new block-by-block strategy for CREATE DATABASE.

  8. Fix replay of create database records on standby

  9. Refactor code for reading and writing relation map files.

  10. Replace RelationOpenSmgr() with RelationGetSmgr().

  11. Refactor the fsync queue for wider use.