Horton

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

Paul Guo <pguo@pivotal.io>

View thread View on postgr.es
From: Paul Guo <pguo@pivotal.io>
To: Asim R P <apraveen@pivotal.io>
Cc: PostgreSQL mailing lists <pgsql-hackers@postgresql.org>
Date: 2019-04-22T04:36:43Z
Lists: pgsql-hackers 
Please see my replies inline. Thanks.

On Fri, Apr 19, 2019 at 12:38 PM Asim R P <apraveen@pivotal.io> wrote:

> On Wed, Apr 17, 2019 at 1:27 PM Paul Guo <pguo@pivotal.io> wrote:
> >
> > create db with tablespace
> > drop database
> > drop tablespace.
>
> Essentially, that sequence of operations causes crash recovery to fail
> if the "drop tablespace" transaction was committed before crashing.
> This is a bug in crash recovery in general and should be reproducible
> without configuring a standby.  Is that right?
>

No. In general, checkpoint is done for drop_db/create_db/drop_tablespace on
master.
That makes the file/directory update-to-date if I understand the related
code correctly.
For standby, checkpoint redo does not ensure that.


>
> Your patch creates missing directories in the destination.  Don't we
> need to create the tablespace symlink under pg_tblspc/?  I would
>

 'create db with tablespace' redo log does not include the tablespace real
directory information.
Yes, we could add in it into the xlog, but that seems to be an overdesign.


> prefer extending the invalid page mechanism to deal with this, as
> suggested by Ashwin off-list.  It will allow us to avoid creating

directories and files only to remove them shortly afterwards when the
> drop database and drop tablespace records are replayed.
>
>
'invalid page' mechanism seems to be more proper for missing pages of a
file. For
missing directories, we could, of course, hack to use that (e.g. reading
any page of
a relfile in that database) to make sure the tablespace create code
(without symlink)
safer (It assumes those directories will be deleted soon).

More feedback about all of the previous discussed solutions is welcome.

Commits

  1. Improve recently-added test reliability

  2. Fix new recovery test for log_error_verbosity=verbose case

  3. Fix test instability

  4. Fix replay of create database records on standby

  5. Allow "in place" tablespaces.

  6. Fix get_dirent_type() for Windows junction points.

  7. Revert "Fix replay of create database records on standby"

  8. Add end-to-end testing of pg_basebackup's tar-format output.

  9. Make DROP DATABASE command generate less WAL records.

  10. Consolidate methods for translating a Perl path to a Windows path.