Horton

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: alvherre@2ndquadrant.com
Cc: michael@paquier.xyz, daniel@yesql.se, guopa@vmware.com, tgl@sss.pgh.pa.us, robertmhaas@gmail.com, ibrar.ahmad@gmail.com, masao.fujii@oss.nttdata.com, a.lubennikova@postgrespro.ru, pasim@vmware.com, thomas.munro@gmail.com, horiguchi.kyotaro@lab.ntt.co.jp, pgsql-hackers@postgresql.org
Date: 2021-11-12T07:43:27Z
Lists: pgsql-hackers

Attachments

At Thu, 11 Nov 2021 11:13:52 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in 
> At Wed, 10 Nov 2021 09:14:30 -0300, Alvaro Herrera <alvherre@2ndquadrant.com> wrote in 
> > Can you use PostgreSQL::Test::Utils::tempdir_short() for those
> > tablespaces?
> 
> Thanks for the suggestion!
> 
> It works for a live cluster. But doesn't work for backups, since I
> find no way to relate a tablespace directory with a backup directory
> not using a symlink.  One way would be taking a backup with tentative
> tablespace directory in the short-named temporary directory then move
> it into the backup direcotry. I'm going that way for now.

This is that.

0001 adds several routines to handle tablespace directories, and adds
tablespace support to backup/_backup_fs.

We don't know an oid corresponding to a tablespace directory before
actually assigning the oid to the tablespace.  So we cannot name a
tablespace directory after the oid.  On the other hand, after defining
the tablespace, cold data files don't tell the real directory name of
the tablespace directory for an oid or a tablespace name, unless we
have readlink.

The function dir_readlink added to Utils.pm is that. Honestly I don't
like the way function works. It uses "cmd /c "dir /A:L $dir"" to
collect information of junctions. I'm not sure that the type label
"<JUNCTION>" is immutable among locales but at least it is shown as
"<JUNCTION>" on Japanese (CP-932) environment. I didn't actually
tested it on Windows and msys environment ...yet.

Premising the availability of the function, we can name tablespace
directories from meaningful words.

The directory to store tablespace directories can be a temporary
directory, but with that way it is needed to create a symlink to find
those directories from a backup.  I chose to place tablespace
directories directly under backup directory.

The attached first file is a revised (or remade) version of tablespace
support for TAP test.

The second is the version adapted to the revised framework. (I
confirmed that the test actually detects the error.)

The third is not changed at all.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Improve recently-added test reliability

  2. Fix new recovery test for log_error_verbosity=verbose case

  3. Fix test instability

  4. Fix replay of create database records on standby

  5. Allow "in place" tablespaces.

  6. Fix get_dirent_type() for Windows junction points.

  7. Revert "Fix replay of create database records on standby"

  8. Add end-to-end testing of pg_basebackup's tar-format output.

  9. Make DROP DATABASE command generate less WAL records.

  10. Consolidate methods for translating a Perl path to a Windows path.