Horton

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

Kyotaro Horiguchi <horikyota.ntt@gmail.com>

View thread View on postgr.es
From: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
To: dilipbalaut@gmail.com
Cc: robertmhaas@gmail.com, alvherre@alvh.no-ip.org, michael@paquier.xyz, rjuju123@gmail.com, pgsql-hackers@postgresql.org
Date: 2022-04-05T02:16:44Z
Lists: pgsql-hackers 
At Mon, 4 Apr 2022 21:14:27 +0530, Dilip Kumar <dilipbalaut@gmail.com> wrote in 
> On Mon, Apr 4, 2022 at 2:25 PM Kyotaro Horiguchi
> <horikyota.ntt@gmail.com> wrote:
> >
> > At Mon, 04 Apr 2022 17:29:48 +0900 (JST), Kyotaro Horiguchi <horikyota.ntt@gmail.com> wrote in
> > > I haven't found how the patch caused creation of a relation file that
> > > is to be removed soon.  However, I find that v19 patch fails by maybe
> > > due to some change in Cluster.pm.  It takes a bit more time to check
> > > that..
> >
> > I was a bit away, of course the wal-logged create database interfares
> > with the patch here. But I haven't found that why it stops creating
> > database directory under pg_tblspc.
> 
> I did not understand what is the exact problem here, but the database
> directory and the version file are created under the default
> tablespace of the target database.  However, other than the default
> tablespace of the database, the database directory will be created
> along with the smgrcreate() so that we do not create an unnecessary
> directory under the tablespace where we do not have any data to be
> copied.

Thanks. Yeah, I suspected something like that but I didn't find a
difference in the code I suspected to be related with, but it's was
wrong.  I took wrong steps trying to reveal that state and faced the
wrong error message. With the correct steps, I could see that
Storage/CREATE creates pg_tblspc/<directory>.

So, if we create missing tablespace directory, we have no way
otherthan creating it directly in pg_tblspc, which is violating the
rule that there shouldn't be real directory in pg_tblspc (when
allow_in_place_tablespaces is false).

So, I have the following points in my mind for now.

- We create the directory "since we know it is just tentative state".

- Then, check that no directory in pg_tblspc when reaching consistency
  when allow_in_place_tablespaces is false.

- Leave the log_invalid_page() mechanism alone as it is always result
  in a corrpt page if a differential WAL record is applied on a newly
  created page that should have been exist.

However, while working on it, I found that I found that recovery faces
missing tablespace directories *after* reaching consistency.  I'm
examining that further.

regards.

-- 
Kyotaro Horiguchi
NTT Open Source Software Center

Commits

  1. Improve recently-added test reliability

  2. Fix new recovery test for log_error_verbosity=verbose case

  3. Fix test instability

  4. Fix replay of create database records on standby

  5. Allow "in place" tablespaces.

  6. Fix get_dirent_type() for Windows junction points.

  7. Revert "Fix replay of create database records on standby"

  8. Add end-to-end testing of pg_basebackup's tar-format output.

  9. Make DROP DATABASE command generate less WAL records.

  10. Consolidate methods for translating a Perl path to a Windows path.