Horton

Thread

Commits

  1. Fix pg_rewind with in-place tablespaces when source is remote

  1. pg_rewind fails with in-place tablespace

    Rui Zhao <xiyuan.zr@alibaba-inc.com> — 2023-07-19T13:31:35Z

    Hello postgres hackers,
 Recently I encountered an issue: pg_rewind fails when dealing with in-place tablespace. The problem seems to be that pg_rewind is treating in-place tablespace as symbolic link, while in fact it should be treated as directory.
 Here is the output of pg_rewind:
pg_rewind: error: file "pg_tblspc/16385" is of different type in source and target
 To help reproduce the failure, I have attached a tap test. And I am pleased to say that I have also identified a solution for this problem, which I have included in the patch.
 Thank you for your attention to this matter.
Best regards,
Rui Zhao

  2. Re: pg_rewind fails with in-place tablespace

    Michael Paquier <michael@paquier.xyz> — 2023-07-25T07:36:42Z

    On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
>  Recently I encountered an issue: pg_rewind fails when dealing with
>  in-place tablespace. The problem seems to be that pg_rewind is
>  treating in-place tablespace as symbolic link, while in fact it
>  should be treated as directory. 
>  Here is the output of pg_rewind:
> pg_rewind: error: file "pg_tblspc/16385" is of different type in
> source and target 
>  To help reproduce the failure, I have attached a tap test. And I am
>  pleased to say that I have also identified a solution for this
>  problem, which I have included in the patch. 
>  Thank you for your attention to this matter.

Issue reproduced here, and agreed that we'd better do something about
that.  I am not sure if your patch is right for the job though, but
I'll try to study that a bit more.
--
Michael

  3. Re: pg_rewind fails with in-place tablespace

    Michael Paquier <michael@paquier.xyz> — 2023-07-28T07:54:56Z

    On Tue, Jul 25, 2023 at 04:36:42PM +0900, Michael Paquier wrote:
> On Wed, Jul 19, 2023 at 09:31:35PM +0800, 赵锐(惜元) wrote:
>>  To help reproduce the failure, I have attached a tap test. And I am
>>  pleased to say that I have also identified a solution for this
>>  problem, which I have included in the patch. 
>>  Thank you for your attention to this matter.
> 
> Issue reproduced here, and agreed that we'd better do something about
> that.  I am not sure if your patch is right for the job though, but
> I'll try to study that a bit more.

It took me some time to remember that for the case of a local source
we'd finish by using recurse_dir() and consider the in-place
tablespace as a regular directory, so a fix located in
libpq_traverse_files() sounds good to me.

+           if (strncmp(link_target, "pg_tblspc/", strlen("pg_tblspc/")) == 0)
+               type = FILE_TYPE_DIRECTORY;
+           else
+               type = FILE_TYPE_SYMLINK;

However this is not consistent with the other places where we detect
if an in-place tablespace is used, like pg_basebackup.c, where we rely
on the fact that the tablespace path is a relative path, using
is_absolute_path() to make the difference between a normal and
in-place tablespace.  I would choose consistency and do the same here,
checking if we have an absolute or relative path, depending on the
result of pg_tablespace_location().

Testing only for the creation of the tablespace is fine for the sake
of the report, but I would slightly more here and create a table on
this tablespace with some data, and a check_query() once pg_rewind is
done.

I am finishing with the attached.  Thoughts?
--
Michael

  4. Re: pg_rewind fails with in-place tablespace

    Michael Paquier <michael@paquier.xyz> — 2023-07-30T22:48:58Z

    On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached.  Thoughts?

Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers.  If there are opinions in favor
of a backpatch, feel free of course.
--
Michael

  5. 回复：pg_rewind fails with in-place tablespace

    Rui Zhao <xiyuan.zr@alibaba-inc.com> — 2023-07-31T02:07:44Z

    Sorry for the delay in responding to this matter as I have been waiting for another similar subject to approved by a moderator.
Upon review, I am satisfied with the proposed solution and believe that checking absolute path is better than hard coding with "pg_tblspc/". I think we have successfully resolved this issue in the pg_rewind case.
However, I would like to bring your attention to another issue: pg_upgrade fails with in-place tablespace. Another issue is still waiting for approved. I have tested all the tools in src/bin with in-place tablespace, and I believe this is the final issue.
Thank you for your understanding and assistance.
Best regard,
Rui Zhao
------------------------------------------------------------------
发件人：Michael Paquier <michael@paquier.xyz>
发送时间：2023年7月31日(星期一) 06:49
收件人：赵锐(惜元) <xiyuan.zr@alibaba-inc.com>
抄　送：pgsql-hackers <pgsql-hackers@lists.postgresql.org>; Thomas Munro <thomas.munro@gmail.com>
主　题：Re: pg_rewind fails with in-place tablespace
On Fri, Jul 28, 2023 at 04:54:56PM +0900, Michael Paquier wrote:
> I am finishing with the attached. Thoughts?
Applied this one as bf22792 on HEAD, without a backpatch as in-place
tablespaces are around for developers. If there are opinions in favor
of a backpatch, feel free of course.
--
Michael

  6. Re: 回复：pg_rewind fails with in-place tablespace

    Michael Paquier <michael@paquier.xyz> — 2023-07-31T02:14:13Z

    On Mon, Jul 31, 2023 at 10:07:44AM +0800, Rui Zhao wrote:
> However, I would like to bring your attention to another issue:
> pg_upgrade fails with in-place tablespace. Another issue is still
> waiting for approved. I have tested all the tools in src/bin with
> in-place tablespace, and I believe this is the final issue. 

No problem.  Please feel free to start a new thread about that, I'm
okay to look at what you would like to propose.  Adding a test in
002_pg_upgrade.pl where the pg_upgrade runs happen would be a good
thing to have, I guess.
--
Michael