Horton

Thread

Commits

  1. pg_basebackup: Fix cross-platform tablespace relocation.

  1. cross-platform pg_basebackup

    Robert Haas <robertmhaas@gmail.com> — 2022-10-20T15:11:17Z

    Suppose that, for some reason, you want to use pg_basebackup on a
Linux machine to back up a database cluster on a Windows machine.
Suppose further that you attempt to use the -T option. Then you might
run afoul of this check:

    /*
     * This check isn't absolutely necessary.  But all tablespaces are created
     * with absolute directories, so specifying a non-absolute path here would
     * just never match, possibly confusing users.  It's also good to be
     * consistent with the new_dir check.
     */
    if (!is_absolute_path(cell->old_dir))
        pg_fatal("old directory is not an absolute path in tablespace
mapping: %s",
                 cell->old_dir);

The problem is that the definition of is_absolute_path() here differs
depending on whether you are on Windows or not. So this code is, I
think, subtly incorrect. What it is testing is whether the
user-specified pathname is an absolute pathname *on the local machine*
whereas what it should be testing is whether the user-specified
pathname is an absolute pathname *on the remote machine*. There's no
problem if both sides are Windows or neither side is Windows, but if
the remote side is and the local side isn't, then something like
-TC:\foo=/backup/foo will fail. As far as I know, there's no reason
why that shouldn't be permitted to work.

What this check is actually intending to prevent, I believe, is
something like -T../mytablespace=/bkp/ts1, because that wouldn't
actually work: the value in the list will be an absolute path. The
tablespace wouldn't get remapped, and the user might be confused about
why it didn't, so it is good that we tell them what they did wrong.
However, I think we could relax the check a little bit, something
along the lines of !is_nonwindows_absolute_path(cell->old_dir) &&
!is_windows_absolute_path(dir). We can't actually know whether the
remote side is Windows or non-Windows, but if the string we're given
is plausibly an absolute path under either set of conventions, it's
probably fine to just search the list for it and see if it shows up.

This would have the disadvantage that if a Linux user creates a
tablespace directory inside $PGDATA and gives it a name like
/home/rhaas/pgdata/C:\Program Files\PostgreSQL\Data, and then attempts
a backup with '-TC:\Program Files\PostgreSQL\Data=/tmp/ts1' it will
not relocate the tablespace, yet the user won't get a message
explaining why. I'm prepared to dismiss that scenario as "not a real
use case".

Thoughts?

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  2. Re: cross-platform pg_basebackup

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-10-20T16:17:12Z

    Robert Haas <robertmhaas@gmail.com> writes:
> However, I think we could relax the check a little bit, something
> along the lines of !is_nonwindows_absolute_path(cell->old_dir) &&
> !is_windows_absolute_path(dir). We can't actually know whether the
> remote side is Windows or non-Windows, but if the string we're given
> is plausibly an absolute path under either set of conventions, it's
> probably fine to just search the list for it and see if it shows up.

Seems reasonable.

> This would have the disadvantage that if a Linux user creates a
> tablespace directory inside $PGDATA and gives it a name like
> /home/rhaas/pgdata/C:\Program Files\PostgreSQL\Data, and then attempts
> a backup with '-TC:\Program Files\PostgreSQL\Data=/tmp/ts1' it will
> not relocate the tablespace, yet the user won't get a message
> explaining why. I'm prepared to dismiss that scenario as "not a real
> use case".

Agreed.

			regards, tom lane

  3. Re: cross-platform pg_basebackup

    Robert Haas <robertmhaas@gmail.com> — 2022-10-20T17:04:48Z

    On Thu, Oct 20, 2022 at 12:17 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > However, I think we could relax the check a little bit, something
> > along the lines of !is_nonwindows_absolute_path(cell->old_dir) &&
> > !is_windows_absolute_path(dir). We can't actually know whether the
> > remote side is Windows or non-Windows, but if the string we're given
> > is plausibly an absolute path under either set of conventions, it's
> > probably fine to just search the list for it and see if it shows up.
>
> Seems reasonable.

Cool. Here's a patch.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  4. Re: cross-platform pg_basebackup

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-10-20T17:28:06Z

    Robert Haas <robertmhaas@gmail.com> writes:
> Cool. Here's a patch.

LGTM, except I'd be inclined to ensure that all the macros
are function-style, ie

+#define IS_DIR_SEP(ch) IS_NONWINDOWS_DIR_SEP(ch)

not just

+#define IS_DIR_SEP IS_NONWINDOWS_DIR_SEP

I don't recall the exact rules, but I know that the second style
can lead to expanding the macro in more cases, which we likely
don't want.  It also seems like better documentation to show
the expected arguments.

			regards, tom lane

  5. Re: cross-platform pg_basebackup

    Robert Haas <robertmhaas@gmail.com> — 2022-10-20T18:47:21Z

    On Thu, Oct 20, 2022 at 1:28 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Robert Haas <robertmhaas@gmail.com> writes:
> > Cool. Here's a patch.
>
> LGTM, except I'd be inclined to ensure that all the macros
> are function-style, ie
>
> +#define IS_DIR_SEP(ch) IS_NONWINDOWS_DIR_SEP(ch)
>
> not just
>
> +#define IS_DIR_SEP IS_NONWINDOWS_DIR_SEP
>
> I don't recall the exact rules, but I know that the second style
> can lead to expanding the macro in more cases, which we likely
> don't want.  It also seems like better documentation to show
> the expected arguments.

OK, thanks. v2 attached.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  6. Re: cross-platform pg_basebackup

    Andrew Dunstan <andrew@dunslane.net> — 2022-10-20T19:12:12Z

    On 2022-10-20 Th 14:47, Robert Haas wrote:
> On Thu, Oct 20, 2022 at 1:28 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Robert Haas <robertmhaas@gmail.com> writes:
>>> Cool. Here's a patch.
>> LGTM, except I'd be inclined to ensure that all the macros
>> are function-style, ie
>>
>> +#define IS_DIR_SEP(ch) IS_NONWINDOWS_DIR_SEP(ch)
>>
>> not just
>>
>> +#define IS_DIR_SEP IS_NONWINDOWS_DIR_SEP
>>
>> I don't recall the exact rules, but I know that the second style
>> can lead to expanding the macro in more cases, which we likely
>> don't want.  It also seems like better documentation to show
>> the expected arguments.
> OK, thanks. v2 attached.
>


Looks good.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  7. Re: cross-platform pg_basebackup

    davinder singh <davindersingh2692@gmail.com> — 2022-10-21T08:14:33Z

    Hi,
Patch v2 looks good to me, I have tested it, and pg_basebackup works fine
across the platforms (Windows to Linux and Linux to Windows).
Syntax used for testing
$ pg_basebackup -h remote_server_ip -p 5432 -U user_name -D backup/data -T
olddir=newdir

I have also tested with non-absolute paths, it behaves as expected.

On Fri, Oct 21, 2022 at 12:42 AM Andrew Dunstan <andrew@dunslane.net> wrote:

>
> On 2022-10-20 Th 14:47, Robert Haas wrote:
> > On Thu, Oct 20, 2022 at 1:28 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >> Robert Haas <robertmhaas@gmail.com> writes:
> >>> Cool. Here's a patch.
> >> LGTM, except I'd be inclined to ensure that all the macros
> >> are function-style, ie
> >>
> >> +#define IS_DIR_SEP(ch) IS_NONWINDOWS_DIR_SEP(ch)
> >>
> >> not just
> >>
> >> +#define IS_DIR_SEP IS_NONWINDOWS_DIR_SEP
> >>
> >> I don't recall the exact rules, but I know that the second style
> >> can lead to expanding the macro in more cases, which we likely
> >> don't want.  It also seems like better documentation to show
> >> the expected arguments.
> > OK, thanks. v2 attached.
> >
>
>
> Looks good.
>
>
> cheers
>
>
> andrew
>
> --
> Andrew Dunstan
> EDB: https://www.enterprisedb.com
>
>
>
>

-- 
Regards,
Davinder
EnterpriseDB: http://www.enterprisedb.com

  8. Re: cross-platform pg_basebackup

    Robert Haas <robertmhaas@gmail.com> — 2022-10-21T13:15:39Z

    On Fri, Oct 21, 2022 at 4:14 AM davinder singh
<davindersingh2692@gmail.com> wrote:
> Hi,
> Patch v2 looks good to me, I have tested it, and pg_basebackup works fine across the platforms (Windows to Linux and Linux to Windows).
> Syntax used for testing
> $ pg_basebackup -h remote_server_ip -p 5432 -U user_name -D backup/data -T olddir=newdir
>
> I have also tested with non-absolute paths, it behaves as expected.

Cool. Thanks to you, Andrew, and Tom for reviewing.

Committed and back-patched to all supported branches.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  9. Re: cross-platform pg_basebackup

    Julien Rouhaud <rjuju123@gmail.com> — 2022-10-24T02:44:26Z

    Hi,

On Fri, Oct 21, 2022 at 09:15:39AM -0400, Robert Haas wrote:
>
> Committed and back-patched to all supported branches.

Is there any additional things to be taken care of or should
https://commitfest.postgresql.org/40/3954/ be closed?

  10. Re: cross-platform pg_basebackup

    Robert Haas <robertmhaas@gmail.com> — 2022-10-28T16:07:13Z

    On Sun, Oct 23, 2022 at 10:44 PM Julien Rouhaud <rjuju123@gmail.com> wrote:
> On Fri, Oct 21, 2022 at 09:15:39AM -0400, Robert Haas wrote:
> > Committed and back-patched to all supported branches.
>
> Is there any additional things to be taken care of or should
> https://commitfest.postgresql.org/40/3954/ be closed?

As far as I know we're done. I have closed that entry.

-- 
Robert Haas
EDB: http://www.enterprisedb.com