Horton

Thread

Commits

  1. Doc: improve description of dump/restore's --clean and --if-exists.

  1. `pg_restore --if-exists` clarification

    The Post Office <noreply@postgresql.org> — 2023-09-28T12:49:26Z

    The following documentation comment has been logged on the website:

Page: https://www.postgresql.org/docs/16/app-pgrestore.html
Description:

Good Morning,

In the `pg_restore` docs
(https://www.postgresql.org/docs/current/app-pgrestore.html), `--if-exists`
states that it is to

> Use conditional commands (i.e., add an IF EXISTS clause) 
> to drop database objects. This option is not valid unless 
> --clean is also specified.", but not being a SQL expert, 
> I'm having a hard time deciphering this.

Suggestion: Would you consider adding the sentence (or something similar)?

> "--clean makes pg_restore drop all objects first,
> and --if-exists prevents that non-existent objects
> cause a failure." 
Source: https://stackoverflow.com/a/75136163/1498178

This is succinct, and tells exactly what one needs to know. Thank you and
have a great day!

Appreciatively,
Attila

  2. Re: `pg_restore --if-exists` clarification

    Kirk Parker <khp@equatoria.us> — 2023-09-28T14:32:24Z

    On Thu, Sep 28, 2023, 05:52 PG Doc comments form <noreply@postgresql.org>
wrote:

> The following documentation comment has been logged on the website:
>
> Page: https://www.postgresql.org/docs/16/app-pgrestore.html
> Description:
>
> Good Morning,
>
> In the `pg_restore` docs
> (https://www.postgresql.org/docs/current/app-pgrestore.html),
> `--if-exists`
> states that it is to
>
> > Use conditional commands (i.e., add an IF EXISTS clause)
> > to drop database objects. This option is not valid unless
> > --clean is also specified.", but not being a SQL expert,
> > I'm having a hard time deciphering this.
>
> Suggestion: Would you consider adding the sentence (or something similar)?
>
> > "--clean makes pg_restore drop all objects first,
> > and --if-exists prevents that non-existent objects
> > cause a failure."
> Source: https://stackoverflow.com/a/75136163/1498178
>
> This is succinct, and tells exactly what one needs to know. Thank you and
> have a great day!
>
> Appreciatively,
> Attila
>

But "failure" is not what happens.  If you read the part regarding --clean,
you will see that it says:

(Unless --if-exists is used, this might generate some *harmless error
messages*, if any objects were not present in the destination
database.) *[emphasis
added]*
with extra emphasis on the word "harmless" --  no failure is caused; the
restore proceeds just fine.
If anything were to change in this regard, it might be better to reconsider
what we call the message (i.e.what language pg_restore emits in this
scenario.)  It's true, I suppose, in a literal sense that it's an error in
that pg_restore couldn't drop a table when instructed to, where no such
table exists.  But pragmatically it doesn't matter, so why not reclassify
this as a "warning" or a "notice"?
--Kirk

>

  3. Re: `pg_restore --if-exists` clarification

    Gulyás Attila <toraritte@gmail.com> — 2023-09-28T16:12:20Z

    So `--if-exists` simply suppresses any notice / warning that would occur?

I'm sorry if I come off as obtuse, but I have literally no idea what
`--clean` and `--if-exists` do when used together. I took the quote from
the Stackoverflow answer on face value, because it sounded plausible and I
can't understand `--if-exists` current description yet. Any clarification
there would be an improvement in my opinion for people in similar
situations (i.e., having little to no SQL knowledge, but still having to do
basic DB admin tasks).

Appreciatively,
Attila

On Thu, Sep 28, 2023 at 12:01 PM Kirk Parker <khp@equatoria.us> wrote:

>
>
> On Thu, Sep 28, 2023, 05:52 PG Doc comments form <noreply@postgresql.org>
> wrote:
>
>> The following documentation comment has been logged on the website:
>>
>> Page: https://www.postgresql.org/docs/16/app-pgrestore.html
>> Description:
>>
>> Good Morning,
>>
>> In the `pg_restore` docs
>> (https://www.postgresql.org/docs/current/app-pgrestore.html),
>> `--if-exists`
>> states that it is to
>>
>> > Use conditional commands (i.e., add an IF EXISTS clause)
>> > to drop database objects. This option is not valid unless
>> > --clean is also specified.", but not being a SQL expert,
>> > I'm having a hard time deciphering this.
>>
>> Suggestion: Would you consider adding the sentence (or something similar)?
>>
>> > "--clean makes pg_restore drop all objects first,
>> > and --if-exists prevents that non-existent objects
>> > cause a failure."
>> Source: https://stackoverflow.com/a/75136163/1498178
>>
>> This is succinct, and tells exactly what one needs to know. Thank you and
>> have a great day!
>>
>> Appreciatively,
>> Attila
>>
>
> But "failure" is not what happens.  If you read the part regarding
> --clean, you will see that it says:
>
> (Unless --if-exists is used, this might generate some *harmless error
> messages*, if any objects were not present in the destination database.) *[emphasis
> added]*
> with extra emphasis on the word "harmless" --  no failure is caused; the
> restore proceeds just fine.
> If anything were to change in this regard, it might be better to
> reconsider what we call the message (i.e.what language pg_restore emits in
> this scenario.)  It's true, I suppose, in a literal sense that it's an
> error in that pg_restore couldn't drop a table when instructed to, where no
> such table exists.  But pragmatically it doesn't matter, so why not
> reclassify this as a "warning" or a "notice"?
> --Kirk
>
>>

  4. Re: `pg_restore --if-exists` clarification

    David G. Johnston <david.g.johnston@gmail.com> — 2023-09-28T16:36:34Z

    On Thursday, September 28, 2023, Gulyás Attila <toraritte@gmail.com> wrote:

> So `--if-exists` simply suppresses any notice / warning that would occur?
>

That switch causes the drop DDL produced by pg_restore to include if exists
clauses.  The server then simply treats the DDL as a no-op if said object
being dropped cannot be found.


>
>> But "failure" is not what happens.  If you read the part regarding
>> --clean, you will see that it says:
>>
>> (Unless --if-exists is used, this might generate some *harmless error
>> messages*, if any objects were not present in the destination database.) *[emphasis
>> added]*
>> with extra emphasis on the word "harmless" --  no failure is caused; the
>> restore proceeds just fine.
>> If anything were to change in this regard, it might be better to
>> reconsider what we call the message (i.e.what language pg_restore emits in
>> this scenario.)  It's true, I suppose, in a literal sense that it's an
>> error in that pg_restore couldn't drop a table when instructed to, where no
>> such table exists.  But pragmatically it doesn't matter, so why not
>> reclassify this as a "warning" or a "notice"?
>>
>
Because pg_restore is just a client and it is repeating back what the
server tells it.  And for the server it is an error to drop an object that
doesn’t exist.

psql and pg_restore, as clients, can choose to ignore the errors they see,
regardless of what kind of error it is, but they don’t take on the added
burden of trying to reclassify errors into something else.

David J.

  5. Re: `pg_restore --if-exists` clarification

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-09-28T17:03:46Z

    "David G. Johnston" <david.g.johnston@gmail.com> writes:
>>> .. But pragmatically it doesn't matter, so why not
>>> reclassify this as a "warning" or a "notice"?

> Because pg_restore is just a client and it is repeating back what the
> server tells it.  And for the server it is an error to drop an object that
> doesn’t exist.
> psql and pg_restore, as clients, can choose to ignore the errors they see,
> regardless of what kind of error it is, but they don’t take on the added
> burden of trying to reclassify errors into something else.

If you actually try this, you'll get output like

...
pg_restore: error: could not execute query: ERROR:  schema "fkpart6" does not exist
Command was: DROP SCHEMA fkpart6;
pg_restore: error: could not execute query: ERROR:  schema "fkpart5" does not exist
Command was: DROP SCHEMA fkpart5;
pg_restore: error: could not execute query: ERROR:  schema "fkpart4" does not exist
Command was: DROP SCHEMA fkpart4;
pg_restore: error: could not execute query: ERROR:  schema "fkpart3" does not exist
Command was: DROP SCHEMA fkpart3;
...
pg_restore: warning: errors ignored on restore: 1488

so the only real problem is that there are (typically) enough of these
to obscure any errors that might be of greater significance.  Still,
genuine restore errors would come out after the blizzard of failed
DROPs, so I don't think there's a big problem in practice.

If we actually wanted to change any behavior here, I think what would
be most profitable to discuss is making --if-exists the default.
Not sure if we want to go there, but if we'd had the DROP IF EXISTS
option all along I bet it would have been done that way.

In any case, it's fair to complain about the documentation.
How about

--clean

   Before restoring database objects, issue commands to DROP all the
   objects that will be restored.  This option is useful for overwriting
   an existing database.  If any of the objects do not exist in the
   destination database, ignorable error messages will be reported,
   unless --if-exists is also specified.

--if-exists

   Use DROP ... IF EXISTS commands to drop objects in --clean mode.
   This suppresses "does not exist" errors that might otherwise be
   reported.  This option is not valid unless --clean is also specified.

			regards, tom lane

  6. Re: `pg_restore --if-exists` clarification

    Gulyás Attila <toraritte@gmail.com> — 2023-09-29T14:30:47Z

    I would love to see Tom's version in the docs! It is only slightly longer
than the current one, but absolutely clear of the purpose of both, and the
behaviour when used in tandem.

Appreciatively,
Attila

On Thu, Sep 28, 2023 at 1:03 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> "David G. Johnston" <david.g.johnston@gmail.com> writes:
> >>> .. But pragmatically it doesn't matter, so why not
> >>> reclassify this as a "warning" or a "notice"?
>
> > Because pg_restore is just a client and it is repeating back what the
> > server tells it.  And for the server it is an error to drop an object
> that
> > doesn’t exist.
> > psql and pg_restore, as clients, can choose to ignore the errors they
> see,
> > regardless of what kind of error it is, but they don’t take on the added
> > burden of trying to reclassify errors into something else.
>
> If you actually try this, you'll get output like
>
> ...
> pg_restore: error: could not execute query: ERROR:  schema "fkpart6" does
> not exist
> Command was: DROP SCHEMA fkpart6;
> pg_restore: error: could not execute query: ERROR:  schema "fkpart5" does
> not exist
> Command was: DROP SCHEMA fkpart5;
> pg_restore: error: could not execute query: ERROR:  schema "fkpart4" does
> not exist
> Command was: DROP SCHEMA fkpart4;
> pg_restore: error: could not execute query: ERROR:  schema "fkpart3" does
> not exist
> Command was: DROP SCHEMA fkpart3;
> ...
> pg_restore: warning: errors ignored on restore: 1488
>
> so the only real problem is that there are (typically) enough of these
> to obscure any errors that might be of greater significance.  Still,
> genuine restore errors would come out after the blizzard of failed
> DROPs, so I don't think there's a big problem in practice.
>
> If we actually wanted to change any behavior here, I think what would
> be most profitable to discuss is making --if-exists the default.
> Not sure if we want to go there, but if we'd had the DROP IF EXISTS
> option all along I bet it would have been done that way.
>
> In any case, it's fair to complain about the documentation.
> How about
>
> --clean
>
>    Before restoring database objects, issue commands to DROP all the
>    objects that will be restored.  This option is useful for overwriting
>    an existing database.  If any of the objects do not exist in the
>    destination database, ignorable error messages will be reported,
>    unless --if-exists is also specified.
>
> --if-exists
>
>    Use DROP ... IF EXISTS commands to drop objects in --clean mode.
>    This suppresses "does not exist" errors that might otherwise be
>    reported.  This option is not valid unless --clean is also specified.
>
>                         regards, tom lane
>

  7. Re: `pg_restore --if-exists` clarification

    Tom Lane <tgl@sss.pgh.pa.us> — 2023-09-29T17:15:21Z

    =?UTF-8?Q?Guly=C3=A1s_Attila?= <toraritte@gmail.com> writes:
> I would love to see Tom's version in the docs! It is only slightly longer
> than the current one, but absolutely clear of the purpose of both, and the
> behaviour when used in tandem.

Hearing no objections, done at

https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=75af0f401f905b947ea14401e8a51f1bae4ac265

			regards, tom lane

  8. Re: `pg_restore --if-exists` clarification

    Gulyás Attila <toraritte@gmail.com> — 2023-09-30T00:02:41Z

    Thank you so much!

Appreciatively,
Attila

On Fri, Sep 29, 2023 at 1:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> =?UTF-8?Q?Guly=C3=A1s_Attila?= <toraritte@gmail.com> writes:
> > I would love to see Tom's version in the docs! It is only slightly longer
> > than the current one, but absolutely clear of the purpose of both, and
> the
> > behaviour when used in tandem.
>
> Hearing no objections, done at
>
>
> https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=75af0f401f905b947ea14401e8a51f1bae4ac265
>
>                         regards, tom lane
>