Horton

Thread

Commits

  1. Skip temp tables from basebackup.

  1. PATCH: Exclude temp relations from base backup

    David Steele <david@pgmasters.net> — 2018-02-28T15:55:10Z

    This is a follow-up patch from the exclude unlogged relations discussion
[1].

The patch excludes temporary relations during a base backup using the
existing looks_like_temp_rel_name() function for identification.

It shares code to identify database directories from [1], so for now
that has been duplicated in this patch to make it independent.  I'll
rebase depending on what gets committed first.

Thanks,
-- 
-David
david@pgmasters.net

[1]
https://www.postgresql.org/message-id/04791bab-cb04-ba43-e9c0-664a4c1ffb2c%40pgmasters.net

  2. Re: PATCH: Exclude temp relations from base backup

    David Steele <david@pgmasters.net> — 2018-03-13T16:34:16Z

    Hi,

On 2/28/18 10:55 AM, David Steele wrote:
> This is a follow-up patch from the exclude unlogged relations discussion
> [1].
> 
> The patch excludes temporary relations during a base backup using the
> existing looks_like_temp_rel_name() function for identification.
> 
> It shares code to identify database directories from [1], so for now
> that has been duplicated in this patch to make it independent.  I'll
> rebase depending on what gets committed first.

Updated the patch to change die() to BAIL_OUT() and use append_to_file()
as suggested for another test patch [1].

Regards,
-- 
-David
david@pgmasters.net

[1]
https://www.postgresql.org/message-id/6bc5d931-5b00-279f-f65a-26e32de400a6%40pgmasters.net

  3. Re: PATCH: Exclude temp relations from base backup

    David Steele <david@pgmasters.net> — 2018-03-23T16:54:44Z

    On 3/13/18 12:34 PM, David Steele wrote:

> Updated the patch to change die() to BAIL_OUT() and use append_to_file()
> as suggested for another test patch.

Updated patch now that the unlogged table exclusions have been committed
[1].

Thanks,
-- 
-David
david@pgmasters.net

[1]
https://www.postgresql.org/message-id/4d9be1c0-5c58-d9a0-7152-2771224910ae%40sigaev.ru

  4. Re: PATCH: Exclude temp relations from base backup

    Teodor Sigaev <teodor@sigaev.ru> — 2018-03-26T15:52:11Z

    Hi!

Will autovacuum (or something else) complain about absense of relfile during 
orphan table deleting? I mean, you get a base backup without temp tables, then 
you try to run postgres on it and will it complain about existing record in 
pg_class and absence of corresponding relfile?


David Steele wrote:
> On 3/13/18 12:34 PM, David Steele wrote:
> 
>> Updated the patch to change die() to BAIL_OUT() and use append_to_file()
>> as suggested for another test patch.
> 
> Updated patch now that the unlogged table exclusions have been committed
> [1].
> 
> Thanks,
> 

-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/

  5. Re: PATCH: Exclude temp relations from base backup

    Stephen Frost <sfrost@snowman.net> — 2018-03-26T17:06:19Z

    Greetings,

* Teodor Sigaev (teodor@sigaev.ru) wrote:
> Will autovacuum (or something else) complain about absense of relfile during
> orphan table deleting? I mean, you get a base backup without temp tables,
> then you try to run postgres on it and will it complain about existing
> record in pg_class and absence of corresponding relfile?

I would certainly hope not considering that's what happens during
regular crash recovery also, so if there's an issue with that, we'd have
a problem in released versions.

There's an independent discussion that was being had recently about how
to make sure those records in pg_class get cleaned up in a reasonable
timeframe and don't lead to problems with wrap-arounds, but that's a
different and pre-existing issue.

Thanks!

Stephen

  6. Re: PATCH: Exclude temp relations from base backup

    David Steele <david@pgmasters.net> — 2018-03-26T17:08:50Z

    On 3/26/18 1:06 PM, Stephen Frost wrote:
> 
> * Teodor Sigaev (teodor@sigaev.ru) wrote:
>> Will autovacuum (or something else) complain about absense of relfile during
>> orphan table deleting? I mean, you get a base backup without temp tables,
>> then you try to run postgres on it and will it complain about existing
>> record in pg_class and absence of corresponding relfile?
> 
> I would certainly hope not considering that's what happens during
> regular crash recovery also, so if there's an issue with that, we'd have
> a problem in released versions.

Agreed.  The logic for pg_basebackup was modeled off RemovePgTempFiles()
which is called at postmaster start.  We are just doing the cleanup in
advance (in the backup only, of course).

Thanks,
-- 
-David
david@pgmasters.net

  7. Re: PATCH: Exclude temp relations from base backup

    Teodor Sigaev <teodor@sigaev.ru> — 2018-03-27T13:34:24Z

    Thank you, pushed

David Steele wrote:
> On 3/26/18 1:06 PM, Stephen Frost wrote:
>>
>> * Teodor Sigaev (teodor@sigaev.ru) wrote:
>>> Will autovacuum (or something else) complain about absense of relfile during
>>> orphan table deleting? I mean, you get a base backup without temp tables,
>>> then you try to run postgres on it and will it complain about existing
>>> record in pg_class and absence of corresponding relfile?
>>
>> I would certainly hope not considering that's what happens during
>> regular crash recovery also, so if there's an issue with that, we'd have
>> a problem in released versions.
> 
> Agreed.  The logic for pg_basebackup was modeled off RemovePgTempFiles()
> which is called at postmaster start.  We are just doing the cleanup in
> advance (in the backup only, of course).
> 
> Thanks,
> 

-- 
Teodor Sigaev                                   E-mail: teodor@sigaev.ru
                                                    WWW: http://www.sigaev.ru/