Horton
Menu

Thread

  1. Remove vacuum_defer_cleanup_age

    Josh Berkus <josh@agliodbs.com> — 2016-10-10T04:51:07Z

    Folks,

Given that hot_standby_feedback is pretty bulletproof now, and a lot of
the work in reducing replay conflicts, I think the utility of
vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
to remove it to 9.6, but it got away from me.

Any objections to removing the option in 10?

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

  2. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-13T00:00:34Z

    On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
> the work in reducing replay conflicts, I think the utility of
> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
> to remove it to 9.6, but it got away from me.
>
> Any objections to removing the option in 10?

I'm not sure I see the point.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  3. Re: Remove vacuum_defer_cleanup_age

    Josh Berkus <josh@agliodbs.com> — 2016-10-18T20:18:07Z

    On 10/12/2016 05:00 PM, Robert Haas wrote:
> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
>> the work in reducing replay conflicts, I think the utility of
>> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
>> to remove it to 9.6, but it got away from me.
>>
>> Any objections to removing the option in 10?
> 
> I'm not sure I see the point.

Redusing the number of configuration variables is an a-priori good.  In
aggregate, the more knobs we have, the harder it is to learn how to
admin Postgres.  Therefore any time a config variable becomes obsolete,
we should remove it.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

  4. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-18T20:28:36Z

    On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkus <josh@agliodbs.com> wrote:
> On 10/12/2016 05:00 PM, Robert Haas wrote:
>> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
>>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
>>> the work in reducing replay conflicts, I think the utility of
>>> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
>>> to remove it to 9.6, but it got away from me.
>>>
>>> Any objections to removing the option in 10?
>>
>> I'm not sure I see the point.
>
> Redusing the number of configuration variables is an a-priori good.  In
> aggregate, the more knobs we have, the harder it is to learn how to
> admin Postgres.  Therefore any time a config variable becomes obsolete,
> we should remove it.

Meh.  I agree that more configuration knobs makes it harder to learn
to configure the system, but we've got enough of them that removing
exactly one isn't going to make a material difference.  Against that,
if you are wrong about it being obsolete and there are actually people
relying on it heavily, those people will be very sad if we remove it,
and unless they read this mailing list, we probably won't find out
until it's too late.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  5. Re: Remove vacuum_defer_cleanup_age

    Josh Berkus <josh@agliodbs.com> — 2016-10-18T20:33:52Z

    On 10/18/2016 01:28 PM, Robert Haas wrote:
> On Tue, Oct 18, 2016 at 4:18 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> On 10/12/2016 05:00 PM, Robert Haas wrote:
>>> On Sun, Oct 9, 2016 at 9:51 PM, Josh Berkus <josh@agliodbs.com> wrote:
>>>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
>>>> the work in reducing replay conflicts, I think the utility of
>>>> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
>>>> to remove it to 9.6, but it got away from me.
>>>>
>>>> Any objections to removing the option in 10?
>>>
>>> I'm not sure I see the point.
>>
>> Redusing the number of configuration variables is an a-priori good.  In
>> aggregate, the more knobs we have, the harder it is to learn how to
>> admin Postgres.  Therefore any time a config variable becomes obsolete,
>> we should remove it.
> 
> Meh.  I agree that more configuration knobs makes it harder to learn
> to configure the system, but we've got enough of them that removing
> exactly one isn't going to make a material difference.  Against that,
> if you are wrong about it being obsolete and there are actually people
> relying on it heavily, those people will be very sad if we remove it,
> and unless they read this mailing list, we probably won't find out
> until it's too late.

Based on that argument, we would never be able to remove any
configuration parameter ever.

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

  6. Re: Remove vacuum_defer_cleanup_age

    Andres Freund <andres@anarazel.de> — 2016-10-18T20:37:46Z

    Hi,

On 2016-10-09 21:51:07 -0700, Josh Berkus wrote:
> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
> the work in reducing replay conflicts, I think the utility of
> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
> to remove it to 9.6, but it got away from me.

HS feedback doesn't e.g. work well with delayed and/or archived replay,
whereas defer_cleanup does.

On the other hand, removing it would make some of the reasoning around
GetOldestXmin() a bit easier.

Andres

  7. Re: Remove vacuum_defer_cleanup_age

    Josh Berkus <josh@agliodbs.com> — 2016-10-18T21:30:45Z

    On 10/18/2016 01:37 PM, Andres Freund wrote:
> Hi,
> 
> On 2016-10-09 21:51:07 -0700, Josh Berkus wrote:
>> Given that hot_standby_feedback is pretty bulletproof now, and a lot of
>> the work in reducing replay conflicts, I think the utility of
>> vacuum_defer_cleanup_age is at an end.  I really meant so submit a patch
>> to remove it to 9.6, but it got away from me.
> 
> HS feedback doesn't e.g. work well with delayed and/or archived replay,
> whereas defer_cleanup does.

Oh, point!  See, that's why I polled, I knew there was something I was
forgetting about.

> On the other hand, removing it would make some of the reasoning around
> GetOldestXmin() a bit easier.

Enough to make it worth breaking the above?

-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

  8. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-19T12:33:20Z

    On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:
> Based on that argument, we would never be able to remove any
> configuration parameter ever.

Well... no.  Based on that argument, we should only remove
configuration parameters if we're fairly certain that they are not
useful any more, which will be rare, but is not never.  I agree that
*if* vacuum_defer_cleanup_age is no longer useful, it should be
removed.  I'm just not convinced that it's truly obsolete, and you
haven't really offered much of an argument for that proposition.  It
does something sufficiently different from hot_standby_feedback that
I'm not sure it's accurate to say that one can substitute for the
other, and indeed, I see Andres has already suggested some scenarios
where it could still be useful.

Actually, I think vacuum_defer_cleanup_age is, and always has been, an
ugly hack.  But for some people it may be the ugly hack that is
letting them continue to use PostgreSQL.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  9. Re: Remove vacuum_defer_cleanup_age

    Bruce Momjian <bruce@momjian.us> — 2016-10-19T12:47:18Z

    On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:
> On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:
> > Based on that argument, we would never be able to remove any
> > configuration parameter ever.
> 
> Well... no.  Based on that argument, we should only remove
> configuration parameters if we're fairly certain that they are not
> useful any more, which will be rare, but is not never.  I agree that
> *if* vacuum_defer_cleanup_age is no longer useful, it should be
> removed.  I'm just not convinced that it's truly obsolete, and you
> haven't really offered much of an argument for that proposition.  It
> does something sufficiently different from hot_standby_feedback that
> I'm not sure it's accurate to say that one can substitute for the
> other, and indeed, I see Andres has already suggested some scenarios
> where it could still be useful.
> 
> Actually, I think vacuum_defer_cleanup_age is, and always has been, an
> ugly hack.  But for some people it may be the ugly hack that is
> letting them continue to use PostgreSQL.

I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
servers --- it cancels transactions rather than delaying cleanup.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

  10. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-19T13:00:06Z

    On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:
> On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:
>> On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:
>> > Based on that argument, we would never be able to remove any
>> > configuration parameter ever.
>>
>> Well... no.  Based on that argument, we should only remove
>> configuration parameters if we're fairly certain that they are not
>> useful any more, which will be rare, but is not never.  I agree that
>> *if* vacuum_defer_cleanup_age is no longer useful, it should be
>> removed.  I'm just not convinced that it's truly obsolete, and you
>> haven't really offered much of an argument for that proposition.  It
>> does something sufficiently different from hot_standby_feedback that
>> I'm not sure it's accurate to say that one can substitute for the
>> other, and indeed, I see Andres has already suggested some scenarios
>> where it could still be useful.
>>
>> Actually, I think vacuum_defer_cleanup_age is, and always has been, an
>> ugly hack.  But for some people it may be the ugly hack that is
>> letting them continue to use PostgreSQL.
>
> I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
> servers --- it cancels transactions rather than delaying cleanup.

I think it's the opposite, isn't it?  vacuum_defer_cleanup_age
prevents cancellations.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  11. Re: Remove vacuum_defer_cleanup_age

    Bruce Momjian <bruce@momjian.us> — 2016-10-19T16:59:56Z

    On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote:
> On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:
> > On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:
> >> On Tue, Oct 18, 2016 at 4:33 PM, Josh Berkus <josh@agliodbs.com> wrote:
> >> > Based on that argument, we would never be able to remove any
> >> > configuration parameter ever.
> >>
> >> Well... no.  Based on that argument, we should only remove
> >> configuration parameters if we're fairly certain that they are not
> >> useful any more, which will be rare, but is not never.  I agree that
> >> *if* vacuum_defer_cleanup_age is no longer useful, it should be
> >> removed.  I'm just not convinced that it's truly obsolete, and you
> >> haven't really offered much of an argument for that proposition.  It
> >> does something sufficiently different from hot_standby_feedback that
> >> I'm not sure it's accurate to say that one can substitute for the
> >> other, and indeed, I see Andres has already suggested some scenarios
> >> where it could still be useful.
> >>
> >> Actually, I think vacuum_defer_cleanup_age is, and always has been, an
> >> ugly hack.  But for some people it may be the ugly hack that is
> >> letting them continue to use PostgreSQL.
> >
> > I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
> > servers --- it cancels transactions rather than delaying cleanup.
> 
> I think it's the opposite, isn't it?  vacuum_defer_cleanup_age
> prevents cancellations.

Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
of xids, that a standby query can run before cancel, like
old_snapshot_threshold, no?  After that, we can cancel standby queries. 
I see hot_standby_feedback as our current behavior on the master where
we never cancel standby queries.

To me, hot_standby_feedback extends no-cleanup-no-cancel from the
standby to the master, while vacuum_defer_cleanup_age behaves like
old_snapshot_threshold in that it causes cancel for long-running
queries.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

  12. Re: Remove vacuum_defer_cleanup_age

    Josh Berkus <josh@agliodbs.com> — 2016-10-19T18:55:30Z

    On 10/19/2016 09:59 AM, Bruce Momjian wrote:
> On Wed, Oct 19, 2016 at 09:00:06AM -0400, Robert Haas wrote:
>> On Wed, Oct 19, 2016 at 8:47 AM, Bruce Momjian <bruce@momjian.us> wrote:
>>> On Wed, Oct 19, 2016 at 08:33:20AM -0400, Robert Haas wrote:

>>>> Actually, I think vacuum_defer_cleanup_age is, and always has been, an
>>>> ugly hack.  But for some people it may be the ugly hack that is
>>>> letting them continue to use PostgreSQL.
>>>
>>> I see vacuum_defer_cleanup_age as old_snapshot_threshold for standby
>>> servers --- it cancels transactions rather than delaying cleanup.
>>
>> I think it's the opposite, isn't it?  vacuum_defer_cleanup_age
>> prevents cancellations.
> 
> Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
> of xids, that a standby query can run before cancel, like
> old_snapshot_threshold, no?  After that, we can cancel standby queries. 
> I see hot_standby_feedback as our current behavior on the master where
> we never cancel standby queries.
> 
> To me, hot_standby_feedback extends no-cleanup-no-cancel from the
> standby to the master, while vacuum_defer_cleanup_age behaves like
> old_snapshot_threshold in that it causes cancel for long-running
> queries.

See Andres' response on this thread.  He's already covered why the
setting is still useful, but why we might want to remove it anyway.


-- 
--
Josh Berkus
Red Hat OSAS
(any opinions are my own)

  13. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-20T00:17:46Z

    On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:
> Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
> of xids, that a standby query can run before cancel, like
> old_snapshot_threshold, no?

No, not really.  It affects the behavior of the master, not the standby.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  14. Re: Remove vacuum_defer_cleanup_age

    Bruce Momjian <bruce@momjian.us> — 2016-10-20T01:09:01Z

    On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote:
> On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:
> > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
> > of xids, that a standby query can run before cancel, like
> > old_snapshot_threshold, no?
> 
> No, not really.  It affects the behavior of the master, not the standby.

But it controls when/if cancels happen on the standby.

-- 
  Bruce Momjian  <bruce@momjian.us>        http://momjian.us
  EnterpriseDB                             http://enterprisedb.com

+ As you are, so once was I.  As I am, so you will be. +
+                      Ancient Roman grave inscription +

  15. Re: Remove vacuum_defer_cleanup_age

    Robert Haas <robertmhaas@gmail.com> — 2016-10-20T14:17:50Z

    On Wed, Oct 19, 2016 at 9:09 PM, Bruce Momjian <bruce@momjian.us> wrote:
> On Wed, Oct 19, 2016 at 08:17:46PM -0400, Robert Haas wrote:
>> On Wed, Oct 19, 2016 at 12:59 PM, Bruce Momjian <bruce@momjian.us> wrote:
>> > Uh, vacuum_defer_cleanup_age sets an upper limit on how long, in terms
>> > of xids, that a standby query can run before cancel, like
>> > old_snapshot_threshold, no?
>>
>> No, not really.  It affects the behavior of the master, not the standby.
>
> But it controls when/if cancels happen on the standby.

True.  I see your point.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company