Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Reduce spurious Hot Standby conflicts from never-visible records.

  1. Hot Standby btree delete records and vacuum_defer_cleanup_age

    Simon Riggs <simon@2ndquadrant.com> — 2010-12-08T15:00:51Z

    Heikki pointed out to me that the btree delete record processing does
not respect vacuum_defer_cleanup_age. It should.

Attached patch to implement that.

Looking to commit in next few hours barring objections/suggestions, to
both HEAD and 9_0_STABLE, in time for next minor release.

-- 
 Simon Riggs           http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services

  2. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2010-12-08T23:10:23Z

    On 08.12.2010 16:00, Simon Riggs wrote:
> Heikki pointed out to me that the btree delete record processing does
> not respect vacuum_defer_cleanup_age. It should.
>
> Attached patch to implement that.
>
> Looking to commit in next few hours barring objections/suggestions, to
> both HEAD and 9_0_STABLE, in time for next minor release.

Please note that it was Noah Misch that raised this a while ago:

http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

  3. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2010-12-08T23:16:26Z

    On 09.12.2010 00:10, Heikki Linnakangas wrote:
> On 08.12.2010 16:00, Simon Riggs wrote:
>> Heikki pointed out to me that the btree delete record processing does
>> not respect vacuum_defer_cleanup_age. It should.
>>
>> Attached patch to implement that.
>>
>> Looking to commit in next few hours barring objections/suggestions, to
>> both HEAD and 9_0_STABLE, in time for next minor release.
>
> Please note that it was Noah Misch that raised this a while ago:
>
> http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php

On closer look, that's not actually the same issue, sorry for the noise..

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

  4. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2010-12-08T23:39:29Z

    On 08.12.2010 16:00, Simon Riggs wrote:
>
> Heikki pointed out to me that the btree delete record processing does
> not respect vacuum_defer_cleanup_age. It should.
>
> Attached patch to implement that.

This doesn't look right to me. btree_xlog_delete_get_latestRemovedXid() 
function calculates the latest XID present on the tuples that we're 
removing b-tree pointers for. btree_xlog_delete_get_latestRemovedXid() 
is used during recovery. vacuum_defer_cleanup_age should take effect in 
the master, not during recovery.

With the patch, btree_xlog_delete_get_latestRemovedXid() returns a value 
that's much smaller than it should. That's just wrong, it means that 
recovery in the standby will incorrectly think that all the removed 
tuples are old and not visible to any running read-only queries anymore, 
and will go ahead and remove the index tuples for them.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

  5. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Simon Riggs <simon@2ndquadrant.com> — 2010-12-09T07:52:11Z

    On Thu, 2010-12-09 at 00:39 +0100, Heikki Linnakangas wrote:

> vacuum_defer_cleanup_age should take effect in 
> the master, not during recovery. 

Hmmm, more to the point, it does take effect on the master and so there
is no need for this at all. What were you thinking? What was I? Doh.

-- 
 Simon Riggs           http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services

  6. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Simon Riggs <simon@2ndquadrant.com> — 2010-12-09T07:55:46Z

    On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote:
> On 09.12.2010 00:10, Heikki Linnakangas wrote:
> > On 08.12.2010 16:00, Simon Riggs wrote:
> >> Heikki pointed out to me that the btree delete record processing does
> >> not respect vacuum_defer_cleanup_age. It should.
> >>
> >> Attached patch to implement that.
> >>
> >> Looking to commit in next few hours barring objections/suggestions, to
> >> both HEAD and 9_0_STABLE, in time for next minor release.
> >
> > Please note that it was Noah Misch that raised this a while ago:
> >
> > http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php
> 
> On closer look, that's not actually the same issue, sorry for the noise..

Heikki, this one *is* important. Will fix. Thanks for the analysis Noah.

-- 
 Simon Riggs           http://www.2ndQuadrant.com/books/
 PostgreSQL Development, 24x7 Support, Training and Services

  7. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Fujii Masao <masao.fujii@gmail.com> — 2011-03-11T04:21:51Z

    On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote:
>> On 09.12.2010 00:10, Heikki Linnakangas wrote:
>> > On 08.12.2010 16:00, Simon Riggs wrote:
>> >> Heikki pointed out to me that the btree delete record processing does
>> >> not respect vacuum_defer_cleanup_age. It should.
>> >>
>> >> Attached patch to implement that.
>> >>
>> >> Looking to commit in next few hours barring objections/suggestions, to
>> >> both HEAD and 9_0_STABLE, in time for next minor release.
>> >
>> > Please note that it was Noah Misch that raised this a while ago:
>> >
>> > http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php
>>
>> On closer look, that's not actually the same issue, sorry for the noise..
>
> Heikki, this one *is* important. Will fix. Thanks for the analysis Noah.

Is this an open item for 9.1?

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center

  8. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Heikki Linnakangas <heikki.linnakangas@enterprisedb.com> — 2011-03-11T07:46:10Z

    On 11.03.2011 06:21, Fujii Masao wrote:
> On Thu, Dec 9, 2010 at 4:55 PM, Simon Riggs<simon@2ndquadrant.com>  wrote:
>> On Thu, 2010-12-09 at 00:16 +0100, Heikki Linnakangas wrote:
>>> On 09.12.2010 00:10, Heikki Linnakangas wrote:
>>>> On 08.12.2010 16:00, Simon Riggs wrote:
>>>>> Heikki pointed out to me that the btree delete record processing does
>>>>> not respect vacuum_defer_cleanup_age. It should.
>>>>>
>>>>> Attached patch to implement that.
>>>>>
>>>>> Looking to commit in next few hours barring objections/suggestions, to
>>>>> both HEAD and 9_0_STABLE, in time for next minor release.
>>>>
>>>> Please note that it was Noah Misch that raised this a while ago:
>>>>
>>>> http://archives.postgresql.org/pgsql-hackers/2010-11/msg01919.php
>>>
>>> On closer look, that's not actually the same issue, sorry for the noise..
>>
>> Heikki, this one *is* important. Will fix. Thanks for the analysis Noah.
>
> Is this an open item for 9.1?

Simon fixed it, commit b9075a6d2f9b07a00262a670dd60272904c79dce.

-- 
   Heikki Linnakangas
   EnterpriseDB   http://www.enterprisedb.com

  9. Re: Hot Standby btree delete records and vacuum_defer_cleanup_age

    Fujii Masao <masao.fujii@gmail.com> — 2011-03-11T08:03:55Z

    On Fri, Mar 11, 2011 at 4:46 PM, Heikki Linnakangas
<heikki.linnakangas@enterprisedb.com> wrote:
>> Is this an open item for 9.1?
>
> Simon fixed it, commit b9075a6d2f9b07a00262a670dd60272904c79dce.

Oh, thanks!

Regards,

-- 
Fujii Masao
NIPPON TELEGRAPH AND TELEPHONE CORPORATION
NTT Open Source Software Center