Horton

Thread

Commits

  1. Add some asserts based on LWLockHeldByMe() for replication slot statistics

  2. Keep replication slot statistics on invalidation

  1. Spurious pgstat_drop_replslot() call

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-03-08T10:19:11Z

    Hi hackers,

Please find attached a tiny patch to remove a $SUBJECT.

Indeed, it does not seem appropriate to remove stats during slot invalidation as
one could still be interested to look at them.

This spurious call has been introduced in be87200efd. I think that initially we
designed to drop slots when a recovery conflict occurred during logical decoding
on a standby. But we changed our mind to invalidate such a slot instead.

The spurious call is probably due to the initial design but has not been removed.

I don't think it's worth to add a test but can do if one feel the need.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  2. Re: Spurious pgstat_drop_replslot() call

    Michael Paquier <michael@paquier.xyz> — 2024-03-08T10:55:39Z

    On Fri, Mar 08, 2024 at 10:19:11AM +0000, Bertrand Drouvot wrote:
> Indeed, it does not seem appropriate to remove stats during slot invalidation as
> one could still be interested to look at them.

Yeah, my take is that this can still be interesting to know, at least
for debugging.  This would limit the stats to be dropped when the slot
is dropped, and that looks like a sound idea.

> This spurious call has been introduced in be87200efd. I think that initially we
> designed to drop slots when a recovery conflict occurred during logical decoding
> on a standby. But we changed our mind to invalidate such a slot instead.
> 
> The spurious call is probably due to the initial design but has not been removed.

This is not a subject that has really been touched on the original
thread mentioned on the commit, so it is a bit hard to be sure.  The
only comment is that a previous version of the patch did the stats
drop in the slot invalidation path at an incorrect location.

> I don't think it's worth to add a test but can do if one feel the need.

Well, that would not be complicated while being cheap, no?  We have a
few paths in the 035 test where we know that a slot has been
invalidated so it is just a matter of querying once
pg_stat_replication_slot and see if some data is still there.
--
Michael

  3. Re: Spurious pgstat_drop_replslot() call

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-03-08T15:04:10Z

    Hi,

On Fri, Mar 08, 2024 at 07:55:39PM +0900, Michael Paquier wrote:
> On Fri, Mar 08, 2024 at 10:19:11AM +0000, Bertrand Drouvot wrote:
> > Indeed, it does not seem appropriate to remove stats during slot invalidation as
> > one could still be interested to look at them.
> 
> Yeah, my take is that this can still be interesting to know, at least
> for debugging.  This would limit the stats to be dropped when the slot
> is dropped, and that looks like a sound idea.

Thanks for looking at it!

> > This spurious call has been introduced in be87200efd. I think that initially we
> > designed to drop slots when a recovery conflict occurred during logical decoding
> > on a standby. But we changed our mind to invalidate such a slot instead.
> > 
> > The spurious call is probably due to the initial design but has not been removed.
> 
> This is not a subject that has really been touched on the original
> thread mentioned on the commit, so it is a bit hard to be sure.  The
> only comment is that a previous version of the patch did the stats
> drop in the slot invalidation path at an incorrect location.

The switch in the patch from "drop" to "invalidation" is in [1], see:

"
Given the precedent of max_slot_wal_keep_size, I think it's wrong to
just drop
the logical slots. Instead we should just mark them as
invalid, 
like InvalidateObsoleteReplicationSlots().

Makes fully sense and done that way in the attached patch.
“

But yeah, hard to be sure why this call is there, at least I don't remember...

> > I don't think it's worth to add a test but can do if one feel the need.
> 
> Well, that would not be complicated while being cheap, no?  We have a
> few paths in the 035 test where we know that a slot has been
> invalidated so it is just a matter of querying once
> pg_stat_replication_slot and see if some data is still there.

We can not be 100% sure that the stats are up to date when the process holding
the active slot is killed. 

So v2 attached adds a test where we ensure first that we have non empty stats
before triggering the invalidation.

[1]: https://www.postgresql.org/message-id/26c6f320-98f0-253c-f8b5-df1e7c1f6315%40amazon.com

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  4. Re: Spurious pgstat_drop_replslot() call

    Michael Paquier <michael@paquier.xyz> — 2024-03-11T07:15:40Z

    On Fri, Mar 08, 2024 at 03:04:10PM +0000, Bertrand Drouvot wrote:
> The switch in the patch from "drop" to "invalidation" is in [1], see:
> 
> "
> Given the precedent of max_slot_wal_keep_size, I think it's wrong to
just drop
> the logical slots. Instead we should just mark them as
invalid, 
> like InvalidateObsoleteReplicationSlots().
> 
> Makes fully sense and done that way in the attached patch.
> “
> 
> But yeah, hard to be sure why this call is there, at least I don't remember...

Yep, noticed that on Friday.

> We can not be 100% sure that the stats are up to date when the process holding
> the active slot is killed. 
> 
> So v2 attached adds a test where we ensure first that we have non empty stats
> before triggering the invalidation.

Ah, that explains the extra poll.  What you have done here makes sense
to me, and the new test fails immediately if I add back the stats drop
in the invalidation code path.

That's a slight change in behavior, unfortunately, and it cannot be
called a bug as this improves the visibility of the stats after an
invalidation, so this is not something that can be backpatched.
--
Michael

  5. Re: Spurious pgstat_drop_replslot() call

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-03-11T07:24:54Z

    Hi,

On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:
> That's a slight change in behavior, unfortunately, and it cannot be
> called a bug as this improves the visibility of the stats after an
> invalidation, so this is not something that can be backpatched.

Yeah, makes sense to me.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  6. Re: Spurious pgstat_drop_replslot() call

    Michael Paquier <michael@paquier.xyz> — 2024-03-12T05:36:58Z

    On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:
> That's a slight change in behavior, unfortunately, and it cannot be
> called a bug as this improves the visibility of the stats after an
> invalidation, so this is not something that can be backpatched.

I've looked again at that and that was OK on a second look.  May I
suggest the attached additions with LWLockHeldByMeInMode to make sure
that the stats are dropped and created while we hold
ReplicationSlotAllocationLock?
--
Michael

  7. Re: Spurious pgstat_drop_replslot() call

    Bertrand Drouvot <bertranddrouvot.pg@gmail.com> — 2024-03-12T06:17:11Z

    Hi,

On Tue, Mar 12, 2024 at 02:36:58PM +0900, Michael Paquier wrote:
> On Mon, Mar 11, 2024 at 04:15:40PM +0900, Michael Paquier wrote:
> > That's a slight change in behavior, unfortunately, and it cannot be
> > called a bug as this improves the visibility of the stats after an
> > invalidation, so this is not something that can be backpatched.
> 
> I've looked again at that and that was OK on a second look.

Thanks!

> May I
> suggest the attached additions with LWLockHeldByMeInMode to make sure
> that the stats are dropped and created while we hold
> ReplicationSlotAllocationLock?

Yeah, makes fully sense and looks good to me.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com

  8. Re: Spurious pgstat_drop_replslot() call

    Alexander Law <exclusion@gmail.com> — 2024-03-12T09:00:00Z

    Hello Bertrand and Michael,

12.03.2024 09:17, Bertrand Drouvot wrote:
>
>> May I
>> suggest the attached additions with LWLockHeldByMeInMode to make sure
>> that the stats are dropped and created while we hold
>> ReplicationSlotAllocationLock?
> Yeah, makes fully sense and looks good to me.

Sorry for a bit off-topic, but I've remembered an anomaly with a similar
assert:
https://www.postgresql.org/message-id/17947-b9554521ad963c9c%40postgresql.org

Maybe you would find it worth considering while working in this area...
(I've just run that reproducer on b36fbd9f8 and confirmed that the
assertion failure is still here.)

Best regards,
Alexander

  9. Re: Spurious pgstat_drop_replslot() call

    Michael Paquier <michael@paquier.xyz> — 2024-03-12T22:37:22Z

    On Tue, Mar 12, 2024 at 12:00:00PM +0300, Alexander Lakhin wrote:
> Sorry for a bit off-topic, but I've remembered an anomaly with a similar
> assert:
> https://www.postgresql.org/message-id/17947-b9554521ad963c9c%40postgresql.org

Thanks for the reminder.  The invalidation path with the stats drop is
only in 16~.

> Maybe you would find it worth considering while working in this area...
> (I've just run that reproducer on b36fbd9f8 and confirmed that the
> assertion failure is still here.)

Indeed, something needs to happen.  I am not surprised that it still
reproduces; nothing has changed with the locking of the stats entries.
:/
--
Michael