Horton

Re: Introduce XID age and inactive timeout based replication slot invalidation

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

View thread
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-21T05:23:54Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a test for commit ac0e33136a using the injection point.

  2. Invalidate inactive replication slots.

  3. Fix incorrect slot type in BuildTupleHashTableExt

  4. Allow synced slots to have their inactive_since.

  5. Change last_inactive_time to inactive_since in pg_replication_slots.

  6. Track last_inactive_time in pg_replication_slots.

  7. Track invalidation_reason in pg_replication_slots.

  8. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  9. Add a failover option to subscriptions.

  10. Allow setting failover property in the replication command.

  11. Allow to enable failover property for replication slots via SQL API.

  12. Track conflict_reason in pg_replication_slots.

  13. Log messages for replication slot acquisition and release.

  14. Remove vacuum_defer_cleanup_age

  15. Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

  16. meson: Add initial version of meson based build system

On Thu, Mar 21, 2024 at 9:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> I also don't see any obvious problem with such an API. However, this
> is not a good time to invent new APIs. Let's keep the feature simple
> and then we can extend it in the next version after more discussion
> and probably by that time we will get some feedback from the field as
> well.

I couldn't agree more.

> > > But the issue is that it would make it inconsistent with the new inactivetimeout
> > > in the subscription that is added in "v12-0005".
> >
> > Can you please elaborate what the inconsistency it causes with inactivetimeout?
> >
> I think the inconsistency can arise from the fact that on publisher
> one can change the inactive_timeout for the slot corresponding to a
> subscription but the subscriber won't know, so it will still show the
> old value.

Understood.

> If we want we can document this as a limitation and let
> users be aware of it. However, I feel at this stage, let's not even
> expose this from the subscription or maybe we can discuss it once/if
> we are done with other patches. Anyway, if one wants to use this
> feature with a subscription, she can create a slot first on the
> publisher with inactive_timeout value and then associate such a slot
> with a required subscription.

If we are not exposing it via subscription (meaning, we don't consider
v13-0004 and v13-0005 patches), I feel we can have a new SQL API
pg_alter_replication_slot(int inactive_timeout) for now just altering
the inactive_timeout of a given slot.

With this approach, one can do either of the following:
1) Create a slot with SQL API with inactive_timeout set, and use it
for subscriptions or for streaming standbys.
2) Create a slot with SQL API without inactive_timeout set, use it for
subscriptions or for streaming standbys, and set inactive_timeout
later via pg_alter_replication_slot() depending on how the slot is
consumed
3) Create a subscription with create_slot=true, and set
inactive_timeout via pg_alter_replication_slot() depending on how the
slot is consumed.

This approach seems consistent and minimal to start with.

If we agree on this, I'll drop both 0004 and 0005 that are allowing
inactive_timeout to be set via replication commands and via
create/alter subscription respectively, and implement
pg_alter_replication_slot().

FWIW, adding the new SQL API pg_alter_replication_slot() isn't that hard.

-- 
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com