Horton

Re: Introduce XID age and inactive timeout based replication slot invalidation

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

View thread
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-21T06:07:24Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a test for commit ac0e33136a using the injection point.

  2. Invalidate inactive replication slots.

  3. Fix incorrect slot type in BuildTupleHashTableExt

  4. Allow synced slots to have their inactive_since.

  5. Change last_inactive_time to inactive_since in pg_replication_slots.

  6. Track last_inactive_time in pg_replication_slots.

  7. Track invalidation_reason in pg_replication_slots.

  8. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  9. Add a failover option to subscriptions.

  10. Allow setting failover property in the replication command.

  11. Allow to enable failover property for replication slots via SQL API.

  12. Track conflict_reason in pg_replication_slots.

  13. Log messages for replication slot acquisition and release.

  14. Remove vacuum_defer_cleanup_age

  15. Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

  16. meson: Add initial version of meson based build system

Hi,

On Thu, Mar 21, 2024 at 10:53:54AM +0530, Bharath Rupireddy wrote:
> On Thu, Mar 21, 2024 at 9:07 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > > But the issue is that it would make it inconsistent with the new inactivetimeout
> > > > in the subscription that is added in "v12-0005".
> > >
> > > Can you please elaborate what the inconsistency it causes with inactivetimeout?
> > >
> > I think the inconsistency can arise from the fact that on publisher
> > one can change the inactive_timeout for the slot corresponding to a
> > subscription but the subscriber won't know, so it will still show the
> > old value.

Yeah, that was what I had in mind.

> > If we want we can document this as a limitation and let
> > users be aware of it. However, I feel at this stage, let's not even
> > expose this from the subscription or maybe we can discuss it once/if
> > we are done with other patches.

I agree, it's important to expose it for things like "failover" but I think we
can get rid of it for the timeout one.

>>  Anyway, if one wants to use this
> > feature with a subscription, she can create a slot first on the
> > publisher with inactive_timeout value and then associate such a slot
> > with a required subscription.

Right.

> 
> If we are not exposing it via subscription (meaning, we don't consider
> v13-0004 and v13-0005 patches), I feel we can have a new SQL API
> pg_alter_replication_slot(int inactive_timeout) for now just altering
> the inactive_timeout of a given slot.

Agree, that seems more "natural" that going through a replication connection.

> With this approach, one can do either of the following:
> 1) Create a slot with SQL API with inactive_timeout set, and use it
> for subscriptions or for streaming standbys.

Yes.

> 2) Create a slot with SQL API without inactive_timeout set, use it for
> subscriptions or for streaming standbys, and set inactive_timeout
> later via pg_alter_replication_slot() depending on how the slot is
> consumed

Yes.

> 3) Create a subscription with create_slot=true, and set
> inactive_timeout via pg_alter_replication_slot() depending on how the
> slot is consumed.

Yes.

We could also do the above 3 and altering the timeout with a replication
connection but the SQL API seems more natural to me.

> 
> This approach seems consistent and minimal to start with.
> 
> If we agree on this, I'll drop both 0004 and 0005 that are allowing
> inactive_timeout to be set via replication commands and via
> create/alter subscription respectively, and implement
> pg_alter_replication_slot().

+1 on this.

> FWIW, adding the new SQL API pg_alter_replication_slot() isn't that hard.

Also I think we should ensure that one could "only" alter the timeout property
for the time being (if not that could lead to the subscription inconsistency 
mentioned above).

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com