Horton

Re: Introduce XID age and inactive timeout based replication slot invalidation

Nathan Bossart <nathandbossart@gmail.com>

View thread
From: Nathan Bossart <nathandbossart@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>, Álvaro Herrera <alvherre@alvh.no-ip.org>, Nisha Moond <nisha.moond412@gmail.com>, Peter Smith <smithpb2250@gmail.com>, vignesh C <vignesh21@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, shveta malik <shveta.malik@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Ajin Cherian <itsajin@gmail.com>, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2025-02-17T16:48:49Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a test for commit ac0e33136a using the injection point.

  2. Invalidate inactive replication slots.

  3. Fix incorrect slot type in BuildTupleHashTableExt

  4. Allow synced slots to have their inactive_since.

  5. Change last_inactive_time to inactive_since in pg_replication_slots.

  6. Track last_inactive_time in pg_replication_slots.

  7. Track invalidation_reason in pg_replication_slots.

  8. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  9. Add a failover option to subscriptions.

  10. Allow setting failover property in the replication command.

  11. Allow to enable failover property for replication slots via SQL API.

  12. Track conflict_reason in pg_replication_slots.

  13. Log messages for replication slot acquisition and release.

  14. Remove vacuum_defer_cleanup_age

  15. Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

  16. meson: Add initial version of meson based build system

On Mon, Feb 17, 2025 at 07:57:22AM +0530, Amit Kapila wrote:
> On Wed, Feb 12, 2025 at 1:16 PM Zhijie Hou (Fujitsu)
> <houzj.fnst@fujitsu.com> wrote:
>> On Wednesday, February 12, 2025 11:56 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
>> > Also, we previously didn't have a good experience with XID-based threshold
>> > parameters like vacuum_defer_cleanup_age as mentioned by Robert (1).
>> > AFAICU from the previous discussion we need a time-based parameter and we
>> > didn't rule out xid_age based parameter as another parameter.

I am not sure I buy the comparison with vacuum_defer_cleanup_age.  That is
a very different feature than max_slot_xid_age, and we still have a number
of XID-based parameters (vacuum_freeze_table_age, vacuum_freeze_min_age,
vacuum_failsafe_age, the multixact versions of those parameters, and the
autovacuum versions).

>> Yeah, I think the primary purpose of this time-based option is to invalidate dormant
>> replication slots that have been inactive for a long period, in which case the
>> slots are no longer useful.
>>
>> Such slots can remain if a subscriber is down due to a system error or
>> inaccessible because of network issues. If this situation persists, it might be
>> more practical to recreate the subscriber rather than attempt to recover the
>> node and wait for it to catch up, which could be time-consuming.
>>
>> Parameters like max_slot_wal_keep_size and max_slot_xid_id_age do not
>> differentiate between active and inactive replication slots. Some customers I
>> met are hesitant about using these settings, as they can sometimes invalidate
>> a slot unnecessarily and break the replication.

Sure, an inactive-timeout feature won't break replication, but it's also
not going to be terribly effective against wraparound-related issues.  It
seems weird to me to allow an active replication slot to take priority over
imminent storage/XID issues it causes.

> Alvaro, Nathan, do let us know if you would like to discuss more on
> the use case for this new GUC idle_replication_slot_timeout?
> Otherwise, we can proceed with this patch.

I guess I'm not mortally opposed to it.  I just think we really need
proper backstops against the storage/XID issues more than we need this one,
and I don't want it to be mistaken for a solution to those problems.

-- 
nathan