Re: Introduce XID age and inactive timeout based replication slot invalidation
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Commits
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add a test for commit ac0e33136a using the injection point.
- 8a695d7998be 18.0 landed
-
Invalidate inactive replication slots.
- ac0e33136abc 18.0 landed
-
Fix incorrect slot type in BuildTupleHashTableExt
- d96d1d5152f3 18.0 cited
-
Allow synced slots to have their inactive_since.
- 6f132ed693b6 17.0 landed
-
Change last_inactive_time to inactive_since in pg_replication_slots.
- 6d49c8d4b4f4 17.0 landed
-
Track last_inactive_time in pg_replication_slots.
- a11f330b5584 17.0 landed
-
Track invalidation_reason in pg_replication_slots.
- 6ae701b4378d 17.0 landed
-
Add option force_initdb to PostgreSQL::Test::Cluster:init()
- ff9e1e764fcc 17.0 cited
-
Add a failover option to subscriptions.
- 776621a5e479 17.0 cited
-
Allow setting failover property in the replication command.
- 73292404370c 17.0 cited
-
Allow to enable failover property for replication slots via SQL API.
- c393308b69d2 17.0 cited
-
Track conflict_reason in pg_replication_slots.
- 007693f2a3ac 17.0 cited
-
Log messages for replication slot acquisition and release.
- 7c3fb505b14e 17.0 cited
-
Remove vacuum_defer_cleanup_age
- 1118cd37eb61 16.0 cited
-
Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids
- be504a3e974d 16.0 cited
-
meson: Add initial version of meson based build system
- e6927270cd18 16.0 cited
Attachments
- v14-0001-Track-invalidation_reason-in-pg_replication_slot.patch (application/x-patch) patch v14-0001
- v14-0002-Track-last_inactive_at-in-pg_replication_slots.patch (application/x-patch) patch v14-0002
- v14-0003-Allow-setting-inactive_timeout-for-replication-s.patch (application/x-patch) patch v14-0003
- v14-0004-Introduce-new-SQL-funtion-pg_alter_replication_s.patch (application/x-patch) patch v14-0004
- v14-0005-Allow-setting-inactive_timeout-in-the-replicatio.patch (application/x-patch) patch v14-0005
- v14-0006-Add-inactive_timeout-based-replication-slot-inva.patch (application/x-patch) patch v14-0006
On Fri, Mar 22, 2024 at 12:39 PM Bertrand Drouvot <bertranddrouvot.pg@gmail.com> wrote: > > > > Please find the v14-0001 patch for now. > > Thanks! > > > LGTM. Let's wait for Bertrand to see if he has more comments on 0001 > > and then I'll push it. > > LGTM too. Thanks. Here I'm implementing the following: 0001 Track invalidation_reason in pg_replication_slots 0002 Track last_inactive_at in pg_replication_slots 0003 Allow setting inactive_timeout for replication slots via SQL API 0004 Introduce new SQL funtion pg_alter_replication_slot 0005 Allow setting inactive_timeout in the replication command 0006 Add inactive_timeout based replication slot invalidation 1. Keep it last_inactive_at as a shared memory variable, but always set it at restart if the slot's inactive_timeout has non-zero value and reset it as soon as someone acquires that slot so that if the slot doesn't get acquired till inactive_timeout, checkpointer will invalidate the slot. 2. Ensure with pg_alter_replication_slot one could "only" alter the timeout property for the time being, if not that could lead to the subscription inconsistency. 3. Have some notes in the CREATE and ALTER SUBSCRIPTION docs about using an existing slot to leverage inactive_timeout feature. 4. last_inactive_at should also be set to the current time during slot creation because if one creates a slot and does nothing with it then it's the time it starts to be inactive. 5. We don't set last_inactive_at to GetCurrentTimestamp() for failover slots. 6. Leave the patch that added support for inactive_timeout in subscriptions. Please see the attached v14 patch set. No change in the attached v14-0001 from the previous patch. -- Bharath Rupireddy PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com