Horton

Re: Introduce XID age and inactive timeout based replication slot invalidation

Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>

View thread
From: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-18T04:28:42Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a test for commit ac0e33136a using the injection point.

  2. Invalidate inactive replication slots.

  3. Fix incorrect slot type in BuildTupleHashTableExt

  4. Allow synced slots to have their inactive_since.

  5. Change last_inactive_time to inactive_since in pg_replication_slots.

  6. Track last_inactive_time in pg_replication_slots.

  7. Track invalidation_reason in pg_replication_slots.

  8. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  9. Add a failover option to subscriptions.

  10. Allow setting failover property in the replication command.

  11. Allow to enable failover property for replication slots via SQL API.

  12. Track conflict_reason in pg_replication_slots.

  13. Log messages for replication slot acquisition and release.

  14. Remove vacuum_defer_cleanup_age

  15. Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

  16. meson: Add initial version of meson based build system

On Sat, Mar 16, 2024 at 3:55 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> > What's proposed with max_slot_xid_age is that during checkpoint we
> > look at slot's xmin and catalog_xmin, and the current system txn id.
> > Then, if the XID age of (xmin, catalog_xmin) and current_xid crosses
> > max_slot_xid_age, we invalidate the slot.
> >
>
> I can see that in your patch (in function
> InvalidatePossiblyObsoleteSlot()). As per my understanding, we need
> something similar for slot xids in ComputeXidHorizons() as we are
> doing WAL in KeepLogSeg(). In KeepLogSeg(), we compute the minimum LSN
> location required by slots and then adjust it for
> 'max_slot_wal_keep_size'. On similar lines, currently in
> ComputeXidHorizons(), we compute the minimum xid required by slots
> (procArray->replication_slot_xmin and
> procArray->replication_slot_catalog_xmin) but then don't adjust it for
> 'max_slot_xid_age'. I could be missing something in this but it is
> better to keep discussing this

After invalidating slots because of max_slot_xid_age, the
procArray->replication_slot_xmin and
procArray->replication_slot_catalog_xmin are recomputed immediately in
InvalidateObsoleteReplicationSlots->ReplicationSlotsComputeRequiredXmin->ProcArraySetReplicationSlotXmin.
And, later the XID horizons in ComputeXidHorizons are computed before
the vacuum on each table via GetOldestNonRemovableTransactionId.
Aren't these enough? Do you want the XID horizons recomputed
immediately, something like the below?

/* Invalidate replication slots based on xmin or catalog_xmin age */
if (max_slot_xid_age > 0)
{
    if (InvalidateObsoleteReplicationSlots(RS_INVAL_XID_AGE,
                                           0, InvalidOid,
                                           InvalidTransactionId))
    {
        ComputeXidHorizonsResult horizons;

        /*
         * Some slots have been invalidated; update the XID horizons
         * as a side-effect.
         */
        ComputeXidHorizons(&horizons);
    }
}

--
Bharath Rupireddy
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com