Horton

Re: Introduce XID age and inactive timeout based replication slot invalidation

Bertrand Drouvot <bertranddrouvot.pg@gmail.com>

View thread
From: Bertrand Drouvot <bertranddrouvot.pg@gmail.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-03-21T05:53:48Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add a test for commit ac0e33136a using the injection point.

  2. Invalidate inactive replication slots.

  3. Fix incorrect slot type in BuildTupleHashTableExt

  4. Allow synced slots to have their inactive_since.

  5. Change last_inactive_time to inactive_since in pg_replication_slots.

  6. Track last_inactive_time in pg_replication_slots.

  7. Track invalidation_reason in pg_replication_slots.

  8. Add option force_initdb to PostgreSQL::Test::Cluster:init()

  9. Add a failover option to subscriptions.

  10. Allow setting failover property in the replication command.

  11. Allow to enable failover property for replication slots via SQL API.

  12. Track conflict_reason in pg_replication_slots.

  13. Log messages for replication slot acquisition and release.

  14. Remove vacuum_defer_cleanup_age

  15. Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids

  16. meson: Add initial version of meson based build system

Hi,

On Thu, Mar 21, 2024 at 08:47:18AM +0530, Amit Kapila wrote:
> On Wed, Mar 20, 2024 at 1:51 PM Bertrand Drouvot
> <bertranddrouvot.pg@gmail.com> wrote:
> >
> > On Wed, Mar 20, 2024 at 12:48:55AM +0530, Bharath Rupireddy wrote:
> > >
> > > 2. last_inactive_at and inactive_timeout are now tracked in on-disk
> > > replication slot data structure.
> >
> > Should last_inactive_at be tracked on disk? Say the engine is down for a period
> > of time > inactive_timeout then the slot will be invalidated after the engine
> > re-start (if no activity before we invalidate the slot). Should the time the
> > engine is down be counted as "inactive" time? I've the feeling it should not, and
> > that we should only take into account inactive time while the engine is up.
> >
> 
> Good point. The question is how do we achieve this without persisting
> the 'last_inactive_at'? Say, 'last_inactive_at' for a particular slot
> had some valid value before we shut down but it still didn't cross the
> configured 'inactive_timeout' value, so, we won't be able to
> invalidate it. Now, after the restart, as we don't know the
> last_inactive_at's value before the shutdown, we will initialize it
> with 0 (this is what Bharath seems to have done in the latest
> v13-0002* patch). After this, even if walsender or backend never
> acquires the slot, we won't invalidate it. OTOH, if we track
> 'last_inactive_at' on the disk, after, restart, we could initialize it
> to the current time if the value is non-zero. Do you have any better
> ideas?
> 

I think that setting last_inactive_at when we restart makes sense if the slot
has been active previously. I think the idea is because it's holding xmin/catalog_xmin
and that we don't want to prevent rows removal longer that the timeout.

So what about relying on xmin/catalog_xmin instead that way?

- For physical slots if xmin is set then set last_inactive_at to the current
time at restart (else zero).

- For logical slot, it's not the same as the catalog_xmin is set at the slot
creation time. So what about setting last_inactive_at at the current time at 
restart but also at creation time for logical slot? (Setting it to zero at
creation time (as we do in v13) does not look right, given the fact that it's
"already" holding a catalog_xmin).

That way, we'd ensure that we are not holding rows for longer that the timeout
and we don't need to persist last_inactive_at.

Regards,

-- 
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com