RE: Introduce XID age and inactive timeout based replication slot invalidation

Zhijie Hou (Fujitsu) <houzj.fnst@fujitsu.com>

From: "Zhijie Hou (Fujitsu)" <houzj.fnst@fujitsu.com>

To: Nisha Moond <nisha.moond412@gmail.com>

Cc: Amit Kapila <amit.kapila16@gmail.com>, vignesh C <vignesh21@gmail.com>, Shlok Kyal <shlok.kyal.oss@gmail.com>, "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>, shveta malik <shveta.malik@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Ajin Cherian <itsajin@gmail.com>, Bertrand Drouvot <bertranddrouvot.pg@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Nathan Bossart <nathandbossart@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Peter Smith <smithpb2250@gmail.com>

Date: 2025-02-08T06:58:45Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Add a test for commit ac0e33136a using the injection point.
- 8a695d7998be 18.0 landed
Invalidate inactive replication slots.
- ac0e33136abc 18.0 landed
Fix incorrect slot type in BuildTupleHashTableExt
- d96d1d5152f3 18.0 cited
Allow synced slots to have their inactive_since.
- 6f132ed693b6 17.0 landed
Change last_inactive_time to inactive_since in pg_replication_slots.
- 6d49c8d4b4f4 17.0 landed
Track last_inactive_time in pg_replication_slots.
- a11f330b5584 17.0 landed
Track invalidation_reason in pg_replication_slots.
- 6ae701b4378d 17.0 landed
Add option force_initdb to PostgreSQL::Test::Cluster:init()
- ff9e1e764fcc 17.0 cited
Add a failover option to subscriptions.
- 776621a5e479 17.0 cited
Allow setting failover property in the replication command.
- 73292404370c 17.0 cited
Allow to enable failover property for replication slots via SQL API.
- c393308b69d2 17.0 cited
Track conflict_reason in pg_replication_slots.
- 007693f2a3ac 17.0 cited
Log messages for replication slot acquisition and release.
- 7c3fb505b14e 17.0 cited
Remove vacuum_defer_cleanup_age
- 1118cd37eb61 16.0 cited
Fix corruption due to vacuum_defer_cleanup_age underflowing 64bit xids
- be504a3e974d 16.0 cited
meson: Add initial version of meson based build system
- e6927270cd18 16.0 cited

On Friday, February 7, 2025 9:06 PM Nisha Moond <nisha.moond412@gmail.com> wrote:
> 
> Attached v72 patches, addressed the above comments as well as Vignesh's
> comments in [2].
>  - There are no new changes in patch-002.

Thanks for updating the patch, I have few review comments:

1.
> InvalidateObsoleteReplicationSlots(ReplicationSlotInvalidationCause cause,

I think the type of first parameter 'cause' is not appropriate anymore since
it's now a bitmap flag instead of an enum.

2.
> -StaticAssertDecl(lengthof(SlotInvalidationCauses) == (RS_INVAL_MAX_CAUSES + 1),
> -				 "array length mismatch");
> +#define	RS_INVAL_MAX_CAUSES (sizeof(InvalidationCauses) / sizeof(InvalidationCauses[0]))

I'd like to confirm if the current value of the RS_INVAL_MAX_CAUSES is correct.
Previously, the value is arrary_length - 1, while now it seems equal to the
arrary_length.

And ISTM we could directly call lengthof() here.

3.

+			if (cause & RS_INVAL_HORIZON)
+			{
+				if (!SlotIsLogical(s))
+					goto invalidation_marked;

I am not sure if this logic is correct. Even if the slot would not be
invalidated due to RS_INVAL_HORIZON, we should continue to check other causes.

Besides, instead of using a goto, I personally prefer to move all these codes
into a separate function which would return a single invalidation cause.

4.
-	if (InvalidateObsoleteReplicationSlots(RS_INVAL_WAL_REMOVED,
+	if (InvalidateObsoleteReplicationSlots(RS_INVAL_WAL_REMOVED | RS_INVAL_IDLE_TIMEOUT,
 										   _logSegNo, InvalidOid,
 										   InvalidTransactionId))

I think this change could trigger an unnecessary WAL position re-calculation when
slots are invalidated only due to RS_INVAL_IDLE_TIMEOUT. But since it would not be
a frequent operation so I am OK to leave it unless we have better ideas.

Best Regards,
Hou zj