Re: Add 64-bit XIDs into PostgreSQL 15

Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>

From: Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>
To: y.sokolov@postgrespro.ru, PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-07-01T11:08:12Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →
  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

Attachments

Yuriy, thank you for the explanation!

 > But they are not allowed to move tuples because concurrent
 > backends allowed to read tuples from the page in exactly same moment.

Now I understand that we cannot repair fragmentation if we don’t have a 
buffer cleanup lock on the page.

Nevertheless, using the heap_page_prune_and_freeze function without 
repairing fragmentation still leads to the inconsistency problem, see below.

 > No! Because patch uses flag in WAL record to instruct "redo"-side to omit
 > fragmentation as well if needed.

Shortly, using the XLHP_REPAIR_FRAGMENTATION flag equal to `false` is 
the direct road to page inconsistency.

Explaining thoroughly, in the case of omitting fragmentation we can end 
up with an inconsistent page state on "redo"-side. Please, see the 
attachment #1, depicting further explanation. Inconsistency can occur 
because of a difference between functions heap_page_prune_and_freeze 
("do"-side) and heap_xlog_prune_freeze ("redo"-side). The difference is 
that the former function performs heap_prune_satisfies_vacuum before 
pruning, and the latter one does not perform it. It can result in the 
next situation:
DO-side:
     heap_page_prune_and_freeze =>
         1) heap_prune_satisfies_vacuum => HEAP_XMAX_COMMITTED=1 for 
some tuples
         2) heap_page_prune_execute => prune some tuples
         3) Do not repair fragmentation
Result: garbage on the page comprising of pruned tuples having 
HEAP_XMAX_COMMITTED=1

REDO-side:
     heap_xlog_prune_freeze =>
         2) heap_page_prune_execute => prune tuple/tuples
         3) Do not repair fragmentation
Result: garbage with pruned tuples having HEAP_XMAX_COMMITTED=0

And that difference in the HEAP_XMAX_COMMITED is the inconsistency, even 
when it is in garbage on a page. And, probably, it is not the only 
example of the problem. I discovered this situation at xid64v58 since I 
had invoked heap_page_prune_and_freeze(repairFragmentation=FALSE)` from 
everywhere I used it.

In regard to the xid64v61-v63, it invokes 
`heap_page_prune_and_freeze(repairFragmentation=TRUE)` from all places 
of code. But the patch still invokes 
`heap_page_prune_and_freeze(repairFragmentation=FALSE)` from the 
`freeze_single_heap_page`. And we can potentially lead a page to 
inconsistency here. If we cannot, please, tell us why we cannot.

Unfortunately, I have not made a test revealing this problem.