Re: Add 64-bit XIDs into PostgreSQL 15
Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>
From: Evgeny Voropaev <evgeny.voropaev@tantorlabs.com>
To: y.sokolov@postgrespro.ru,
PostgreSQL Hackers <pgsql-hackers@postgresql.org>
Date: 2025-07-01T11:08:12Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Add SLRU tests for 64-bit page case
- a60b8a58f435 17.0 landed
-
Make use FullTransactionId in 2PC filenames
- 5a1dfde8334b 17.0 landed
-
Use larger segment file names for pg_notify
- 2cdf131c46e6 17.0 landed
-
Index SLRUs by 64-bit integers rather than by 32-bit integers
- 4ed8f0913bfd 17.0 landed
Attachments
Yuriy, thank you for the explanation!
> But they are not allowed to move tuples because concurrent
> backends allowed to read tuples from the page in exactly same moment.
Now I understand that we cannot repair fragmentation if we don’t have a
buffer cleanup lock on the page.
Nevertheless, using the heap_page_prune_and_freeze function without
repairing fragmentation still leads to the inconsistency problem, see below.
> No! Because patch uses flag in WAL record to instruct "redo"-side to omit
> fragmentation as well if needed.
Shortly, using the XLHP_REPAIR_FRAGMENTATION flag equal to `false` is
the direct road to page inconsistency.
Explaining thoroughly, in the case of omitting fragmentation we can end
up with an inconsistent page state on "redo"-side. Please, see the
attachment #1, depicting further explanation. Inconsistency can occur
because of a difference between functions heap_page_prune_and_freeze
("do"-side) and heap_xlog_prune_freeze ("redo"-side). The difference is
that the former function performs heap_prune_satisfies_vacuum before
pruning, and the latter one does not perform it. It can result in the
next situation:
DO-side:
heap_page_prune_and_freeze =>
1) heap_prune_satisfies_vacuum => HEAP_XMAX_COMMITTED=1 for
some tuples
2) heap_page_prune_execute => prune some tuples
3) Do not repair fragmentation
Result: garbage on the page comprising of pruned tuples having
HEAP_XMAX_COMMITTED=1
REDO-side:
heap_xlog_prune_freeze =>
2) heap_page_prune_execute => prune tuple/tuples
3) Do not repair fragmentation
Result: garbage with pruned tuples having HEAP_XMAX_COMMITTED=0
And that difference in the HEAP_XMAX_COMMITED is the inconsistency, even
when it is in garbage on a page. And, probably, it is not the only
example of the problem. I discovered this situation at xid64v58 since I
had invoked heap_page_prune_and_freeze(repairFragmentation=FALSE)` from
everywhere I used it.
In regard to the xid64v61-v63, it invokes
`heap_page_prune_and_freeze(repairFragmentation=TRUE)` from all places
of code. But the patch still invokes
`heap_page_prune_and_freeze(repairFragmentation=FALSE)` from the
`freeze_single_heap_page`. And we can potentially lead a page to
inconsistency here. If we cannot, please, tell us why we cannot.
Unfortunately, I have not made a test revealing this problem.