Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Finnerty, Jim <jfinnert@amazon.com>

View thread
From: "Finnerty, Jim" <jfinnert@amazon.com>
To: Stephen Frost <sfrost@snowman.net>, Maxim Orlov <orlovmg@gmail.com>
Cc: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-04T22:22:50Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers


On 1/4/22, 2:35 PM, "Stephen Frost" <sfrost@snowman.net> wrote:

>>
>>    Not saying that I've got any idea how to fix that case offhand, and we
>>    don't really support such a thing today as the server would just stop
>>    instead, ...
>>   Perhaps that's a
>>   worthwhile tradeoff for being able to generally avoid having to vacuum
>>   and deal with transaction wrap-around, but I have to wonder if there
>>   might be a better answer.  
>>

For the target use cases that PostgreSQL is designed for, it's a very worthwhile tradeoff in my opinion.  Such long-running transactions need to be killed.

Re: -- If after upgrade page has no free space for special data, tuples are
   converted to "double xmax" format: xmin became virtual
   FrozenTransactionId, xmax occupies the whole 64bit.
   Page converted to new format when vacuum frees enough space.

I'm concerned about the maintainability impact of having 2 new on-disk page formats.  It's already complex enough with XIDs and multixact-XIDs.

If the lack of space for the two epochs in the special data area is a problem only in an upgrade scenario, why not resolve the problem before completing the upgrade process like a kind of post-process pg_repack operation that converts all "double xmax" pages to the "double-epoch" page format?  i.e. maybe the "double xmax" representation is needed as an intermediate representation during upgrade, but after upgrade completes successfully there are no pages with the "double-xmax" representation.  This would eliminate a whole class of coding errors and would make the code dealing with 64-bit XIDs simpler and more maintainable.

    /Jim