Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Finnerty, Jim <jfinnert@amazon.com>

View thread
From: "Finnerty, Jim" <jfinnert@amazon.com>
To: Simon Riggs <simon.riggs@enterprisedb.com>
Cc: Maxim Orlov <orlovmg@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2022-01-06T16:20:10Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

Re:    Most software has a one-stage upgrade model. What you propose would
    have us install 2 things, with a step in-between, which makes it
    harder to manage.

The intended benefit would be that the code doesn't need to handle the possibility of 2 different XID representations for the indefinite future.  

I agree that VACUUM would be the preferred tool to make room for the special data area so that there is no need to install a separate tool, though, whether this work happens before or after the upgrade. 

Re: 1. Upgrade, with important aspect not-enabled-yet, but everything else working - all required software is delivered in one shot, with fast upgrade

Let's clarify what happens during upgrade.  What format are the pages in immediately after the upgrade? 

    2. As each table is VACUUMed, we confirm/clean/groom data blocks so
    each table is individually confirmed as being ready. The pace that
    this happens at is under user control.

What are VACUUM's new responsibilities in this phase?  VACUUM needs a new task that confirms when there exists no heap page for a table that is not ready.

If upgrade put all the pages into either double-xmax or double-epoch representation, then VACUUM's responsibility could be to split the double-xmax pages into the double-epoch representation and verify when there exists no double-xmax pages.

    3. When all tables have been prepared, then restart to allow xid64 format usage

Let's also clarify what happens at restart time.

If we were to do the upgrade before preparing space in advance, is there a way to ever remove the code that knows about the double-xmax XID representation?