Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Peter Geoghegan <pg@bowt.ie>

View thread
From: Peter Geoghegan <pg@bowt.ie>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Chris Travers <chris@orioledata.com>, Aleksander Alekseev <aleksander@timescale.com>, pgsql-hackers@lists.postgresql.org, Chris Travers <chris.travers@gmail.com>, Fedor Sigaev <teodor@sigaev.ru>, Alexander Korotkov <aekorotkov@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Nikita Glukhov <n.gluhov@postgrespro.ru>, Yura Sokolov <y.sokolov@postgrespro.ru>, Maxim Orlov <orlovmg@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>
Date: 2022-11-28T23:15:01Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

On Mon, Nov 28, 2022 at 1:52 PM Peter Geoghegan <pg@bowt.ie> wrote:
> I'm not claiming to know how to build this "better xidStopLimit
> mechanism", by the way. I'm not seriously proposing it. Mostly I'm
> just saying that the question "where do you draw the line if not at 2
> billion XIDs?" is a very pertinent question. It is not a question that
> is made any less valid by the fact that we already know that 2 billion
> XIDs is pretty far from optimal in almost all cases. Having some limit
> based on something is likely to be more effective than having no limit
> based on nothing.
>
> Admittedly this argument works a lot better with the failsafe than it
> does with xidStopLimit. Both are removed by the patch.

Come to think of it, if you were going to do something like this it
would probably work by throttling XID allocations, with a gradual
ramp-up. It would rarely get to the point that the system refused to
allocate XIDs completely.

It's not fundamentally unreasonable to force the application to live
within its means by throttling. Feedback that slows down the rate of
writes is much more common in the LSM tree world, within systems like
MyRocks [1], where the risk of the system being destabilized by debt
is more pressing.

As I said, I don't think that this is a particularly promising way of
addressing problems with Postgres XID space exhaustion, since I
believe that the underlying issue isn't usually a simple lack of "XID
space slack capacity". But if you assume that I'm wrong here (if you
assume that we very often don't have the ability to freeze lazily
enough), then ISTM that throttling or feedback to stall new writes is
a very reasonable option. In fact, it's practically mandatory.

[1] https://docs.google.com/presentation/d/1WgP-SlKay5AnSoVDSvOIzmu7edMmtYhdywoa0oAR4JQ/edit#slide=id.g8839c9d71b_0_79
-- 
Peter Geoghegan