Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Chris Travers <chris@orioledata.com>

View thread
From: Chris Travers <chris@orioledata.com>
To: Bruce Momjian <bruce@momjian.us>
Cc: Robert Haas <robertmhaas@gmail.com>, Aleksander Alekseev <aleksander@timescale.com>, pgsql-hackers@lists.postgresql.org, Chris Travers <chris.travers@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Fedor Sigaev <teodor@sigaev.ru>, Alexander Korotkov <aekorotkov@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Nikita Glukhov <n.gluhov@postgrespro.ru>, Yura Sokolov <y.sokolov@postgrespro.ru>, Maxim Orlov <orlovmg@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>
Date: 2022-11-30T02:36:44Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

On Tue, Nov 29, 2022 at 5:57 PM Bruce Momjian <bruce@momjian.us> wrote:

> On Tue, Nov 29, 2022 at 11:41:07AM -0500, Robert Haas wrote:
> > My argument is that removing xidStopLimit is totally fine, because it
> > only serves to stop the database. What to do about xidWarnLimit is a
> > slightly more complex question. Certainly it can't be left untouched,
> > because warning that we're about to shut down the database for lack of
> > allocatable XIDs is not sensible if there is no such lack and we
> > aren't going to shut it down. But I'm also not sure if the model is
> > right. Doing nothing for a long time and then warning in every
> > transaction when some threshold is crossed is an extreme behavior
> > change. Right now that's somewhat justified because we're about to hit
> > a brick wall at full speed, but if we remove the brick wall and
> > replace it with a gentle pelting with rotten eggs, it's unclear that a
> > similarly strenuous reaction is the right model. But that's also not
> > to say that we should do nothing at all.
>
> Yeah, we would probably need to warn on every 1 million transactions or
> something.
>
>
My proposal would be to make the threshold configurable and start warning
on every transaction after that.  There are a couple reasons to do that.

The first is that noisy warnings are extremely easy to see.  You get them
in cron emails, from psql, in the db logs etc.  Having them every million
makes them harder to catch.

The point here is not to ensure there are no problems, but to make sure
that an existing layer in the current swiss cheese model of safety doesn't
go away.  Will it stop all problems?  No.  But the current warning strategy
is effective, given how many times we hear of cases of people having to
take drastic action to avoid impending xid wraparound.

If someone has an insert only database and maye doesn't want to ever
freeze, they can set the threshold to -1 or something.  I would suggest
keeping the default as at 2 billion to be in line with existing limitations
and practices.  People can then adjust as they see fit.

Warning text might be something like "XID Lag Threshold Exceeded.  Is
autovacuum clearing space and keeping up?"


> --
>   Bruce Momjian  <bruce@momjian.us>        https://momjian.us
>   EDB                                      https://enterprisedb.com
>
> Embrace your flaws.  They make you human, rather than perfect,
> which you will never be.
>