Horton

Re: Add 64-bit XIDs into PostgreSQL 15

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Chris Travers <chris@orioledata.com>
Cc: Aleksander Alekseev <aleksander@timescale.com>, pgsql-hackers@lists.postgresql.org, Chris Travers <chris.travers@gmail.com>, Peter Geoghegan <pg@bowt.ie>, Fedor Sigaev <teodor@sigaev.ru>, Alexander Korotkov <aekorotkov@gmail.com>, Konstantin Knizhnik <knizhnik@garret.ru>, Nikita Glukhov <n.gluhov@postgrespro.ru>, Yura Sokolov <y.sokolov@postgrespro.ru>, Maxim Orlov <orlovmg@gmail.com>, Pavel Borisov <pashkin.elfe@gmail.com>, Simon Riggs <simon.riggs@enterprisedb.com>
Date: 2022-11-29T16:41:07Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add SLRU tests for 64-bit page case

  2. Make use FullTransactionId in 2PC filenames

  3. Use larger segment file names for pg_notify

  4. Index SLRUs by 64-bit integers rather than by 32-bit integers

On Tue, Nov 29, 2022 at 8:03 AM Chris Travers <chris@orioledata.com> wrote:
> To be clear, I never suggested shutting down the database.  What I have suggested is that repurposing the current approaching-xid-wraparound warnings to start complaining loudly when a threshold is exceeded would be helpful.   I think it makes sense to make that threshold configurable especially if we eventually have people running bloat-free table structures.
>
> There are two fundamental problems here.  The first is that if, as you say, a table is progressively bloating and we are getting further and further behind on vacuuming and freezing, something is seriously wrong and we need to do something about it.  In these cases, I my experience is that vacuuming and related tools tend to suffer degraded performance, and determining how to solve the problem takes quite a bit more time than a routine bloat issue would.  So what I am arguing against is treating the problem just as a bloat issue.  If you get there due to vacuum being slow, something else is wrong and you are probably going to have to find and fix that as well in order to catch up.  At least that's my experience.
>
> I don't object to the db continuing to run, allocate xids etc.  What I object to is it doing so in silently where things are almost certainly going very wrong.

OK. My feeling is that the set of things we can do to warn the user is
somewhat limited. I'm open to trying our best, but we need to have
reasonable expectations. Sophisticated users will be monitoring for
problems even if we do nothing to warn, and dumb ones won't look at
their logs. Any feature that proposes to warn must aim at the uses who
are smart enough to check the logs but dumb enough not to have any
more sophisticated monitoring. Such users certainly exist and are not
even uncommon, but they aren't the only kind by a long shot.

My argument is that removing xidStopLimit is totally fine, because it
only serves to stop the database. What to do about xidWarnLimit is a
slightly more complex question. Certainly it can't be left untouched,
because warning that we're about to shut down the database for lack of
allocatable XIDs is not sensible if there is no such lack and we
aren't going to shut it down. But I'm also not sure if the model is
right. Doing nothing for a long time and then warning in every
transaction when some threshold is crossed is an extreme behavior
change. Right now that's somewhat justified because we're about to hit
a brick wall at full speed, but if we remove the brick wall and
replace it with a gentle pelting with rotten eggs, it's unclear that a
similarly strenuous reaction is the right model. But that's also not
to say that we should do nothing at all.

-- 
Robert Haas
EDB: http://www.enterprisedb.com