Horton

Re: BUG #17257: (auto)vacuum hangs within lazy_scan_prune()

Peter Geoghegan <pg@bowt.ie>

View thread
From: Peter Geoghegan <pg@bowt.ie>
To: Noah Misch <noah@leadboat.com>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, robertmhaas@gmail.com, Alexander Lakhin <exclusion@gmail.com>, PostgreSQL mailing lists <pgsql-bugs@lists.postgresql.org>, Andres Freund <andres@anarazel.de>
Date: 2024-01-10T19:57:34Z
Lists: pgsql-bugs

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Ensure vacuum removes all visibly dead tuples older than OldestXmin

  2. Combine freezing and pruning steps in VACUUM

  3. Handle non-chain tuples outside of heap_prune_chain()

  4. Fix false reports in pg_visibility

  5. Remove retry loop in heap_page_prune().

  6. vacuumlazy.c: document vistest and OldestXmin.

  7. Deduplicate choice of horizon for a relation procarray.c.

  8. Remove tupgone special case from vacuumlazy.c.

  9. Simplify state managed by VACUUM.

  10. Recycle nbtree pages deleted during same VACUUM.

  11. snapshot scalability: Don't compute global horizons while building snapshots.

  12. Raise error when affecting tuple moved into different partition.

On Wed, Jan 10, 2024 at 2:38 PM Noah Misch <noah@leadboat.com> wrote:
> I don't know.  That particular system experienced the infinite loop only once.

While I certainly can't recreate the problem on demand, it has been
seen on this same application far more than once.

> > I'm referring to calls such as the
> > "GetOldestNonRemovableTransactionId(NULL)" and
> > "GlobalVisCheckRemovableFullXid()" calls that take place inside
> > _bt_pendingfsm_finalize(). It's not like we do stuff like that in very
> > many other places.
>
> I see what you mean about the rarity and potential importance of
> "GetOldestNonRemovableTransactionId(NULL)".  There's just one other caller,
> vac_update_datfrozenxid(), which calls it for an unrelated cause.

I just noticed another detail that adds significant weight to this
theory: it looks like the problem is hit on the first tuple located on
the first heap page that VACUUM scans *after* it completes its first
round of index vacuuming (I'm inferring this from vacrel state,
particular its lpdead_items instrumentation counter). The dead_items
array is as large as possible here (just under 1 GiB), and
lpdead_items is 178956692 (which uses up all of our dead_items space).
VACUUM scans tens of gigabytes of heap pages before it begins this
initial round of index vacuuming (according to vacrel->scanned_pages).

What are the chances that all of this is just a coincidence? Low, I'd say.

-- 
Peter Geoghegan