Horton

Re: Removing more vacuumlazy.c special cases, relfrozenxid optimizations

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Peter Geoghegan <pg@bowt.ie>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-02-18T20:54:19Z
Lists: pgsql-hackers 
On Fri, Feb 18, 2022 at 3:41 PM Peter Geoghegan <pg@bowt.ie> wrote:
> Concerns about my general approach to this project (and even the
> Postgres 14 VACUUM work) were expressed by Robert and Andres over on
> the "Nonrandom scanned_pages distorts pg_class.reltuples set by
> VACUUM" thread. Some of what was said honestly shocked me. It now
> seems unwise to pursue this project on my original timeline. I even
> thought about shelving it indefinitely (which is still on the table).
>
> I propose the following compromise: the least contentious patch alone
> will be in scope for Postgres 15, while the other patches will not be.
> I'm referring to the first patch from v8, which adds dynamic tracking
> of the oldest extant XID in each heap table, in order to be able to
> use it as our new relfrozenxid. I can't imagine that I'll have
> difficulty convincing Andres of the merits of this idea, for one,
> since it was his idea in the first place. It makes a lot of sense,
> independent of any change to how and when we freeze.
>
> The first patch is tricky, but at least it won't require elaborate
> performance validation. It doesn't change any of the basic performance
> characteristics of VACUUM. It sometimes allows us to advance
> relfrozenxid to a value beyond FreezeLimit (typically only possible in
> an aggressive VACUUM), which is an intrinsic good. If it isn't
> effective then the overhead seems very unlikely to be noticeable. It's
> pretty much a strictly additive improvement.
>
> Are there any objections to this plan?

I really like the idea of reducing the scope of what is being changed
here, and I agree that eagerly advancing relfrozenxid carries much
less risk than the other changes.

I'd like to have a clearer idea of exactly what is in each of the
remaining patches before forming a final opinion.

What's tricky about 0001? Does it change any other behavior, either as
a necessary component of advancing relfrozenxid more eagerly, or
otherwise?

If there's a way you can make the precise contents of 0002 and 0003
more clear, I would like that, too.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

Commits

  1. Have VACUUM warn on relfrozenxid "in the future".

  2. vacuumlazy.c: Further consolidate resource allocation.

  3. Generalize how VACUUM skips all-frozen pages.

  4. Set relfrozenxid to oldest extant XID seen by VACUUM.

  5. Doc: Add relfrozenxid Tip to XID wraparound section.

  6. vacuumlazy.c: document vistest and OldestXmin.

  7. Increase hash_mem_multiplier default to 2.0.

  8. Consolidate VACUUM xid cutoff logic.

  9. Add VACUUM instrumentation for scanned pages, relfrozenxid.

  10. Simplify lazy_scan_heap's handling of scanned pages.

  11. Try to stabilize reloptions test, again.

  12. Unify VACUUM VERBOSE and autovacuum logging.

  13. Fix possible HOT corruption when RECENTLY_DEAD changes to DEAD while pruning.

  14. pg_resetxlog: add option to set oldest xid & use by pg_upgrade

  15. Teach VACUUM to bypass unnecessary index vacuuming.

  16. Centralize horizon determination for temp tables, fixing bug due to skew.

  17. pg_surgery: Try to stabilize regression tests.

  18. Add "split after new tuple" nbtree optimization.

  19. Fix bugs in vacuum of shared rels, by keeping their relcache entries current.

  20. Avoid useless truncation attempts during VACUUM.

  21. Only skip pages marked as clean in the visibility map, if the last 32

  22. Fix recently-understood problems with handling of XID freezing, particularly