Re: GUC for cleanup indexes threshold.

Jim Nasby <jim.nasby@bluetreble.com>

From: Jim Nasby <Jim.Nasby@BlueTreble.com>
To: Robert Haas <robertmhaas@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>
Cc: Masahiko Sawada <sawada.mshk@gmail.com>, Simon Riggs <simon@2ndquadrant.com>, "Ideriha, Takeshi" <ideriha.takeshi@jp.fujitsu.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>
Date: 2017-02-24T20:16:16Z
Lists: pgsql-hackers
On 2/24/17 11:26 AM, Robert Haas wrote:
> I think we need to come up with some set of tests to figure out what
> actually works well in practice here.  Theories are a good starting
> point, but good vacuum behavior is really important, and a patch that
> changes it ought to be backed up by at least some experimental
> evidence.

I think something else worth considering is that if we had some method 
of mapping heap TIDs back to indexes then a lot (all?) of these problems 
would go away. 10+ years ago the idea of keeping such a mapping would 
probably be untenable, but with resource forks and how much cheaper 
storage is maybe that's no longer the case.

For btree I think this could be done by keeping a second btree ordered 
by ctid that points either to index entries or even just to whole index 
pages. At ~ 20 bytes per entry, even a 1B row index would take ~20GB.

Page splits are obviously a big issue. Maybe it's safe to update the 
ctid map for every item that gets moved when a split happens.

Would a ctid map work for other indexes as well?
-- 
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)


Commits

  1. Fix upper limit for vacuum_cleanup_index_scale_factor

  2. Increase upper limit for vacuum_cleanup_index_scale_factor

  3. Fixes for vacuum_cleanup_index_scale_factor GUC option

  4. Skip full index scan during cleanup of B-tree indexes when possible