Re: Patch: Write Amplification Reduction Method (WARM)

On Wed, Mar 29, 2017 at 3:42 AM, Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:

> I pushed 0002 after some makeup, since it's just cosmetic and not
> controversial.


Thanks. I think your patch of tracking interesting attributes seems ok too
after the performance issue was addressed. Even though we can still improve
that further, at least Mithun confirmed that there is no significant
regression anymore and in fact for one artificial case, patch does better
than even master.


> Here's 0003 rebased on top of it.
>
> (Also, I took out the gin and gist changes: it would be wrong to change
> that unconditionally, because the 0xFFFF pattern appears in indexes that
> would be pg_upgraded.  We need a different strategy, if we want to
> enable WARM on GiST/GIN indexes.)
>
>
Yeah, those changes would have broken pg-upgraded clusters. So looks good.
But the rebased patch throws an assertion failure.
ItemPointerGetOffsetNumberNoCheck will mask the first 3 bits and return the
rest, but since GIN continues to store ip_posid greater than
OffsetNumberMask, the masking causes  problems. May be we can teach
GinItemPointerGetOffsetNumber to fetch the flags separately and add them
back to what ItemPointerGetOffsetNumberNoCheck returns. This avoids
referencing ip_posid directly from this code.

BTW we have messed up patch names a bit here. You applied 0003 from v21 and
rebased 0004. But the rebased patch was
named 0001-Free-3-bits-of-ItemPointerData.ip_posid.patch. I'm reverting
back to the earlier used names. So rebased v22 set of patches attached.

0001_interesting_attrs_v22.patch - Alvaro's patch of simplifying attr
checks. I think this has settled down

0002_track_root_lp_v22 - We probably need to decide whether its worth
saving a bit in tuple header for additional work during WARM update of
finding root tuple.

0004_Free-3-bits-of-ItemPointerData.ip_posid_v22 - A slight update to
Alvaro's rebased version posted yesterday

0005_warm_updates_v22 - Main WARM patch. Addresses all review comments so
far and includes fixes for toasted value handling

0007_vacuum_enhancements_v22 - VACUUM enhancements to control WARM cleanup.
This now also includes changes made to memory usage. The dead tuples and
warm chains are tracked in a single work area, from two ends. When these
ends meet, we do a round of index cleanup. IMO this should give us most
optimal utilisation of available memory depending on whether we are doing
WARM cleanup or not and percentage of dead tuples and warm chains.

0006_warm_taptests_v22 - Alvaro reported lack of Makefile. It also seemed
that he wants to rename it to avoid "warm" reference. So done that, but
Alvaro is seeing hangs with the tests in his environment, so probably needs
some investigation. It works for me with IPC::Run 0.94

Thanks,
Pavan

-- 
 Pavan Deolasee                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services