Horton

Re: Patch: Write Amplification Reduction Method (WARM)

Alvaro Herrera <alvherre@2ndquadrant.com>

View thread View on postgr.es
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Pavan Deolasee <pavan.deolasee@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Bruce Momjian <bruce@momjian.us>, Jaime Casanova <jaime.casanova@2ndquadrant.com>, Haribabu Kommi <kommi.haribabu@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-03-14T19:19:36Z
Lists: pgsql-hackers 
Pavan Deolasee wrote:

> BTW I wanted to share some more numbers from a recent performance test. I
> thought it's important because the latest patch has fully functional chain
> conversion code as well as all WAL-logging related pieces are in place
> too. I ran these tests on a box borrowed from Tomas (thanks!).  This has
> 64GB RAM and 350GB SSD with 1GB on-board RAM. I used the same test setup
> that I used for the first test results reported on this thread i.e. a
> modified pgbench_accounts table with additional columns and additional
> indexes (one index on abalance so that every UPDATE is a potential WARM
> update).
> 
> In a test where table + indexes exceeds RAM, running for 8hrs and
> auto-vacuum parameters set such that we get 2-3 autovacuums on the table
> during the test, we see WARM delivering more than 100% TPS as compared to
> master. In this graph, I've plotted a moving average of TPS and the spikes
> that we see coincides with the checkpoints (checkpoint_timeout is set to
> 20mins and max_wal_size large enough to avoid any xlog-based checkpoints).
> The spikes are more prominent on WARM but I guess that's purely because it
> delivers much higher TPS. I haven't shown here but I see WARM updates close
> to 65-70% of the total updates. Also there is significant reduction in WAL
> generated per txn.

Impressive results.  Labels on axes would improve readability of the chart :-)

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Implement SortSupport for macaddr data type

  2. Simplify check of modified attributes in heap_update

  3. Remove direct uses of ItemPointer.{ip_blkid,ip_posid}

  4. Fix CatalogTupleInsert/Update abstraction for case of shared indstate.

  5. Provide CatalogTupleDelete() as a wrapper around simple_heap_delete().

  6. Band-aid fix for incorrect use of view options as StdRdOptions.

  7. Update visibility map in the second phase of vacuum.

  8. Avoid having two copies of the HOT-chain search logic.

  9. Postgres95 1.01 Distribution - Virgin Sources