Horton

Re: Speed up Clog Access by increasing CLOG buffers

Tomas Vondra <tomas.vondra@2ndquadrant.com>

View thread View on postgr.es
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Dilip Kumar <dilipbalaut@gmail.com>, Andres Freund <andres@anarazel.de>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2016-09-29T01:10:30Z
Lists: pgsql-hackers 
On 09/29/2016 01:59 AM, Robert Haas wrote:
> On Wed, Sep 28, 2016 at 6:45 PM, Tomas Vondra
> <tomas.vondra@2ndquadrant.com> wrote:
>> So, is 300 too little? I don't think so, because Dilip saw some benefit from
>> that. Or what scale factor do we think is needed to reproduce the benefit?
>> My machine has 256GB of ram, so I can easily go up to 15000 and still keep
>> everything in RAM. But is it worth it?
>
> Dunno. But it might be worth a test or two at, say, 5000, just to
> see if that makes any difference.
>

OK, I have some benchmarks to run on that machine, but I'll do a few 
tests with scale 5000 - probably sometime next week. I don't think the 
delay matters very much, as it's clear the patch will end up with RwF in 
this CF round.

> I feel like we must be missing something here.  If Dilip is seeing
> huge speedups and you're seeing nothing, something is different, and
> we don't know what it is.  Even if the test case is artificial, it
> ought to be the same when one of you runs it as when the other runs
> it.  Right?
>

Yes, definitely - we're missing something important, I think. One 
difference is that Dilip is using longer runs, but I don't think that's 
a problem (as I demonstrated how stable the results are).

I wonder what CPU model is Dilip using - I know it's x86, but not which 
generation it is. I'm using E5-4620 v1 Xeon, perhaps Dilip is using a 
newer model and it makes a difference (although that seems unlikely).

regards

-- 
Tomas Vondra                  http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Use group updates when setting transaction status in clog.

  2. Improve 64bit atomics support.

  3. Add ProcArrayGroupUpdate wait event.

  4. Make the different Unix-y semaphore implementations ABI-compatible.

  5. Fix broken ALTER INDEX documentation

  6. Code and docs review for commit 3187d6de0e5a9e805b27c48437897e8c39071d45.

  7. Partition the freelist for shared dynahash tables.

  8. Correct StartupSUBTRANS for page wraparound

  9. Make idle backends exit if the postmaster dies.

  10. contrib/sslinfo: add ssl_extension_info SRF

  11. Reduce ProcArrayLock contention by removing backends in batches.

  12. Fix `make installcheck` for serializable transactions.

  13. Lockless StrategyGetBuffer clock sweep hot path.

  14. Reduce sinval synchronization overhead.