Horton

Re: [HACKERS] [POC] Faster processing at Gather node

Andres Freund <andres@anarazel.de>

View thread View on postgr.es
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Rafia Sabih <rafia.sabih@enterprisedb.com>, PostgreSQL Developers <pgsql-hackers@postgresql.org>
Date: 2018-02-27T21:06:38Z
Lists: pgsql-hackers 
Hi,

On 2018-02-27 16:03:17 -0500, Robert Haas wrote:
> On Wed, Feb 7, 2018 at 1:41 PM, Andres Freund <andres@anarazel.de> wrote:
> > Well, it's more than just systems like that - for 64bit atomics we
> > sometimes do fall back to spinlock based atomics on 32bit systems, even
> > if they support 32 bit atomics.
> 
> I built with -m32 on my laptop and tried "select aid, count(*) from
> pgbench_accounts group by 1 having count(*) > 1" on pgbench at scale
> factor 100 with pgbench_accounts_pkey dropped and
> max_parallel_workers_per_gather set to 10 on (a) commit
> 0b5e33f667a2042d7022da8bef31a8be5937aad1 (I know this is a little old,
> but I think it doesn't matter), (b) same plus
> shm-mq-less-spinlocks-v3, and (c) same plus shm-mq-less-spinlocks-v3
> and shm-mq-reduce-receiver-latch-set-v2.
> 
> (a) 16563.790 ms, 16625.257 ms, 16496.062 ms
> (b) 17217.051 ms, 17157.745 ms, 17225.755 ms [median to median +3.9% vs. (a)]
> (c) 15491.947 ms, 15455.840 ms, 15452.649 ms [median to median -7.0%
> vs. (a), -10.2% vs (b)]
> 
> Do you think that's a problem?  If it is, what do you think we should
> do about it?  It seems to me that it's probably OK because (1) with
> both patches we still come out ahead and (2) 32-bit systems will
> presumably continue to become rarer as time goes on, but you might
> disagree.

No, I think this is fairly reasonable. A fairly extreme usecase on a 32
bit machine regressing a bit, while gaining peformance in other case?
That works for me.


> OK, I'll try to check how feasible that would be.

cool.

Greetings,

Andres Freund

Commits

  1. C comment: fix wording about shared memory message queue

  2. shm_mq: Have the receiver set the sender's less frequently.

  3. shm_mq: Reduce spinlock usage.

  4. Remove use of byte-masking macros in record_image_cmp

  5. Remove memory leak protection from Gather and Gather Merge nodes.

  6. Avoid projecting tuples unnecessarily in Gather and Gather Merge.

  7. Tweak use of ExecContextForcesOids by Gather (Merge).

  8. Pass eflags down to parallel workers.

  9. Disable installcheck tests for test_session_hooks

  10. First-draft release notes for 10.1.

  11. Remove TupleDesc remapping logic from tqueue.c.

  12. Avoid memory leaks when a GatherMerge node is rescanned.

  13. Code review for nodeGatherMerge.c.

  14. Add a Gather Merge executor node.

  15. Push scan/join target list beneath Gather when possible.