Horton

Re: Parallel Append implementation

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Amit Khandekar <amitdkhan.pg@gmail.com>, Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-04-04T12:01:32Z
Lists: pgsql-hackers 
On Tue, Apr 4, 2017 at 12:47 AM, Andres Freund <andres@anarazel.de> wrote:
> I don't think the parallel seqscan is comparable in complexity with the
> parallel append case.  Each worker there does the same kind of work, and
> if one of them is behind, it'll just do less.  But correct sizing will
> be more important with parallel-append, because with non-partial
> subplans the work is absolutely *not* uniform.

Sure, that's a problem, but I think it's still absolutely necessary to
ramp up the maximum "effort" (in terms of number of workers)
logarithmically.  If you just do it by costing, the winning number of
workers will always be the largest number that we think we'll be able
to put to use - e.g. with 100 branches of relatively equal cost we'll
pick 100 workers.  That's not remotely sane.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Update parallel.sgml for Parallel Append

  2. Support Parallel Append plan nodes.

  3. Remove BufFile's isTemp flag.

  4. Improve comments for parallel executor estimation functions.

  5. Separate reinitialization of shared parallel-scan state from ExecReScan.

  6. Eat XIDs more efficiently in recovery TAP test.

  7. Avoid syntax error on platforms that have neither LOCALE_T nor ICU.

  8. Preparatory refactoring for parallel merge join support.