Horton

Re: Parallel Append implementation

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Ashutosh Bapat <ashutosh.bapat@enterprisedb.com>
Cc: Amit Khandekar <amitdkhan.pg@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2017-02-16T14:45:37Z
Lists: pgsql-hackers 
On Wed, Feb 15, 2017 at 11:15 PM, Ashutosh Bapat
<ashutosh.bapat@enterprisedb.com> wrote:
> If the user is ready throw 200 workers and if the subplans can use
> them to speed up the query 200 times (obviously I am exaggerating),
> why not to use those? When the user set
> max_parallel_workers_per_gather to that high a number, he meant it to
> be used by a gather, and that's what we should be doing.

The reason is because of what Amit Khandekar wrote in his email -- you
get a result with a partitioned table that is wildly inconsistent with
the result you get for an unpartitioned table.  You could equally well
argue that if the user sets max_parallel_workers_per_gather to 200,
and there's a parallel sequential scan of an 8MB table to be
performed, we ought to use all 200 workers for that.  But the planner
in fact estimates a much lesser number of workers, because using 200
workers for that task wastes a lot of resources for no real
performance benefit.  If you partition that 8MB table into 100 tables
that are each 80kB, that shouldn't radically increase the number of
workers that get used.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Update parallel.sgml for Parallel Append

  2. Support Parallel Append plan nodes.

  3. Remove BufFile's isTemp flag.

  4. Improve comments for parallel executor estimation functions.

  5. Separate reinitialization of shared parallel-scan state from ExecReScan.

  6. Eat XIDs more efficiently in recovery TAP test.

  7. Avoid syntax error on platforms that have neither LOCALE_T nor ICU.

  8. Preparatory refactoring for parallel merge join support.