Horton

Re: Parallel copy

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: vignesh C <vignesh21@gmail.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Rafia Sabih <rafia.pghackers@gmail.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Ants Aasma <ants@cybertec.at>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alastair Turner <minion@decodable.me>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-07-23T03:52:42Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow WaitLatch() to be used without a latch.

  2. Add %P to log_line_prefix for parallel group leader

  3. Include replication origins in SQL functions for commit timestamp

  4. Avoid useless buffer allocations during binary COPY FROM.

On Thu, Jul 23, 2020 at 8:51 AM Bharath Rupireddy <
bharath.rupireddyforpostgres@gmail.com> wrote:

> On Wed, Jul 22, 2020 at 7:56 PM vignesh C <vignesh21@gmail.com> wrote:
> >
> > Thanks for reviewing and providing the comments Ashutosh.
> > Please find my thoughts below:
> >
> > On Fri, Jul 17, 2020 at 7:18 PM Ashutosh Sharma <ashu.coek88@gmail.com>
> wrote:
> > >
> > > Some review comments (mostly) from the leader side code changes:
> > >
> > > 3) Should we allow Parallel Copy when the insert method is
> CIM_MULTI_CONDITIONAL?
> > >
> > > +   /* Check if the insertion mode is single. */
> > > +   if (FindInsertMethod(cstate) == CIM_SINGLE)
> > > +       return false;
> > >
> > > I know we have added checks in CopyFrom() to ensure that if any
> trigger (before row or instead of) is found on any of partition being
> loaded with data, then COPY FROM operation would fail, but does it mean
> that we are okay to perform parallel copy on partitioned table. Have we
> done some performance testing with the partitioned table where the data in
> the input file needs to be routed to the different partitions?
> > >
> >
> > Partition data is handled like what Amit had told in one of earlier
> mails [1].  My colleague Bharath has run performance test with partition
> table, he will be sharing the results.
> >
>
> I ran tests for partitioned use cases - results are similar to that of non
> partitioned cases[1].
>

I could see the gain up to 10-11 times for non-partitioned cases [1], can
we use similar test case here as well (with one of the indexes on text
column or having gist index) to see its impact?

[1] -
https://www.postgresql.org/message-id/CALj2ACVR4WE98Per1H7ajosW8vafN16548O2UV8bG3p4D3XnPg%40mail.gmail.com

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com