Horton

Re: Parallel copy

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Robert Haas <robertmhaas@gmail.com>, Ants Aasma <ants@cybertec.at>, vignesh C <vignesh21@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alastair Turner <minion@decodable.me>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-06-04T02:40:07Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow WaitLatch() to be used without a latch.

  2. Add %P to log_line_prefix for parallel group leader

  3. Include replication origins in SQL functions for commit timestamp

  4. Avoid useless buffer allocations during binary COPY FROM.

On Thu, Jun 4, 2020 at 12:09 AM Andres Freund <andres@anarazel.de> wrote:
>
> Hi,
>
> On 2020-06-03 12:13:14 -0400, Robert Haas wrote:
> > On Mon, May 18, 2020 at 12:48 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > > In the above case, even though we are executing a single command from
> > > the user perspective, but the currentCommandId will be four after the
> > > command.  One increment will be for the copy command and the other
> > > three increments are for locking tuple in PK table
> > > (tab_fk_referenced_chk) for three tuples in FK table
> > > (tab_fk_referencing_chk).  Now, for parallel workers, it is
> > > (theoretically) possible that the three tuples are processed by three
> > > different workers which don't get synced as of now.  The question was
> > > do we see any kind of problem with this and if so can we just sync it
> > > up at the end of parallelism.
>
> > I strongly disagree with the idea of "just sync(ing) it up at the end
> > of parallelism". That seems like a completely unprincipled approach to
> > the problem. Either the command counter increment is important or it's
> > not. If it's not important, maybe we can arrange to skip it in the
> > first place. If it is important, then it's probably not OK for each
> > backend to be doing it separately.
>
> That scares me too. These command counter increments definitely aren't
> unnecessary in the general case.
>

Yeah, this is what we want to understand?  Can you explain how they
are useful here?  AFAIU, heap_lock_tuple doesn't use commandid while
storing the transaction information of xact while locking the tuple.

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com