Horton

Re: Parallel copy

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Ants Aasma <ants@cybertec.at>, vignesh C <vignesh21@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alastair Turner <minion@decodable.me>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-06-03T18:38:59Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow WaitLatch() to be used without a latch.

  2. Add %P to log_line_prefix for parallel group leader

  3. Include replication origins in SQL functions for commit timestamp

  4. Avoid useless buffer allocations during binary COPY FROM.

Hi,

On 2020-06-03 12:13:14 -0400, Robert Haas wrote:
> On Mon, May 18, 2020 at 12:48 AM Amit Kapila <amit.kapila16@gmail.com> wrote:
> > In the above case, even though we are executing a single command from
> > the user perspective, but the currentCommandId will be four after the
> > command.  One increment will be for the copy command and the other
> > three increments are for locking tuple in PK table
> > (tab_fk_referenced_chk) for three tuples in FK table
> > (tab_fk_referencing_chk).  Now, for parallel workers, it is
> > (theoretically) possible that the three tuples are processed by three
> > different workers which don't get synced as of now.  The question was
> > do we see any kind of problem with this and if so can we just sync it
> > up at the end of parallelism.

> I strongly disagree with the idea of "just sync(ing) it up at the end
> of parallelism". That seems like a completely unprincipled approach to
> the problem. Either the command counter increment is important or it's
> not. If it's not important, maybe we can arrange to skip it in the
> first place. If it is important, then it's probably not OK for each
> backend to be doing it separately.

That scares me too. These command counter increments definitely aren't
unnecessary in the general case.

Even in the example you share above, aren't we potentially going to
actually lock rows multiple times from within the same transaction,
instead of once?  If the command counter increments from within
ri_trigger.c aren't visible to other parallel workers/leader, we'll not
correctly understand that a locked row is invisible to heap_lock_tuple,
because we're not using a new enough snapshot (by dint of not having a
new enough cid).

I've not dug through everything that'd potentially cause, but it seems
pretty clearly a no-go from here.

Greetings,

Andres Freund