Horton

Re: Parallel copy

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Ants Aasma <ants@cybertec.at>, vignesh C <vignesh21@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alastair Turner <minion@decodable.me>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2020-05-12T05:00:47Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow WaitLatch() to be used without a latch.

  2. Add %P to log_line_prefix for parallel group leader

  3. Include replication origins in SQL functions for commit timestamp

  4. Avoid useless buffer allocations during binary COPY FROM.

On Mon, May 11, 2020 at 11:52 PM Robert Haas <robertmhaas@gmail.com> wrote:
>
> > Case-3:
> > In copy command, for performing foreign key checks, we take KEY SHARE
> > lock on primary key table rows which inturn will increment the command
> > counter and updates the snapshot.  Now, as we share the snapshots at
> > the beginning of the command, we can't allow it to be changed later.
> > So, unless we do something special for it, I think we can't allow
> > parallelism in such cases.
>
> This sounds like much more of a problem to me; it'd be a significant
> restriction that would kick in routine cases where the user isn't
> doing anything particularly exciting. The command counter presumably
> only needs to be updated once per command, so maybe we could do that
> before we start parallelism. However, I think we would need to have
> some kind of dynamic memory structure to which new combo CIDs can be
> added by any member of the group, and then discovered by other members
> of the group later. At the end of the parallel operation, the leader
> must discover any combo CIDs added by others to that table before
> destroying it, even if it has no immediate use for the information. We
> can't allow a situation where the group members have inconsistent
> notions of which combo CIDs exist or what their mappings are, and if
> KEY SHARE locks are being taken, new combo CIDs could be created.
>

AFAIU, we don't generate combo CIDs for this case.  See below code in
heap_lock_tuple():

/*
* Store transaction information of xact locking the tuple.
*
* Note: Cmax is meaningless in this context, so don't set it; this avoids
* possibly generating a useless combo CID.  Moreover, if we're locking a
* previously updated tuple, it's important to preserve the Cmax.
*
* Also reset the HOT UPDATE bit, but only if there's no update; otherwise
* we would break the HOT chain.
*/
tuple->t_data->t_infomask &= ~HEAP_XMAX_BITS;
tuple->t_data->t_infomask2 &= ~HEAP_KEYS_UPDATED;
tuple->t_data->t_infomask |= new_infomask;
tuple->t_data->t_infomask2 |= new_infomask2;

I don't understand why we need to do something special for combo CIDs
if they are not generated during this operation?

-- 
With Regards,
Amit Kapila.
EnterpriseDB: http://www.enterprisedb.com