Horton

Re: Parallel copy

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: vignesh C <vignesh21@gmail.com>
Cc: Ashutosh Sharma <ashu.coek88@gmail.com>, Greg Nancarrow <gregn4422@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: 2020-09-28T09:31:21Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow WaitLatch() to be used without a latch.

  2. Add %P to log_line_prefix for parallel group leader

  3. Include replication origins in SQL functions for commit timestamp

  4. Avoid useless buffer allocations during binary COPY FROM.

On Tue, Sep 22, 2020 at 2:44 PM vignesh C <vignesh21@gmail.com> wrote:
>
> Thanks Ashutosh for your comments.
>
> On Wed, Sep 16, 2020 at 6:36 PM Ashutosh Sharma <ashu.coek88@gmail.com> wrote:
> >
> > Hi Vignesh,
> >
> > I've spent some time today looking at your new set of patches and I've
> > some thoughts and queries which I would like to put here:
> >
> > Why are these not part of the shared cstate structure?
> >
> >     SerializeString(pcxt, PARALLEL_COPY_KEY_NULL_PRINT, cstate->null_print);
> >     SerializeString(pcxt, PARALLEL_COPY_KEY_DELIM, cstate->delim);
> >     SerializeString(pcxt, PARALLEL_COPY_KEY_QUOTE, cstate->quote);
> >     SerializeString(pcxt, PARALLEL_COPY_KEY_ESCAPE, cstate->escape);
> >
>
> I have used shared_cstate mainly to share the integer & bool data
> types from the leader to worker process. The above data types are of
> char* data type, I will not be able to use it like how I could do it
> for integer type. So I preferred to send these as separate keys to the
> worker. Thoughts?
>

I think the way you have written will work but if we go with
Ashutosh's proposal it will look elegant and in the future, if we need
to share more strings as part of cstate structure then that would be
easier. You can probably refer to EstimateParamListSpace,
SerializeParamList, and RestoreParamList to see how we can share
different types of data in one key.

-- 
With Regards,
Amit Kapila.