Re: Parallel copy

vignesh C <vignesh21@gmail.com>

From: vignesh C <vignesh21@gmail.com>

To: Amit Kapila <amit.kapila16@gmail.com>

Cc: Greg Nancarrow <gregn4422@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, Rafia Sabih <rafia.pghackers@gmail.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>, Ants Aasma <ants@cybertec.at>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Alastair Turner <minion@decodable.me>, Thomas Munro <thomas.munro@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>

Date: 2020-10-14T09:53:57Z

Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

Allow WaitLatch() to be used without a latch.
- 733fa9aa51c5 14.0 cited
Add %P to log_line_prefix for parallel group leader
- b8fdee7d0ca8 14.0 cited
Include replication origins in SQL functions for commit timestamp
- b1e48bbe64a4 14.0 cited
Avoid useless buffer allocations during binary COPY FROM.
- cd22d3cdb9bd 14.0 cited

Attachments

v7-0001-Copy-code-readjustment-to-support-parallel-copy.patch (text/x-patch) patch v7-0001
v7-0002-Framework-for-leader-worker-in-parallel-copy.patch (text/x-patch) patch v7-0002
v7-0003-Allow-copy-from-command-to-process-data-from-file.patch (text/x-patch) patch v7-0003
v7-0004-Documentation-for-parallel-copy.patch (text/x-patch) patch v7-0004
v7-0005-Tests-for-parallel-copy.patch (text/x-patch) patch v7-0005
v7-0006-Parallel-Copy-For-Binary-Format-Files.patch (text/x-patch) patch v7-0006

On Fri, Oct 9, 2020 at 12:10 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
>
> While looking at the latest code, I observed below issue in patch
> v6-0003-Allow-copy-from-command-to-process-data-from-file:
>
> + /* Estimate the size for shared information for PARALLEL_COPY_KEY_CSTATE */
> + est_cstateshared = MAXALIGN(sizeof(SerializedParallelCopyState));
> + shm_toc_estimate_chunk(&pcxt->estimator, est_cstateshared);
> + shm_toc_estimate_keys(&pcxt->estimator, 1);
> +
> + strsize = EstimateCstateSize(pcxt, cstate, attnamelist, &whereClauseStr,
> + &rangeTableStr, &attnameListStr,
> + &notnullListStr, &nullListStr,
> + &convertListStr);
>
> Here, do we need to separately estimate the size of
> SerializedParallelCopyState when it is also done in
> EstimateCstateSize?

This is not required, this has been removed in the attached patches.

Regards,
Vignesh
EnterpriseDB: http://www.enterprisedb.com