Re: COPY FROM WHEN condition
Tomas Vondra <tomas.vondra@2ndquadrant.com>
From: Tomas Vondra <tomas.vondra@2ndquadrant.com>
To: David Rowley <david.rowley@2ndquadrant.com>
Cc: Andres Freund <andres@anarazel.de>,
Surafel Temesgen <surafel3000@gmail.com>,
Alvaro Herrera <alvherre@2ndquadrant.com>, Adam Berlin
<berlin.ab@gmail.com>,
PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-01-29T14:25:59Z
Lists: pgsql-hackers
On 1/29/19 8:18 AM, David Rowley wrote:
> ...
> Here are my performance tests of with and without your change to the
> memory contexts (I missed where you posted your results).
>
> $ cat bench.pl
> for (my $i=0; $i < 8912891; $i++) {
> print "1\n1\n2\n2\n";
> }
> 36a1281f86c: (with your change)
>
> postgres=# copy listp from program $$perl ~/bench.pl$$ delimiter '|';
> COPY 35651564
> Time: 26825.142 ms (00:26.825)
> Time: 27202.117 ms (00:27.202)
> Time: 26266.705 ms (00:26.267)
>
> 4be058fe9ec: (before your change)
>
> postgres=# copy listp from program $$perl ~/bench.pl$$ delimiter '|';
> COPY 35651564
> Time: 25645.460 ms (00:25.645)
> Time: 25698.193 ms (00:25.698)
> Time: 25737.251 ms (00:25.737)
>
How do I reproduce this? I don't see this test in the thread you linked,
so I'm not sure how many partitions you were using, what's the structure
of the table etc.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Commits
-
Remove unused struct member, enforce multi_insert callback presence.
- 57a7a3adfe4e 12.0 landed
-
Separate per-batch and per-tuple memory contexts in COPY
- 36a1281f86c0 12.0 landed
-
Fix handling of volatile expressions in COPY FROM ... WHERE
- 4a8283d0ec5a 12.0 landed
-
Allow COPY FROM to filter data using WHERE conditions
- 31f3817402da 12.0 landed
-
Remove obsolete netbsd dynloader code
- b68ff3ea672c 12.0 cited