Horton

Re: Fdw batch insert error out when set batch_size > 65535

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Cc: Tomas Vondra <tomas.vondra@enterprisedb.com>, "houzj.fnst@fujitsu.com" <houzj.fnst@fujitsu.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2021-06-09T06:05:46Z
Lists: pgsql-hackers 
Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> writes:
>> I've added a simple regression test to postgres_fdw, testing that batch
>> sizes > 65535 work fine, and pushed the fix.

> I was earlier thinking of adding one, but stopped because it might
> increase the regression test execution time. It looks like that's true
> - with and without the test case it takes 17 sec and 4 sec
> respectively on my dev system which is 4X slower. I'm not sure if this
> is okay.

The cost, versus the odds of ever detecting a problem, doesn't
seem like a good tradeoff.

			regards, tom lane

Commits

  1. Remove expensive test of postgres_fdw batch inserts

  2. Adjust batch size in postgres_fdw to not use too many parameters

  3. Implement support for bulk inserts in postgres_fdw