Adjust batch size in postgres_fdw to not use too many parameters
Tomas Vondra <tomas.vondra@postgresql.org>
Adjust batch size in postgres_fdw to not use too many parameters The FE/BE protocol identifies parameters with an Int16 index, which limits the maximum number of parameters per query to 65535. With batching added to postges_fdw this limit is much easier to hit, as the whole batch is essentially a single query, making this error much easier to hit. The failures are a bit unpredictable, because it also depends on the number of columns in the query. So instead of just failing, this patch tweaks the batch_size to not exceed the maximum number of parameters. Reported-by: Hou Zhijie <houzj.fnst@cn.fujitsu.com> Reviewed-by: Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com> Discussion: https://postgr.es/m/OS0PR01MB571603973C0AC2874AD6BF2594299%40OS0PR01MB5716.jpnprd01.prod.outlook.com
Files
| Path | Change | +/− |
|---|---|---|
| contrib/postgres_fdw/expected/postgres_fdw.out | modified | +11 −0 |
| contrib/postgres_fdw/postgres_fdw.c | modified | +9 −2 |
| contrib/postgres_fdw/sql/postgres_fdw.sql | modified | +7 −0 |
| doc/src/sgml/postgres-fdw.sgml | modified | +11 −0 |
| src/interfaces/libpq/fe-exec.c | modified | +12 −9 |
| src/interfaces/libpq/libpq-fe.h | modified | +2 −0 |
Documentation touched
Discussion
- Fdw batch insert error out when set batch_size > 65535 33 messages · 2021-05-21 → 2023-07-03