Re: Emitting JSON to file using COPY TO
Andrew Dunstan <andrew@dunslane.net>
From: Andrew Dunstan <andrew@dunslane.net>
To: jian he <jian.universality@gmail.com>
Cc: Joe Conway <mail@joeconway.com>, Junwang Zhao <zhjwpku@gmail.com>,
Florents Tselai <florents.tselai@gmail.com>,
"Andrey M. Borodin" <x4mmm@yandex-team.ru>,
Dean Rasheed <dean.a.rasheed@gmail.com>,
Daniel Verite <daniel@manitou-mail.org>, Davin Shearer <davin@apache.org>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2026-03-08T19:44:51Z
Lists: pgsql-hackers, pgsql-general
On 2026-03-08 Su 12:16 PM, jian he wrote:
> hi.
>
> V27-0002 is still not bullet-proof.
>
> drop table if exists t1;
> create table t1(a int);
> insert into t1 values (1);
> copy (select * from t1) to stdout json;
> {"a":1}
> WARNING: resource was not closed: TupleDesc 0x7171d0ca3440 (18239,-1)
>
> Also see ExecAssignScanProjectionInfo->ExecConditionalAssignProjectionInfo
> So in v28-0002, I changed to
> + /*
> + * composite_to_json() requires a stable TupleDesc. Since the slot's
> + * descriptor (slot->tts_tupleDescriptor) can change during the execution
> + * of a SELECT query, we use cstate->queryDesc->tupDesc instead. This
> + * precaution is only necessary when the output slot's TupleDesc is of
> + * type RECORDOID.
> + */
> + if (!cstate->rel && slot->tts_tupleDescriptor->tdtypeid == RECORDOID)
> + slot->tts_tupleDescriptor = cstate->queryDesc->tupDesc;
Hmm. But should we be scribbling on slot->tts_tupleDescriptor like that?
How about something like this?:
- * Full table or query without column list. Ensure the slot uses
- * cstate->tupDesc so that the datum is stamped with the right type;
- * for queries output type is RECORDOID this must be the blessed
- * descriptor so that composite_to_json can look it up via
- * lookup_rowtype_tupdesc.
+ * Full table or query without column list. For queries, the slot's
+ * TupleDesc may carry RECORDOID, which is not registered in the
type
+ * cache and would cause composite_to_json's lookup_rowtype_tupdesc
+ * call to fail. Build a HeapTuple stamped with the blessed
+ * descriptor so the type can be looked up correctly.
*/
if (!cstate->rel && slot->tts_tupleDescriptor->tdtypeid ==
RECORDOID)
- slot->tts_tupleDescriptor = cstate->queryDesc->tupDesc;
+ {
+ HeapTuple tup;
- rowdata = ExecFetchSlotHeapTupleDatum(slot);
+ tup = heap_form_tuple(cstate->tupDesc,
+ slot->tts_values,
+ slot->tts_isnull);
+ rowdata = HeapTupleGetDatum(tup);
+ }
+ else
+ {
+ rowdata = ExecFetchSlotHeapTupleDatum(slot);
+ }
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Commits
-
Add option force_array for COPY JSON FORMAT
- 4c0390ac53b7 19 (unreleased) landed
-
json format for COPY TO
- 7dadd38cda95 19 (unreleased) landed
-
introduce CopyFormat, refactor CopyFormatOptions
- a2145605ee3d 19 (unreleased) landed
-
Doc: add IDs to copy.sgml's <varlistentry> and <refsect1>
- e4018f891dec 19 (unreleased) cited
-
Refactor COPY TO to use format callback functions.
- 2e4127b6d2d8 18.0 cited