Re: Emitting JSON to file using COPY TO
jian he <jian.universality@gmail.com>
From: jian he <jian.universality@gmail.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Joe Conway <mail@joeconway.com>, Junwang Zhao <zhjwpku@gmail.com>, Florents Tselai <florents.tselai@gmail.com>,
"Andrey M. Borodin" <x4mmm@yandex-team.ru>, Dean Rasheed <dean.a.rasheed@gmail.com>,
Daniel Verite <daniel@manitou-mail.org>, Davin Shearer <davin@apache.org>,
PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2026-03-09T03:48:26Z
Lists: pgsql-hackers, pgsql-general
Attachments
- v29-0002-json-format-for-COPY-TO.patch (text/x-patch) patch v29-0002
- v29-0001-introduce-CopyFormat-refactor-CopyFormatOptions.patch (text/x-patch) patch v29-0001
- v29-0003-Add-option-force_array-for-COPY-JSON-FORMAT.patch (text/x-patch) patch v29-0003
On Mon, Mar 9, 2026 at 3:44 AM Andrew Dunstan <andrew@dunslane.net> wrote:
>
> Hmm. But should we be scribbling on slot->tts_tupleDescriptor like that?
> How about something like this?:
>
> - * Full table or query without column list. Ensure the slot uses
> - * cstate->tupDesc so that the datum is stamped with the right type;
> - * for queries output type is RECORDOID this must be the blessed
> - * descriptor so that composite_to_json can look it up via
> - * lookup_rowtype_tupdesc.
> + * Full table or query without column list. For queries, the slot's
> + * TupleDesc may carry RECORDOID, which is not registered in the
> type
> + * cache and would cause composite_to_json's lookup_rowtype_tupdesc
> + * call to fail. Build a HeapTuple stamped with the blessed
> + * descriptor so the type can be looked up correctly.
> */
> if (!cstate->rel && slot->tts_tupleDescriptor->tdtypeid ==
> RECORDOID)
> - slot->tts_tupleDescriptor = cstate->queryDesc->tupDesc;
> + {
> + HeapTuple tup;
>
> - rowdata = ExecFetchSlotHeapTupleDatum(slot);
> + tup = heap_form_tuple(cstate->tupDesc,
> + slot->tts_values,
> + slot->tts_isnull);
> + rowdata = HeapTupleGetDatum(tup);
> + }
> + else
> + {
> + rowdata = ExecFetchSlotHeapTupleDatum(slot);
> + }
>
This is better. I've tried to get rid of json_projvalues and json_projnulls.
Just using heap_form_tuple, but it won't work.
I incorporated the v28-0004 COPY column list into v9-0002.
With this patch set, we added four fields to the struct CopyToStateData.
+ StringInfo json_buf; /* reusable buffer for JSON output,
+ * initialized in BeginCopyTo */
+ TupleDesc tupDesc; /* Descriptor for JSON output; for a column
+ * list this is a projected descriptor */
+ Datum *json_projvalues; /* pre-allocated projection values, or
+ * NULL */
+ bool *json_projnulls; /* pre-allocated projection nulls, or NULL */
Using the script in
https://www.postgresql.org/message-id/CACJufxFFZqxC3p4WjpTEi4riaJm%3DpADX%2Bpy0yQ0%3DRWTn5cqK3Q%40mail.gmail.com
I tested it again on macOS and Linux, and there are no regressions for
COPY TO with the TEXT and CSV formats.
--
jian
https://www.enterprisedb.com/
Commits
-
Add option force_array for COPY JSON FORMAT
- 4c0390ac53b7 19 (unreleased) landed
-
json format for COPY TO
- 7dadd38cda95 19 (unreleased) landed
-
introduce CopyFormat, refactor CopyFormatOptions
- a2145605ee3d 19 (unreleased) landed
-
Doc: add IDs to copy.sgml's <varlistentry> and <refsect1>
- e4018f891dec 19 (unreleased) cited
-
Refactor COPY TO to use format callback functions.
- 2e4127b6d2d8 18.0 cited