Re: Emitting JSON to file using COPY TO

jian he <jian.universality@gmail.com>

From: jian he <jian.universality@gmail.com>
To: Andrew Dunstan <andrew@dunslane.net>
Cc: Joe Conway <mail@joeconway.com>, Junwang Zhao <zhjwpku@gmail.com>, Florents Tselai <florents.tselai@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>, Dean Rasheed <dean.a.rasheed@gmail.com>, Daniel Verite <daniel@manitou-mail.org>, Davin Shearer <davin@apache.org>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2026-03-09T03:48:26Z
Lists: pgsql-hackers, pgsql-general

Attachments

On Mon, Mar 9, 2026 at 3:44 AM Andrew Dunstan <andrew@dunslane.net> wrote:
>
> Hmm. But should we be scribbling on slot->tts_tupleDescriptor like that?
> How about something like this?:
>
> -        * Full table or query without column list.  Ensure the slot uses
> -        * cstate->tupDesc so that the datum is stamped with the right type;
> -        * for queries output type is RECORDOID this must be the blessed
> -        * descriptor so that composite_to_json can look it up via
> -        * lookup_rowtype_tupdesc.
> +        * Full table or query without column list.  For queries, the slot's
> +        * TupleDesc may carry RECORDOID, which is not registered in the
> type
> +        * cache and would cause composite_to_json's lookup_rowtype_tupdesc
> +        * call to fail.  Build a HeapTuple stamped with the blessed
> +        * descriptor so the type can be looked up correctly.
>           */
>          if (!cstate->rel && slot->tts_tupleDescriptor->tdtypeid ==
> RECORDOID)
> -           slot->tts_tupleDescriptor = cstate->queryDesc->tupDesc;
> +       {
> +           HeapTuple   tup;
>
> -       rowdata = ExecFetchSlotHeapTupleDatum(slot);
> +           tup = heap_form_tuple(cstate->tupDesc,
> +                                 slot->tts_values,
> +                                 slot->tts_isnull);
> +           rowdata = HeapTupleGetDatum(tup);
> +       }
> +       else
> +       {
> +           rowdata = ExecFetchSlotHeapTupleDatum(slot);
> +       }
>
This is better. I've tried to get rid of json_projvalues and json_projnulls.
Just using heap_form_tuple, but it won't work.

I incorporated the v28-0004 COPY column list into v9-0002.
With this patch set, we added four fields to the struct CopyToStateData.

+    StringInfo    json_buf;        /* reusable buffer for JSON output,
+                                 * initialized in BeginCopyTo */
+    TupleDesc    tupDesc;        /* Descriptor for JSON output; for a column
+                                 * list this is a projected descriptor */
+    Datum       *json_projvalues;    /* pre-allocated projection values, or
+                                     * NULL */
+    bool       *json_projnulls; /* pre-allocated projection nulls, or NULL */

Using the script in
https://www.postgresql.org/message-id/CACJufxFFZqxC3p4WjpTEi4riaJm%3DpADX%2Bpy0yQ0%3DRWTn5cqK3Q%40mail.gmail.com
I tested it again on macOS and Linux, and there are no regressions for
COPY TO with the TEXT and CSV formats.



--
jian
https://www.enterprisedb.com/

Commits

  1. Add option force_array for COPY JSON FORMAT

  2. json format for COPY TO

  3. introduce CopyFormat, refactor CopyFormatOptions

  4. Doc: add IDs to copy.sgml's <varlistentry> and <refsect1>

  5. Refactor COPY TO to use format callback functions.