Horton

Re: Allow logical replication to copy tables in binary format

Amit Kapila <amit.kapila16@gmail.com>

View thread
From: Amit Kapila <amit.kapila16@gmail.com>
To: "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>
Cc: Melih Mutlu <m.melihmutlu@gmail.com>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>, "Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>, Euler Taveira <euler@eulerto.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-02-27T09:01:37Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow logical replication to copy tables in binary format.

  2. Common function for percent placeholder replacement

  3. Doc: add XML ID attributes to <sectN> and <varlistentry> tags.

  4. Re-order disable_on_error in tab-complete.

  5. Make subscription tests pass with log_error_verbosity=verbose

  6. Weaken type-OID-matching checks in array_recv and record_recv.

  7. Allow logical replication to transfer data in binary format.

On Mon, Feb 20, 2023 at 3:37 PM shiy.fnst@fujitsu.com
<shiy.fnst@fujitsu.com> wrote:
>
> On Thu, Feb 16, 2023 8:48 PM Amit Kapila <amit.kapila16@gmail.com> wrote:
> >
> > So, doesn't this mean that there is no separate failure mode during
> > the initial copy? I am clarifying this to see if the patch really
> > needs a separate copy_format option for initial sync?
> >
>
> In the case that the data type doesn't have binary output function, for apply
> phase, the column will be sent in text format (see logicalrep_write_tuple()) and
> it works fine. But with copy_format = binary, the walsender exits with an
> error.
>
...
...
>
> Then I got the following error in the publisher log.
>
> walsender ERROR:  no binary output function available for type public.myvarchar
> walsender STATEMENT:  COPY public.tbl1 (a) TO STDOUT  WITH (FORMAT binary)
>

Thanks for sharing the example. I think to address this user can
create a SUBSCRIPTION with 'binary = false' and then after the initial
copy enables it with ALTER SUBSCRIPTION.  Personally, I feel it is not
required to have a separate option to allow copy in binary mode. Note,
where there is some use for it but having more options for similar
work is also confusing as users need to pay attention to different
options and their values. It won't be difficult to add such an option
in the future if we see such cases and or users specifically require
something like this.

-- 
With Regards,
Amit Kapila.