Horton

Re: Allow logical replication to copy tables in binary format

Jelte Fennema-Nio <postgres@jeltef.nl>

View thread
From: Jelte Fennema <postgres@jeltef.nl>
To: "Hayato Kuroda (Fujitsu)" <kuroda.hayato@fujitsu.com>
Cc: Melih Mutlu <m.melihmutlu@gmail.com>, "Takamichi Osumi (Fujitsu)" <osumi.takamichi@fujitsu.com>, "shiy.fnst@fujitsu.com" <shiy.fnst@fujitsu.com>, Euler Taveira <euler@eulerto.com>, Amit Kapila <amit.kapila16@gmail.com>, pgsql-hackers <pgsql-hackers@postgresql.org>, Bharath Rupireddy <bharath.rupireddyforpostgres@gmail.com>
Date: 2023-02-22T10:43:23Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Allow logical replication to copy tables in binary format.

  2. Common function for percent placeholder replacement

  3. Doc: add XML ID attributes to <sectN> and <varlistentry> tags.

  4. Re-order disable_on_error in tab-complete.

  5. Make subscription tests pass with log_error_verbosity=verbose

  6. Weaken type-OID-matching checks in array_recv and record_recv.

  7. Allow logical replication to transfer data in binary format.

> If in future version the general data type is added, the copy command in binary
> format will not work well, right? It is because the inferior version does not have
> recv/send functions for added type. It means that there is a possibility that
> replication between different versions may be failed if binary format is specified.
> Therefore, I think we should accept copy_format = binary only when the major
> version of servers are the same.

I don't think it's necessary to check versions. Yes, there are
situations where binary will fail across major versions. But in many
cases it does not. To me it seems the responsibility of the operator
to evaluate this risk. And if the operator chooses wrong and uses
binary copy across incompatible versions, then it will still fail hard
in that case during the copy phase (so still a very early error). So I
don't see a reason to check pre-emptively, afaict it will only
disallow some valid usecases and introduce more code.

Furthermore no major version check is done for "binary = true" either
(afaik). The only additional failure scenario that copy_format=binary
introduces is when one of the types does not implement a send function
on the source. With binary=true, this would continue to work, but with
copy_format=binary this stops working. All other failure scenarios
that binary encoding of types introduces apply to both binary=true and
copy_format=binary (the only difference being in which phase of the
replication these failures happen, the apply or the copy phase).

> I'm not sure the combination of "copy_format = binary" and "copy_data = false"
> should be accepted or not. How do you think?

It seems quite useless indeed to specify the format of a copy that won't happen.