Re: Binary support for pgoutput plugin

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dave Cramer <davecramer@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, Daniel Gustafsson <daniel@yesql.se>, Petr Jelinek <petr@2ndquadrant.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, Michael Paquier <michael@paquier.xyz>, Dmitry Dolgov <9erthalion6@gmail.com>, Thomas Munro <thomas.munro@gmail.com>, Tomas Vondra <tomas.vondra@2ndquadrant.com>, Petr Jelinek <petr.jelinek@2ndquadrant.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>
Date: 2020-07-18T16:53:32Z
Lists: pgsql-hackers

Attachments

I've pushed this patch, with a number of adjustments, some cosmetic
and some not so much (no pg_dump support!?).  We're not quite
done though ...

Dave Cramer <davecramer@gmail.com> writes:
> So looking at how to confirm that the subscriber has receive functions for
> all of the types.
> AFAICT we don't have that information since the publication determines what
> is sent?

Yeah, at the point where we need to send the option, we seem not to have a
lot of info.  In practice, if the sender has a typsend function, the only
way the subscriber doesn't have a matching typreceive function is if it's
an older PG version.  I think it's sufficient to document that you can't
use binary mode in that case, so that's what I did.  (Note that
getTypeBinaryInputInfo will say "no binary input function available for
type %s" in such a case, so that seemed like adequate error handling.)

> On Tue, 14 Jul 2020 at 22:47, Andres Freund <andres@anarazel.de> wrote:
>> An oid mismatch error without knowing what that's about isn't very
>> helpful either.
>> How about adding an errcontext that shows the "source type oid", the
>> target type oid & type name and, for records, the column name of the
>> target table? That'd make this a lot easier to debug.

> This code line 482 in proto.c attempts to limit what is sent in binary. We
> could certainly be more restrictive here.

I think Andres' point is to be *less* restrictive.  I left that logic
as-is in the committed patch, but we could do something like the attached
to improve the situation.

			regards, tom lane

Commits

  1. Weaken type-OID-matching checks in array_recv and record_recv.

  2. Fix construction of updated-columns bitmap in logical replication.

  3. Allow logical replication to transfer data in binary format.

  4. Phase 2 pgindent run for v12.