Horton

Re: pg_dump / copy bugs with "big lines" ?

Michael Paquier <michael.paquier@gmail.com>

View thread View on postgr.es
From: Michael Paquier <michael.paquier@gmail.com>
To: Alvaro Herrera <alvherre@2ndquadrant.com>
Cc: Daniel Verite <daniel@manitou-mail.org>, Robert Haas <robertmhaas@gmail.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, Ronan Dunklau <ronan.dunklau@dalibo.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2016-03-03T01:31:26Z
Lists: pgsql-hackers 
On Thu, Mar 3, 2016 at 12:47 AM, Alvaro Herrera
<alvherre@2ndquadrant.com> wrote:
> Well, the CopyData message has an Int32 field for the message length.
> I don't know the FE/BE protocol very well but I suppose each row
> corresponds to one CopyData message, or perhaps each column corresponds
> to one CopyData message.  In either case, it's not possible to go beyond
> 2GB without changing the protocol ...

Based on what I know from this stuff (OOM libpq and other stuff
remnants), one 'd' message means one row. fe-protocol3.c and
CopySendEndOfRow in backend's copy.c are confirming that as well. I am
indeed afraid that having extra logic to get chunks of data will
require extending the protocol with a new message type for this
purpose.
-- 
Michael

Commits

  1. Fix overflow check in StringInfo; add missing casts

  2. Permit dump/reload of not-too-large >1GB tuples