Horton

Re: pg_dump / copy bugs with "big lines" ?

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: "Daniel Verite" <daniel@manitou-mail.org>
Cc: "Alvaro Herrera" <alvherre@2ndquadrant.com>, "Robert Haas" <robertmhaas@gmail.com>, "Jim Nasby" <Jim.Nasby@bluetreble.com>, "Ronan Dunklau" <ronan.dunklau@dalibo.com>, "pgsql-hackers" <pgsql-hackers@postgresql.org>
Date: 2016-03-01T16:50:36Z
Lists: pgsql-hackers 
"Daniel Verite" <daniel@manitou-mail.org> writes:
> 	Alvaro Herrera wrote:
>> If others can try this patch to ensure it enables pg_dump to work on
>> their databases, it would be great.

> It doesn't seem to help if one field exceeds 1Gb, for instance when
> inflated by a bin->hex translation.

It's not going to be possible to fix that without enormously invasive
changes (affecting individual datatype I/O functions, for example).
And in general, fields approaching that size are going to give you
problems in all kinds of ways, not only COPY.

I think we should be satisfied if we can make COPY deal with the sum
of a line's fields exceeding 1GB.

			regards, tom lane

Commits

  1. Fix overflow check in StringInfo; add missing casts

  2. Permit dump/reload of not-too-large >1GB tuples