Horton

Re: pg_dump / copy bugs with "big lines" ?

Alvaro Herrera <alvherre@2ndquadrant.com>

View thread View on postgr.es
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Daniel Verite <daniel@manitou-mail.org>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, Robert Haas <robertmhaas@gmail.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, Ronan Dunklau <ronan.dunklau@dalibo.com>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2016-03-18T15:48:30Z
Lists: pgsql-hackers 
Daniel Verite wrote:

> To go past that problem, I've tried tweaking the StringInfoData
> used for COPY FROM, like the original patch does in CopyOneRowTo. 
> 
> It turns out that it fails a bit later when trying to make a tuple
> from the big line, in heap_form_tuple():
> 
>   tuple = (HeapTuple) palloc0(HEAPTUPLESIZE + len);
> 
> which fails because (HEAPTUPLESIZE + len) is again considered
> an invalid size, the  size being 1468006476 in my test.

Um, it seems reasonable to make this one be a huge-zero-alloc:

	MemoryContextAllocExtended(CurrentMemoryContext, HEAPTUPLESIZE + len,
				   MCXT_ALLOC_HUGE | MCXT_ALLOC_ZERO)

-- 
Álvaro Herrera                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Fix overflow check in StringInfo; add missing casts

  2. Permit dump/reload of not-too-large >1GB tuples