Horton

Re: Improving on MAX_CONVERSION_GROWTH

Robert Haas <robertmhaas@gmail.com>

View thread View on postgr.es
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2019-09-27T15:53:56Z
Lists: pgsql-hackers 
On Fri, Sep 27, 2019 at 11:40 AM Andres Freund <andres@anarazel.de> wrote:
> Note that one of the additional reasons for the 1GB limit is that it
> protects against int overflows. I'm somewhat unconvinced that that's a
> sensible approach, but ...

It's not crazy. People using 'int' rather casually just as they use
'palloc' rather casually, without necessarily thinking about what
could go wrong at the edges. I don't have any beef with that as a
general strategy; I just think we should be trying to do better in the
cases where it negatively affects the user experience.

> It's worthwhile to note that additional passes over data are often quite
> expensive, memory latency hasn't shrunk that much in last decade or
> so. I have frequently seen all the memcpys from one StringInfo/char*
> into another StringInfo show up in profiles.

OK.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

  1. Avoid unnecessary out-of-memory errors during encoding conversion.

  2. Allow repalloc() to give back space when a large chunk is downsized.