Horton
Menu

Re: increasing the default WAL segment size

Robert Haas <robertmhaas@gmail.com>

View thread
From: Robert Haas <robertmhaas@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2016-09-20T20:18:02Z
Lists: pgsql-hackers 
On Tue, Sep 20, 2016 at 4:09 PM, Andres Freund <andres@anarazel.de> wrote:
> On 2016-09-20 16:05:44 -0400, Robert Haas wrote:
>> On Tue, Sep 20, 2016 at 2:49 PM, Peter Eisentraut
>> <peter.eisentraut@2ndquadrant.com> wrote:
>> > On 8/24/16 9:31 PM, Robert Haas wrote:
>> >> I'd like to propose that we increase the default WAL segment size,
>> >> which is currently 16MB.
>> >
>> > While the discussion about the best default value is ongoing, maybe we
>> > should at least *allow* some larger sizes, for testing out.  Currently,
>> > configure says "Allowed values are 1,2,4,8,16,32,64.".  What might be a
>> > good new upper limit?
>
> I'm doubtful it's worth increasing this.
>
>> 1GB?
>
> That sounds way too big to me. WAL file allocation would trigger pretty
> massive IO storms during zeroing, max_wal_size is going to be hard to
> tune, the amounts of dirty data during bulk loads is going to be very
> hard to control.  If somebody wants to do something like this they
> better be well informed enough to override a #define.

EnterpriseDB has customers generating multiple TB of WAL per day.
Even with a 1GB segment size, some of them will fill multiple files
per minute.  At the current limit of 64MB, a few of them would still
fill more than one file per second.  That is not sane.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.