Horton
Menu

Re: increasing the default WAL segment size

Peter Eisentraut <peter.eisentraut@2ndquadrant.com>

View thread
From: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
To: Beena Emerson <memissemerson@gmail.com>
Cc: Amit Kapila <amit.kapila16@gmail.com>, Simon Riggs <simon@2ndquadrant.com>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Stephen Frost <sfrost@snowman.net>, David Steele <david@pgmasters.net>, tushar <tushar.ahuja@enterprisedb.com>, Prabhat Sahu <prabhat.sahu@enterprisedb.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>
Date: 2017-04-05T12:36:44Z
Lists: pgsql-hackers 
On 4/5/17 06:04, Beena Emerson wrote:
>     I suggest the next step is to dial up the allowed segment size in
>     configure and run some tests about what a reasonable maximum value could
>     be.  I did a little bit of that, but somewhere around 256 MB, things got
>     really slow.
> 
> 
> Would it be better if just increase the limit to 128MB for now?
> In next we can change the WAL file name format and expand the range?

I don't think me saying it felt a bit slow around 256 MB is a proper
technical analysis that should lead to the conclusion that that upper
limit should be 128 MB. ;-)

This tells me that there is a lot of explore and test here before we
should let it loose on users.

I think the best we should do now is spend a bit of time exploring
whether/how larger values of segment size behave, and bump the hardcoded
configure limit if we get positive results.  Everything else should
probably be postponed.

(Roughly speaking, to get started, this would mean compiling with
--with-wal-segsize 16, 32, 64, 128, 256, run make check-world both
sequentially and in parallel, and take note of a) passing, b) run time,
c) disk space.)

-- 
Peter Eisentraut              http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.