Horton
Menu

Re: increasing the default WAL segment size

Simon Riggs <simon@2ndquadrant.com>

View thread
From: Simon Riggs <simon@2ndquadrant.com>
To: Amit Kapila <amit.kapila16@gmail.com>
Cc: Simon Riggs <simon@2ndquadrant.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, PostgreSQL-development <pgsql-hackers@postgresql.org>, Beena Emerson <memissemerson@gmail.com>
Date: 2017-01-03T13:59:44Z
Lists: pgsql-hackers 
On 3 January 2017 at 13:45, Amit Kapila <amit.kapila16@gmail.com> wrote:
> On Tue, Jan 3, 2017 at 6:41 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
>> On 2 January 2017 at 21:23, Jim Nasby <Jim.Nasby@bluetreble.com> wrote:
>>
>>> It's not clear from the thread that there is consensus that this feature is desired. In particular, the performance aspects of changing segment size from a C constant to a variable are in question. Someone with access to large hardware should test that. Andres[1] and Robert[2] did suggest that the option could be changed to a bitshift, which IMHO would also solve some sanity-checking issues.
>>
>> Overall, Robert has made a good case. The only discussion now is about
>> the knock-on effects it causes.
>>
>> One concern that has only barely been discussed is the effect of
>> zero-ing new WAL files. That is a linear effect and will adversely
>> effect performance as WAL segment size increases.
>>
>
> Sorry, but I am not able to understand why this is a problem?  The
> bigger the size of WAL segment, lesser the number of files.  So IIUC,
> then it can only impact if zero-ing two 16MB files is cheaper than
> zero-ing one 32MB file.  Is that your theory or you have something
> else in mind?

The issue I see is that at present no backend needs to do more than
16MB of zeroing at one time, so the impact on response time is
reduced. If we start doing zeroing in larger chunks than the impact on
response times will increase. So instead of regular blips we have one
large blip, less often. I think the latter will be worse, but welcome
measurements that show that performance is smooth and regular with
large files sizes.

-- 
Simon Riggs                http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.