Horton
Menu

Re: increasing the default WAL segment size

Claudio Freire <klaussfreire@gmail.com>

View thread
From: Claudio Freire <klaussfreire@gmail.com>
To: Simon Riggs <simon@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2016-08-25T16:55:58Z
Lists: pgsql-hackers 
On Thu, Aug 25, 2016 at 12:21 PM, Simon Riggs <simon@2ndquadrant.com> wrote:
> On 25 August 2016 at 02:31, Robert Haas <robertmhaas@gmail.com> wrote:
>
>> Furthermore, there is an enforced, synchronous fsync at the end of
>> every segment, which actually does hurt performance on write-heavy
>> workloads.[2] Of course, if that were the only reason to consider
>> increasing the segment size, it would probably make more sense to just
>> try to push that extra fsync into the background, but that's not
>> really the case.  From what I hear, the gigantic number of files is a
>> bigger pain point.
>
> I think we should fully describe the problem before finding a solution.
>
> This is too big a change to just tweak a value without discussing the
> actual issue.
>
> And if the problem is as described, how can a change of x4 be enough
> to make it worth the pain of change? I think you're already admitting
> it can't be worth it by discussing initdb configuration.
>
> If we do have the pain of change, should we also consider making WAL
> files variable length? What do we gain by having the files all the
> same size? ISTM better to have WAL files that vary in length up to 1GB
> in size.
>
> (This is all about XLOG_SEG_SIZE; I presume XLOG_BLCKSZ can stay as it
> is, right?)

Avoiding variable sizes does avoid some failure modes on the
filesystem side in the face of crashes/power loss.

So making them variable size, while possible, wouldn't be simple at
all (it would involve figuring out the way filesystems behave when
facing crash when a file is changing in size).

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.