Horton
Menu

Re: increasing the default WAL segment size

Beena Emerson <memissemerson@gmail.com>

View thread
From: Beena Emerson <memissemerson@gmail.com>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Kuntal Ghosh <kuntalghosh.2007@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Stephen Frost <sfrost@snowman.net>, David Steele <david@pgmasters.net>, tushar <tushar.ahuja@enterprisedb.com>, Prabhat Sahu <prabhat.sahu@enterprisedb.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>
Date: 2017-03-27T19:36:13Z
Lists: pgsql-hackers

Attachments

Hello,

On Sat, Mar 25, 2017 at 10:32 PM, Peter Eisentraut <
peter.eisentraut@2ndquadrant.com> wrote:

> At this point, I suggest splitting this patch up into several
> potentially less controversial pieces.
>
> One big piece is that we currently don't support segment sizes larger
> than 64 GB, for various internal arithmetic reasons.  Your patch appears
> to address that.  So I suggest isolating that.  Assuming it works
> correctly, I think there would be no great concern about it.
>
> The next piece would be making the various tools aware of varying
> segment sizes without having to rely on a built-in value.
>
> The third piece would then be the rest that allows you to set the size
> at initdb
>
> If we take these in order, we would make it easier to test various sizes
> and see if there are any more unforeseen issues when changing sizes.  It
> would also make it easier to do performance testing so we can address
> the original question of what the default size should be.
>

PFA the patches divided into 3 parts:

02-increase-max-wal-segsize.patch - Increases the wal-segsize and changes
the internal representation of max_wal_size and min_wal_size to mb.

03-modify-tools.patch - Makes XLogSegSize into a variable, currently set as
XLOG_SEG_SIZE and modifies the tools to fetch the size instead of using
inbuilt value.

04-initdb-walsegsize.patch - Adds the initdb option to set wal-segsize and
make related changes. Update pg_test_fsync to use DEFAULT_XLOG_SEG_SIZE
instead of XLOG_SEG_SIZE


> One concern I have is that your patch does not contain any tests.  There
> should probably be lots of tests.


05-initdb_tests.patch adds tap tests to initialize cluster with different
wal_segment_size and then check the config values. What other tests do you
have in mind? Checking the various tools?

--
Beena Emerson

EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.