Horton
Menu

Re: increasing the default WAL segment size

Andres Freund <andres@anarazel.de>

View thread
From: Andres Freund <andres@anarazel.de>
To: Beena Emerson <memissemerson@gmail.com>
Cc: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2016-12-20T08:28:47Z
Lists: pgsql-hackers 
Hi,

On 2016-12-19 15:14:50 +0530, Beena Emerson wrote:
> The attached patch removes --with-wal-segsize configure option and adds a
> new initdb option --wal-segsize. The module initdb passes the wal-segsize
> value into an environment variable which is used to overwrite the guc value
> wal_ segment_size and set the internal variables : XLogSegSize and
> XLOG_SEG_SIZE (xlog_internal.h). The default wal_segment_size is not
> changed but I have increased the maximum size to 1GB.
> 
> Since  XLOG_SEG_SIZE is now variable, it could not be used directly in
> src/bin modules and few macros and few changes had to be made:

I do think this has the potential for negative performance
implications. Consider code like
			/* skip over the page header */
			if (CurrPos % XLogSegSize == 0)
			{
				CurrPos += SizeOfXLogLongPHD;
				currpos += SizeOfXLogLongPHD;
			}
			else
right now that's doable in an efficient manner, because XLogSegSize is
constant. If it's a variable and the compiler doesn't even know it's a
power-of-two, it'll have to do a full "div" - and that's quite easily
noticeable in a lot of cases.

Now it could entirely be that the costs of this will be swamped by
everything else, but I'd not want to rely on it.

I think we need tests with concurrent large-file copies. And then also
look at the profile to see whether the relevant places become new
hotspots (not that we introduce something that's just hidden for now).


We might be able to do a bit better, efficency wise, by storing
XLogSegSize as a "shift factor". I.e. the 16M setting would be 24
(i.e. XLogSegSize would be defined as 1 << 24).

Greetings,

Andres Freund

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.