Horton
Menu

Re: increasing the default WAL segment size

Beena Emerson <memissemerson@gmail.com>

View thread
From: Beena Emerson <memissemerson@gmail.com>
To: Andres Freund <andres@anarazel.de>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2016-12-20T09:08:57Z
Lists: pgsql-hackers 
Hello Andres,

On Tue, Dec 20, 2016 at 1:58 PM, Andres Freund <andres@anarazel.de> wrote:

> Hi,
>
> On 2016-12-19 15:14:50 +0530, Beena Emerson wrote:
> > The attached patch removes --with-wal-segsize configure option and adds a
> > new initdb option --wal-segsize. The module initdb passes the wal-segsize
> > value into an environment variable which is used to overwrite the guc
> value
> > wal_ segment_size and set the internal variables : XLogSegSize and
> > XLOG_SEG_SIZE (xlog_internal.h). The default wal_segment_size is not
> > changed but I have increased the maximum size to 1GB.
> >
> > Since  XLOG_SEG_SIZE is now variable, it could not be used directly in
> > src/bin modules and few macros and few changes had to be made:
>
> I do think this has the potential for negative performance
> implications. Consider code like
>                         /* skip over the page header */
>                         if (CurrPos % XLogSegSize == 0)
>                         {
>                                 CurrPos += SizeOfXLogLongPHD;
>                                 currpos += SizeOfXLogLongPHD;
>                         }
>                         else
> right now that's doable in an efficient manner, because XLogSegSize is
> constant. If it's a variable and the compiler doesn't even know it's a
> power-of-two, it'll have to do a full "div" - and that's quite easily
> noticeable in a lot of cases.
>
> Now it could entirely be that the costs of this will be swamped by
> everything else, but I'd not want to rely on it.
>
> I think we need tests with concurrent large-file copies. And then also
> look at the profile to see whether the relevant places become new
> hotspots (not that we introduce something that's just hidden for now).


> We might be able to do a bit better, efficency wise, by storing
> XLogSegSize as a "shift factor". I.e. the 16M setting would be 24
> (i.e. XLogSegSize would be defined as 1 << 24).
>
>
Thank you.
I did not realize we could face such problems. I will perform the tests to
check the performance and do the required changes.


--

Beena Emerson

Have a Great Day!

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.