Horton
Menu

Re: increasing the default WAL segment size

David Steele <david@pgmasters.net>

View thread
From: David Steele <david@pgmasters.net>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Stephen Frost <sfrost@snowman.net>, Beena Emerson <memissemerson@gmail.com>, tushar <tushar.ahuja@enterprisedb.com>, Prabhat Sahu <prabhat.sahu@enterprisedb.com>, Ashutosh Sharma <ashu.coek88@gmail.com>, Jim Nasby <Jim.Nasby@bluetreble.com>, Kuntal Ghosh <kuntalghosh.2007@gmail.com>, Andres Freund <andres@anarazel.de>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2017-03-22T22:05:16Z
Lists: pgsql-hackers 
Hi Robert,

On 3/22/17 3:45 PM, Robert Haas wrote:
> On Wed, Mar 22, 2017 at 3:24 PM, David Steele <david@pgmasters.net> wrote:
>>> One of the reasons to go with the LSN is that we would actually be
>>> maintaining what happens when the WAL files are 16MB in size.
>>>
>>> David's initial expectation was this for 64MB WAL files:
>>>
>>> 000000010000000000000040
>>> 000000010000000000000080
>>> 0000000100000000000000CO
>>> 000000010000000100000000
>>
>>
>> This is the 1GB sequence, actually, but idea would be the same for 64MB
>> files.
>
> Wait, really?  I thought you abandoned this approach because there's
> then no principled way to handle WAL segments of less than the default
> size.

I did say that, but I thought I had hit on a compromise.

But, as I originally pointed out the hex characters in the filename are 
not aligned correctly for > 8 bits (< 16MB segments) and using different 
alignments just made it less consistent.

It would be OK if we were willing to drop the 1,2,4,8 segment sizes 
because then the alignment would make sense and not change the current 
16MB sequence.

Even then, there are some interesting side effects.  For 1GB segments 
the "0000000100000001000000C0" segment would include LSNs 1/C0000000 
through 1/FFFFFFFF.  This is correct but is not an obvious filename to 
LSN mapping, at least for LSNs that appear later in the segment.

-- 
-David
david@pgmasters.net

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Make WAL segment size configurable at initdb time.

  2. Perform only one ReadControlFile() during startup.

  3. Introduce BYTES unit for GUCs.

  4. Remove useless duplicate inclusions of system header files.

  5. Refactor other replication commands to use DestRemoteSimple.

  6. Add a SHOW command to the replication command language.

  7. Add a new DestReceiver for printing tuples without catalog access.

  8. Support fls().

  9. Extend yesterday's patch making BLCKSZ and RELSEG_SIZE configurable to also

  10. Commit the reasonably uncontroversial parts of J.R. Nield's PITR patch, to

  11. XLOG (also known as WAL -:)) Bootstrap/Startup/Shutdown.

  12. Transaction log manager core code.