Horton

Re: [PING] fallocate() causes btrfs to never compress postgresql files

Thomas Munro <thomas.munro@gmail.com>

View thread
From: Thomas Munro <thomas.munro@gmail.com>
To: Dimitrios Apostolou <jimis@gmx.net>
Cc: Tomas Vondra <tomas@vondra.me>, pgsql-hackers@lists.postgresql.org, Andres Freund <andres@anarazel.de>, Melanie Plageman <melanieplageman@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, David Rowley <dgrowleyml@gmail.com>
Date: 2025-06-09T08:33:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add file_extend_method=posix_fallocate,write_zeros.

  2. Add smgrzeroextend(), FileZero(), FileFallocate()

Attachments

On Tue, Jun 3, 2025 at 1:58 AM Dimitrios Apostolou <jimis@gmx.net> wrote:
> This sounds like the best solution IMO. People can then experiment with
> different settings and filesystems, and that way we also learn in the
> process. Thank you for the effort and patches so far.

OK, here's a basic patch to experiment with.  You can set:

file_extend_method = fallocate,ftruncate,write
file_extend_method_threshold = 8 # (below 8 always write, 0 means never write)

To really make COPY fly we also need to get write combining and AIO
going (we've had this working with various prototypes, but it all
missed the boat for v18 which can only do that stuff for reads).  Then
you'll have concurrent 128kB or up to 1MB writes trundling along in
the background which I guess should work pretty nicely for stuff like
BTRFS/ZFS and compression and all that jazz.