Horton

Re: [PING] fallocate() causes btrfs to never compress postgresql files

Dimitrios Apostolou <jimis@gmx.net>

View thread
From: Dimitrios Apostolou <jimis@gmx.net>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tomas Vondra <tomas@vondra.me>, pgsql-hackers@lists.postgresql.org, Andres Freund <andres@anarazel.de>, Melanie Plageman <melanieplageman@gmail.com>, Heikki Linnakangas <hlinnaka@iki.fi>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>, David Rowley <dgrowleyml@gmail.com>
Date: 2025-06-02T10:14:01Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Add file_extend_method=posix_fallocate,write_zeros.

  2. Add smgrzeroextend(), FileZero(), FileFallocate()

On Sun, 1 Jun 2025, Thomas Munro wrote:

> Or for a completely different approach: I wonder if ftruncate() would
> be more efficient on COW systems anyway.  The minimum thing we need is
> for the file system to remember the new size, 'cause, erm, we don't.
> All the rest is probably a waste of cycles, since they reserve real
> space (or fail to) later in the checkpointer or whatever process
> eventually writes the data out.

FWIW I asked the btrfs devs. From 
https://github.com/kdave/btrfs-progs/pull/976
I quote Qu Wenruo:

> Only for falloc(), not ftruncate().
> 
> The PREALLOC inode flag is added for any preallocated file extent, 
> meanwhile truncate only creates holes.
>
> truncate is fast but it's really different from fallocate by there is 
> nothing really allocated.
> 
> This means the later writes will need to allocate their own data 
> extents. This is fine and even preferred for btrfs, but may lead to 
> performance drop for more traditional fses.
> 
> We're in an era that fs features are not longer that generic, fallocate 
> is just one example, in fact fallocate will cause more problems more 
> than no compression.
>
> It's really a deep rabbit hole, and is not something simple true or 
> false questions.


In other words, btrfs will not try to allocate anything with ftruncate(), 
it will just mark the new space as a "hole". As such, the file is not 
marked as "PREALLOC" which is what disables compression. Of course there 
is no guarantee that further writes will succeed, and as quoted above, 
other (non-COW) filesystems might be slower writing the 
ftruncate()-allocated space.


Regards,
Dimitris