Horton

Re: should frontend tools use syncfs() ?

Justin Pryzby <pryzby@telsasoft.com>

View thread View on postgr.es
From: Justin Pryzby <pryzby@telsasoft.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, David Steele <david@pgmasters.net>, Fujii Masao <masao.fujii@oss.nttdata.com>, Bruce Momjian <bruce@momjian.us>, Paul Guo <guopa@vmware.com>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Brown <michael.brown@discourse.org>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-10-02T15:41:54Z
Lists: pgsql-hackers 
On Thu, Sep 30, 2021 at 05:08:24PM +1300, Thomas Munro wrote:
> On Thu, Sep 30, 2021 at 4:49 PM Michael Paquier <michael@paquier.xyz> wrote:
> > fsync_pgdata() is going to manipulate many inodes anyway, because
> > that's a code path designed to do so.  If we know that syncfs() is
> > just going to be better, I'd rather just call it by default if
> > available and not add new switches to all the frontend tools in need
> > of flushing the data folder, switches that are not documented in your
> > patch.
> 
> If we want this it should be an option, because it flushes out data
> other than the pgdata dir, and it doesn't report errors on old
> kernels.

I ran into bad performance of initdb --sync-only shortly after adding it to my
db migration script, so added initdb --syncfs.

I found that with sufficiently recent coreutils, I can do what's wanted by calling 
/bin/sync -f /datadir

Since it's not integrated into initdb, it's necessary to include each
tablespace and wal.

-- 
Justin

Commits

  1. Adjust documentation for syncfs().

  2. Improve the naming in wal_sync_method code.

  3. Allow using syncfs() in frontend utilities.

  4. Add support for syncfs() in frontend support functions.

  5. Make enum for sync methods available to frontend code.

  6. Move PG_TEMP_FILE* macros to file_utils.h.

  7. Change client-side fsync_fname() to report errors fatally