Re: fdatasync performance problem with large number of DB files

Fujii Masao <masao.fujii@oss.nttdata.com>

From: Fujii Masao <masao.fujii@oss.nttdata.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Bruce Momjian <bruce@momjian.us>, Paul Guo <guopa@vmware.com>, Tom Lane <tgl@sss.pgh.pa.us>, Michael Brown <michael.brown@discourse.org>, pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2021-03-19T02:22:59Z
Lists: pgsql-hackers

On 2021/03/19 10:37, Thomas Munro wrote:
> On Fri, Mar 19, 2021 at 2:16 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>> PS: For illustration/discussion, I've also attached a "none" patch.  I
>> also couldn't resist rebasing my "wal" mode patch, which I plan to
>> propose for PG15 because there is not enough time left for this
>> release.
> 
> Erm... I attached the wrong version by mistake.  Here's a better one.

Thanks for updating the patch! It looks good to me!
I have one minor comment for the patch.

+		elog(LOG, "could not open %s: %m", path);
+		return;
+	}
+	if (syncfs(fd) < 0)
+		elog(LOG, "could not sync filesystem for \"%s\": %m", path);

Since these are neither internal errors nor low-level debug messages, ereport() should be used for them rather than elog()? For example,

		ereport(LOG,
				(errcode_for_file_access(),
				 errmsg("could not open \"%s\": %m", path)))

Regards,

-- 
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION



Commits

  1. Change recovery_init_sync_method to PGC_SIGHUP.

  2. Provide recovery_init_sync_method=syncfs.