Thread
-
test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T20:55:12Z
I have modified test_fsync to use test labels that match wal_sync_method values, and and added more tests for open_sync with different sizes. This should make the program easier for novices to understand. Here is a test run for Ubuntu 11.04: $ ./test_fsync 2000 operations per test Compare file sync methods using one 8k write: (in wal_sync_method preference order, except fdatasync is Linux's default) open_datasync (non-direct I/O)* 85.127 ops/sec open_datasync (direct I/O) 87.119 ops/sec fdatasync 81.006 ops/sec fsync 82.621 ops/sec fsync_writethrough n/a open_sync (non-direct I/O)* 84.412 ops/sec open_sync (direct I/O) 91.006 ops/sec * This non-direct I/O mode is not used by Postgres. Compare file sync methods using two 8k writes: (in wal_sync_method preference order, except fdatasync is Linux's default) open_datasync (non-direct I/O)* 42.721 ops/sec open_datasync (direct I/O) 45.296 ops/sec fdatasync 76.665 ops/sec fsync 78.361 ops/sec fsync_writethrough n/a open_sync (non-direct I/O)* 42.311 ops/sec open_sync (direct I/O) 45.247 ops/sec * This non-direct I/O mode is not used by Postgres. Compare open_sync with different write sizes: (This is designed to compare the cost of writing 16k in different write open_sync sizes.) 1 16k open_sync write 86.740 ops/sec 2 8k open_sync writes 44.709 ops/sec 4 4k open_sync writes 22.096 ops/sec 8 2k open_sync writes 10.856 ops/sec 16 1k open_sync writes 5.434 ops/sec Test if fsync on non-write file descriptor is honored: (If the times are similar, fsync() can sync data written on a different descriptor.) write, fsync, close 86.802 ops/sec write, close, fsync 85.766 ops/sec Non-sync'ed 8k writes: write 83.068 ops/sec Applied patch attached. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
A.M. <agentm@themactionfaction.com> — 2011-01-18T21:57:39Z
On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > I have modified test_fsync to use test labels that match wal_sync_method > values, and and added more tests for open_sync with different sizes. > This should make the program easier for novices to understand. Here is > a test run for Ubuntu 11.04: > > $ ./test_fsync > 2000 operations per test > > Compare file sync methods using one 8k write: > (in wal_sync_method preference order, except fdatasync > is Linux's default) > open_datasync (non-direct I/O)* 85.127 ops/sec > open_datasync (direct I/O) 87.119 ops/sec > fdatasync 81.006 ops/sec > fsync 82.621 ops/sec > fsync_writethrough n/a > open_sync (non-direct I/O)* 84.412 ops/sec > open_sync (direct I/O) 91.006 ops/sec > * This non-direct I/O mode is not used by Postgres. I am curious how this is targeted at novices. A naive user might enable the "fastest" option which could be exactly wrong. For this to be useful to novices, I suspect the tool will need to generate platform-specific suggestions, no? Cheers, M
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T22:16:03Z
A.M. wrote: > > On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > > > I have modified test_fsync to use test labels that match wal_sync_method > > values, and and added more tests for open_sync with different sizes. > > This should make the program easier for novices to understand. Here is > > a test run for Ubuntu 11.04: > > > > $ ./test_fsync > > 2000 operations per test > > > > Compare file sync methods using one 8k write: > > (in wal_sync_method preference order, except fdatasync > > is Linux's default) > > open_datasync (non-direct I/O)* 85.127 ops/sec > > open_datasync (direct I/O) 87.119 ops/sec > > fdatasync 81.006 ops/sec > > fsync 82.621 ops/sec > > fsync_writethrough n/a > > open_sync (non-direct I/O)* 84.412 ops/sec > > open_sync (direct I/O) 91.006 ops/sec > > * This non-direct I/O mode is not used by Postgres. > > I am curious how this is targeted at novices. A naive user might enable > the "fastest" option which could be exactly wrong. For this to be useful > to novices, I suspect the tool will need to generate platform-specific > suggestions, no? Uh, why isn't the fastest option right for them? It is hardware/kernel specific when you run it --- how could it be better? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
A.M. <agentm@themactionfaction.com> — 2011-01-18T22:19:14Z
On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: > A.M. wrote: >> >> On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: >> >>> I have modified test_fsync to use test labels that match wal_sync_method >>> values, and and added more tests for open_sync with different sizes. >>> This should make the program easier for novices to understand. Here is >>> a test run for Ubuntu 11.04: >>> >>> $ ./test_fsync >>> 2000 operations per test >>> >>> Compare file sync methods using one 8k write: >>> (in wal_sync_method preference order, except fdatasync >>> is Linux's default) >>> open_datasync (non-direct I/O)* 85.127 ops/sec >>> open_datasync (direct I/O) 87.119 ops/sec >>> fdatasync 81.006 ops/sec >>> fsync 82.621 ops/sec >>> fsync_writethrough n/a >>> open_sync (non-direct I/O)* 84.412 ops/sec >>> open_sync (direct I/O) 91.006 ops/sec >>> * This non-direct I/O mode is not used by Postgres. >> >> I am curious how this is targeted at novices. A naive user might enable >> the "fastest" option which could be exactly wrong. For this to be useful >> to novices, I suspect the tool will need to generate platform-specific >> suggestions, no? > > Uh, why isn't the fastest option right for them? It is hardware/kernel > specific when you run it --- how could it be better? Because the fastest option may not be syncing to disk. For example, the only option that makes sense on OS X is fsync_writethrough- it would be helpful if the tool pointed that out (on OS X only, obviously). Cheers, M
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T22:21:00Z
A.M. wrote: > > On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: > > > A.M. wrote: > >> > >> On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: > >> > >>> I have modified test_fsync to use test labels that match wal_sync_method > >>> values, and and added more tests for open_sync with different sizes. > >>> This should make the program easier for novices to understand. Here is > >>> a test run for Ubuntu 11.04: > >>> > >>> $ ./test_fsync > >>> 2000 operations per test > >>> > >>> Compare file sync methods using one 8k write: > >>> (in wal_sync_method preference order, except fdatasync > >>> is Linux's default) > >>> open_datasync (non-direct I/O)* 85.127 ops/sec > >>> open_datasync (direct I/O) 87.119 ops/sec > >>> fdatasync 81.006 ops/sec > >>> fsync 82.621 ops/sec > >>> fsync_writethrough n/a > >>> open_sync (non-direct I/O)* 84.412 ops/sec > >>> open_sync (direct I/O) 91.006 ops/sec > >>> * This non-direct I/O mode is not used by Postgres. > >> > >> I am curious how this is targeted at novices. A naive user might enable > >> the "fastest" option which could be exactly wrong. For this to be useful > >> to novices, I suspect the tool will need to generate platform-specific > >> suggestions, no? > > > > Uh, why isn't the fastest option right for them? It is hardware/kernel > > specific when you run it --- how could it be better? > > Because the fastest option may not be syncing to disk. For example, > the only option that makes sense on OS X is fsync_writethrough- it > would be helpful if the tool pointed that out (on OS X only, obviously). Yes, that would be a serious problem. :-( I am not sure how we would address this --- your point is a good one. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
A.M. <agentm@themactionfaction.com> — 2011-01-18T22:33:22Z
On Jan 18, 2011, at 5:21 PM, Bruce Momjian wrote: > A.M. wrote: >> >> On Jan 18, 2011, at 5:16 PM, Bruce Momjian wrote: >> >>> A.M. wrote: >>>> >>>> On Jan 18, 2011, at 3:55 PM, Bruce Momjian wrote: >>>> >>>>> I have modified test_fsync to use test labels that match wal_sync_method >>>>> values, and and added more tests for open_sync with different sizes. >>>>> This should make the program easier for novices to understand. Here is >>>>> a test run for Ubuntu 11.04: >>>>> >>>>> $ ./test_fsync >>>>> 2000 operations per test >>>>> >>>>> Compare file sync methods using one 8k write: >>>>> (in wal_sync_method preference order, except fdatasync >>>>> is Linux's default) >>>>> open_datasync (non-direct I/O)* 85.127 ops/sec >>>>> open_datasync (direct I/O) 87.119 ops/sec >>>>> fdatasync 81.006 ops/sec >>>>> fsync 82.621 ops/sec >>>>> fsync_writethrough n/a >>>>> open_sync (non-direct I/O)* 84.412 ops/sec >>>>> open_sync (direct I/O) 91.006 ops/sec >>>>> * This non-direct I/O mode is not used by Postgres. >>>> >>>> I am curious how this is targeted at novices. A naive user might enable >>>> the "fastest" option which could be exactly wrong. For this to be useful >>>> to novices, I suspect the tool will need to generate platform-specific >>>> suggestions, no? >>> >>> Uh, why isn't the fastest option right for them? It is hardware/kernel >>> specific when you run it --- how could it be better? >> >> Because the fastest option may not be syncing to disk. For example, >> the only option that makes sense on OS X is fsync_writethrough- it >> would be helpful if the tool pointed that out (on OS X only, obviously). > > Yes, that would be a serious problem. :-( > > I am not sure how we would address this --- your point is a good one. One general idea I had would be to offer some heuristics such as "this sync rate is comparable to that of one SATA drive" or "comparable to RAID 10 with X drives" or "this rate is likely too fast to be actually be syncing". But then you are stuck with making sure that the heuristics are kept up-to-date, which would be annoying. Otherwise, the only option I see is to detect the kernel and compare against a list of known problematic methods. Perhaps it would be easier to compare against a whitelist. Also, the tool would likely need to parse "mount" output to account for problems with specific filesystems. I am just throwing around some ideas... Cheers, M
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T22:41:33Z
A.M. wrote: > >> Because the fastest option may not be syncing to disk. For example, > >> the only option that makes sense on OS X is fsync_writethrough- it > >> would be helpful if the tool pointed that out (on OS X only, obviously). > > > > Yes, that would be a serious problem. :-( > > > > I am not sure how we would address this --- your point is a good one. > > One general idea I had would be to offer some heuristics such as "this > sync rate is comparable to that of one SATA drive" or "comparable to > RAID 10 with X drives" or "this rate is likely too fast to be actually > be syncing". But then you are stuck with making sure that the heuristics > are kept up-to-date, which would be annoying. That fails for RAID BBUs. > Otherwise, the only option I see is to detect the kernel and compare > against a list of known problematic methods. Perhaps it would be easier > to compare against a whitelist. Also, the tool would likely need to > parse "mount" output to account for problems with specific filesystems. > > I am just throwing around some ideas... That sounds pretty complicated. One idea would be the creation of a wiki where people could post their results, or ideally a tool that could read the output and load it into a database for analysis with other results. -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-18T22:59:56Z
Bruce Momjian <bruce@momjian.us> writes: > I have modified test_fsync to use test labels that match wal_sync_method > values, and and added more tests for open_sync with different sizes. Given that it was unclear whether the first such test was of any value, why are you slowing down the program by adding more? regards, tom lane
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T23:03:24Z
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > I have modified test_fsync to use test labels that match wal_sync_method > > values, and and added more tests for open_sync with different sizes. > > Given that it was unclear whether the first such test was of any value, > why are you slowing down the program by adding more? Greg Smith indicated it has value, so I made it more complete. No? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
Tom Lane <tgl@sss.pgh.pa.us> — 2011-01-18T23:06:09Z
Bruce Momjian <bruce@momjian.us> writes: > Tom Lane wrote: >> Bruce Momjian <bruce@momjian.us> writes: >>> I have modified test_fsync to use test labels that match wal_sync_method >>> values, and and added more tests for open_sync with different sizes. >> Given that it was unclear whether the first such test was of any value, >> why are you slowing down the program by adding more? > Greg Smith indicated it has value, so I made it more complete. No? My recollection of that discussion is a bit different: there wasn't a clear-cut reason to rip it out. But the more tests you add to test_fsync, the less useful it becomes. regards, tom lane
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T23:09:27Z
Tom Lane wrote: > Bruce Momjian <bruce@momjian.us> writes: > > Tom Lane wrote: > >> Bruce Momjian <bruce@momjian.us> writes: > >>> I have modified test_fsync to use test labels that match wal_sync_method > >>> values, and and added more tests for open_sync with different sizes. > > >> Given that it was unclear whether the first such test was of any value, > >> why are you slowing down the program by adding more? > > > Greg Smith indicated it has value, so I made it more complete. No? > > My recollection of that discussion is a bit different: there wasn't a > clear-cut reason to rip it out. But the more tests you add to > test_fsync, the less useful it becomes. Well, this is Greg Smith's text: http://archives.postgresql.org/pgsql-hackers/2011-01/msg01717.php > Might be some value for determining things like what the optimal WAL > block size to use is. All these tests are kind of hard to use > effectively still, I'm not sure if it's time to start trimming tests yet > until we've made more progress on interpreting results first. so I figured the test should be complete; a partial test is pretty useless. What I am thinking is that the program should just run the first test by default (to choose wal_sync_method), and add a -v option to run the additional tests. Yes? -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +
-
Re: test_fsync label adjustments
A.M. <agentm@themactionfaction.com> — 2011-01-18T23:20:11Z
On Jan 18, 2011, at 5:41 PM, Bruce Momjian wrote: > A.M. wrote: >>>> Because the fastest option may not be syncing to disk. For example, >>>> the only option that makes sense on OS X is fsync_writethrough- it >>>> would be helpful if the tool pointed that out (on OS X only, obviously). >>> >>> Yes, that would be a serious problem. :-( >>> >>> I am not sure how we would address this --- your point is a good one. >> >> One general idea I had would be to offer some heuristics such as "this >> sync rate is comparable to that of one SATA drive" or "comparable to >> RAID 10 with X drives" or "this rate is likely too fast to be actually >> be syncing". But then you are stuck with making sure that the heuristics >> are kept up-to-date, which would be annoying. > > That fails for RAID BBUs. Well, it's nothing more than a heuristic- it is still nice to know whether or not the fancy hardware RAID I just setup is similar to Josh Berkus' RAID setup or a single SATA drive (which would hint at a misconfiguration). As you said, perhaps a wiki is better for this. But a wiki won't integrate with this tool, which I why I would hesitate to point novices to this tool... should the tool point to the wiki? > >> Otherwise, the only option I see is to detect the kernel and compare >> against a list of known problematic methods. Perhaps it would be easier >> to compare against a whitelist. Also, the tool would likely need to >> parse "mount" output to account for problems with specific filesystems. >> >> I am just throwing around some ideas... > > That sounds pretty complicated. One idea would be the creation of a > wiki where people could post their results, or ideally a tool that could > read the output and load it into a database for analysis with other > results. The OS X example is pretty cut-and-dry- it would be nice if there were some kind of hints in the tool pointing in the right direction, or at least a few words of warning: "the fastest option may not be the safest- read the docs". Cheers, M
-
Re: test_fsync label adjustments
Bruce Momjian <bruce@momjian.us> — 2011-01-18T23:36:31Z
A.M. wrote: > > On Jan 18, 2011, at 5:41 PM, Bruce Momjian wrote: > > > A.M. wrote: > >>>> Because the fastest option may not be syncing to disk. For example, > >>>> the only option that makes sense on OS X is fsync_writethrough- it > >>>> would be helpful if the tool pointed that out (on OS X only, obviously). > >>> > >>> Yes, that would be a serious problem. :-( > >>> > >>> I am not sure how we would address this --- your point is a good one. > >> > >> One general idea I had would be to offer some heuristics such as "this > >> sync rate is comparable to that of one SATA drive" or "comparable to > >> RAID 10 with X drives" or "this rate is likely too fast to be actually > >> be syncing". But then you are stuck with making sure that the heuristics > >> are kept up-to-date, which would be annoying. > > > > That fails for RAID BBUs. > > Well, it's nothing more than a heuristic- it is still nice to know whether or not the fancy hardware RAID I just setup is similar to Josh Berkus' RAID setup or a single SATA drive (which would hint at a misconfiguration). As you said, perhaps a wiki is better for this. But a wiki won't integrate with this tool, which I why I would hesitate to point novices to this tool... should the tool point to the wiki? > > > > >> Otherwise, the only option I see is to detect the kernel and compare > >> against a list of known problematic methods. Perhaps it would be easier > >> to compare against a whitelist. Also, the tool would likely need to > >> parse "mount" output to account for problems with specific filesystems. > >> > >> I am just throwing around some ideas... > > > > That sounds pretty complicated. One idea would be the creation of a > > wiki where people could post their results, or ideally a tool that could > > read the output and load it into a database for analysis with other > > results. > > The OS X example is pretty cut-and-dry- it would be nice if there were > some kind of hints in the tool pointing in the right direction, or at > least a few words of warning: "the fastest option may not be the safest- > read the docs". We have a wal reliability section in the docs that attempts to address this: http://developer.postgresql.org/pgdocs/postgres/wal-reliability.html -- Bruce Momjian <bruce@momjian.us> http://momjian.us EnterpriseDB http://enterprisedb.com + It's impossible for everything to be true. +