Horton

Thread

  1. Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-03-10T02:20:09Z

    "Mikheev, Vadim" <vmikheev@SECTORBASE.COM> writes:
> Tom, could you run this test for different block sizes?
> Up to 32*8k?
>> 
>> You mean changing the amount written per write(), while holding the
>> total file size constant, right?

> Yes. Currently XLogWrite writes 8k blocks one by one. From what I've seen
> on Solaris we can use O_DSYNC there without changing XLogWrite to
> write() more than 1 block (if > 1 block is available for writing).
> But on other platforms write(BLOCKS_TO_WRITE * 8k) + fsync() probably will
> be
> faster than BLOCKS_TO_WRITE * write(8k) (for file opened with O_DSYNC)
> if BLOCKS_TO_WRITE > 1.
> I just wonder with what BLOCKS_TO_WRITE we'll see same times for both
> approaches.

Okay, I changed the program to
	char zbuffer[8192 * BLOCKS];
(all else the same)

and on HPUX 10.20 I get

$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c
$ time a.out

real    1m18.48s
user    0m0.04s
sys     0m34.69s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c
$ time a.out

real    0m35.10s
user    0m0.01s
sys     0m9.08s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c
$ time a.out

real    0m29.75s
user    0m0.01s
sys     0m5.23s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c
$ time a.out

real    0m22.77s
user    0m0.01s
sys     0m1.80s
$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c
$ time a.out

real    0m22.08s
user    0m0.01s
sys     0m1.25s


$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c
$ time a.out

real    0m20.64s
user    0m0.02s
sys     0m0.67s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c
$ time a.out

real    0m20.72s
user    0m0.01s
sys     0m0.57s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=32 tfsync.c
$ time a.out

real    0m20.59s
user    0m0.01s
sys     0m0.61s
$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c
$ time a.out

real    0m20.86s
user    0m0.01s
sys     0m0.69s

So I also see that there is no benefit to writing more than one block at
a time with ODSYNC.  And even at half a meg per write, DSYNC is slower
than ODSYNC with 8K per write!  Note the fairly high system-time
consumption for DSYNC, too.  I think this is not so much a matter of a
really good ODSYNC implementation, as a really bad DSYNC one ...

			regards, tom lane

  2. Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-03-10T02:41:24Z

    More numbers, these from a Powerbook G3 laptop running Linux 2.2:

[tgl@g3 tmp]$ uname -a
Linux g3 2.2.18-4hpmac #1 Thu Dec 21 15:16:15 MST 2000 ppc unknown

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m32.418s
user	0m0.020s
sys	0m14.020s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m10.894s
user	0m0.000s
sys	0m4.030s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m7.211s
user	0m0.000s
sys	0m2.200s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m4.441s
user	0m0.020s
sys	0m0.870s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m4.488s
user	0m0.000s
sys	0m0.640s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m3.725s
user	0m0.000s
sys	0m0.310s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m3.785s
user	0m0.000s
sys	0m0.290s

[tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c
[tgl@g3 tmp]$ time ./a.out

real	0m3.753s
user	0m0.010s
sys	0m0.300s


Starting to look like we should just use ODSYNC where available, and
forget about dumping more per write ...

			regards, tom lane

  3. Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace

    Denis Perchine <dyp@perchine.com> — 2001-03-10T07:17:11Z

    On Saturday 10 March 2001 08:41, Tom Lane wrote:
> More numbers, these from a Powerbook G3 laptop running Linux 2.2:

Eeegghhh. Sorry... But where did you get O_DSYNC on Linux?????
Maybe here?

bits/fcntl.h: # define O_DSYNC      O_SYNC

There is no any O_DSYNC in the kernel... Even in the latest 2.4.x.

> [tgl@g3 tmp]$ uname -a
> Linux g3 2.2.18-4hpmac #1 Thu Dec 21 15:16:15 MST 2000 ppc unknown
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=1 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m32.418s
> user	0m0.020s
> sys	0m14.020s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=4 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m10.894s
> user	0m0.000s
> sys	0m4.030s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=8 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m7.211s
> user	0m0.000s
> sys	0m2.200s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=32 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m4.441s
> user	0m0.020s
> sys	0m0.870s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_DSYNC -DBLOCKS=64 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m4.488s
> user	0m0.000s
> sys	0m0.640s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=1 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m3.725s
> user	0m0.000s
> sys	0m0.310s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=4 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m3.785s
> user	0m0.000s
> sys	0m0.290s
>
> [tgl@g3 tmp]$ gcc -Wall -O -DINIT_WRITE -DUSE_ODSYNC -DBLOCKS=64 tfsync.c
> [tgl@g3 tmp]$ time ./a.out
>
> real	0m3.753s
> user	0m0.010s
> sys	0m0.300s
>
>
> Starting to look like we should just use ODSYNC where available, and
> forget about dumping more per write ...
>
> 			regards, tom lane
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo@postgresql.org

-- 
Sincerely Yours,
Denis Perchine

----------------------------------
E-Mail: dyp@perchine.com
HomePage: http://www.perchine.com/dyp/
FidoNet: 2:5000/120.5
----------------------------------

  4. Re: AW: AW: AW: WAL does not recover gracefully from ou t-of -dis k-sp ace

    Tom Lane <tgl@sss.pgh.pa.us> — 2001-03-10T16:12:08Z

    Denis Perchine <dyp@perchine.com> writes:
> On Saturday 10 March 2001 08:41, Tom Lane wrote:
>> More numbers, these from a Powerbook G3 laptop running Linux 2.2:

> Eeegghhh. Sorry... But where did you get O_DSYNC on Linux?????

> bits/fcntl.h: # define O_DSYNC      O_SYNC

Hm, must be.  Okay, so those two sets of numbers should be taken as
fsync() and O_SYNC respectively.  Still the conclusion seems pretty
clear: the open() options are way more efficient than calling fsync()
separately.

			regards, tom lane