800gb_table_synchronize_seqscans_test.txt

text/plain

Filename: 800gb_table_synchronize_seqscans_test.txt
Type: text/plain
Part: 0
Message: Re: Parallel Seq Scan vs kernel read ahead
drowley@amd3990x:~$ cat bench.sql
select * from t where a < 0;

800GB table test to see if synchronize_seqscans make any difference. block size choice is 65536 pages at once for an 800GB table.

drowley@amd3990x:~$ ./run.sh 
VACUUM
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)

LOG:  parameter "synchronize_seqscans" changed to "on"
progress: 600.1 s, 0.0 tps, lat 338852.008 ms stddev 5.778
progress: 1200.1 s, 0.0 tps, lat 351535.978 ms stddev 22050.829
progress: 1800.0 s, 0.0 tps, lat 368101.666 ms stddev 12233.796
progress: 2400.1 s, 0.0 tps, lat 375688.326 ms stddev 603.229
progress: 3000.1 s, 0.0 tps, lat 372042.342 ms stddev 6508.545
progress: 3600.1 s, 0.0 tps, lat 382646.664 ms stddev 1783.991
progress: 4200.1 s, 0.0 tps, lat 361988.824 ms stddev 8863.446
progress: 4800.1 s, 0.0 tps, lat 399223.008 ms stddev 748.818
progress: 5400.1 s, 0.0 tps, lat 389623.176 ms stddev 9490.910
progress: 6000.1 s, 0.0 tps, lat 375013.275 ms stddev 2082.368
progress: 6600.1 s, 0.0 tps, lat 377468.443 ms stddev 2939.086
progress: 7200.1 s, 0.0 tps, lat 395761.169 ms stddev 33343.449
progress: 7800.1 s, 0.0 tps, lat 391496.694 ms stddev 525.280
progress: 8400.1 s, 0.0 tps, lat 384499.730 ms stddev 5190.770
progress: 9000.1 s, 0.0 tps, lat 370066.998 ms stddev 15298.661
progress: 9600.1 s, 0.0 tps, lat 367759.910 ms stddev 8883.792
progress: 10200.1 s, 0.0 tps, lat 358205.864 ms stddev 6157.981
progress: 10800.0 s, 0.0 tps, lat 379475.568 ms stddev 345.029
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 10800 s
number of transactions actually processed: 174
latency average = 372738.134 ms
latency stddev = 20030.941 ms
tps = 0.016091 (including connections establishing)
tps = 0.016091 (excluding connections establishing)
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)
LOG:  parameter "synchronize_seqscans" changed to "off"

progress: 600.1 s, 0.0 tps, lat 324110.243 ms stddev 4.264
progress: 1200.1 s, 0.0 tps, lat 320839.894 ms stddev 1461.685
progress: 1800.1 s, 0.0 tps, lat 319347.057 ms stddev 1216.770
progress: 2400.1 s, 0.0 tps, lat 317994.946 ms stddev 120.483
progress: 3000.1 s, 0.0 tps, lat 318296.652 ms stddev 249.609
progress: 3600.1 s, 0.0 tps, lat 318480.157 ms stddev 33.142
progress: 4200.0 s, 0.0 tps, lat 318053.507 ms stddev 22.743
progress: 4800.0 s, 0.0 tps, lat 317685.890 ms stddev 287.964
progress: 5400.1 s, 0.0 tps, lat 318927.415 ms stddev 5.574
progress: 6000.1 s, 0.0 tps, lat 318765.897 ms stddev 1292.618
progress: 6600.1 s, 0.0 tps, lat 320286.378 ms stddev 396.735
progress: 7200.0 s, 0.0 tps, lat 320036.296 ms stddev 1657.181
progress: 7800.1 s, 0.0 tps, lat 322015.642 ms stddev 12.878
progress: 8400.0 s, 0.0 tps, lat 322036.471 ms stddev 70.883
progress: 9000.0 s, 0.0 tps, lat 321968.224 ms stddev 29.876
progress: 9600.0 s, 0.0 tps, lat 322147.525 ms stddev 19.800
progress: 10200.1 s, 0.0 tps, lat 322089.086 ms stddev 8.655
progress: 10800.0 s, 0.0 tps, lat 321981.674 ms stddev 150.991
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 10800 s
number of transactions actually processed: 204
latency average = 320204.028 ms
latency stddev = 1917.304 ms
tps = 0.018738 (including connections establishing)
tps = 0.018738 (excluding connections establishing)


Capping the maximum step size to 8192 gives:
I only ran these for 30 mins each. 

drowley@amd3990x:~$ ./run.sh 
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)

(LOG:  parameter "synchronize_seqscans" changed to "on")

progress: 600.1 s, 0.0 tps, lat 321954.177 ms stddev 1.010
progress: 1200.1 s, 0.0 tps, lat 321841.054 ms stddev 184.794
progress: 1800.0 s, 0.0 tps, lat 322171.921 ms stddev 265.924
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 1800 s
number of transactions actually processed: 36
latency average = 321969.172 ms
latency stddev = 239.107 ms
tps = 0.018635 (including connections establishing)
tps = 0.018635 (excluding connections establishing)
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)

(LOG:  parameter "synchronize_seqscans" changed to "off")

progress: 600.1 s, 0.0 tps, lat 321585.626 ms stddev 13.019
progress: 1200.1 s, 0.0 tps, lat 321501.842 ms stddev 88.430
progress: 1800.0 s, 0.0 tps, lat 321340.781 ms stddev 36.354
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 1800 s
number of transactions actually processed: 36
latency average = 321389.523 ms
latency stddev = 178.607 ms
tps = 0.018669 (including connections establishing)
tps = 0.018669 (excluding connections establishing)


master branch:

drowley@amd3990x:~$ ./run.sh 
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)

(LOG:  parameter "synchronize_seqscans" changed to "on")

progress: 600.1 s, 0.0 tps, lat 0.000 ms stddev 0.000
progress: 1200.1 s, 0.0 tps, lat 1059885.963 ms stddev 0.916
progress: 1800.0 s, 0.0 tps, lat 0.000 ms stddev 0.000
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 1800 s
number of transactions actually processed: 12
latency average = 1070226.162 ms
latency stddev = 10340.200 ms
tps = 0.005606 (including connections establishing)
tps = 0.005606 (excluding connections establishing)
ALTER SYSTEM
 pg_reload_conf 
----------------
 t
(1 row)

(LOG:  parameter "synchronize_seqscans" changed to "off")

progress: 600.1 s, 0.0 tps, lat 0.000 ms stddev 0.000
progress: 1200.1 s, 0.0 tps, lat 1092429.699 ms stddev 0.388
progress: 1800.0 s, 0.0 tps, lat 0.000 ms stddev 0.000
transaction type: bench.sql
scaling factor: 1
query mode: simple
number of clients: 6
number of threads: 6
duration: 1800 s
number of transactions actually processed: 12
latency average = 1085846.859 ms
latency stddev = 6582.840 ms
tps = 0.005526 (including connections establishing)
tps = 0.005526 (excluding connections establishing)