Re: Performance degradation on concurrent COPY into a single relation in PG16.
Jakub Wartak <jakub.wartak@enterprisedb.com>
From: Jakub Wartak <jakub.wartak@enterprisedb.com>
To: Masahiko Sawada <sawada.mshk@gmail.com>
Cc: Heikki Linnakangas <hlinnaka@iki.fi>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2023-07-03T09:53:56Z
Lists: pgsql-hackers
Hi Masahiko, Out of curiosity I've tried and it is reproducible as you have stated : XFS @ 4.18.0-425.10.1.el8_7.x86_64: [root@rockyora ~]# time ./test test.1 1 total 200000 fallocate 200000 filewrite 0 real 0m5.868s user 0m0.035s sys 0m5.716s [root@rockyora ~]# time ./test test.2 2 total 200000 fallocate 100000 filewrite 100000 real 0m25.858s user 0m0.108s sys 0m3.596s [root@rockyora ~]# time ./test test.3 2 total 200000 fallocate 100000 filewrite 100000 real 0m25.927s user 0m0.091s sys 0m3.621s [root@rockyora ~]# time ./test test.4 1 total 200000 fallocate 200000 filewrite 0 real 0m3.044s user 0m0.043s sys 0m2.934s According to iostat and blktrace -d /dev/sda -o - | blkparse -i - output , the XFS issues sync writes while ext4 does not, xfs looks like constant loop of sync writes (D) by kworker/2:1H-kblockd: [..] 8,0 2 34172 24.115364875 312 D WS 44624928 + 16 [kworker/2:1H] 8,0 2 34173 24.115482679 0 C WS 44624928 + 16 [0] 8,0 2 34174 24.115548251 6501 A WS 42525760 + 16 <- (253,0) 34225216 8,0 2 34175 24.115548660 6501 A WS 44624960 + 16 <- (8,2) 42525760 8,0 2 34176 24.115549111 6501 Q WS 44624960 + 16 [test] 8,0 2 34177 24.115551351 6501 G WS 44624960 + 16 [test] 8,0 2 34178 24.115552111 6501 I WS 44624960 + 16 [test] 8,0 2 34179 24.115559713 312 D WS 44624960 + 16 [kworker/2:1H] 8,0 2 34180 24.115677217 0 C WS 44624960 + 16 [0] 8,0 2 34181 24.115743150 6501 A WS 42525792 + 16 <- (253,0) 34225248 8,0 2 34182 24.115743502 6501 A WS 44624992 + 16 <- (8,2) 42525792 8,0 2 34183 24.115743949 6501 Q WS 44624992 + 16 [test] 8,0 2 34184 24.115746175 6501 G WS 44624992 + 16 [test] 8,0 2 34185 24.115746918 6501 I WS 44624992 + 16 [test] 8,0 2 34186 24.115754492 312 D WS 44624992 + 16 [kworker/2:1H] So it looks like you are onto something. Regards, -J.
Commits
-
Avoid edge case in pg_visibility test with small shared_buffers
- 408209d6a9ae 16.3 landed
- 3a4837fc809a 17.0 landed
-
Fix bulk table extension when copying into multiple partitions
- 0002feb82096 16.1 landed
- 22655aa23132 17.0 landed
-
hio: Take number of prior relation extensions into account
- 82a4edabd272 17.0 landed
- d37ab378b6e7 16.0 landed
-
Fix performance regression in pg_strtointNN_safe functions
- 4e2e75cd29eb 16.0 landed
- 3845577cb55e 17.0 landed
-
Fix performance problem with new COPY DEFAULT code
- c1308ce2d922 16.0 landed
- b635ac03e802 17.0 landed
-
hio: Use ExtendBufferedRelBy() to extend tables more efficiently
- 00d1e02be249 16.0 cited
-
Add VACUUM/ANALYZE BUFFER_USAGE_LIMIT option
- 1cbbee033857 16.0 cited