Re: Logging parallel worker draught
Benoit Lobréau <benoit.lobreau@dalibo.com>
From: Benoit Lobréau <benoit.lobreau@dalibo.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: "Imseih (AWS), Sami" <simseih@amazon.com>,
Alvaro Herrera <alvherre@alvh.no-ip.org>, Robert Haas
<robertmhaas@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>,
Melanie Plageman <melanieplageman@gmail.com>,
"Andrey M. Borodin" <x4mmm@yandex-team.ru>
Date: 2024-08-28T12:58:51Z
Lists: pgsql-hackers
Attachments
- 0001-Add-logging-for-parallel-worker-usage.patch (text/x-patch) patch 0001
Hi, Here is a new version of the patch. Sorry for the long delay, I was hit by a motivation drought and was quite busy otherwise. The guc is now called `log_parallel_workers` and has three possible values: * "none": disables logging * "all": logs parallel worker info for all parallel queries or utilities * "failure": logs only when the number of parallel workers planned couldn't be reached. For this, I added several members to the EState struct. Each gather node / gather merge node updates the values and the offending queries are displayed during standard_ExecutorEnd. For CREATE INDEX / REINDEX on btree and brin, I check the parallel context struct (pcxt) during _bt_end_parallel() or _brin_end_parallel() and display a log message when needed. For vacuum, I do the same in parallel_vacuum_end(). I added some information to the error message for parallel queries as an experiment. I find it useful, but it can be removed, if you re not convinced. 2024-08-27 15:59:11.089 CEST [54585] LOG: 1 parallel nodes planned (1 obtained all their workers, 0 obtained none), 2 workers planned (2 workers launched) 2024-08-27 15:59:11.089 CEST [54585] STATEMENT: EXPLAIN (ANALYZE) SELECT i, avg(j) FROM test_pql GROUP BY i; 2024-08-27 15:59:14.006 CEST [54585] LOG: 2 parallel nodes planned (0 obtained all their workers, 1 obtained none), 4 workers planned (1 workers launched) 2024-08-27 15:59:14.006 CEST [54585] STATEMENT: EXPLAIN (ANALYZE) SELECT i, avg(j) FROM test_pql GROUP BY i UNION SELECT i, avg(j) FROM test_pql GROUP BY i; For CREATE INDEX / REDINDEX / VACUUMS: 2024-08-27 15:58:59.769 CEST [54521] LOG: 1 workers planned (0 workers launched) 2024-08-27 15:58:59.769 CEST [54521] STATEMENT: REINDEX TABLE test_pql; Do you think this is better? I am not sure if a struct is needed to store the es_nworkers* and if the modification I did to parallel.h is ok. Thanks to: Jehan-Guillaume de Rorthais, Guillaume Lelarge and Franck Boudehen for the help and motivation boost. (sorry for the spam, I had to resend the mail to the list) -- Benoit Lobréau Consultant http://dalibo.com