Re: Logging parallel worker draught

Benoit Lobréau <benoit.lobreau@dalibo.com>

From: Benoit Lobréau <benoit.lobreau@dalibo.com>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: "Imseih (AWS), Sami" <simseih@amazon.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Robert Haas <robertmhaas@gmail.com>, Amit Kapila <amit.kapila16@gmail.com>, Melanie Plageman <melanieplageman@gmail.com>, "Andrey M. Borodin" <x4mmm@yandex-team.ru>
Date: 2024-08-28T12:58:51Z
Lists: pgsql-hackers

Attachments

Hi,

Here is a new version of the patch. Sorry for the long delay, I was hit 
by a motivation drought and was quite busy otherwise.

The guc is now called `log_parallel_workers` and has three possible values:

* "none": disables logging
* "all": logs parallel worker info for all parallel queries or utilities
* "failure": logs only when the number of parallel workers planned 
couldn't be reached.

For this, I added several members to the EState struct.

Each gather node / gather merge node updates the values and the 
offending queries are displayed during standard_ExecutorEnd.

For CREATE INDEX / REINDEX on btree and brin, I check the parallel 
context struct (pcxt) during _bt_end_parallel() or _brin_end_parallel() 
and display a log message when needed.

For vacuum, I do the same in parallel_vacuum_end().

I added some information to the error message for parallel queries as an 
experiment. I find it useful, but it can be removed, if you re not 
convinced.

2024-08-27 15:59:11.089 CEST [54585] LOG:  1 parallel nodes planned (1 
obtained all their workers, 0 obtained none), 2 workers planned (2 
workers launched)
2024-08-27 15:59:11.089 CEST [54585] STATEMENT:  EXPLAIN (ANALYZE)
		SELECT i, avg(j) FROM test_pql GROUP BY i;

2024-08-27 15:59:14.006 CEST [54585] LOG:  2 parallel nodes planned (0 
obtained all their workers, 1 obtained none), 4 workers planned (1 
workers launched)
2024-08-27 15:59:14.006 CEST [54585] STATEMENT:  EXPLAIN (ANALYZE)
		SELECT i, avg(j) FROM test_pql GROUP BY i
		UNION
		SELECT i, avg(j) FROM test_pql GROUP BY i;

For CREATE INDEX / REDINDEX / VACUUMS:

2024-08-27 15:58:59.769 CEST [54521] LOG:  1 workers planned (0 workers 
launched)
2024-08-27 15:58:59.769 CEST [54521] STATEMENT:  REINDEX TABLE test_pql;

Do you think this is better?

I am not sure if a struct is needed to store the es_nworkers* and if the 
modification I did to parallel.h is ok.

Thanks to: Jehan-Guillaume de Rorthais, Guillaume Lelarge and Franck 
Boudehen for the help and motivation boost.

(sorry for the spam, I had to resend the mail to the list)

-- 
Benoit Lobréau
Consultant
http://dalibo.com