Support to define custom wait events for extensions
Masahiro Ikeda <ikedamsh@oss.nttdata.com>
From: Masahiro Ikeda <ikedamsh@oss.nttdata.com>
To: pgsql-hackers@lists.postgresql.org
Date: 2023-06-15T06:06:01Z
Lists: pgsql-hackers
Attachments
- 0001-Support-to-define-custom-wait-events-for-extensions.patch (text/x-diff) patch 0001
- 0003-Add-docs-to-define-custom-wait-events.patch (text/x-diff) patch 0003
- 0002-Add-a-extension-to-test-custom-wait-event.patch (text/x-diff) patch 0002
Hi,
Currently, only one PG_WAIT_EXTENSION event can be used as a
wait event for extensions. Therefore, in environments with multiple
extensions are installed, it could take time to identify which
extension is the bottleneck.
So, I'd like to support new APIs to define custom wait events
for extensions. It's discussed in [1].
I made patches to realize it. Although I have some TODOs,
I want to know your feedbacks. Please feel free to comment.
# Implementation of new APIs
I implemented 2 new APIs for extensions.
* RequestNamedExtensionWaitEventTranche()
* GetNamedExtensionWaitEventTranche()
Extensions can request custom wait events by calling
RequestNamedExtensionWaitEventTranche(). After that, use
GetNamedExtensionWaitEventTranche() to get the wait event information.
The APIs usage example by extensions are following.
```
shmem_request_hook = shmem_request;
shmem_startup_hook = shmem_startup;
static void
shmem_request(void)
{
/* request a custom wait event */
RequestNamedExtensionWaitEventTranche("custom_wait_event");
}
static void
shmem_startup(void)
{
/* get the wait event information */
custom_wait_event =
GetNamedExtensionWaitEventTranche("custom_wait_event");
}
void
extension_funtion()
{
(void) WaitLatch(MyLatch,
WL_LATCH_SET | WL_TIMEOUT | WL_EXIT_ON_PM_DEATH,
10L * 1000,
custom_wait_event); /* notify core with custom wait event */
ResetLatch(MyLatch);
}
```
# Implementation overview
I referenced the implementation of
RequestNamedLWLockTranche()/GetNamedLWLockTranche().
(0001-Support-to-define-custom-wait-events-for-extensions.patch)
Extensions calls RequestNamedExtensionWaitEventTranche() in
shmem_request_hook() to request wait events to be used by each
extension.
In the core, the requested wait events are dynamically registered in
shared
memory. The extension then obtains the wait event information with
GetNamedExtensionWaitEventTranche() and uses the value to notify the
core
that it is waiting.
When a string representing of the wait event is requested,
it returns the name defined by calling
RequestNamedExtensionWaitEventTranche().
# PoC extension
I created the PoC extension and you can use it, as shown here:
(0002-Add-a-extension-to-test-custom-wait-event.patch)
1. start PostgreSQL with the following configuration
shared_preload_libraries = 'inject_wait_event'
2. check wait events periodically
psql-1=# SELECT query, wait_event_type, wait_event FROM pg_stat_activity
WHERE backend_type = 'client backend' AND pid != pg_backend_pid() ;
psql-1=# \watch
3. execute a function to inject a wait event
psql-2=# CREATE EXTENSION inject_wait_event;
psql-2=# SELECT inject_wait_event();
4. check the custom wait event
You can see the following results of psql-1.
(..snip..)
query | wait_event_type | wait_event
-----------------------------+-----------------+------------
SELECT inject_wait_event(); | Extension | Extension
(1 row)
(..snip..)
(...about 10 seconds later ..)
query | wait_event_type | wait_event
-----------------------------+-----------------+-------------------
SELECT inject_wait_event(); | Extension | custom_wait_event
# requested wait event by the extension!
(1 row)
(..snip..)
# TODOs
* tests on windows (since I tested on Ubuntu 20.04 only)
* add custom wait events for existing contrib modules (ex. postgres_fdw)
* add regression code (but, it seems to be difficult)
* others? (Please let me know)
[1]
https://www.postgresql.org/message-id/81290a48-b25c-22a5-31a6-3feff5864fe3%40gmail.com
Regards,
--
Masahiro Ikeda
NTT DATA CORPORATION
Commits
-
Change custom wait events to use dynamic shared hash tables
- af720b4c50a1 17.0 landed
-
Support custom wait events for wait event type "Extension"
- c9af05465307 17.0 landed
-
Add WAIT_EVENT_{CLASS,ID}_MASK in wait_event.c
- 7395a90db87b 17.0 landed
-
worker_spi: Switch to TAP tests
- 320c311fda91 17.0 cited
-
Simplify some conditions related to [LW]Lock in generate-wait_event_types.pl
- c17164aec88c 17.0 landed