Re: [Proposal] Add foreign-server health checks infrastructure
Önder Kalacı <onderkalaci@gmail.com>
From: Önder Kalacı <onderkalaci@gmail.com>
To: "kuroda.hayato@fujitsu.com" <kuroda.hayato@fujitsu.com>
Cc: Andres Freund <andres@anarazel.de>,
Fujii Masao <masao.fujii@oss.nttdata.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, "Shinya11.Kato@oss.nttdata.com" <Shinya11.Kato@oss.nttdata.com>,
"zyu@yugabyte.com" <zyu@yugabyte.com>, Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Date: 2022-10-03T08:59:01Z
Lists: pgsql-hackers
Hi Hayato Kuroda,
Thanks for the patch. I think there are some non-fdw extensions out there
which could benefit from this logic. That's why I want to first learn some
more about high-level design/goals of the patch a little more.
+/*
> + * Call callbacks for checking remote servers.
> + */
> +void
> +CallCheckingRemoteServersCallbacks(void)
Why do we run this periodically, but not when a specific connection is
going to be used? Wouldn't running it periodically prevent detecting some
already-closed sockets at the time of the connection used (e.g., we checked
10 seconds ago, the server died 5 seconds ago)?
In other words, what is the trade-off for calling
pgfdw_connection_check_internal() inside GetConnection() when we are about
to use a "cached" connection? I think that might simplify the patch as well.
+/*
> + * Helper function for pgfdw_connection_check
> + */
> +static bool
> +pgfdw_connection_check_internal(PGconn *conn)
> +{
Can we have this function/logic on Postgres core, so that other extensions
can also use?
+ AddWaitEventToSet(eventset, WL_SOCKET_CLOSED, PQsocket(conn), NULL,
> NULL);
What if PQsocket(conn) returns -1? Maybe we move certain connection state
checks into pgfdw_connection_check_internal() such that it is more generic?
I can think of checks like: conn!=NULL, PQsocket(conn) != PGINVALID_SOCKET,
PQstatus == CONNECTION_OK
+ eventset = CreateWaitEventSet(CurrentMemoryContext, 1);
> + AddWaitEventToSet(eventset, WL_SOCKET_CLOSED, PQsocket(conn), NULL,
> NULL);
> +
> + WaitEventSetWait(eventset, 0, &events, 1, 0);
> +
> + if (events.events & WL_SOCKET_CLOSED)
> + {
> + FreeWaitEventSet(eventset);
> + return false;
> + }
> + FreeWaitEventSet(eventset);
Do you see any performance implications of creating/freeing waitEventSets
per check? I wonder if we can somehow re-use the same waitEventSet by
modifyWaitEvent? I guess no, but still, if this check causes a performance
implication, can we somehow cache 1 waitEventSet per connection?
Thanks,
Onder KALACI
Developing the Citus extension @Microsoft
Commits
-
postgres_fdw: Extend postgres_fdw_get_connections to return user name.
- 4f08ab554577 18.0 landed
-
postgres_fdw: Fix bug in connection status check.
- 454aab4b738e 18.0 landed
-
postgres_fdw: Add connection status check to postgres_fdw_get_connections().
- 857df3cef7be 18.0 landed
-
postgres_fdw: Add "used_in_xact" column to postgres_fdw_get_connections().
- c297a47c5f8d 18.0 landed
-
doc: Enhance documentation for postgres_fdw_get_connections() output columns.
- 284c030a10b8 18.0 landed
-
Avoid reference to nonexistent array element in ExecInitAgg().
- 92957ed98c5c 16.0 cited
-
Add WL_SOCKET_CLOSED for socket shutdown events.
- 50e570a59e7f 15.0 cited
-
postgres_fdw: reestablish new connection if cached one is detected as broken.
- 32a9c0bdf493 14.0 cited