Thread

1. Add http connection service file functionality

   Andrew Jackson <andrewjackson947@gmail.com> — 2026-03-25T01:13:24Z

   Hello,
   
   This patch adds an exported function to the libpq-oauth shared object
   file that uses libcurl to look up connection service files
   from an HTTP address instead of just on the local filesystem.
   The goal here is to provide the ability for managed service
   operators a single source of truth for connection details.
   This enables a form of built-in libpq service discovery
   format. This would allow administrators to add, remove, and
   change hosts in multi host connection strings without
   coordinating with every end user who may hardcode their
   connection strings in a lot of different places.
   
   Currently libpq has functionality which accomplishes some of
   the above by allowing entry of connection parameters into
   LDAP servers[0], though this cannot be specified directly in
   a connection string (though there is a patch that adds this
   functionality [1]). Another potential issue here is that
   setting up LDAP infrastructure is a lot less accessible to
   many administrators than setting up an HTTP web server.
   
   The current state of this patch is very rough and is being
   presented as more of a RFC than anything else.
   Some obvious issues:
   1. Lots of duplicated logic between the parse_service_file_curl
      and parseServiceFile.
   2. Bundling this functionality in with libpq-oauth.so seems odd.
      It would probably make more sense to rename libpq-oauth.so to
      libpq-oauth.so to libpq-libcurl.so or create an entirely new
      .so file for this logic.
   
   Despite these shortcomings this approach may be a more natural
   alternative to previous attempts [2, 3] at allowing administrators
   to mix read-only/read-write nodes into overloaded A records.
   
   Would appreciate any feedback.
   
   Thanks,
   Andrew Jackson
   
   
   
   

2. Re: Add http connection service file functionality

   Andrew Jackson <andrewjackson947@gmail.com> — 2026-03-25T01:43:49Z

   Hello,
   
   Apologies, forgot to attach the past and the links to the references i made.
   
   [0]: https://www.postgresql.org/docs/current/libpq-ldap.html
   [1]: https://www.postgresql.org/message-id/flat/CAKK5BkFOFGfKJNbTuYBvE0PfpHmW8iZEmdNogaCYqjAOhtNgDg@mail.gmail.com
   [2]: https://www.postgresql.org/message-id/flat/CAKK5BkESSc69sp2TiTWHvvOHCUey0rDWXSrR9pinyRqyfamUYg@mail.gmail.com
   [3]: https://www.postgresql.org/message-id/AM9PR09MB49008B02CDF003054D5D4E00977DA@AM9PR09MB4900.eurprd09.prod.outlook.com
   
   
   Thanks
   Andrew Jackson
   
   On Tue, Mar 24, 2026 at 8:13 PM Andrew Jackson
   <andrewjackson947@gmail.com> wrote:
   >
   > Hello,
   >
   > This patch adds an exported function to the libpq-oauth shared object
   > file that uses libcurl to look up connection service files
   > from an HTTP address instead of just on the local filesystem.
   > The goal here is to provide the ability for managed service
   > operators a single source of truth for connection details.
   > This enables a form of built-in libpq service discovery
   > format. This would allow administrators to add, remove, and
   > change hosts in multi host connection strings without
   > coordinating with every end user who may hardcode their
   > connection strings in a lot of different places.
   >
   > Currently libpq has functionality which accomplishes some of
   > the above by allowing entry of connection parameters into
   > LDAP servers[0], though this cannot be specified directly in
   > a connection string (though there is a patch that adds this
   > functionality [1]). Another potential issue here is that
   > setting up LDAP infrastructure is a lot less accessible to
   > many administrators than setting up an HTTP web server.
   >
   > The current state of this patch is very rough and is being
   > presented as more of a RFC than anything else.
   > Some obvious issues:
   > 1. Lots of duplicated logic between the parse_service_file_curl
   >    and parseServiceFile.
   > 2. Bundling this functionality in with libpq-oauth.so seems odd.
   >    It would probably make more sense to rename libpq-oauth.so to
   >    libpq-oauth.so to libpq-libcurl.so or create an entirely new
   >    .so file for this logic.
   >
   > Despite these shortcomings this approach may be a more natural
   > alternative to previous attempts [2, 3] at allowing administrators
   > to mix read-only/read-write nodes into overloaded A records.
   >
   > Would appreciate any feedback.
   >
   > Thanks,
   > Andrew Jackson
   

3. Re: Add http connection service file functionality

   Andrew Jackson <andrewjackson947@gmail.com> — 2026-04-06T21:39:38Z

   Hello,
   
   I went ahead and repurposed this patch to implement connection
   parameter lookup similar to how it is currently done for LDAP. This
   means this will only look up the info for a single service and is only
   accessible via the pg_service.conf file. I also tested builds for with
   and without oauth configured and it builds so I think the CI issues
   should be fixed now. I also expanded unit tests significantly.
   
   Also I forgot to mention in my last emails that the implementation for
   the libcurl logic was mostly borrowed from the libcurl docs[0].
   
   Thanks,
   Andrew Jackson
   
   [0]: https://curl.se/libcurl/c/fopen.html