Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. meson: Fix install-quiet after clean

  2. oauth: Run Autoconf tests with correct compiler flags

  3. Link libpq with libdl if the platform needs that.

  4. Doc: correct spelling of meson switch.

  5. oauth: Correct SSL dependency for libpq-oauth.a

  6. oauth: Fix Autoconf build on macOS

  7. oauth: Move the builtin flow into a separate module

  8. Remove a stray "pgrminclude" annotation

  9. oauth: Simplify copy of PGoauthBearerRequest

  10. oauth: Improve validator docs on interruptibility

  11. oauth: Disallow synchronous DNS in libcurl

  12. oauth: Fix postcondition for set_timer on macOS

  13. oauth: Use IPv4-only issuer in oauth_validator tests

  14. Work around OAuth/EVFILT_TIMER quirk on NetBSD.

  15. oauth: Fix incorrect const markers in struct

  16. Add missing entry to oauth_validator test .gitignore

  17. cirrus: Temporarily fix libcurl link error

  18. Add support for OAUTHBEARER SASL mechanism

  19. libpq: Handle asynchronous actions during SASL

  20. require_auth: prepare for multiple SASL mechanisms

  21. Move PG_MAX_AUTH_TOKEN_LENGTH to libpq/auth.h

  22. Make SASL max message length configurable

  23. jsonapi: fully initialize dummy lexer

  24. common/jsonapi: support libpq as a client

  25. Remove fe_memutils from libpgcommon_shlib

  26. Revert ECPG's use of pnstrdup()

  27. Explicitly require password for SCRAM exchange

  28. Refactor SASL exchange to return tri-state status

  1. [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-06-08T16:37:46Z

    Hi all,

We've been working on ways to expand the list of third-party auth
methods that Postgres provides. Some example use cases might be "I want
to let anyone with a Google account read this table" or "let anyone who
belongs to this GitHub organization connect as a superuser".

Attached is a proof of concept that implements pieces of OAuth 2.0
federated authorization, via the OAUTHBEARER SASL mechanism from RFC
7628 [1]. Currently, only Linux is supported due to some ugly hacks in
the backend.

The architecture can support the following use cases, as long as your
OAuth issuer of choice implements the necessary specs, and you know how
to write a validator for your issuer's bearer tokens:

- Authentication only, where an external validator uses the bearer
token to determine the end user's identity, and Postgres decides
whether that user ID is authorized to connect via the standard pg_ident
user mapping.

- Authorization only, where the validator uses the bearer token to
determine the allowed roles for the end user, and then checks to make
sure that the connection's role is one of those. This bypasses pg_ident
and allows pseudonymous connections, where Postgres doesn't care who
you are as long as the token proves you're allowed to assume the role
you want.

- A combination, where the validator provides both an authn_id (for
later audits of database access) and an authorization decision based on
the bearer token and role provided.

It looks kinda like this during use:

    $ psql 'host=example.org oauth_client_id=f02c6361-0635-...'
    Visit https://oauth.example.org/login and enter the code: FPQ2-M4BG

= Quickstart =

For anyone who likes building and seeing green tests ASAP.

Prerequisite software:
- iddawc v0.9.9 [2], library and dev headers, for client support
- Python 3, for the test suite only

(Some newer distributions have dev packages for iddawc, but mine did
not.)

Configure using --with-oauth (and, if you've installed iddawc into a
non-standard location, be sure to use --with-includes and --with-
libraries. Make sure either rpath or LD_LIBRARY_PATH will get you what
you need). Install as usual.

To run the test suite, make sure the contrib/authn_id extension is
installed, then init and start your dev cluster. No other configuration
is required; the test will do it for you. Switch to the src/test/python
directory, point your PG* envvars to a superuser connection on the
cluster (so that a "bare" psql will connect automatically), and run
`make installcheck`.

= Production Setup =

(but don't use this in production, please)

Actually setting up a "real" system requires knowing the specifics of
your third-party issuer of choice. Your issuer MUST implement OpenID
Discovery and the OAuth Device Authorization flow! Seriously, check
this before spending a lot of time writing a validator against an
issuer that can't actually talk to libpq.

The broad strokes are as follows:

1. Register a new public client with your issuer to get an OAuth client
ID for libpq. You'll use this as the oauth_client_id in the connection
string. (If your issuer doesn't support public clients and gives you a
client secret, you can use the oauth_client_secret connection parameter
to provide that too.)

The client you register must be able to use a device authorization
flow; some issuers require additional setup for that.

2. Set up your HBA with the 'oauth' auth method, and set the 'issuer'
and 'scope' options. 'issuer' is the base URL identifying your third-
party issuer (for example, https://accounts.google.com), and 'scope' is
the set of OAuth scopes that the client and server will need to
authenticate and/or authorize the user (e.g. "openid email").

So a sample HBA line might look like

    host all all samehost oauth issuer="https://accounts.google.com" scope="openid email"

3. In postgresql.conf, set up an oauth_validator_command that's capable
of verifying bearer tokens and implements the validator protocol. This
is the hardest part. See below.

= Design =

On the client side, I've implemented the Device Authorization flow (RFC
8628, [3]). What this means in practice is that libpq reaches out to a
third-party issuer (e.g. Google, Azure, etc.), identifies itself with a
client ID, and requests permission to act on behalf of the end user.
The issuer responds with a login URL and a one-time code, which libpq
presents to the user using the notice hook. The end user then navigates
to that URL, presents their code, authenticates to the issuer, and
grants permission for libpq to retrieve a bearer token. libpq grabs a
token and sends it to the server for verification.

(The bearer token, in this setup, is essentially a plaintext password,
and you must secure it like you would a plaintext password. The token
has an expiration date and can be explicitly revoked, which makes it
slightly better than a password, but this is still a step backwards
from something like SCRAM with channel binding. There are ways to bind
a bearer token to a client certificate [4], which would mitigate the
risk of token theft -- but your issuer has to support that, and I
haven't found much support in the wild.)

The server side is where things get more difficult for the DBA. The
OAUTHBEARER spec has this to say about the server side implementation:

   The server validates the response according to the specification for
   the OAuth Access Token Types used.

And here's what the Bearer Token specification [5] says:

   This document does not specify the encoding or the contents of the
   token; hence, detailed recommendations about the means of
   guaranteeing token integrity protection are outside the scope of
   this document.

It's the Wild West. Every issuer does their own thing in their own
special way. Some don't really give you a way to introspect information
about a bearer token at all, because they assume that the issuer of the
token and the consumer of the token are essentially the same service.
Some major players provide their own custom libraries, implemented in
your-language-of-choice, to deal with their particular brand of magic.

So I punted and added the oauth_validator_command GUC. A token
validator command reads the bearer token from a file descriptor that's
passed to it, then does whatever magic is necessary to validate that
token and find out who owns it. Optionally, it can look at the role
that's being connected and make sure that the token authorizes the user
to actually use that role. Then it says yea or nay to Postgres, and
optionally tells the server who the user is so that their ID can be
logged and mapped through pg_ident.

(See the commit message in 0005 for a full description of the protocol.
The test suite also has two toy implementations that illustrate the
protocol, but they provide zero security.)

This is easily the worst part of the patch, not only because my
implementation is a bad hack on OpenPipeStream(), but because it
balances the security of the entire system on the shoulders of a DBA
who does not have time to read umpteen OAuth specifications cover to
cover. More thought and coding effort is needed here, but I didn't want
to gold-plate a bad design. I'm not sure what alternatives there are
within the rules laid out by OAUTHBEARER. And the system is _extremely_
flexible, in the way that only code that's maintained by somebody else
can be.

= Patchset Roadmap =

The seven patches can be grouped into three:

1. Prep

  0001 decouples the SASL code from the SCRAM implementation.
  0002 makes it possible to use common/jsonapi from the frontend.
  0003 lets the json_errdetail() result be freed, to avoid leaks.

2. OAUTHBEARER Implementation

  0004 implements the client with libiddawc.
  0005 implements server HBA support and oauth_validator_command.

3. Testing

  0006 adds a simple test extension to retrieve the authn_id.
  0007 adds the Python test suite I've been developing against.

The first three patches are, hopefully, generally useful outside of
this implementation, and I'll plan to register them in the next
commitfest. The middle two patches are the "interesting" pieces, and
I've split them into client and server for ease of understanding,
though neither is particularly useful without the other.

The last two patches grew out of a test suite that I originally built
to be able to exercise NSS corner cases at the protocol/byte level. It
was incredibly helpful during implementation of this new SASL
mechanism, since I could write the client and server independently of
each other and get high coverage of broken/malicious implementations.
It's based on pytest and Construct, and the Python 3 requirement might
turn some away, but I wanted to include it in case anyone else wanted
to hack on the code. src/test/python/README explains more.

= Thoughts/Reflections =

...in no particular order.

I picked OAuth 2.0 as my first experiment in federated auth mostly
because I was already familiar with pieces of it. I think SAML (via the
SAML20 mechanism, RFC 6595) would be a good companion to this proof of
concept, if there is general interest in federated deployments.

I don't really like the OAUTHBEARER spec, but I'm not sure there's a
better alternative. Everything is left as an exercise for the reader.
It's not particularly extensible. Standard OAuth is built for
authorization, not authentication, and from reading the RFC's history,
it feels like it was a hack to just get something working. New
standards like OpenID Connect have begun to fill in the gaps, but the
SASL mechanisms have not kept up. (The OPENID20 mechanism is, to my
understanding, unrelated/obsolete.) And support for helpful OIDC
features seems to be spotty in the real world.

The iddawc dependency for client-side OAuth was extremely helpful to
develop this proof of concept quickly, but I don't think it would be an
appropriate component to build a real feature on. It's extremely
heavyweight -- it incorporates a huge stack of dependencies, including
a logging framework and a web server, to implement features we would
probably never use -- and it's fairly difficult to debug in practice.
If a device authorization flow were the only thing that libpq needed to
support natively, I think we should just depend on a widely used HTTP
client, like libcurl or neon, and implement the minimum spec directly
against the existing test suite.

There are a huge number of other authorization flows besides Device
Authorization; most would involve libpq automatically opening a web
browser for you. I felt like that wasn't an appropriate thing for a
library to do by default, especially when one of the most important
clients is a command-line application. Perhaps there could be a hook
for applications to be able to override the builtin flow and substitute
their own.

Since bearer tokens are essentially plaintext passwords, the relevant
specs require the use of transport-level protection, and I think it'd
be wise for the client to require TLS to be in place before performing
the initial handshake or sending a token.

Not every OAuth issuer is also an OpenID Discovery provider, so it's
frustrating that OAUTHBEARER (which is purportedly an OAuth 2.0
feature) requires OIDD for real-world implementations. Perhaps we could
hack around this with a data: URI or something.

The client currently performs the OAuth login dance every single time a
connection is made, but a proper OAuth client would cache its tokens to
reuse later, and keep an eye on their expiration times. This would make
daily use a little more like that of Kerberos, but we would have to
design a way to create and secure a token cache on disk.

If you've read this far, thank you for your interest, and I hope you
enjoy playing with it!

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc7628
[2] https://github.com/babelouest/iddawc
[3] https://datatracker.ietf.org/doc/html/rfc8628
[4] https://datatracker.ietf.org/doc/html/rfc8705
[5] https://datatracker.ietf.org/doc/html/rfc6750#section-5.2

  2. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Michael Paquier <michael@paquier.xyz> — 2021-06-18T04:07:06Z

    On Tue, Jun 08, 2021 at 04:37:46PM +0000, Jacob Champion wrote:
> 1. Prep
> 
>   0001 decouples the SASL code from the SCRAM implementation.
>   0002 makes it possible to use common/jsonapi from the frontend.
>   0003 lets the json_errdetail() result be freed, to avoid leaks.
>
> The first three patches are, hopefully, generally useful outside of
> this implementation, and I'll plan to register them in the next
> commitfest. The middle two patches are the "interesting" pieces, and
> I've split them into client and server for ease of understanding,
> though neither is particularly useful without the other.

Beginning with the beginning, could you spawn two threads for the
jsonapi rework and the SASL/SCRAM business?  I agree that these look
independently useful.  Glad to see someone improving the code with
SASL and SCRAM which are too inter-dependent now.  I saw in the RFCs
dedicated to OAUTH the need for the JSON part as well.

+#  define check_stack_depth()
+#  ifdef JSONAPI_NO_LOG
+#    define json_log_and_abort(...) \
+   do { fprintf(stderr, __VA_ARGS__); exit(1); } while(0)
+#  else
In patch 0002, this is the wrong approach.  libpq will not be able to
feed on such reports, and you cannot use any of the APIs from the
palloc() family either as these just fail on OOM.  libpq should be
able to know about the error, and would fill in the error back to the
application.  This abstraction is not necessary on HEAD as
pg_verifybackup is fine with this level of reporting.  My rough guess
is that we will need to split the existing jsonapi.c into two files,
one that can be used in shared libraries and a second that handles the 
errors.

+           /* TODO: SASL_EXCHANGE_FAILURE with output is forbidden in SASL */
            if (result == SASL_EXCHANGE_SUCCESS)
                sendAuthRequest(port,
                            AUTH_REQ_SASL_FIN,
                            output,
                            outputlen);
Perhaps that's an issue we need to worry on its own?  I didn't recall
this part..
--
Michael

  3. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Heikki Linnakangas <hlinnaka@iki.fi> — 2021-06-18T08:31:00Z

    On 08/06/2021 19:37, Jacob Champion wrote:
> We've been working on ways to expand the list of third-party auth
> methods that Postgres provides. Some example use cases might be "I want
> to let anyone with a Google account read this table" or "let anyone who
> belongs to this GitHub organization connect as a superuser".

Cool!

> The iddawc dependency for client-side OAuth was extremely helpful to
> develop this proof of concept quickly, but I don't think it would be an
> appropriate component to build a real feature on. It's extremely
> heavyweight -- it incorporates a huge stack of dependencies, including
> a logging framework and a web server, to implement features we would
> probably never use -- and it's fairly difficult to debug in practice.
> If a device authorization flow were the only thing that libpq needed to
> support natively, I think we should just depend on a widely used HTTP
> client, like libcurl or neon, and implement the minimum spec directly
> against the existing test suite.

You could punt and let the application implement that stuff. I'm 
imagining that the application code would look something like this:

conn = PQconnectStartParams(...);
for (;;)
{
     status = PQconnectPoll(conn)
     switch (status)
     {
         case CONNECTION_SASL_TOKEN_REQUIRED:
             /* open a browser for the user, get token */
             token = open_browser()
             PQauthResponse(token);
             break;
         ...
     }
}

It would be nice to have a simple default implementation, though, for 
psql and all the other client applications that come with PostgreSQL itself.

> If you've read this far, thank you for your interest, and I hope you
> enjoy playing with it!

A few small things caught my eye in the backend oauth_exchange function:

> +       /* Handle the client's initial message. */
> +       p = strdup(input);

this strdup() should be pstrdup().

In the same function, there are a bunch of reports like this:

>                    ereport(ERROR,
> +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
> +                           errmsg("malformed OAUTHBEARER message"),
> +                           errdetail("Comma expected, but found character \"%s\".",
> +                                     sanitize_char(*p))));

I don't think the double quotes are needed here, because sanitize_char 
will return quotes if it's a single character. So it would end up 
looking like this: ... found character "'x'".

- Heikki

  4. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-06-22T23:22:31Z

    On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> On 08/06/2021 19:37, Jacob Champion wrote:
> > We've been working on ways to expand the list of third-party auth
> > methods that Postgres provides. Some example use cases might be "I want
> > to let anyone with a Google account read this table" or "let anyone who
> > belongs to this GitHub organization connect as a superuser".
> 
> Cool!

Glad you think so! :D

> > The iddawc dependency for client-side OAuth was extremely helpful to
> > develop this proof of concept quickly, but I don't think it would be an
> > appropriate component to build a real feature on. It's extremely
> > heavyweight -- it incorporates a huge stack of dependencies, including
> > a logging framework and a web server, to implement features we would
> > probably never use -- and it's fairly difficult to debug in practice.
> > If a device authorization flow were the only thing that libpq needed to
> > support natively, I think we should just depend on a widely used HTTP
> > client, like libcurl or neon, and implement the minimum spec directly
> > against the existing test suite.
> 
> You could punt and let the application implement that stuff. I'm 
> imagining that the application code would look something like this:
> 
> conn = PQconnectStartParams(...);
> for (;;)
> {
>      status = PQconnectPoll(conn)
>      switch (status)
>      {
>          case CONNECTION_SASL_TOKEN_REQUIRED:
>              /* open a browser for the user, get token */
>              token = open_browser()
>              PQauthResponse(token);
>              break;
>          ...
>      }
> }

I was toying with the idea of having a callback for libpq clients,
where they could take full control of the OAuth flow if they wanted to.
Doing it inline with PQconnectPoll seems like it would work too. It has
a couple of drawbacks that I can see:

- If a client isn't currently using a poll loop, they'd have to switch
to one to be able to use OAuth connections. Not a difficult change, but
considering all the other hurdles to making this work, I'm hoping to
minimize the hoop-jumping.

- A client would still have to receive a bunch of OAuth parameters from
some new libpq API in order to construct the correct URL to visit, so
the overall complexity for implementers might be higher than if we just
passed those params directly in a callback.

> It would be nice to have a simple default implementation, though, for 
> psql and all the other client applications that come with PostgreSQL itself.

I agree. I think having a bare-bones implementation in libpq itself
would make initial adoption *much* easier, and then if specific
applications wanted to have richer control over an authorization flow,
then they could implement that themselves with the aforementioned
callback.

The Device Authorization flow was the most minimal working
implementation I could find, since it doesn't require a web browser on
the system, just the ability to print a prompt to the console. But if
anyone knows of a better flow for this use case, I'm all ears.

> > If you've read this far, thank you for your interest, and I hope you
> > enjoy playing with it!
> 
> A few small things caught my eye in the backend oauth_exchange function:
> 
> > +       /* Handle the client's initial message. */
> > +       p = strdup(input);
> 
> this strdup() should be pstrdup().

Thanks, I'll fix that in the next re-roll.

> In the same function, there are a bunch of reports like this:
> 
> >                    ereport(ERROR,
> > +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > +                           errmsg("malformed OAUTHBEARER message"),
> > +                           errdetail("Comma expected, but found character \"%s\".",
> > +                                     sanitize_char(*p))));
> 
> I don't think the double quotes are needed here, because sanitize_char 
> will return quotes if it's a single character. So it would end up 
> looking like this: ... found character "'x'".

I'll fix this too. Thanks!

--Jacob

  5. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-06-22T23:26:03Z

    On Fri, 2021-06-18 at 13:07 +0900, Michael Paquier wrote:
> On Tue, Jun 08, 2021 at 04:37:46PM +0000, Jacob Champion wrote:
> > 1. Prep
> > 
> >   0001 decouples the SASL code from the SCRAM implementation.
> >   0002 makes it possible to use common/jsonapi from the frontend.
> >   0003 lets the json_errdetail() result be freed, to avoid leaks.
> > 
> > The first three patches are, hopefully, generally useful outside of
> > this implementation, and I'll plan to register them in the next
> > commitfest. The middle two patches are the "interesting" pieces, and
> > I've split them into client and server for ease of understanding,
> > though neither is particularly useful without the other.
> 
> Beginning with the beginning, could you spawn two threads for the
> jsonapi rework and the SASL/SCRAM business?

Done [1, 2]. I've copied your comments into those threads with my
responses, and I'll have them registered in commitfest shortly.

Thanks!
--Jacob

[1] https://www.postgresql.org/message-id/3d2a6f5d50e741117d6baf83eb67ebf1a8a35a11.camel%40vmware.com
[2] https://www.postgresql.org/message-id/a250d475ba1c0cc0efb7dfec8e538fcc77cdcb8e.camel%40vmware.com

  6. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Michael Paquier <michael@paquier.xyz> — 2021-06-23T06:10:46Z

    On Tue, Jun 22, 2021 at 11:26:03PM +0000, Jacob Champion wrote:
> Done [1, 2]. I've copied your comments into those threads with my
> responses, and I'll have them registered in commitfest shortly.

Thanks!
--
Michael

  7. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-08-25T18:41:39Z

    On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> > 
> > A few small things caught my eye in the backend oauth_exchange function:
> > 
> > > +       /* Handle the client's initial message. */
> > > +       p = strdup(input);
> > 
> > this strdup() should be pstrdup().
> 
> Thanks, I'll fix that in the next re-roll.
> 
> > In the same function, there are a bunch of reports like this:
> > 
> > >                    ereport(ERROR,
> > > +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > +                           errmsg("malformed OAUTHBEARER message"),
> > > +                           errdetail("Comma expected, but found character \"%s\".",
> > > +                                     sanitize_char(*p))));
> > 
> > I don't think the double quotes are needed here, because sanitize_char 
> > will return quotes if it's a single character. So it would end up 
> > looking like this: ... found character "'x'".
> 
> I'll fix this too. Thanks!

v2, attached, incorporates Heikki's suggested fixes and also rebases on
top of latest HEAD, which had the SASL refactoring changes committed
last month.

The biggest change from the last patchset is 0001, an attempt at
enabling jsonapi in the frontend without the use of palloc(), based on
suggestions by Michael and Tom from last commitfest. I've also made
some improvements to the pytest suite. No major changes to the OAuth
implementation yet.

--Jacob

  8. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Zhihong Yu <zyu@yugabyte.com> — 2021-08-25T22:25:03Z

    On Wed, Aug 25, 2021 at 11:42 AM Jacob Champion <pchampion@vmware.com>
wrote:

> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> > >
> > > A few small things caught my eye in the backend oauth_exchange
> function:
> > >
> > > > +       /* Handle the client's initial message. */
> > > > +       p = strdup(input);
> > >
> > > this strdup() should be pstrdup().
> >
> > Thanks, I'll fix that in the next re-roll.
> >
> > > In the same function, there are a bunch of reports like this:
> > >
> > > >                    ereport(ERROR,
> > > > +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > > +                           errmsg("malformed OAUTHBEARER message"),
> > > > +                           errdetail("Comma expected, but found
> character \"%s\".",
> > > > +                                     sanitize_char(*p))));
> > >
> > > I don't think the double quotes are needed here, because sanitize_char
> > > will return quotes if it's a single character. So it would end up
> > > looking like this: ... found character "'x'".
> >
> > I'll fix this too. Thanks!
>
> v2, attached, incorporates Heikki's suggested fixes and also rebases on
> top of latest HEAD, which had the SASL refactoring changes committed
> last month.
>
> The biggest change from the last patchset is 0001, an attempt at
> enabling jsonapi in the frontend without the use of palloc(), based on
> suggestions by Michael and Tom from last commitfest. I've also made
> some improvements to the pytest suite. No major changes to the OAuth
> implementation yet.
>
> --Jacob
>
Hi,
For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :

+   /* Clean up. */
+   termJsonLexContext(&lex);

At the end of termJsonLexContext(), empty is copied to lex. For stack
based JsonLexContext, the copy seems unnecessary.
Maybe introduce a boolean parameter for termJsonLexContext() to signal that
the copy can be omitted ?

+#ifdef FRONTEND
+       /* make sure initialization succeeded */
+       if (lex->strval == NULL)
+           return JSON_OUT_OF_MEMORY;

Should PQExpBufferBroken(lex->strval) be used for the check ?

Thanks

  9. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Zhihong Yu <zyu@yugabyte.com> — 2021-08-25T23:24:19Z

    On Wed, Aug 25, 2021 at 3:25 PM Zhihong Yu <zyu@yugabyte.com> wrote:

>
>
> On Wed, Aug 25, 2021 at 11:42 AM Jacob Champion <pchampion@vmware.com>
> wrote:
>
>> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
>> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
>> > >
>> > > A few small things caught my eye in the backend oauth_exchange
>> function:
>> > >
>> > > > +       /* Handle the client's initial message. */
>> > > > +       p = strdup(input);
>> > >
>> > > this strdup() should be pstrdup().
>> >
>> > Thanks, I'll fix that in the next re-roll.
>> >
>> > > In the same function, there are a bunch of reports like this:
>> > >
>> > > >                    ereport(ERROR,
>> > > > +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
>> > > > +                           errmsg("malformed OAUTHBEARER message"),
>> > > > +                           errdetail("Comma expected, but found
>> character \"%s\".",
>> > > > +                                     sanitize_char(*p))));
>> > >
>> > > I don't think the double quotes are needed here, because
>> sanitize_char
>> > > will return quotes if it's a single character. So it would end up
>> > > looking like this: ... found character "'x'".
>> >
>> > I'll fix this too. Thanks!
>>
>> v2, attached, incorporates Heikki's suggested fixes and also rebases on
>> top of latest HEAD, which had the SASL refactoring changes committed
>> last month.
>>
>> The biggest change from the last patchset is 0001, an attempt at
>> enabling jsonapi in the frontend without the use of palloc(), based on
>> suggestions by Michael and Tom from last commitfest. I've also made
>> some improvements to the pytest suite. No major changes to the OAuth
>> implementation yet.
>>
>> --Jacob
>>
> Hi,
> For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
>
> +   /* Clean up. */
> +   termJsonLexContext(&lex);
>
> At the end of termJsonLexContext(), empty is copied to lex. For stack
> based JsonLexContext, the copy seems unnecessary.
> Maybe introduce a boolean parameter for termJsonLexContext() to signal
> that the copy can be omitted ?
>
> +#ifdef FRONTEND
> +       /* make sure initialization succeeded */
> +       if (lex->strval == NULL)
> +           return JSON_OUT_OF_MEMORY;
>
> Should PQExpBufferBroken(lex->strval) be used for the check ?
>
> Thanks
>
Hi,
For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :

+   i_init_session(&session);
+
+   if (!conn->oauth_client_id)
+   {
+       /* We can't talk to a server without a client identifier. */
+       appendPQExpBufferStr(&conn->errorMessage,
+                            libpq_gettext("no oauth_client_id is set for
the connection"));
+       goto cleanup;

Can conn->oauth_client_id check be performed ahead of i_init_session() ?
That way, ```goto cleanup``` can be replaced with return.

+       if (!error_code || (strcmp(error_code, "authorization_pending")
+                           && strcmp(error_code, "slow_down")))

What if, in the future, there is error code different from the above two
which doesn't represent "OAuth token retrieval failed" scenario ?

For client_initial_response(),

+   token_buf = createPQExpBuffer();
+   if (!token_buf)
+       goto cleanup;

If token_buf is NULL, there doesn't seem to be anything to free. We can
return directly.

Cheers

  10. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-08-26T16:13:08Z

    On Wed, 2021-08-25 at 15:25 -0700, Zhihong Yu wrote:
> 
> Hi,
> For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
> 
> +   /* Clean up. */
> +   termJsonLexContext(&lex); 
> 
> At the end of termJsonLexContext(), empty is copied to lex. For stack
> based JsonLexContext, the copy seems unnecessary.
> Maybe introduce a boolean parameter for termJsonLexContext() to
> signal that the copy can be omitted ?

Do you mean heap-based? i.e. destroyJsonLexContext() does an
unnecessary copy before free? Yeah, in that case it's not super useful,
but I think I'd want some evidence that the performance hit matters
before optimizing it.

Are there any other internal APIs that take a boolean parameter like
that? If not, I think we'd probably just want to remove the copy
entirely if it's a problem.

> +#ifdef FRONTEND
> +       /* make sure initialization succeeded */
> +       if (lex->strval == NULL)
> +           return JSON_OUT_OF_MEMORY;
> 
> Should PQExpBufferBroken(lex->strval) be used for the check ?

It should be okay to continue if the strval is broken but non-NULL,
since it's about to be reset. That has the fringe benefit of allowing
the function to go as far as possible without failing, though that's
probably a pretty weak justification.

In practice, do you think that the probability of success is low enough
that we should just short-circuit and be done with it?

On Wed, 2021-08-25 at 16:24 -0700, Zhihong Yu wrote:
> 
> For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :
> 
> +   i_init_session(&session);
> +
> +   if (!conn->oauth_client_id)
> +   {
> +       /* We can't talk to a server without a client identifier. */
> +       appendPQExpBufferStr(&conn->errorMessage,
> +                            libpq_gettext("no oauth_client_id is set for the connection"));
> +       goto cleanup;
> 
> Can conn->oauth_client_id check be performed ahead
> of i_init_session() ? That way, ```goto cleanup``` can be replaced
> with return.

Yeah, I think that makes sense. FYI, this is probably one of the
functions that will be rewritten completely once iddawc is removed.

> +       if (!error_code || (strcmp(error_code, "authorization_pending")
> +                           && strcmp(error_code, "slow_down")))
> 
> What if, in the future, there is error code different from the above
> two which doesn't represent "OAuth token retrieval failed" scenario ?

We'd have to update our code; that would be a breaking change to the
Device Authorization spec. Here's what it says today [1]:

   The "authorization_pending" and "slow_down" error codes define
   particularly unique behavior, as they indicate that the OAuth client
   should continue to poll the token endpoint by repeating the token
   request (implementing the precise behavior defined above).  If the
   client receives an error response with any other error code, it MUST
   stop polling and SHOULD react accordingly, for example, by displaying
   an error to the user.

> For client_initial_response(),
> 
> +   token_buf = createPQExpBuffer();
> +   if (!token_buf)
> +       goto cleanup;
> 
> If token_buf is NULL, there doesn't seem to be anything to free. We
> can return directly.

That's true today, but implementations have a habit of changing. I
personally prefer not to introduce too many exit points from a function
that's already using goto. In my experience, that makes future
maintenance harder.

Thanks for the reviews! Have you been able to give the patchset a try
with an OAuth deployment?

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc8628#section-3.5

  11. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Zhihong Yu <zyu@yugabyte.com> — 2021-08-26T16:20:17Z

    On Thu, Aug 26, 2021 at 9:13 AM Jacob Champion <pchampion@vmware.com> wrote:

> On Wed, 2021-08-25 at 15:25 -0700, Zhihong Yu wrote:
> >
> > Hi,
> > For v2-0001-common-jsonapi-support-FRONTEND-clients.patch :
> >
> > +   /* Clean up. */
> > +   termJsonLexContext(&lex);
> >
> > At the end of termJsonLexContext(), empty is copied to lex. For stack
> > based JsonLexContext, the copy seems unnecessary.
> > Maybe introduce a boolean parameter for termJsonLexContext() to
> > signal that the copy can be omitted ?
>
> Do you mean heap-based? i.e. destroyJsonLexContext() does an
> unnecessary copy before free? Yeah, in that case it's not super useful,
> but I think I'd want some evidence that the performance hit matters
> before optimizing it.
>
> Are there any other internal APIs that take a boolean parameter like
> that? If not, I think we'd probably just want to remove the copy
> entirely if it's a problem.
>
> > +#ifdef FRONTEND
> > +       /* make sure initialization succeeded */
> > +       if (lex->strval == NULL)
> > +           return JSON_OUT_OF_MEMORY;
> >
> > Should PQExpBufferBroken(lex->strval) be used for the check ?
>
> It should be okay to continue if the strval is broken but non-NULL,
> since it's about to be reset. That has the fringe benefit of allowing
> the function to go as far as possible without failing, though that's
> probably a pretty weak justification.
>
> In practice, do you think that the probability of success is low enough
> that we should just short-circuit and be done with it?
>
> On Wed, 2021-08-25 at 16:24 -0700, Zhihong Yu wrote:
> >
> > For v2-0002-libpq-add-OAUTHBEARER-SASL-mechanism.patch :
> >
> > +   i_init_session(&session);
> > +
> > +   if (!conn->oauth_client_id)
> > +   {
> > +       /* We can't talk to a server without a client identifier. */
> > +       appendPQExpBufferStr(&conn->errorMessage,
> > +                            libpq_gettext("no oauth_client_id is set
> for the connection"));
> > +       goto cleanup;
> >
> > Can conn->oauth_client_id check be performed ahead
> > of i_init_session() ? That way, ```goto cleanup``` can be replaced
> > with return.
>
> Yeah, I think that makes sense. FYI, this is probably one of the
> functions that will be rewritten completely once iddawc is removed.
>
> > +       if (!error_code || (strcmp(error_code, "authorization_pending")
> > +                           && strcmp(error_code, "slow_down")))
> >
> > What if, in the future, there is error code different from the above
> > two which doesn't represent "OAuth token retrieval failed" scenario ?
>
> We'd have to update our code; that would be a breaking change to the
> Device Authorization spec. Here's what it says today [1]:
>
>    The "authorization_pending" and "slow_down" error codes define
>    particularly unique behavior, as they indicate that the OAuth client
>    should continue to poll the token endpoint by repeating the token
>    request (implementing the precise behavior defined above).  If the
>    client receives an error response with any other error code, it MUST
>    stop polling and SHOULD react accordingly, for example, by displaying
>    an error to the user.
>
> > For client_initial_response(),
> >
> > +   token_buf = createPQExpBuffer();
> > +   if (!token_buf)
> > +       goto cleanup;
> >
> > If token_buf is NULL, there doesn't seem to be anything to free. We
> > can return directly.
>
> That's true today, but implementations have a habit of changing. I
> personally prefer not to introduce too many exit points from a function
> that's already using goto. In my experience, that makes future
> maintenance harder.
>
> Thanks for the reviews! Have you been able to give the patchset a try
> with an OAuth deployment?
>
> --Jacob
>
> [1] https://datatracker.ietf.org/doc/html/rfc8628#section-3.5

Hi,
bq. destroyJsonLexContext() does an unnecessary copy before free? Yeah, in
that case it's not super useful,
but I think I'd want some evidence that the performance hit matters before
optimizing it.

Yes I agree.

bq. In practice, do you think that the probability of success is low enough
that we should just short-circuit and be done with it?

Haven't had a chance to try your patches out yet.
I will leave this to people who are more familiar with OAuth
implementation(s).

bq.  I personally prefer not to introduce too many exit points from a
function that's already using goto.

Fair enough.

Cheers

  12. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Michael Paquier <michael@paquier.xyz> — 2021-08-27T02:32:32Z

    On Thu, Aug 26, 2021 at 04:13:08PM +0000, Jacob Champion wrote:
> Do you mean heap-based? i.e. destroyJsonLexContext() does an
> unnecessary copy before free? Yeah, in that case it's not super useful,
> but I think I'd want some evidence that the performance hit matters
> before optimizing it.

As an authentication code path, the impact is minimal and my take on
that would be to keep the code simple.  Now if you'd really wish to
stress that without relying on the backend, one simple way is to use
pgbench -C -n with a mostly-empty script (one meta-command) coupled
with some profiling.
--
Michael

  13. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2021-08-31T20:48:43Z

    On Fri, 2021-08-27 at 11:32 +0900, Michael Paquier wrote:
> Now if you'd really wish to
> stress that without relying on the backend, one simple way is to use
> pgbench -C -n with a mostly-empty script (one meta-command) coupled
> with some profiling.

Ah, thanks! I'll add that to the toolbox.

--Jacob

  14. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2022-03-04T19:13:42Z

    Hi all,

v3 rebases this patchset over the top of Samay's pluggable auth
provider API [1], included here as patches 0001-3. The final patch in
the set ports the server implementation from a core feature to a
contrib module; to switch between the two approaches, simply leave out
that final patch.

There are still some backend changes that must be made to get this
working, as pointed out in 0009, and obviously libpq support still
requires code changes.

--Jacob

[1] https://www.postgresql.org/message-id/flat/CAJxrbyxTRn5P8J-p%2BwHLwFahK5y56PhK28VOb55jqMO05Y-DJw%40mail.gmail.com

  15. Re: [PoC] Federated Authn/z with OAUTHBEARER

    samay sharma <smilingsamay@gmail.com> — 2022-03-22T21:48:08Z

    Hi Jacob,

Thank you for porting this on top of the pluggable auth methods API. I've
addressed the feedback around other backend changes in my latest patch, but
the client side changes still remain. I had a few questions to understand
them better.

(a) What specifically do the client side changes in the patch implement?
(b) Are the changes you made on the client side specific to OAUTH or are
they about making SASL more generic? As an additional question, if someone
wanted to implement something similar on top of your patch, would they
still have to make client side changes?

Regards,
Samay

On Fri, Mar 4, 2022 at 11:13 AM Jacob Champion <pchampion@vmware.com> wrote:

> Hi all,
>
> v3 rebases this patchset over the top of Samay's pluggable auth
> provider API [1], included here as patches 0001-3. The final patch in
> the set ports the server implementation from a core feature to a
> contrib module; to switch between the two approaches, simply leave out
> that final patch.
>
> There are still some backend changes that must be made to get this
> working, as pointed out in 0009, and obviously libpq support still
> requires code changes.
>
> --Jacob
>
> [1]
> https://www.postgresql.org/message-id/flat/CAJxrbyxTRn5P8J-p%2BwHLwFahK5y56PhK28VOb55jqMO05Y-DJw%40mail.gmail.com
>

  16. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2022-03-22T22:44:12Z

    On Tue, 2022-03-22 at 14:48 -0700, samay sharma wrote:
> Thank you for porting this on top of the pluggable auth methods API.
> I've addressed the feedback around other backend changes in my latest
> patch, but the client side changes still remain. I had a few
> questions to understand them better.
> 
> (a) What specifically do the client side changes in the patch implement?

Hi Samay,

The client-side changes are an implementation of the OAuth 2.0 Device
Authorization Grant [1] in libpq. The majority of the OAuth logic is
handled by the third-party iddawc library.

The server tells the client what OIDC provider to contact, and then
libpq prompts you to log into that provider on your
smartphone/browser/etc. using a one-time code. After you give libpq
permission to act on your behalf, the Bearer token gets sent to libpq
via a direct connection, and libpq forwards it to the server so that
the server can determine whether you're allowed in.

> (b) Are the changes you made on the client side specific to OAUTH or
> are they about making SASL more generic?

The original patchset included changes to make SASL more generic. Many
of those changes have since been merged, and the remaining code is
mostly OAuth-specific, but there are still improvements to be made.
(And there's some JSON crud to sift through in the first couple of
patches. I'm still mad that the OAUTHBEARER spec requires clients to
parse JSON in the first place.)

> As an additional question,
> if someone wanted to implement something similar on top of your
> patch, would they still have to make client side changes?

Any new SASL mechanisms require changes to libpq at this point. You
need to implement a new pg_sasl_mech, modify pg_SASL_init() to select
the mechanism correctly, and add whatever connection string options you
need, along with the associated state in pg_conn. Patch 0004 has all
the client-side magic for OAUTHBEARER.

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc8628

  17. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <pchampion@vmware.com> — 2022-03-26T00:00:22Z

    On Fri, 2022-03-04 at 19:13 +0000, Jacob Champion wrote:
> v3 rebases this patchset over the top of Samay's pluggable auth
> provider API [1], included here as patches 0001-3.

v4 rebases over the latest version of the pluggable auth patchset
(included as 0001-4). Note that there's a recent conflict as
of d4781d887; use an older commit as the base (or wait for the other
thread to be updated).

--Jacob

  18. Re: [PoC] Federated Authn/z with OAUTHBEARER

    mahendrakar s <mahendrakarforpg@gmail.com> — 2022-09-20T05:03:10Z

    Hi  Hackers,

We are trying to implement AAD(Azure AD) support in PostgreSQL and it
can be achieved with support of the OAuth method. To support AAD on
top of OAuth in a generic fashion (i.e for all other OAuth providers),
we are proposing this patch. It basically exposes two new hooks (one
for error reporting and one for OAuth provider specific token
validation) and passing OAuth bearer token to backend. It also adds
support for client credentials flow of OAuth additional to device code
flow which Jacob has proposed.

The changes for each component are summarized below.

1.     Provider-specific extension:
        Each OAuth provider implements their own token validator as an
extension. Extension registers an OAuth provider hook which is matched
to a line in the HBA file.

2.     Add support to pass on the OAuth bearer token. In this
obtaining the bearer token is left to 3rd party application or user.

        ./psql -U <username> -d 'dbname=postgres
oauth_client_id=<client_id> oauth_bearer_token=<token>

3.     HBA: An additional param ‘provider’ is added for the oauth method.
        Defining "oauth" as method + passing provider, issuer endpoint
and expected audience

        * * * * oauth   provider=<token validation extension>
issuer=.... scope=....

4.     Engine Backend:
        Support for generic OAUTHBEARER type, requesting client to
provide token and passing to token for provider-specific extension.

5.     Engine Frontend: Two-tiered approach.
           a)      libpq transparently passes on the token received
from 3rd party client as is to the backend.
           b)      libpq optionally compiled for the clients which
explicitly need libpq to orchestrate OAuth communication with the
issuer (it depends heavily on 3rd party library iddawc as Jacob
already pointed out. The library seems to be supporting all the OAuth
flows.)

Please let us know your thoughts as the proposed method supports
different OAuth flows with the use of provider specific hooks. We
think that the proposal would be useful for various OAuth providers.

Thanks,
Mahendrakar.


On Tue, 20 Sept 2022 at 10:18, Jacob Champion <pchampion@vmware.com> wrote:
>
> On Tue, 2021-06-22 at 23:22 +0000, Jacob Champion wrote:
> > On Fri, 2021-06-18 at 11:31 +0300, Heikki Linnakangas wrote:
> > >
> > > A few small things caught my eye in the backend oauth_exchange function:
> > >
> > > > +       /* Handle the client's initial message. */
> > > > +       p = strdup(input);
> > >
> > > this strdup() should be pstrdup().
> >
> > Thanks, I'll fix that in the next re-roll.
> >
> > > In the same function, there are a bunch of reports like this:
> > >
> > > >                    ereport(ERROR,
> > > > +                          (errcode(ERRCODE_PROTOCOL_VIOLATION),
> > > > +                           errmsg("malformed OAUTHBEARER message"),
> > > > +                           errdetail("Comma expected, but found character \"%s\".",
> > > > +                                     sanitize_char(*p))));
> > >
> > > I don't think the double quotes are needed here, because sanitize_char
> > > will return quotes if it's a single character. So it would end up
> > > looking like this: ... found character "'x'".
> >
> > I'll fix this too. Thanks!
>
> v2, attached, incorporates Heikki's suggested fixes and also rebases on
> top of latest HEAD, which had the SASL refactoring changes committed
> last month.
>
> The biggest change from the last patchset is 0001, an attempt at
> enabling jsonapi in the frontend without the use of palloc(), based on
> suggestions by Michael and Tom from last commitfest. I've also made
> some improvements to the pytest suite. No major changes to the OAuth
> implementation yet.
>
> --Jacob

  19. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-20T23:19:31Z

    Hi Mahendrakar, thanks for your interest and for the patch!

On Mon, Sep 19, 2022 at 10:03 PM mahendrakar s
<mahendrakarforpg@gmail.com> wrote:
> The changes for each component are summarized below.
>
> 1.     Provider-specific extension:
>         Each OAuth provider implements their own token validator as an
> extension. Extension registers an OAuth provider hook which is matched
> to a line in the HBA file.

How easy is it to write a Bearer validator using C? My limited
understanding was that most providers were publishing libraries in
higher-level languages.

Along those lines, sample validators will need to be provided, both to
help in review and to get the pytest suite green again. (And coverage
for the new code is important, too.)

> 2.     Add support to pass on the OAuth bearer token. In this
> obtaining the bearer token is left to 3rd party application or user.
>
>         ./psql -U <username> -d 'dbname=postgres
> oauth_client_id=<client_id> oauth_bearer_token=<token>

This hurts, but I think people are definitely going to ask for it, given
the frightening practice of copy-pasting these (incredibly sensitive
secret) tokens all over the place... Ideally I'd like to implement
sender constraints for the Bearer token, to *prevent* copy-pasting (or,
you know, outright theft). But I'm not sure that sender constraints are
well-implemented yet for the major providers.

> 3.     HBA: An additional param ‘provider’ is added for the oauth method.
>         Defining "oauth" as method + passing provider, issuer endpoint
> and expected audience
>
>         * * * * oauth   provider=<token validation extension>
> issuer=.... scope=....

Naming aside (this conflicts with Samay's previous proposal, I think), I
have concerns about the implementation. There's this code:

> +		if (oauth_provider && oauth_provider->name)
> +		{
> +			ereport(ERROR,
> +				(errmsg("OAuth provider \"%s\" is already loaded.",
> +					oauth_provider->name)));
> +		}

which appears to prevent loading more than one global provider. But
there's also code that deals with a provider list? (Again, it'd help to
have test code covering the new stuff.)

>            b)      libpq optionally compiled for the clients which
> explicitly need libpq to orchestrate OAuth communication with the
> issuer (it depends heavily on 3rd party library iddawc as Jacob
> already pointed out. The library seems to be supporting all the OAuth
> flows.)

Speaking of iddawc, I don't think it's a dependency we should choose to
rely on. For all the code that it has, it doesn't seem to provide
compatibility with several real-world providers.

Google, for one, chose not to follow the IETF spec it helped author, and
iddawc doesn't support its flavor of Device Authorization. At another
point, I think iddawc tried to decode Azure's Bearer tokens, which is
incorrect...

I haven't been able to check if those problems have been fixed in a
recent version, but if we're going to tie ourselves to a huge
dependency, I'd at least like to believe that said dependency is
battle-tested and solid, and personally I don't feel like iddawc is.

> -	auth_method = I_TOKEN_AUTH_METHOD_NONE;
> -	if (conn->oauth_client_secret && *conn->oauth_client_secret)
> -		auth_method = I_TOKEN_AUTH_METHOD_SECRET_BASIC;

This code got moved, but I'm not sure why? It doesn't appear to have
made a change to the logic.

> +	if (conn->oauth_client_secret && *conn->oauth_client_secret)
> +	{
> +		session_response_type = I_RESPONSE_TYPE_CLIENT_CREDENTIALS;
> +	}

Is this an Azure-specific requirement? Ideally a public client (which
psql is) shouldn't have to provide a secret to begin with, if I
understand that bit of the protocol correctly. I think Google also
required provider-specific changes in this part of the code, and
unfortunately I don't think they looked the same as yours.

We'll have to figure all that out... Standards are great; everyone has
one of their own. :)

Thanks,
--Jacob

  20. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-21T16:03:22Z

    On Tue, Sep 20, 2022 at 4:19 PM Jacob Champion <jchampion@timescale.com> wrote:
> > 2.     Add support to pass on the OAuth bearer token. In this
> > obtaining the bearer token is left to 3rd party application or user.
> >
> >         ./psql -U <username> -d 'dbname=postgres
> > oauth_client_id=<client_id> oauth_bearer_token=<token>
>
> This hurts, but I think people are definitely going to ask for it, given
> the frightening practice of copy-pasting these (incredibly sensitive
> secret) tokens all over the place...

After some further thought -- in this case, you already have an opaque
Bearer token (and therefore you already know, out of band, which
provider needs to be used), you're willing to copy-paste it from
whatever service you got it from, and you have an extension plugged
into Postgres on the backend that verifies this Bearer blob using some
procedure that Postgres knows nothing about.

Why do you need the OAUTHBEARER mechanism logic at that point? Isn't
that identical to a custom password scheme? It seems like that could
be handled completely by Samay's pluggable auth proposal.

--Jacob

  21. RE: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovskiy <andrey.chudnovskiy@microsoft.com> — 2022-09-21T22:10:25Z

    We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.

Libpq implementing OAUTHBEARER is needed for community/3rd party tools to have user-friendly authentication experience:
1. For community client tools, like pg_admin, psql etc. 
   Example experience: pg_admin would be able to open a popup dialog to authenticate customer and keep refresh token to avoid asking the user frequently.
2. For 3rd party connectors supporting generic OAUTH with any provider. Useful for datawiz clients, like Tableau or ETL tools. Those can support both user and client OAUTH flows.

Libpq passing toked directly from an upstream client is useful in other scenarios:
1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.

Thanks!
Andrey.

-----Original Message-----
From: Jacob Champion <jchampion@timescale.com> 
Sent: Wednesday, September 21, 2022 9:03 AM
To: mahendrakar s <mahendrakarforpg@gmail.com>
Cc: pgsql-hackers@postgresql.org; smilingsamay@gmail.com; andres@anarazel.de; Andrey Chudnovskiy <Andrey.Chudnovskiy@microsoft.com>; Mahendrakar Srinivasarao <mahendrakars@microsoft.com>
Subject: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

[You don't often get email from jchampion@timescale.com. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

On Tue, Sep 20, 2022 at 4:19 PM Jacob Champion <jchampion@timescale.com> wrote:
> > 2.     Add support to pass on the OAuth bearer token. In this
> > obtaining the bearer token is left to 3rd party application or user.
> >
> >         ./psql -U <username> -d 'dbname=postgres 
> > oauth_client_id=<client_id> oauth_bearer_token=<token>
>
> This hurts, but I think people are definitely going to ask for it, 
> given the frightening practice of copy-pasting these (incredibly 
> sensitive
> secret) tokens all over the place...

After some further thought -- in this case, you already have an opaque Bearer token (and therefore you already know, out of band, which provider needs to be used), you're willing to copy-paste it from whatever service you got it from, and you have an extension plugged into Postgres on the backend that verifies this Bearer blob using some procedure that Postgres knows nothing about.

Why do you need the OAUTHBEARER mechanism logic at that point? Isn't that identical to a custom password scheme? It seems like that could be handled completely by Samay's pluggable auth proposal.

--Jacob

  22. Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-21T22:31:29Z

    On Wed, Sep 21, 2022 at 3:10 PM Andrey Chudnovskiy
<Andrey.Chudnovskiy@microsoft.com> wrote:
> We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.

Right, I agree that we could potentially do both.

> Libpq passing toked directly from an upstream client is useful in other scenarios:
> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
> 2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.

What I don't understand is how the OAUTHBEARER mechanism helps you in
this case. You're short-circuiting the negotiation where the server
tells the client what provider to use and what scopes to request, and
instead you're saying "here's a secret string, just take it and
validate it with magic."

I realize the ability to pass an opaque token may be useful, but from
the server's perspective, I don't see what differentiates it from the
password auth method plus a custom authenticator plugin. Why pay for
the additional complexity of OAUTHBEARER if you're not going to use
it?

--Jacob

  23. Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-09-22T04:55:08Z

    First, My message from corp email wasn't displayed in the thread,
That is what Jacob replied to, let me post it here for context:

> We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtain one.
>
> Libpq implementing OAUTHBEARER is needed for community/3rd party tools to have user-friendly authentication experience:
>
> 1. For community client tools, like pg_admin, psql etc.
>   Example experience: pg_admin would be able to open a popup dialog to authenticate customers and keep refresh tokens to avoid asking the user frequently.
> 2. For 3rd party connectors supporting generic OAUTH with any provider. Useful for datawiz clients, like Tableau or ETL tools. Those can support both user and client OAUTH flows.
>
> Libpq passing toked directly from an upstream client is useful in other scenarios:
> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.
> 2. Resource-tight (like IoT) clients. Those can be compiled without the optional libpq flag not including the iddawc or other dependency.

-----------------------------------------------------------------------------------------------------
On this:

> What I don't understand is how the OAUTHBEARER mechanism helps you in
> this case. You're short-circuiting the negotiation where the server
> tells the client what provider to use and what scopes to request, and
> instead you're saying "here's a secret string, just take it and
> validate it with magic."
>
> I realize the ability to pass an opaque token may be useful, but from
> the server's perspective, I don't see what differentiates it from the
> password auth method plus a custom authenticator plugin. Why pay for
> the additional complexity of OAUTHBEARER if you're not going to use
> it?

Yes, passing a token as a new auth method won't make much sense in
isolation. However:
1. Since OAUTHBEARER is supported in the ecosystem, passing a token as
a way to authenticate with OAUTHBEARER is more consistent (IMO), then
passing it as a password.
2. Validation on the backend side doesn't depend on whether the token
is obtained by libpq or transparently passed by the upstream client.
3. Single OAUTH auth method on the server side for both scenarios,
would allow both enterprise clients with their own Token acquisition
and community clients using libpq flows to connect as the same PG
users/roles.

On Wed, Sep 21, 2022 at 8:36 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Wed, Sep 21, 2022 at 3:10 PM Andrey Chudnovskiy
> <Andrey.Chudnovskiy@microsoft.com> wrote:
> > We can support both passing the token from an upstream client and libpq implementing OAUTH2 protocol to obtaining one.
>
> Right, I agree that we could potentially do both.
>
> > Libpq passing toked directly from an upstream client is useful in other scenarios:
> > 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advance provider-specific token acquisition flows.
> > 2. Resource-tight (like IoT) clients. Those can be compiled without optional libpq flag not including the iddawc or other dependency.
>
> What I don't understand is how the OAUTHBEARER mechanism helps you in
> this case. You're short-circuiting the negotiation where the server
> tells the client what provider to use and what scopes to request, and
> instead you're saying "here's a secret string, just take it and
> validate it with magic."
>
> I realize the ability to pass an opaque token may be useful, but from
> the server's perspective, I don't see what differentiates it from the
> password auth method plus a custom authenticator plugin. Why pay for
> the additional complexity of OAUTHBEARER if you're not going to use
> it?
>
> --Jacob
>
>
>
>

  24. Re: [EXTERNAL] Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-22T21:53:55Z

    On 9/21/22 21:55, Andrey Chudnovsky wrote:
> First, My message from corp email wasn't displayed in the thread,

I see it on the public archives [1]. Your client is choosing some pretty
confusing quoting tactics, though, which you may want to adjust. :D

I have what I'll call some "skeptical curiosity" here -- you don't need
to defend your use cases to me by any means, but I'd love to understand
more about them.

> Yes, passing a token as a new auth method won't make much sense in
> isolation. However:
> 1. Since OAUTHBEARER is supported in the ecosystem, passing a token as
> a way to authenticate with OAUTHBEARER is more consistent (IMO), then
> passing it as a password.

Agreed. It's probably not a very strong argument for the new mechanism,
though, especially if you're not using the most expensive code inside it.

> 2. Validation on the backend side doesn't depend on whether the token
> is obtained by libpq or transparently passed by the upstream client.

Sure.

> 3. Single OAUTH auth method on the server side for both scenarios,
> would allow both enterprise clients with their own Token acquisition
> and community clients using libpq flows to connect as the same PG
> users/roles.

Okay, this is a stronger argument. With that in mind, I want to revisit
your examples and maybe provide some counterproposals:

>> Libpq passing toked directly from an upstream client is useful in other scenarios:
>> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.

I can see that providing a token directly would help you work around
limitations in libpq's "standard" OAuth flows, whether we use iddawc or
not. And it's cheap in terms of implementation. But I have a feeling it
would fall apart rapidly with error cases, where the server is giving
libpq information via the OAUTHBEARER mechanism, but libpq can only
communicate to your wrapper through human-readable error messages on stderr.

This seems like clear motivation for client-side SASL plugins (which
were also discussed on Samay's proposal thread). That's a lot more
expensive to implement in libpq, but if it were hypothetically
available, wouldn't you rather your provider-specific code be able to
speak OAUTHBEARER directly with the server?

>> 2. Resource-tight (like IoT) clients. Those can be compiled without the optional libpq flag not including the iddawc or other dependency.

I want to dig into this much more; resource-constrained systems are near
and dear to me. I can see two cases here:

Case 1: The device is an IoT client that wants to connect on its own
behalf. Why would you want to use OAuth in that case? And how would the
IoT device get its Bearer token to begin with? I'm much more used to
architectures that provision high-entropy secrets for this, whether
they're incredibly long passwords per device (in which case,
channel-bound SCRAM should be a fairly strong choice?) or client certs
(which can be better decentralized, but make for a lot of bookkeeping).

If the answer to that is, "we want an IoT client to be able to connect
using the same role as a person", then I think that illustrates a clear
need for SASL negotiation. That would let the IoT client choose
SCRAM-*-PLUS or EXTERNAL, and the person at the keyboard can choose
OAUTHBEARER. Then we have incredible flexibility, because you don't have
to engineer one mechanism to handle them all.

Case 2: The constrained device is being used as a jump point. So there's
an actual person at a keyboard, trying to get into a backend server
(maybe behind a firewall layer, etc.), and the middlebox is either not
web-connected or is incredibly tiny for some reason. That might be a
good use case for a copy-pasted Bearer token, but is there actual demand
for that use case? What motivation would you (or your end user) have for
choosing a fairly heavy, web-centric authentication method in such a
constrained environment?

Are there other resource-constrained use cases I've missed?

Thanks,
--Jacob

[1]
https://www.postgresql.org/message-id/MN0PR21MB31694BAC193ECE1807FD45358F4F9%40MN0PR21MB3169.namprd21.prod.outlook.com

  25. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-23T22:39:19Z

    On Fri, Mar 25, 2022 at 5:00 PM Jacob Champion <pchampion@vmware.com> wrote:
> v4 rebases over the latest version of the pluggable auth patchset
> (included as 0001-4). Note that there's a recent conflict as
> of d4781d887; use an older commit as the base (or wait for the other
> thread to be updated).

Here's a newly rebased v5. (They're all zipped now, which I probably
should have done a while back, sorry.)

- As before, 0001-4 are the pluggable auth set; they've now diverged
from the official version over on the other thread [1].
- I'm not sure that 0005 is still completely coherent after the
rebase, given the recent changes to jsonapi.c. But for now, the tests
are green, and that should be enough to keep the conversation going.
- 0008 will hopefully be obsoleted when the SYSTEM_USER proposal [2] lands.

Thanks,
--Jacob

[1] https://www.postgresql.org/message-id/CAJxrbyxgFzfqby%2BVRCkeAhJnwVZE50%2BZLPx0JT2TDg9LbZtkCg%40mail.gmail.com
[2] https://www.postgresql.org/message-id/flat/7e692b8c-0b11-45db-1cad-3afc5b57409f@amazon.com

  26. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-09-27T01:39:28Z

    >>> Libpq passing toked directly from an upstream client is useful in other scenarios:
>>> 1. Enterprise clients, built with .Net / Java and using provider-specific authentication libraries, like MSAL for AAD. Those can also support more advanced provider-specific token acquisition flows.

> I can see that providing a token directly would help you work around
> limitations in libpq's "standard" OAuth flows, whether we use iddawc or
> not. And it's cheap in terms of implementation. But I have a feeling it
> would fall apart rapidly with error cases, where the server is giving
> libpq information via the OAUTHBEARER mechanism, but libpq can only
> communicate to your wrapper through human-readable error messages on stderr.

For the providing token directly, that would be primarily used for
scenarios where the same party controls both the server and the client
side wrapper.
I.e. The client knows how to get a token for a particular principal
and doesn't need any additional information other than human readable
messages.
Please clarify the scenarios where you see this falling apart.

I can provide an example in the cloud world. We (Azure) as well as
other providers offer ways to obtain OAUTH tokens for
Service-to-Service communication at IAAS / PAAS level.
on Azure "Managed Identity" feature integrated in Compute VM allows a
client to make a local http call to get a token. VM itself manages the
certificate livecycle, as well as implements the corresponding OAUTH
flow.
This capability is used by both our 1st party PAAS offerings, as well
as 3rd party services deploying on VMs or managed K8S clusters.
Here, the client doesn't need libpq assistance in obtaining the token.

> This seems like clear motivation for client-side SASL plugins (which
> were also discussed on Samay's proposal thread). That's a lot more
> expensive to implement in libpq, but if it were hypothetically
> available, wouldn't you rather your provider-specific code be able to
> speak OAUTHBEARER directly with the server?

I generally agree that pluggable auth layers in libpq could be
beneficial. However, as you pointed out in Samay's thread, that would
require a new distribution model for libpq / clients to optionally
include provider-specific logic.

My optimistic plan here would be to implement several core OAUTH flows
in libpq core which would be generic enough to support major
enterprise OAUTH providers:
1. Client Credentials flow (Client_id + Client_secret) for backend applications.
2. Authorization Code Flow with PKCE and/or Device code flow for GUI
applications.

(2.) above would require a protocol between libpq and upstream clients
to exchange several messages.
Your patch includes a way for libpq to deliver to the client a message
about the next authentication steps, so planned to build on top of
that.

A little about scenarios, we look at.
What we're trying to achieve here is an easy integration path for
multiple players in the ecosystem:
- Managed PaaS Postgres providers (both us and multi-cloud solutions)
- SaaS providers deploying postgres on IaaS/PaaS providers' clouds
- Tools - pg_admin, psql and other ones.
- BI, ETL, Federation and other scenarios where postgres is used as
the data source.

If we can offer a provider agnostic solution for Backend <=> libpq <=>
Upstreal client path, we can have all players above build support for
OAUTH credentials, managed by the cloud provider of their choice.

For us, that would mean:
- Better administrator experience with pg_admin / psql handling of the
AAD (Azure Active Directory) authentication flows.
- Path for integration solutions using Postgres to build AAD
authentication in their management experience.
- Ability to use AAD identity provider for any Postgres deployments
other than our 1st party PaaS offering.
- Ability to offer github as the identity provider for PaaS Postgres offering.

Other players in the ecosystem above would be able to get the same benefits.

Does that make sense and possible without provider specific libpq plugin?

-------------------------
On resource constrained scenarios.
> I want to dig into this much more; resource-constrained systems are near
> and dear to me. I can see two cases here:

I just referred to the ability to compile libpq without extra
dependencies to save some kilobytes.
Not sure if OAUTH is widely used in those cases. It involves overhead
anyway, and requires the device to talk to an additional party (OAUTH
provider).
Likely Cert authentication is easier.
If needed, it can get libpq with full OAUTH support and use a client
code. But I didn't think about this scenario.

On Fri, Sep 23, 2022 at 3:39 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Fri, Mar 25, 2022 at 5:00 PM Jacob Champion <pchampion@vmware.com> wrote:
> > v4 rebases over the latest version of the pluggable auth patchset
> > (included as 0001-4). Note that there's a recent conflict as
> > of d4781d887; use an older commit as the base (or wait for the other
> > thread to be updated).
>
> Here's a newly rebased v5. (They're all zipped now, which I probably
> should have done a while back, sorry.)
>
> - As before, 0001-4 are the pluggable auth set; they've now diverged
> from the official version over on the other thread [1].
> - I'm not sure that 0005 is still completely coherent after the
> rebase, given the recent changes to jsonapi.c. But for now, the tests
> are green, and that should be enough to keep the conversation going.
> - 0008 will hopefully be obsoleted when the SYSTEM_USER proposal [2] lands.
>
> Thanks,
> --Jacob
>
> [1] https://www.postgresql.org/message-id/CAJxrbyxgFzfqby%2BVRCkeAhJnwVZE50%2BZLPx0JT2TDg9LbZtkCg%40mail.gmail.com
> [2] https://www.postgresql.org/message-id/flat/7e692b8c-0b11-45db-1cad-3afc5b57409f@amazon.com

  27. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-27T21:45:55Z

    On Mon, Sep 26, 2022 at 6:39 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> For the providing token directly, that would be primarily used for
> scenarios where the same party controls both the server and the client
> side wrapper.
> I.e. The client knows how to get a token for a particular principal
> and doesn't need any additional information other than human readable
> messages.
> Please clarify the scenarios where you see this falling apart.

The most concrete example I can see is with the OAUTHBEARER error
response. If you want to eventually handle differing scopes per role,
or different error statuses (which the proof-of-concept currently
hardcodes as `invalid_token`), then the client can't assume it knows
what the server is going to say there. I think that's true even if you
control both sides and are hardcoding the provider.

How should we communicate those pieces to a custom client when it's
passing a token directly? The easiest way I can see is for the custom
client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
If you had to parse the libpq error message, I don't think that'd be
particularly maintainable.

> I can provide an example in the cloud world. We (Azure) as well as
> other providers offer ways to obtain OAUTH tokens for
> Service-to-Service communication at IAAS / PAAS level.
> on Azure "Managed Identity" feature integrated in Compute VM allows a
> client to make a local http call to get a token. VM itself manages the
> certificate livecycle, as well as implements the corresponding OAUTH
> flow.
> This capability is used by both our 1st party PAAS offerings, as well
> as 3rd party services deploying on VMs or managed K8S clusters.
> Here, the client doesn't need libpq assistance in obtaining the token.

Cool. To me that's the strongest argument yet for directly providing
tokens to libpq.

> My optimistic plan here would be to implement several core OAUTH flows
> in libpq core which would be generic enough to support major
> enterprise OAUTH providers:
> 1. Client Credentials flow (Client_id + Client_secret) for backend applications.
> 2. Authorization Code Flow with PKCE and/or Device code flow for GUI
> applications.

As long as it's clear to DBAs when to use which flow (because existing
documentation for that is hit-and-miss), I think it's reasonable to
eventually support multiple flows. Personally my preference would be
to start with one or two core flows, and expand outward once we're
sure that we do those perfectly. Otherwise the explosion of knobs and
buttons might be overwhelming, both to users and devs.

Related to the question of flows is the client implementation library.
I've mentioned that I don't think iddawc is production-ready. As far
as I'm aware, there is only one certified OpenID relying party written
in C, and that's... an Apache server plugin. That leaves us either
choosing an untested library, scouring the web for a "tested" library
(and hoping we're right in our assessment), or implementing our own
(which is going to tamp down enthusiasm for supporting many flows,
though that has its own set of benefits). If you know of any reliable
implementations with a C API, please let me know.

> (2.) above would require a protocol between libpq and upstream clients
> to exchange several messages.
> Your patch includes a way for libpq to deliver to the client a message
> about the next authentication steps, so planned to build on top of
> that.

Specifically it delivers that message to an end user. If you want a
generic machine client to be able to use that, then we'll need to talk
about how.

> A little about scenarios, we look at.
> What we're trying to achieve here is an easy integration path for
> multiple players in the ecosystem:
> - Managed PaaS Postgres providers (both us and multi-cloud solutions)
> - SaaS providers deploying postgres on IaaS/PaaS providers' clouds
> - Tools - pg_admin, psql and other ones.
> - BI, ETL, Federation and other scenarios where postgres is used as
> the data source.
>
> If we can offer a provider agnostic solution for Backend <=> libpq <=>
> Upstreal client path, we can have all players above build support for
> OAUTH credentials, managed by the cloud provider of their choice.

Well... I don't quite understand why we'd go to the trouble of
providing a provider-agnostic communication solution only to have
everyone write their own provider-specific client support. Unless
you're saying Microsoft would provide an officially blessed plugin for
the *server* side only, and Google would provide one of their own, and
so on.

The server side authorization is the only place where I think it makes
sense to specialize by default. libpq should remain agnostic, with the
understanding that we'll need to make hard decisions when a major
provider decides not to follow a spec.

> For us, that would mean:
> - Better administrator experience with pg_admin / psql handling of the
> AAD (Azure Active Directory) authentication flows.
> - Path for integration solutions using Postgres to build AAD
> authentication in their management experience.
> - Ability to use AAD identity provider for any Postgres deployments
> other than our 1st party PaaS offering.
> - Ability to offer github as the identity provider for PaaS Postgres offering.

GitHub is unfortunately a bit tricky, unless they've started
supporting OpenID recently?

> Other players in the ecosystem above would be able to get the same benefits.
>
> Does that make sense and possible without provider specific libpq plugin?

If the players involved implement the flows and follow the specs, yes.
That's a big "if", unfortunately. I think GitHub and Google are two
major players who are currently doing things their own way.

> I just referred to the ability to compile libpq without extra
> dependencies to save some kilobytes.
> Not sure if OAUTH is widely used in those cases. It involves overhead
> anyway, and requires the device to talk to an additional party (OAUTH
> provider).
> Likely Cert authentication is easier.
> If needed, it can get libpq with full OAUTH support and use a client
> code. But I didn't think about this scenario.

Makes sense. Thanks!

--Jacob

  28. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-09-30T14:47:34Z

    > The most concrete example I can see is with the OAUTHBEARER error
> response. If you want to eventually handle differing scopes per role,
> or different error statuses (which the proof-of-concept currently
> hardcodes as `invalid_token`), then the client can't assume it knows
> what the server is going to say there. I think that's true even if you
> control both sides and are hardcoding the provider.

Ok, I see the point. It's related to the topic of communication
between libpq and the upstream client.


> How should we communicate those pieces to a custom client when it's
> passing a token directly? The easiest way I can see is for the custom
> client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> If you had to parse the libpq error message, I don't think that'd be
> particularly maintainable.

I agree that parsing the message is not a sustainable way.
Could you provide more details on the SASL plugin approach you propose?

Specifically, is this basically a set of extension hooks for the client
side?
With the need for the client to be compiled with the plugins based on
the set of providers it needs.


> Well... I don't quite understand why we'd go to the trouble of
> providing a provider-agnostic communication solution only to have
> everyone write their own provider-specific client support. Unless
> you're saying Microsoft would provide an officially blessed plugin for
> the *server* side only, and Google would provide one of their own, and
> so on.

Yes, via extensions. Identity providers can open source extensions to
use their auth services outside of first party PaaS offerings.
For 3rd party Postgres PaaS or on premise deployments.


> The server side authorization is the only place where I think it makes
> sense to specialize by default. libpq should remain agnostic, with the
> understanding that we'll need to make hard decisions when a major
> provider decides not to follow a spec.

Completely agree with agnostic libpq. Though needs validation with
several major providers to know if this is possible.


> Specifically it delivers that message to an end user. If you want a
> generic machine client to be able to use that, then we'll need to talk
> about how.

Yes, that's what needs to be decided.
In both Device code and Authorization code scenarios, libpq and the
client would need to exchange a couple of pieces of metadata.
Plus, after success, the client should be able to access a refresh token
for further use.

Can we implement a generic protocol like for this between libpq and the
clients?

  29. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-09-30T20:45:29Z

    On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> > How should we communicate those pieces to a custom client when it's
> > passing a token directly? The easiest way I can see is for the custom
> > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > If you had to parse the libpq error message, I don't think that'd be
> > particularly maintainable.
>
> I agree that parsing the message is not a sustainable way.
> Could you provide more details on the SASL plugin approach you propose?
>
> Specifically, is this basically a set of extension hooks for the client side?
> With the need for the client to be compiled with the plugins based on
> the set of providers it needs.

That's a good question. I can see two broad approaches, with maybe
some ability to combine them into a hybrid:

1. If there turns out to be serious interest in having libpq itself
handle OAuth natively (with all of the web-facing code that implies,
and all of the questions still left to answer), then we might be able
to provide a "token hook" in the same way that we currently provide a
passphrase hook for OpenSSL keys. By default, libpq would use its
internal machinery to take the provider details, navigate its builtin
flow, and return the Bearer token. If you wanted to override that
behavior as a client, you could replace the builtin flow with your
own, by registering a set of callbacks.

2. Alternatively, OAuth support could be provided via a mechanism
plugin for some third-party SASL library (GNU libgsasl, Cyrus
libsasl2). We could provide an OAuth plugin in contrib that handles
the default flow. Other providers could publish their alternative
plugins to completely replace the OAUTHBEARER mechanism handling.

Approach (2) would make for some duplicated effort since every
provider has to write code to speak the OAUTHBEARER protocol. It might
simplify provider-specific distribution, since (at least for Cyrus) I
think you could build a single plugin that supports both the client
and server side. But it would be a lot easier to unknowingly (or
knowingly) break the spec, since you'd control both the client and
server sides. There would be less incentive to interoperate.

Finally, we could potentially take pieces from both, by having an
official OAuth mechanism plugin that provides a client-side hook to
override the flow. I have no idea if the benefits would offset the
costs of a plugin-for-a-plugin style architecture. And providers would
still be free to ignore it and just provide a full mechanism plugin
anyway.

> > Well... I don't quite understand why we'd go to the trouble of
> > providing a provider-agnostic communication solution only to have
> > everyone write their own provider-specific client support. Unless
> > you're saying Microsoft would provide an officially blessed plugin for
> > the *server* side only, and Google would provide one of their own, and
> > so on.
>
> Yes, via extensions. Identity providers can open source extensions to
> use their auth services outside of first party PaaS offerings.
> For 3rd party Postgres PaaS or on premise deployments.

Sounds reasonable.

> > The server side authorization is the only place where I think it makes
> > sense to specialize by default. libpq should remain agnostic, with the
> > understanding that we'll need to make hard decisions when a major
> > provider decides not to follow a spec.
>
> Completely agree with agnostic libpq. Though needs validation with
> several major providers to know if this is possible.

Agreed.

> > Specifically it delivers that message to an end user. If you want a
> > generic machine client to be able to use that, then we'll need to talk
> > about how.
>
> Yes, that's what needs to be decided.
> In both Device code and Authorization code scenarios, libpq and the
> client would need to exchange a couple of pieces of metadata.
> Plus, after success, the client should be able to access a refresh token for further use.
>
> Can we implement a generic protocol like for this between libpq and the clients?

I think we can probably prototype a callback hook for approach (1)
pretty quickly. (2) is a lot more work and investigation, but it's
work that I'm interested in doing (when I get the time). I think there
are other very good reasons to consider a third-party SASL library,
and some good lessons to be learned, even if the community decides not
to go down that road.

Thanks,
--Jacob

  30. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-10-03T18:04:27Z

    > I think we can probably prototype a callback hook for approach (1)
> pretty quickly. (2) is a lot more work and investigation, but it's
> work that I'm interested in doing (when I get the time). I think there
> are other very good reasons to consider a third-party SASL library,
> and some good lessons to be learned, even if the community decides not
> to go down that road.

Makes sense. We will work on (1.) and do some check if there are any
blockers for a shared solution to support github and google.

On Fri, Sep 30, 2022 at 1:45 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > > How should we communicate those pieces to a custom client when it's
> > > passing a token directly? The easiest way I can see is for the custom
> > > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > > If you had to parse the libpq error message, I don't think that'd be
> > > particularly maintainable.
> >
> > I agree that parsing the message is not a sustainable way.
> > Could you provide more details on the SASL plugin approach you propose?
> >
> > Specifically, is this basically a set of extension hooks for the client side?
> > With the need for the client to be compiled with the plugins based on
> > the set of providers it needs.
>
> That's a good question. I can see two broad approaches, with maybe
> some ability to combine them into a hybrid:
>
> 1. If there turns out to be serious interest in having libpq itself
> handle OAuth natively (with all of the web-facing code that implies,
> and all of the questions still left to answer), then we might be able
> to provide a "token hook" in the same way that we currently provide a
> passphrase hook for OpenSSL keys. By default, libpq would use its
> internal machinery to take the provider details, navigate its builtin
> flow, and return the Bearer token. If you wanted to override that
> behavior as a client, you could replace the builtin flow with your
> own, by registering a set of callbacks.
>
> 2. Alternatively, OAuth support could be provided via a mechanism
> plugin for some third-party SASL library (GNU libgsasl, Cyrus
> libsasl2). We could provide an OAuth plugin in contrib that handles
> the default flow. Other providers could publish their alternative
> plugins to completely replace the OAUTHBEARER mechanism handling.
>
> Approach (2) would make for some duplicated effort since every
> provider has to write code to speak the OAUTHBEARER protocol. It might
> simplify provider-specific distribution, since (at least for Cyrus) I
> think you could build a single plugin that supports both the client
> and server side. But it would be a lot easier to unknowingly (or
> knowingly) break the spec, since you'd control both the client and
> server sides. There would be less incentive to interoperate.
>
> Finally, we could potentially take pieces from both, by having an
> official OAuth mechanism plugin that provides a client-side hook to
> override the flow. I have no idea if the benefits would offset the
> costs of a plugin-for-a-plugin style architecture. And providers would
> still be free to ignore it and just provide a full mechanism plugin
> anyway.
>
> > > Well... I don't quite understand why we'd go to the trouble of
> > > providing a provider-agnostic communication solution only to have
> > > everyone write their own provider-specific client support. Unless
> > > you're saying Microsoft would provide an officially blessed plugin for
> > > the *server* side only, and Google would provide one of their own, and
> > > so on.
> >
> > Yes, via extensions. Identity providers can open source extensions to
> > use their auth services outside of first party PaaS offerings.
> > For 3rd party Postgres PaaS or on premise deployments.
>
> Sounds reasonable.
>
> > > The server side authorization is the only place where I think it makes
> > > sense to specialize by default. libpq should remain agnostic, with the
> > > understanding that we'll need to make hard decisions when a major
> > > provider decides not to follow a spec.
> >
> > Completely agree with agnostic libpq. Though needs validation with
> > several major providers to know if this is possible.
>
> Agreed.
>
> > > Specifically it delivers that message to an end user. If you want a
> > > generic machine client to be able to use that, then we'll need to talk
> > > about how.
> >
> > Yes, that's what needs to be decided.
> > In both Device code and Authorization code scenarios, libpq and the
> > client would need to exchange a couple of pieces of metadata.
> > Plus, after success, the client should be able to access a refresh token for further use.
> >
> > Can we implement a generic protocol like for this between libpq and the clients?
>
> I think we can probably prototype a callback hook for approach (1)
> pretty quickly. (2) is a lot more work and investigation, but it's
> work that I'm interested in doing (when I get the time). I think there
> are other very good reasons to consider a third-party SASL library,
> and some good lessons to be learned, even if the community decides not
> to go down that road.
>
> Thanks,
> --Jacob

  31. Re: [PoC] Federated Authn/z with OAUTHBEARER

    mahendrakar s <mahendrakarforpg@gmail.com> — 2022-11-23T09:58:32Z

    Hi,


We validated on  libpq handling OAuth natively with different flows
with different OIDC certified providers.

Flows: Device Code, Client Credentials and Refresh Token.
Providers: Microsoft, Google and Okta.
Also validated with OAuth provider Github.

We propose using OpenID Connect (OIDC) as the protocol, instead of
OAuth, as it is:
- Discovery mechanism to bridge the differences and provide metadata.
- Stricter protocol and certification process to reliably identify
which providers can be supported.
- OIDC is designed for authentication, while the main purpose of OAUTH is to
authorize applications on behalf of the user.

Github is not OIDC certified, so won’t be supported with this proposal.
However, it may be supported in the future through the ability for the
extension to provide custom discovery document content.

OpenID configuration has a well-known discovery mechanism
for the provider configuration URI which is
defined in OpenID Connect. It allows libpq to fetch
metadata about provider (i.e endpoints, supported grants, response types, etc).

In the attached patch (based on V2 patch in the thread and does not
contain Samay's changes):
- Provider can configure issuer url and scope through the options hook.)
- Server passes on an open discovery url and scope to libpq.
- Libpq handles OAuth flow based on the flow_type sent in the
connection string [1].
- Added callbacks to notify a structure to client tools if OAuth flow
requires user interaction.
- Pg backend uses hooks to validate bearer token.

Note that authentication code flow with PKCE for GUI clients is not
implemented yet.

Proposed next steps:
- Broaden discussion to reach agreement on the approach.
- Implement libpq changes without iddawc
- Prototype GUI flow with pgAdmin

Thanks,
Mahendrakar.

[1]:
connection string for refresh token flow:
./psql -U <user> -d 'dbname=postgres oauth_client_id=<client_id>
oauth_flow_type=<flowtype>  oauth_refresh_token=<refresh token>'

On Mon, 3 Oct 2022 at 23:34, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > I think we can probably prototype a callback hook for approach (1)
> > pretty quickly. (2) is a lot more work and investigation, but it's
> > work that I'm interested in doing (when I get the time). I think there
> > are other very good reasons to consider a third-party SASL library,
> > and some good lessons to be learned, even if the community decides not
> > to go down that road.
>
> Makes sense. We will work on (1.) and do some check if there are any
> blockers for a shared solution to support github and google.
>
> On Fri, Sep 30, 2022 at 1:45 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Fri, Sep 30, 2022 at 7:47 AM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > > How should we communicate those pieces to a custom client when it's
> > > > passing a token directly? The easiest way I can see is for the custom
> > > > client to speak the OAUTHBEARER protocol directly (e.g. SASL plugin).
> > > > If you had to parse the libpq error message, I don't think that'd be
> > > > particularly maintainable.
> > >
> > > I agree that parsing the message is not a sustainable way.
> > > Could you provide more details on the SASL plugin approach you propose?
> > >
> > > Specifically, is this basically a set of extension hooks for the client side?
> > > With the need for the client to be compiled with the plugins based on
> > > the set of providers it needs.
> >
> > That's a good question. I can see two broad approaches, with maybe
> > some ability to combine them into a hybrid:
> >
> > 1. If there turns out to be serious interest in having libpq itself
> > handle OAuth natively (with all of the web-facing code that implies,
> > and all of the questions still left to answer), then we might be able
> > to provide a "token hook" in the same way that we currently provide a
> > passphrase hook for OpenSSL keys. By default, libpq would use its
> > internal machinery to take the provider details, navigate its builtin
> > flow, and return the Bearer token. If you wanted to override that
> > behavior as a client, you could replace the builtin flow with your
> > own, by registering a set of callbacks.
> >
> > 2. Alternatively, OAuth support could be provided via a mechanism
> > plugin for some third-party SASL library (GNU libgsasl, Cyrus
> > libsasl2). We could provide an OAuth plugin in contrib that handles
> > the default flow. Other providers could publish their alternative
> > plugins to completely replace the OAUTHBEARER mechanism handling.
> >
> > Approach (2) would make for some duplicated effort since every
> > provider has to write code to speak the OAUTHBEARER protocol. It might
> > simplify provider-specific distribution, since (at least for Cyrus) I
> > think you could build a single plugin that supports both the client
> > and server side. But it would be a lot easier to unknowingly (or
> > knowingly) break the spec, since you'd control both the client and
> > server sides. There would be less incentive to interoperate.
> >
> > Finally, we could potentially take pieces from both, by having an
> > official OAuth mechanism plugin that provides a client-side hook to
> > override the flow. I have no idea if the benefits would offset the
> > costs of a plugin-for-a-plugin style architecture. And providers would
> > still be free to ignore it and just provide a full mechanism plugin
> > anyway.
> >
> > > > Well... I don't quite understand why we'd go to the trouble of
> > > > providing a provider-agnostic communication solution only to have
> > > > everyone write their own provider-specific client support. Unless
> > > > you're saying Microsoft would provide an officially blessed plugin for
> > > > the *server* side only, and Google would provide one of their own, and
> > > > so on.
> > >
> > > Yes, via extensions. Identity providers can open source extensions to
> > > use their auth services outside of first party PaaS offerings.
> > > For 3rd party Postgres PaaS or on premise deployments.
> >
> > Sounds reasonable.
> >
> > > > The server side authorization is the only place where I think it makes
> > > > sense to specialize by default. libpq should remain agnostic, with the
> > > > understanding that we'll need to make hard decisions when a major
> > > > provider decides not to follow a spec.
> > >
> > > Completely agree with agnostic libpq. Though needs validation with
> > > several major providers to know if this is possible.
> >
> > Agreed.
> >
> > > > Specifically it delivers that message to an end user. If you want a
> > > > generic machine client to be able to use that, then we'll need to talk
> > > > about how.
> > >
> > > Yes, that's what needs to be decided.
> > > In both Device code and Authorization code scenarios, libpq and the
> > > client would need to exchange a couple of pieces of metadata.
> > > Plus, after success, the client should be able to access a refresh token for further use.
> > >
> > > Can we implement a generic protocol like for this between libpq and the clients?
> >
> > I think we can probably prototype a callback hook for approach (1)
> > pretty quickly. (2) is a lot more work and investigation, but it's
> > work that I'm interested in doing (when I get the time). I think there
> > are other very good reasons to consider a third-party SASL library,
> > and some good lessons to be learned, even if the community decides not
> > to go down that road.
> >
> > Thanks,
> > --Jacob

  32. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-11-23T20:05:37Z

    On 11/23/22 01:58, mahendrakar s wrote:
> We validated on  libpq handling OAuth natively with different flows
> with different OIDC certified providers.
> 
> Flows: Device Code, Client Credentials and Refresh Token.
> Providers: Microsoft, Google and Okta.

Great, thank you!

> Also validated with OAuth provider Github.

(How did you get discovery working? I tried this and had to give up
eventually.)

> We propose using OpenID Connect (OIDC) as the protocol, instead of
> OAuth, as it is:
> - Discovery mechanism to bridge the differences and provide metadata.
> - Stricter protocol and certification process to reliably identify
> which providers can be supported.
> - OIDC is designed for authentication, while the main purpose of OAUTH is to
> authorize applications on behalf of the user.

How does this differ from the previous proposal? The OAUTHBEARER SASL
mechanism already relies on OIDC for discovery. (I think that decision
is confusing from an architectural and naming standpoint, but I don't
think they really had an alternative...)

> Github is not OIDC certified, so won’t be supported with this proposal.
> However, it may be supported in the future through the ability for the
> extension to provide custom discovery document content.

Right.

> OpenID configuration has a well-known discovery mechanism
> for the provider configuration URI which is
> defined in OpenID Connect. It allows libpq to fetch
> metadata about provider (i.e endpoints, supported grants, response types, etc).

Sure, but this is already how the original PoC works. The test suite
implements an OIDC provider, for instance. Is there something different
to this that I'm missing?

> In the attached patch (based on V2 patch in the thread and does not
> contain Samay's changes):
> - Provider can configure issuer url and scope through the options hook.)
> - Server passes on an open discovery url and scope to libpq.
> - Libpq handles OAuth flow based on the flow_type sent in the
> connection string [1].
> - Added callbacks to notify a structure to client tools if OAuth flow
> requires user interaction.
> - Pg backend uses hooks to validate bearer token.

Thank you for the sample!

> Note that authentication code flow with PKCE for GUI clients is not
> implemented yet.
> 
> Proposed next steps:
> - Broaden discussion to reach agreement on the approach.

High-level thoughts on this particular patch (I assume you're not
looking for low-level implementation comments yet):

0) The original hook proposal upthread, I thought, was about allowing
libpq's flow implementation to be switched out by the application. I
don't see that approach taken here. It's fine if that turned out to be a
bad idea, of course, but this patch doesn't seem to match what we were
talking about.

1) I'm really concerned about the sudden explosion of flows. We went
from one flow (Device Authorization) to six. It's going to be hard
enough to validate that *one* flow is useful and can be securely
deployed by end users; I don't think we're going to be able to maintain
six, especially in combination with my statement that iddawc is not an
appropriate dependency for us.

I'd much rather give applications the ability to use their own OAuth
code, and then maintain within libpq only the flows that are broadly
useful. This ties back to (0) above.

2) Breaking the refresh token into its own pseudoflow is, I think,
passing the buck onto the user for something that's incredibly security
sensitive. The refresh token is powerful; I don't really want it to be
printed anywhere, let alone copy-pasted by the user. Imagine the
phishing opportunities.

If we want to support refresh tokens, I believe we should be developing
a plan to cache and secure them within the client. They should be used
as an accelerator for other flows, not as their own flow.

3) I don't like the departure from the OAUTHBEARER mechanism that's
presented here. For one, since I can't see a sample plugin that makes
use of the "flow type" magic numbers that have been added, I don't
really understand why the extension to the mechanism is necessary.

For two, if we think OAUTHBEARER is insufficient, the people who wrote
it would probably like to hear about it. Claiming support for a spec,
and then implementing an extension without review from the people who
wrote the spec, is not something I'm personally interested in doing.

4) The test suite is still broken, so it's difficult to see these things
in practice for review purposes.

> - Implement libpq changes without iddawc

This in particular will be much easier with a functioning test suite,
and with a smaller number of flows.

> - Prototype GUI flow with pgAdmin

Cool!

Thanks,
--Jacob

  33. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-11-24T03:45:48Z

    > How does this differ from the previous proposal? The OAUTHBEARER SASL
> mechanism already relies on OIDC for discovery. (I think that decision
> is confusing from an architectural and naming standpoint, but I don't
> think they really had an alternative...)
Mostly terminology questions here. OAUTHBEARER SASL appears to be the
spec about using OAUTH2 tokens for Authentication.
While any OAUTH2 can generally work, we propose to specifically
highlight that only OIDC providers can be supported, as we need the
discovery document.
And we won't be able to support Github under that requirement.
Since the original patch used that too - no change on that, just
confirmation that we need OIDC compliance.

> 0) The original hook proposal upthread, I thought, was about allowing
> libpq's flow implementation to be switched out by the application. I
> don't see that approach taken here. It's fine if that turned out to be a
> bad idea, of course, but this patch doesn't seem to match what we were
> talking about.
We still plan to allow the client to pass the token. Which is a
generic way to implement its own OAUTH flows.

> 1) I'm really concerned about the sudden explosion of flows. We went
> from one flow (Device Authorization) to six. It's going to be hard
> enough to validate that *one* flow is useful and can be securely
> deployed by end users; I don't think we're going to be able to maintain
> six, especially in combination with my statement that iddawc is not an
> appropriate dependency for us.

> I'd much rather give applications the ability to use their own OAuth
> code, and then maintain within libpq only the flows that are broadly
> useful. This ties back to (0) above.
We consider the following set of flows to be minimum required:
- Client Credentials - For Service to Service scenarios.
- Authorization Code with PKCE - For rich clients,including pgAdmin.
- Device code - for psql (and possibly other non-GUI clients).
- Refresh code (separate discussion)
Which is pretty much the list described here:
https://oauth.net/2/grant-types/ and in OAUTH2 specs.
Client Credentials is very simple, so does Refresh Code.
If you prefer to pick one of the richer flows, Authorization code for
GUI scenarios is probably much more widely used.
Plus it's easier to implement too, as interaction goes through a
series of callbacks. No polling required.

> 2) Breaking the refresh token into its own pseudoflow is, I think,
> passing the buck onto the user for something that's incredibly security
> sensitive. The refresh token is powerful; I don't really want it to be
> printed anywhere, let alone copy-pasted by the user. Imagine the
> phishing opportunities.

> If we want to support refresh tokens, I believe we should be developing
> a plan to cache and secure them within the client. They should be used
> as an accelerator for other flows, not as their own flow.
It's considered a separate "grant_type" in the specs / APIs.
https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens

For the clients, it would be storing the token and using it to authenticate.
On the question of sensitivity, secure credentials stores are
different for each platform, with a lot of cloud offerings for this.
pgAdmin, for example, has its own way to secure credentials to avoid
asking users for passwords every time the app is opened.
I believe we should delegate the refresh token management to the clients.

>3) I don't like the departure from the OAUTHBEARER mechanism that's
> presented here. For one, since I can't see a sample plugin that makes
> use of the "flow type" magic numbers that have been added, I don't
> really understand why the extension to the mechanism is necessary.
I don't think it's much of a departure, but rather a separation of
responsibilities between libpq and upstream clients.
As libpq can be used in different apps, the client would need
different types of flows/grants.
I.e. those need to be provided to libpq at connection initialization
or some other point.
We will change to "grant_type" though and use string to be closer to the spec.
What do you think is the best way for the client to signal which OAUTH
flow should be used?

On Wed, Nov 23, 2022 at 12:05 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On 11/23/22 01:58, mahendrakar s wrote:
> > We validated on  libpq handling OAuth natively with different flows
> > with different OIDC certified providers.
> >
> > Flows: Device Code, Client Credentials and Refresh Token.
> > Providers: Microsoft, Google and Okta.
>
> Great, thank you!
>
> > Also validated with OAuth provider Github.
>
> (How did you get discovery working? I tried this and had to give up
> eventually.)
>
> > We propose using OpenID Connect (OIDC) as the protocol, instead of
> > OAuth, as it is:
> > - Discovery mechanism to bridge the differences and provide metadata.
> > - Stricter protocol and certification process to reliably identify
> > which providers can be supported.
> > - OIDC is designed for authentication, while the main purpose of OAUTH is to
> > authorize applications on behalf of the user.
>
> How does this differ from the previous proposal? The OAUTHBEARER SASL
> mechanism already relies on OIDC for discovery. (I think that decision
> is confusing from an architectural and naming standpoint, but I don't
> think they really had an alternative...)
>
> > Github is not OIDC certified, so won’t be supported with this proposal.
> > However, it may be supported in the future through the ability for the
> > extension to provide custom discovery document content.
>
> Right.
>
> > OpenID configuration has a well-known discovery mechanism
> > for the provider configuration URI which is
> > defined in OpenID Connect. It allows libpq to fetch
> > metadata about provider (i.e endpoints, supported grants, response types, etc).
>
> Sure, but this is already how the original PoC works. The test suite
> implements an OIDC provider, for instance. Is there something different
> to this that I'm missing?
>
> > In the attached patch (based on V2 patch in the thread and does not
> > contain Samay's changes):
> > - Provider can configure issuer url and scope through the options hook.)
> > - Server passes on an open discovery url and scope to libpq.
> > - Libpq handles OAuth flow based on the flow_type sent in the
> > connection string [1].
> > - Added callbacks to notify a structure to client tools if OAuth flow
> > requires user interaction.
> > - Pg backend uses hooks to validate bearer token.
>
> Thank you for the sample!
>
> > Note that authentication code flow with PKCE for GUI clients is not
> > implemented yet.
> >
> > Proposed next steps:
> > - Broaden discussion to reach agreement on the approach.
>
> High-level thoughts on this particular patch (I assume you're not
> looking for low-level implementation comments yet):
>
> 0) The original hook proposal upthread, I thought, was about allowing
> libpq's flow implementation to be switched out by the application. I
> don't see that approach taken here. It's fine if that turned out to be a
> bad idea, of course, but this patch doesn't seem to match what we were
> talking about.
>
> 1) I'm really concerned about the sudden explosion of flows. We went
> from one flow (Device Authorization) to six. It's going to be hard
> enough to validate that *one* flow is useful and can be securely
> deployed by end users; I don't think we're going to be able to maintain
> six, especially in combination with my statement that iddawc is not an
> appropriate dependency for us.
>
> I'd much rather give applications the ability to use their own OAuth
> code, and then maintain within libpq only the flows that are broadly
> useful. This ties back to (0) above.
>
> 2) Breaking the refresh token into its own pseudoflow is, I think,
> passing the buck onto the user for something that's incredibly security
> sensitive. The refresh token is powerful; I don't really want it to be
> printed anywhere, let alone copy-pasted by the user. Imagine the
> phishing opportunities.
>
> If we want to support refresh tokens, I believe we should be developing
> a plan to cache and secure them within the client. They should be used
> as an accelerator for other flows, not as their own flow.
>
> 3) I don't like the departure from the OAUTHBEARER mechanism that's
> presented here. For one, since I can't see a sample plugin that makes
> use of the "flow type" magic numbers that have been added, I don't
> really understand why the extension to the mechanism is necessary.
>
> For two, if we think OAUTHBEARER is insufficient, the people who wrote
> it would probably like to hear about it. Claiming support for a spec,
> and then implementing an extension without review from the people who
> wrote the spec, is not something I'm personally interested in doing.
>
> 4) The test suite is still broken, so it's difficult to see these things
> in practice for review purposes.
>
> > - Implement libpq changes without iddawc
>
> This in particular will be much easier with a functioning test suite,
> and with a smaller number of flows.
>
> > - Prototype GUI flow with pgAdmin
>
> Cool!
>
> Thanks,
> --Jacob

  34. Re: [PoC] Federated Authn/z with OAUTHBEARER

    mahendrakar s <mahendrakarforpg@gmail.com> — 2022-11-24T08:20:49Z

    Hi Jacob,

I had validated Github by skipping the discovery mechanism and letting
the provider extension pass on the endpoints. This is just for
validation purposes.
If it needs to be supported, then need a way to send the discovery
document from extension.


Thanks,
Mahendrakar.

On Thu, 24 Nov 2022 at 09:16, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > How does this differ from the previous proposal? The OAUTHBEARER SASL
> > mechanism already relies on OIDC for discovery. (I think that decision
> > is confusing from an architectural and naming standpoint, but I don't
> > think they really had an alternative...)
> Mostly terminology questions here. OAUTHBEARER SASL appears to be the
> spec about using OAUTH2 tokens for Authentication.
> While any OAUTH2 can generally work, we propose to specifically
> highlight that only OIDC providers can be supported, as we need the
> discovery document.
> And we won't be able to support Github under that requirement.
> Since the original patch used that too - no change on that, just
> confirmation that we need OIDC compliance.
>
> > 0) The original hook proposal upthread, I thought, was about allowing
> > libpq's flow implementation to be switched out by the application. I
> > don't see that approach taken here. It's fine if that turned out to be a
> > bad idea, of course, but this patch doesn't seem to match what we were
> > talking about.
> We still plan to allow the client to pass the token. Which is a
> generic way to implement its own OAUTH flows.
>
> > 1) I'm really concerned about the sudden explosion of flows. We went
> > from one flow (Device Authorization) to six. It's going to be hard
> > enough to validate that *one* flow is useful and can be securely
> > deployed by end users; I don't think we're going to be able to maintain
> > six, especially in combination with my statement that iddawc is not an
> > appropriate dependency for us.
>
> > I'd much rather give applications the ability to use their own OAuth
> > code, and then maintain within libpq only the flows that are broadly
> > useful. This ties back to (0) above.
> We consider the following set of flows to be minimum required:
> - Client Credentials - For Service to Service scenarios.
> - Authorization Code with PKCE - For rich clients,including pgAdmin.
> - Device code - for psql (and possibly other non-GUI clients).
> - Refresh code (separate discussion)
> Which is pretty much the list described here:
> https://oauth.net/2/grant-types/ and in OAUTH2 specs.
> Client Credentials is very simple, so does Refresh Code.
> If you prefer to pick one of the richer flows, Authorization code for
> GUI scenarios is probably much more widely used.
> Plus it's easier to implement too, as interaction goes through a
> series of callbacks. No polling required.
>
> > 2) Breaking the refresh token into its own pseudoflow is, I think,
> > passing the buck onto the user for something that's incredibly security
> > sensitive. The refresh token is powerful; I don't really want it to be
> > printed anywhere, let alone copy-pasted by the user. Imagine the
> > phishing opportunities.
>
> > If we want to support refresh tokens, I believe we should be developing
> > a plan to cache and secure them within the client. They should be used
> > as an accelerator for other flows, not as their own flow.
> It's considered a separate "grant_type" in the specs / APIs.
> https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens
>
> For the clients, it would be storing the token and using it to authenticate.
> On the question of sensitivity, secure credentials stores are
> different for each platform, with a lot of cloud offerings for this.
> pgAdmin, for example, has its own way to secure credentials to avoid
> asking users for passwords every time the app is opened.
> I believe we should delegate the refresh token management to the clients.
>
> >3) I don't like the departure from the OAUTHBEARER mechanism that's
> > presented here. For one, since I can't see a sample plugin that makes
> > use of the "flow type" magic numbers that have been added, I don't
> > really understand why the extension to the mechanism is necessary.
> I don't think it's much of a departure, but rather a separation of
> responsibilities between libpq and upstream clients.
> As libpq can be used in different apps, the client would need
> different types of flows/grants.
> I.e. those need to be provided to libpq at connection initialization
> or some other point.
> We will change to "grant_type" though and use string to be closer to the spec.
> What do you think is the best way for the client to signal which OAUTH
> flow should be used?
>
> On Wed, Nov 23, 2022 at 12:05 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On 11/23/22 01:58, mahendrakar s wrote:
> > > We validated on  libpq handling OAuth natively with different flows
> > > with different OIDC certified providers.
> > >
> > > Flows: Device Code, Client Credentials and Refresh Token.
> > > Providers: Microsoft, Google and Okta.
> >
> > Great, thank you!
> >
> > > Also validated with OAuth provider Github.
> >
> > (How did you get discovery working? I tried this and had to give up
> > eventually.)
> >
> > > We propose using OpenID Connect (OIDC) as the protocol, instead of
> > > OAuth, as it is:
> > > - Discovery mechanism to bridge the differences and provide metadata.
> > > - Stricter protocol and certification process to reliably identify
> > > which providers can be supported.
> > > - OIDC is designed for authentication, while the main purpose of OAUTH is to
> > > authorize applications on behalf of the user.
> >
> > How does this differ from the previous proposal? The OAUTHBEARER SASL
> > mechanism already relies on OIDC for discovery. (I think that decision
> > is confusing from an architectural and naming standpoint, but I don't
> > think they really had an alternative...)
> >
> > > Github is not OIDC certified, so won’t be supported with this proposal.
> > > However, it may be supported in the future through the ability for the
> > > extension to provide custom discovery document content.
> >
> > Right.
> >
> > > OpenID configuration has a well-known discovery mechanism
> > > for the provider configuration URI which is
> > > defined in OpenID Connect. It allows libpq to fetch
> > > metadata about provider (i.e endpoints, supported grants, response types, etc).
> >
> > Sure, but this is already how the original PoC works. The test suite
> > implements an OIDC provider, for instance. Is there something different
> > to this that I'm missing?
> >
> > > In the attached patch (based on V2 patch in the thread and does not
> > > contain Samay's changes):
> > > - Provider can configure issuer url and scope through the options hook.)
> > > - Server passes on an open discovery url and scope to libpq.
> > > - Libpq handles OAuth flow based on the flow_type sent in the
> > > connection string [1].
> > > - Added callbacks to notify a structure to client tools if OAuth flow
> > > requires user interaction.
> > > - Pg backend uses hooks to validate bearer token.
> >
> > Thank you for the sample!
> >
> > > Note that authentication code flow with PKCE for GUI clients is not
> > > implemented yet.
> > >
> > > Proposed next steps:
> > > - Broaden discussion to reach agreement on the approach.
> >
> > High-level thoughts on this particular patch (I assume you're not
> > looking for low-level implementation comments yet):
> >
> > 0) The original hook proposal upthread, I thought, was about allowing
> > libpq's flow implementation to be switched out by the application. I
> > don't see that approach taken here. It's fine if that turned out to be a
> > bad idea, of course, but this patch doesn't seem to match what we were
> > talking about.
> >
> > 1) I'm really concerned about the sudden explosion of flows. We went
> > from one flow (Device Authorization) to six. It's going to be hard
> > enough to validate that *one* flow is useful and can be securely
> > deployed by end users; I don't think we're going to be able to maintain
> > six, especially in combination with my statement that iddawc is not an
> > appropriate dependency for us.
> >
> > I'd much rather give applications the ability to use their own OAuth
> > code, and then maintain within libpq only the flows that are broadly
> > useful. This ties back to (0) above.
> >
> > 2) Breaking the refresh token into its own pseudoflow is, I think,
> > passing the buck onto the user for something that's incredibly security
> > sensitive. The refresh token is powerful; I don't really want it to be
> > printed anywhere, let alone copy-pasted by the user. Imagine the
> > phishing opportunities.
> >
> > If we want to support refresh tokens, I believe we should be developing
> > a plan to cache and secure them within the client. They should be used
> > as an accelerator for other flows, not as their own flow.
> >
> > 3) I don't like the departure from the OAUTHBEARER mechanism that's
> > presented here. For one, since I can't see a sample plugin that makes
> > use of the "flow type" magic numbers that have been added, I don't
> > really understand why the extension to the mechanism is necessary.
> >
> > For two, if we think OAUTHBEARER is insufficient, the people who wrote
> > it would probably like to hear about it. Claiming support for a spec,
> > and then implementing an extension without review from the people who
> > wrote the spec, is not something I'm personally interested in doing.
> >
> > 4) The test suite is still broken, so it's difficult to see these things
> > in practice for review purposes.
> >
> > > - Implement libpq changes without iddawc
> >
> > This in particular will be much easier with a functioning test suite,
> > and with a smaller number of flows.
> >
> > > - Prototype GUI flow with pgAdmin
> >
> > Cool!
> >
> > Thanks,
> > --Jacob

  35. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-11-29T21:12:21Z

    On 11/23/22 19:45, Andrey Chudnovsky wrote:
> Mostly terminology questions here. OAUTHBEARER SASL appears to be the
> spec about using OAUTH2 tokens for Authentication.
> While any OAUTH2 can generally work, we propose to specifically
> highlight that only OIDC providers can be supported, as we need the
> discovery document.

*If* you're using in-band discovery, yes. But I thought your use case
was explicitly tailored to out-of-band token retrieval:

> The client knows how to get a token for a particular principal
> and doesn't need any additional information other than human readable
> messages.

In that case, isn't OAuth sufficient? There's definitely a need to
document the distinction, but I don't think we have to require OIDC as
long as the client application makes up for the missing information.
(OAUTHBEARER makes the openid-configuration error member optional,
presumably for this reason.)

>> 0) The original hook proposal upthread, I thought, was about allowing
>> libpq's flow implementation to be switched out by the application. I
>> don't see that approach taken here. It's fine if that turned out to be a
>> bad idea, of course, but this patch doesn't seem to match what we were
>> talking about.
> We still plan to allow the client to pass the token. Which is a
> generic way to implement its own OAUTH flows.

Okay. But why push down the implementation into the server?

To illustrate what I mean, here's the architecture of my proposed patchset:

  +-------+                                          +----------+
  |       | -------------- Empty Token ------------> |          |
  | libpq | <----- Error Result (w/ Discovery ) ---- |          |
  |       |                                          |          |
  | +--------+                     +--------------+  |          |
  | | iddawc | <--- [ Flow ] ----> | Issuer/      |  | Postgres |
  | |        | <-- Access Token -- | Authz Server |  |          |
  | +--------+                     +--------------+  |   +-----------+
  |       |                                          |   |           |
  |       | -------------- Access Token -----------> | > | Validator |
  |       | <---- Authorization Success/Failure ---- | < |           |
  |       |                                          |   +-----------+
  +-------+                                          +----------+

In this implementation, there's only one black box: the validator, which
is responsible for taking an access token from an untrusted client,
verifying that it was issued correctly for the Postgres service, and
either 1) determining whether the bearer is authorized to access the
database, or 2) determining the authenticated ID of the bearer so that
the HBA can decide whether they're authorized. (Or both.)

This approach is limited by the flows that we explicitly enable within
libpq and its OAuth implementation library. You mentioned that you
wanted to support other flows, including clients with out-of-band
knowledge, and I suggested:

> If you wanted to override [iddawc's]
> behavior as a client, you could replace the builtin flow with your
> own, by registering a set of callbacks.

In other words, the hooks would replace iddawc in the above diagram.
In my mind, something like this:

     +-------+                                       +----------+
  +------+   | ----------- Empty Token ------------> | Postgres |
  |      | < | <---------- Error Result ------------ |          |
  | Hook |   |                                       |   +-----------+
  |      |   |                                       |   |           |
  +------+ > | ------------ Access Token ----------> | > | Validator |
     |       | <--- Authorization Success/Failure -- | < |           |
     | libpq |                                       |   +-----------+
     +-------+                                       +----------+

Now there's a second black box -- the client hook -- which takes an
OAUTHBEARER error result (which may or may not have OIDC discovery
information) and returns the access token. How it does this is
unspecified -- it'll probably use some OAuth 2.0 flow, but maybe not.
Maybe it sends the user to a web browser; maybe it uses some of the
magic provider-specific libraries you mentioned upthread. It might have
a refresh token cached so it doesn't have to involve the user at all.

Crucially, though, the two black boxes remain independent of each other.
They have well-defined inputs and outputs (the client hook could be
roughly described as "implement get_auth_token()"). Their correctness
can be independently verified against published OAuth specs and/or
provider documentation. And the client application still makes a single
call to PQconnect*().

Compare this to the architecture proposed by your patch:

  Client App
  +----------------------+
  |             +-------+                                +----------+
  |             | libpq |                                | Postgres |
  | PQconnect > |       |                                |   +-------+
  |          +------+   | ------- Flow Type (!) -------> | > |       |
  |     +- < | Hook | < | <------- Error Result -------- | < |       |
  | [ get    +------+   |                                |   |       |
  |   token ]   |       |                                |   |       |
  |     |       |       |                                |   | Hooks |
  |     v       |       |                                |   |       |
  | PQconnect > | ----> | ------ Access Token ---------> | > |       |
  |             |       | <--- Authz Success/Failure --- | < |       |
  |             +-------+                                |   +-------+
  +----------------------+                               +----------+

Rather than decouple things, I think this proposal drives a spike
through the client app, libpq, and the server. Please correct me if I've
misunderstood pieces of the patch, but the following is my view of it:

What used to be a validator hook on the server side now actively
participates in the client-side flow for some reason. (I still don't
understand what the server is supposed to do with that knowledge.
Changing your authz requirements based on the flow the client wants to
use seems like a good way to introduce bugs.)

The client-side hook is now coupled to the application logic: you have
to know to expect an error from the first PQconnect*() call, then check
whatever magic your hook has done for you to be able to set up the
second call to PQconnect*() with the correctly scoped bearer token. So
if you want to switch between the internal libpq OAuth implementation
and your own hook, you have to rewrite your app logic.

On top of all that, the "flow type code" being sent is a custom
extension to OAUTHBEARER that appears to be incompatible with the RFC's
discovery exchange (which is done by sending an empty auth token during
the first round trip).

> We consider the following set of flows to be minimum required:
> - Client Credentials - For Service to Service scenarios.

Okay, that's simple enough that I think it could probably be maintained
inside libpq with minimal cost. At the same time, is it complicated
enough that you need libpq to do it for you?

Maybe once we get the hooks ironed out, it'll be more obvious what the
tradeoff is...

> If you prefer to pick one of the richer flows, Authorization code for
> GUI scenarios is probably much more widely used.
> Plus it's easier to implement too, as interaction goes through a
> series of callbacks. No polling required.

I don't think flows requiring the invocation of web browsers and custom
URL handlers are a clear fit for libpq. For a first draft, at least, I
think that use case should be pushed upward into the client application
via a custom hook.

>> If we want to support refresh tokens, I believe we should be developing
>> a plan to cache and secure them within the client. They should be used
>> as an accelerator for other flows, not as their own flow.
> It's considered a separate "grant_type" in the specs / APIs.
> https://openid.net/specs/openid-connect-core-1_0.html#RefreshTokens

Yes, but that doesn't mean we have to expose it to users via a
connection option. You don't get a refresh token out of the blue; you
get it by going through some other flow, and then you use it in
preference to going through that flow again later.

> For the clients, it would be storing the token and using it to authenticate.
> On the question of sensitivity, secure credentials stores are
> different for each platform, with a lot of cloud offerings for this.
> pgAdmin, for example, has its own way to secure credentials to avoid
> asking users for passwords every time the app is opened.
> I believe we should delegate the refresh token management to the clients.

Delegating to client apps would be fine (and implicitly handled by a
token hook, because the client app would receive the refresh token
directly rather than going through libpq). Delegating to end users, not
so much. Printing a refresh token to stderr as proposed here is, I
think, making things unnecessarily difficult (and/or dangerous) for  users.

>> 3) I don't like the departure from the OAUTHBEARER mechanism that's
>> presented here. For one, since I can't see a sample plugin that makes
>> use of the "flow type" magic numbers that have been added, I don't
>> really understand why the extension to the mechanism is necessary.
> I don't think it's much of a departure, but rather a separation of
> responsibilities between libpq and upstream clients.

Given the proposed architectures above, 1) I think this is further
coupling the components, not separating them; and 2) I can't agree that
an incompatible discovery mechanism is "not much of a departure". If
OAUTHBEARER's functionality isn't good enough for some reason, let's
talk about why.

> As libpq can be used in different apps, the client would need
> different types of flows/grants.
> I.e. those need to be provided to libpq at connection initialization
> or some other point.

Why do libpq (or the server!) need to know those things at all, if
they're not going to implement the flow?

> We will change to "grant_type" though and use string to be closer to the spec.
> What do you think is the best way for the client to signal which OAUTH
> flow should be used?

libpq should not need to know the grant type in use if the client is
bypassing its internal implementation entirely.

Thanks,
--Jacob

  36. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-11-29T21:19:59Z

    On 11/24/22 00:20, mahendrakar s wrote:
> I had validated Github by skipping the discovery mechanism and letting
> the provider extension pass on the endpoints. This is just for
> validation purposes.
> If it needs to be supported, then need a way to send the discovery
> document from extension.

Yeah. I had originally bounced around the idea that we could send a
data:// URL, but I think that opens up problems.

You're supposed to be able to link the issuer URI with the URI you got
the configuration from, and if they're different, you bail out. If a
server makes up its own OpenID configuration, we'd have to bypass that
safety check, and decide what the risks and mitigations are... Not sure
it's worth it.

Especially if you could just lobby GitHub to, say, provide an OpenID
config. (Maybe there's a security-related reason they don't.)

--Jacob

  37. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-12-06T00:15:06Z

    Jacob,
Thanks for your feedback.
I think we can focus on the roles and responsibilities of the components first.
Details of the patch can be elaborated. Like "flow type code" is a
mistake on our side, and we will use the term "grant_type" which is
defined by OIDC spec. As well as details of usage of refresh_token.

> Rather than decouple things, I think this proposal drives a spike
> through the client app, libpq, and the server. Please correct me if I've
> misunderstood pieces of the patch, but the following is my view of it:

> What used to be a validator hook on the server side now actively
> participates in the client-side flow for some reason. (I still don't
> understand what the server is supposed to do with that knowledge.
> Changing your authz requirements based on the flow the client wants to
> use seems like a good way to introduce bugs.)

> The client-side hook is now coupled to the application logic: you have
> to know to expect an error from the first PQconnect*() call, then check
> whatever magic your hook has done for you to be able to set up the
> second call to PQconnect*() with the correctly scoped bearer token. So
> if you want to switch between the internal libpq OAuth implementation
> and your own hook, you have to rewrite your app logic.

Basically Yes. We propose an increase of the server side hook responsibility.
>From just validating the token, to also return the provider root URL
and required audience. And possibly provide more metadata in the
future.
Which is in our opinion aligned with SASL protocol, where the server
side is responsible for telling the client auth requirements based on
the requested role in the startup packet.

Our understanding is that in the original patch that information came
purely from hba, and we propose extension being able to control that
metadata.
As we see extension as being owned by the identity provider, compared
to HBA which is owned by the server administrator or cloud provider.

This change of the roles is based on the vision of 4 independent actor
types in the ecosystem:
1. Identity Providers (Okta, Google, Microsoft, other OIDC providers).
   - Publish open source extensions for PostgreSQL.
   - Don't have to own the server deployments, and must ensure their
extensions can work in any environment. This is where we think
additional hook responsibility helps.
2. Server Owners / PAAS providers (On premise admins, Cloud providers,
multi-cloud PAAS providers).
   - Install extensions and configure HBA to allow clients to
authenticate with the identity providers of their choice.
3. Client Application Developers (Data Wis, integration tools,
PgAdmin, monitoring tools, e.t.c.)
   - Independent from specific Identity providers or server providers.
Write one code for all identity providers.
   - Rely on application deployment owners to configure which OIDC
provider to use across client and server setups.
4. Application Deployment Owners (End customers setting up applications)
   - The only actor actually aware of which identity provider to use.
Configures the stack based on the Identity and PostgreSQL deployments
they have.

The critical piece of the vision is (3.) above is applications
agnostic of the identity providers. Those applications rely on
properly configured servers and rich driver logic (libpq,
com.postgresql, npgsql) to allow their application to popup auth
windows or do service-to-service authentication with any provider. In
our view that would significantly democratize the deployment of OAUTH
authentication in the community.

In order to allow this separation, we propose:
1. HBA + Extension is the single source of truth of Provider root URL
+ Required Audience for each role. If some backfill for missing OIDC
discovery is needed, the provider-specific extension would be
providing it.
2. Client Application knows which grant_type to use in which scenario.
But can be coded without knowledge of a specific provider. So can't
provide discovery details.
3. Driver (libpq, others) - coordinate the authentication flow based
on client grant_type and identity provider metadata to allow client
applications to use any flow with any provider in a unified way.

Yes, this would require a little more complicated flow between
components than in your original patch. And yes, more complexity comes
with more opportunity to make bugs.
However, I see PG Server and Libpq as the places which can have more
complexity. For the purpose of making work for the community
participants easier and simplify adoption.

Does this make sense to you?


On Tue, Nov 29, 2022 at 1:20 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On 11/24/22 00:20, mahendrakar s wrote:
> > I had validated Github by skipping the discovery mechanism and letting
> > the provider extension pass on the endpoints. This is just for
> > validation purposes.
> > If it needs to be supported, then need a way to send the discovery
> > document from extension.
>
> Yeah. I had originally bounced around the idea that we could send a
> data:// URL, but I think that opens up problems.
>
> You're supposed to be able to link the issuer URI with the URI you got
> the configuration from, and if they're different, you bail out. If a
> server makes up its own OpenID configuration, we'd have to bypass that
> safety check, and decide what the risks and mitigations are... Not sure
> it's worth it.
>
> Especially if you could just lobby GitHub to, say, provide an OpenID
> config. (Maybe there's a security-related reason they don't.)
>
> --Jacob

  38. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-12-07T19:06:07Z

    On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> I think we can focus on the roles and responsibilities of the components first.
> Details of the patch can be elaborated. Like "flow type code" is a
> mistake on our side, and we will use the term "grant_type" which is
> defined by OIDC spec. As well as details of usage of refresh_token.

(For the record, whether we call it "flow type" or "grant type"
doesn't address my concern.)

> Basically Yes. We propose an increase of the server side hook responsibility.
> From just validating the token, to also return the provider root URL
> and required audience. And possibly provide more metadata in the
> future.

I think it's okay to have the extension and HBA collaborate to provide
discovery information. Your proposal goes further than that, though,
and makes the server aware of the chosen client flow. That appears to
be an architectural violation: why does an OAuth resource server need
to know the client flow at all?

> Which is in our opinion aligned with SASL protocol, where the server
> side is responsible for telling the client auth requirements based on
> the requested role in the startup packet.

You've proposed an alternative SASL mechanism. There's nothing wrong
with that, per se, but I think it should be clear why we've chosen
something nonstandard.

> Our understanding is that in the original patch that information came
> purely from hba, and we propose extension being able to control that
> metadata.
> As we see extension as being owned by the identity provider, compared
> to HBA which is owned by the server administrator or cloud provider.

That seems reasonable, considering how tightly coupled the Issuer and
the token validation process are.

> 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> multi-cloud PAAS providers).
>    - Install extensions and configure HBA to allow clients to
> authenticate with the identity providers of their choice.

(For a future conversation: they need to set up authorization, too,
with custom scopes or some other magic. It's not enough to check who
the token belongs to; even if Postgres is just using the verified
email from OpenID as an authenticator, you have to also know that the
user authorized the token -- and therefore the client -- to access
Postgres on their behalf.)

> 3. Client Application Developers (Data Wis, integration tools,
> PgAdmin, monitoring tools, e.t.c.)
>    - Independent from specific Identity providers or server providers.
> Write one code for all identity providers.

Ideally, yes, but that only works if all identity providers implement
the same flows in compatible ways. We're already seeing instances
where that's not the case and we'll necessarily have to deal with that
up front.

>    - Rely on application deployment owners to configure which OIDC
> provider to use across client and server setups.
> 4. Application Deployment Owners (End customers setting up applications)
>    - The only actor actually aware of which identity provider to use.
> Configures the stack based on the Identity and PostgreSQL deployments
> they have.

(I have doubts that the roles will be as decoupled in practice as you
have described them, but I'd rather defer that for now.)

> The critical piece of the vision is (3.) above is applications
> agnostic of the identity providers. Those applications rely on
> properly configured servers and rich driver logic (libpq,
> com.postgresql, npgsql) to allow their application to popup auth
> windows or do service-to-service authentication with any provider. In
> our view that would significantly democratize the deployment of OAUTH
> authentication in the community.

That seems to be restating the goal of OAuth and OIDC. Can you explain
how the incompatible change allows you to accomplish this better than
standard implementations?

> In order to allow this separation, we propose:
> 1. HBA + Extension is the single source of truth of Provider root URL
> + Required Audience for each role. If some backfill for missing OIDC
> discovery is needed, the provider-specific extension would be
> providing it.
> 2. Client Application knows which grant_type to use in which scenario.
> But can be coded without knowledge of a specific provider. So can't
> provide discovery details.
> 3. Driver (libpq, others) - coordinate the authentication flow based
> on client grant_type and identity provider metadata to allow client
> applications to use any flow with any provider in a unified way.
>
> Yes, this would require a little more complicated flow between
> components than in your original patch.

Why? I claim that standard OAUTHBEARER can handle all of that. What
does your proposed architecture (the third diagram) enable that my
proposed hook (the second diagram) doesn't?

> And yes, more complexity comes
> with more opportunity to make bugs.
> However, I see PG Server and Libpq as the places which can have more
> complexity. For the purpose of making work for the community
> participants easier and simplify adoption.
>
> Does this make sense to you?

Some of it, but it hasn't really addressed the questions from my last mail.

Thanks,
--Jacob

  39. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-12-07T23:22:51Z

    > I think it's okay to have the extension and HBA collaborate to provide
> discovery information. Your proposal goes further than that, though,
> and makes the server aware of the chosen client flow. That appears to
> be an architectural violation: why does an OAuth resource server need
> to know the client flow at all?

Ok. It may have left there from intermediate iterations. We did
consider making extension drive the flow for specific grant_type, but
decided against that idea. For the same reason you point to.
Is it correct that your main concern about use of grant_type was that
it's propagated to the server? Then yes, we will remove sending it to
the server.

> Ideally, yes, but that only works if all identity providers implement
> the same flows in compatible ways. We're already seeing instances
> where that's not the case and we'll necessarily have to deal with that
> up front.

Yes, based on our analysis OIDC spec is detailed enough, that
providers implementing that one, can be supported with generic code in
libpq / client.
Github specifically won't fit there though. Microsoft Azure AD,
Google, Okta (including Auth0) will.
Theoretically discovery documents can be returned from the extension
(server-side) which is provider specific. Though we didn't plan to
prioritize that.

> That seems to be restating the goal of OAuth and OIDC. Can you explain
> how the incompatible change allows you to accomplish this better than
> standard implementations?

Do you refer to passing grant_type to the server? Which we will get
rid of in the next iteration. Or other incompatible changes as well?

> Why? I claim that standard OAUTHBEARER can handle all of that. What
> does your proposed architecture (the third diagram) enable that my
> proposed hook (the second diagram) doesn't?

The hook proposed on the 2nd diagram effectively delegates all Oauth
flows implementations to the client.
We propose libpq takes care of pulling OpenId discovery and coordination.
Which is effectively Diagram 1 + more flows + server hook providing
root url/audience.

Created the diagrams with all components for 3 flows:
1. Authorization code grant (Clients with Browser access):
  +----------------------+                                         +----------+
  |             +-------+                                          |
       |
  | PQconnect   |       |                                          |
       |
  | [auth_code] |       |                                          |
+-----------+
  |          -> |       | -------------- Empty Token ------------> | >
|           |
  |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth  |
  |             |       |                                          |
|  Hook     |
  |             |       |                                          |
+-----------+
  |             |       |                        +--------------+  |          |
  |             |       | -------[GET]---------> | OIDC         |  | Postgres |
  |          +------+   | <--Provider Metadata-- | Discovery    |  |          |
  |     +- < | Hook | < |                        +--------------+  |
       |
  |     |    +------+   |                                          |
       |
  |     v       |       |                                          |
       |
  |  [get auth  |       |                                          |
       |
  |    code]    |       |                                          |
       |
  |<user action>|       |                                          |
       |
  |     |       |       |                                          |
       |
  |     +       |       |                                          |
       |
  | PQconnect > | +--------+                     +--------------+  |
       |
  |             | | iddawc | <-- [ Auth code ]-> | Issuer/      |  |          |
  |             | |        | <-- Access Token -- | Authz Server |  |          |
  |             | +--------+                     +--------------+  |          |
  |             |       |                                          |
+-----------+
  |             |       | -------------- Access Token -----------> | >
| Validator |
  |             |       | <---- Authorization Success/Failure ---- | <
|   Hook    |
  |          +------+   |                                          |
+-----------+
  |      +-< | Hook |   |                                          |
       |
  |      v   +------+   |                                          |
       |
  |[store       +-------+                                          |
       |
  |  refresh_token]                                                +----------+
  +----------------------+

2. Device code grant
  +----------------------+                                         +----------+
  |             +-------+                                          |
       |
  | PQconnect   |       |                                          |
       |
  | [auth_code] |       |                                          |
+-----------+
  |          -> |       | -------------- Empty Token ------------> | >
|           |
  |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth  |
  |             |       |                                          |
|  Hook     |
  |             |       |                                          |
+-----------+
  |             |       |                        +--------------+  |          |
  |             |       | -------[GET]---------> | OIDC         |  | Postgres |
  |          +------+   | <--Provider Metadata-- | Discovery    |  |          |
  |     +- < | Hook | < |                        +--------------+  |
       |
  |     |    +------+   |                                          |
       |
  |     v       |       |                                          |
       |
  |  [device    | +---------+                     +--------------+ |
       |
  |    code]    | | iddawc  |                     | Issuer/      | |
       |
  |<user action>| |         | --[ Device code ]-> | Authz Server | |
       |
  |             | |<polling>| --[ Device code ]-> |              | |
       |
  |             | |         | --[ Device code ]-> |              | |
       |
  |             | |         |                     |              | |          |
  |             | |         | <-- Access Token -- |              | |          |
  |             | +---------+                     +--------------+ |          |
  |             |       |                                          |
+-----------+
  |             |       | -------------- Access Token -----------> | >
| Validator |
  |             |       | <---- Authorization Success/Failure ---- | <
|   Hook    |
  |          +------+   |                                          |
+-----------+
  |      +-< | Hook |   |                                          |
       |
  |      v   +------+   |                                          |
       |
  |[store       +-------+                                          |
       |
  |  refresh_token]                                                +----------+
  +----------------------+

3. Non-interactive flows (Client Secret / Refresh_Token)
  +----------------------+                                         +----------+
  |             +-------+                                          |
       |
  | PQconnect   |       |                                          |
       |
  | [grant_type]|       |                                          |          |
  |          -> |       |                                          |
+-----------+
  |             |       | -------------- Empty Token ------------> | >
|           |
  |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth  |
  |             |       |                                          |
|  Hook     |
  |             |       |                                          |
+-----------+
  |             |       |                        +--------------+  |          |
  |             |       | -------[GET]---------> | OIDC         |  | Postgres |
  |             |       | <--Provider Metadata-- | Discovery    |  |          |
  |             |       |                        +--------------+  |
       |
  |             |       |                                          |
       |
  |             | +--------+                     +--------------+  |
       |
  |             | | iddawc | <-- [ Secret ]----> | Issuer/      |  |          |
  |             | |        | <-- Access Token -- | Authz Server |  |          |
  |             | +--------+                     +--------------+  |          |
  |             |       |                                          |
+-----------+
  |             |       | -------------- Access Token -----------> | >
| Validator |
  |             |       | <---- Authorization Success/Failure ---- | <
|   Hook    |
  |             |       |                                          |
+-----------+
  |             +-------+                                          +----------+
  +----------------------+

I think what was the most confusing in our latest patch is that
flow_type was passed to the server.
We are not proposing this going forward.

> (For a future conversation: they need to set up authorization, too,
> with custom scopes or some other magic. It's not enough to check who
> the token belongs to; even if Postgres is just using the verified
> email from OpenID as an authenticator, you have to also know that the
> user authorized the token -- and therefore the client -- to access
> Postgres on their behalf.)

My understanding is that metadata in the tokens is provider specific,
so server side hook would be the right place to handle that.
Plus I can envision for some providers it can make sense to make a
remote call to pull some information.

The way we implement Azure AD auth today in PAAS PostgreSQL offering:
- Server administrator uses special extension functions to create
Azure AD enabled PostgreSQL roles.
- PostgreSQL extension maps Roles to unique identity Ids (UID) in the Directory.
- Connection flow: If the token is valid and Role => UID mapping
matches, we authenticate as the Role.
- Then its native PostgreSQL role based access control takes care of privileges.

This is the same for both User- and System-to-system authorization.
Though I assume different providers may treat user- and system-
identities differently. So their extension would handle that.

Thanks!
Andrey.

On Wed, Dec 7, 2022 at 11:06 AM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> > I think we can focus on the roles and responsibilities of the components first.
> > Details of the patch can be elaborated. Like "flow type code" is a
> > mistake on our side, and we will use the term "grant_type" which is
> > defined by OIDC spec. As well as details of usage of refresh_token.
>
> (For the record, whether we call it "flow type" or "grant type"
> doesn't address my concern.)
>
> > Basically Yes. We propose an increase of the server side hook responsibility.
> > From just validating the token, to also return the provider root URL
> > and required audience. And possibly provide more metadata in the
> > future.
>
> I think it's okay to have the extension and HBA collaborate to provide
> discovery information. Your proposal goes further than that, though,
> and makes the server aware of the chosen client flow. That appears to
> be an architectural violation: why does an OAuth resource server need
> to know the client flow at all?
>
> > Which is in our opinion aligned with SASL protocol, where the server
> > side is responsible for telling the client auth requirements based on
> > the requested role in the startup packet.
>
> You've proposed an alternative SASL mechanism. There's nothing wrong
> with that, per se, but I think it should be clear why we've chosen
> something nonstandard.
>
> > Our understanding is that in the original patch that information came
> > purely from hba, and we propose extension being able to control that
> > metadata.
> > As we see extension as being owned by the identity provider, compared
> > to HBA which is owned by the server administrator or cloud provider.
>
> That seems reasonable, considering how tightly coupled the Issuer and
> the token validation process are.
>
> > 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> > multi-cloud PAAS providers).
> >    - Install extensions and configure HBA to allow clients to
> > authenticate with the identity providers of their choice.
>
> (For a future conversation: they need to set up authorization, too,
> with custom scopes or some other magic. It's not enough to check who
> the token belongs to; even if Postgres is just using the verified
> email from OpenID as an authenticator, you have to also know that the
> user authorized the token -- and therefore the client -- to access
> Postgres on their behalf.)
>
> > 3. Client Application Developers (Data Wis, integration tools,
> > PgAdmin, monitoring tools, e.t.c.)
> >    - Independent from specific Identity providers or server providers.
> > Write one code for all identity providers.
>
> Ideally, yes, but that only works if all identity providers implement
> the same flows in compatible ways. We're already seeing instances
> where that's not the case and we'll necessarily have to deal with that
> up front.
>
> >    - Rely on application deployment owners to configure which OIDC
> > provider to use across client and server setups.
> > 4. Application Deployment Owners (End customers setting up applications)
> >    - The only actor actually aware of which identity provider to use.
> > Configures the stack based on the Identity and PostgreSQL deployments
> > they have.
>
> (I have doubts that the roles will be as decoupled in practice as you
> have described them, but I'd rather defer that for now.)
>
> > The critical piece of the vision is (3.) above is applications
> > agnostic of the identity providers. Those applications rely on
> > properly configured servers and rich driver logic (libpq,
> > com.postgresql, npgsql) to allow their application to popup auth
> > windows or do service-to-service authentication with any provider. In
> > our view that would significantly democratize the deployment of OAUTH
> > authentication in the community.
>
> That seems to be restating the goal of OAuth and OIDC. Can you explain
> how the incompatible change allows you to accomplish this better than
> standard implementations?
>
> > In order to allow this separation, we propose:
> > 1. HBA + Extension is the single source of truth of Provider root URL
> > + Required Audience for each role. If some backfill for missing OIDC
> > discovery is needed, the provider-specific extension would be
> > providing it.
> > 2. Client Application knows which grant_type to use in which scenario.
> > But can be coded without knowledge of a specific provider. So can't
> > provide discovery details.
> > 3. Driver (libpq, others) - coordinate the authentication flow based
> > on client grant_type and identity provider metadata to allow client
> > applications to use any flow with any provider in a unified way.
> >
> > Yes, this would require a little more complicated flow between
> > components than in your original patch.
>
> Why? I claim that standard OAUTHBEARER can handle all of that. What
> does your proposed architecture (the third diagram) enable that my
> proposed hook (the second diagram) doesn't?
>
> > And yes, more complexity comes
> > with more opportunity to make bugs.
> > However, I see PG Server and Libpq as the places which can have more
> > complexity. For the purpose of making work for the community
> > participants easier and simplify adoption.
> >
> > Does this make sense to you?
>
> Some of it, but it hasn't really addressed the questions from my last mail.
>
> Thanks,
> --Jacob

  40. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-12-08T04:25:09Z

    That being said, the Diagram 2 would look like this with our proposal:
  +----------------------+                                         +----------+
  |             +-------+                                          | Postgres |
  | PQconnect ->|       |                                          |
       |
  |             |       |                                          |
+-----------+
  |             |       | -------------- Empty Token ------------> | >
|           |
  |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
| Pre-Auth  |
  |          +------+   |                                          |
|  Hook     |
  |     +- < | Hook |   |                                          |
+-----------+
  |     |    +------+   |                                          |          |
  |     v       |       |                                          |
       |
  |  [get token]|       |                                          |
       |
  |     |       |       |                                          |
       |
  |     +       |       |                                          |
+-----------+
  | PQconnect > |       | -------------- Access Token -----------> | >
| Validator |
  |             |       | <---- Authorization Success/Failure ---- | <
|   Hook    |
  |             |       |                                          |
+-----------+
  |             +-------+                                          |
       | +----------------------+
+----------+


With the application taking care of all Token acquisition logic. While
the server-side hook is participating in the pre-authentication reply.

That is definitely a required scenario for the long term and the
easiest to implement in the client core.
And if we can do at least that flow in PG16 it will be a strong
foundation to provide more support for specific grants in libpq going
forward.

Does the diagram above look good to you? We can then start cleaning up
the patch to get that in first.

Thanks!
Andrey.


On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > I think it's okay to have the extension and HBA collaborate to provide
> > discovery information. Your proposal goes further than that, though,
> > and makes the server aware of the chosen client flow. That appears to
> > be an architectural violation: why does an OAuth resource server need
> > to know the client flow at all?
>
> Ok. It may have left there from intermediate iterations. We did
> consider making extension drive the flow for specific grant_type, but
> decided against that idea. For the same reason you point to.
> Is it correct that your main concern about use of grant_type was that
> it's propagated to the server? Then yes, we will remove sending it to
> the server.
>
> > Ideally, yes, but that only works if all identity providers implement
> > the same flows in compatible ways. We're already seeing instances
> > where that's not the case and we'll necessarily have to deal with that
> > up front.
>
> Yes, based on our analysis OIDC spec is detailed enough, that
> providers implementing that one, can be supported with generic code in
> libpq / client.
> Github specifically won't fit there though. Microsoft Azure AD,
> Google, Okta (including Auth0) will.
> Theoretically discovery documents can be returned from the extension
> (server-side) which is provider specific. Though we didn't plan to
> prioritize that.
>
> > That seems to be restating the goal of OAuth and OIDC. Can you explain
> > how the incompatible change allows you to accomplish this better than
> > standard implementations?
>
> Do you refer to passing grant_type to the server? Which we will get
> rid of in the next iteration. Or other incompatible changes as well?
>
> > Why? I claim that standard OAUTHBEARER can handle all of that. What
> > does your proposed architecture (the third diagram) enable that my
> > proposed hook (the second diagram) doesn't?
>
> The hook proposed on the 2nd diagram effectively delegates all Oauth
> flows implementations to the client.
> We propose libpq takes care of pulling OpenId discovery and coordination.
> Which is effectively Diagram 1 + more flows + server hook providing
> root url/audience.
>
> Created the diagrams with all components for 3 flows:
> 1. Authorization code grant (Clients with Browser access):
>   +----------------------+                                         +----------+
>   |             +-------+                                          |
>        |
>   | PQconnect   |       |                                          |
>        |
>   | [auth_code] |       |                                          |
> +-----------+
>   |          -> |       | -------------- Empty Token ------------> | >
> |           |
>   |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth  |
>   |             |       |                                          |
> |  Hook     |
>   |             |       |                                          |
> +-----------+
>   |             |       |                        +--------------+  |          |
>   |             |       | -------[GET]---------> | OIDC         |  | Postgres |
>   |          +------+   | <--Provider Metadata-- | Discovery    |  |          |
>   |     +- < | Hook | < |                        +--------------+  |
>        |
>   |     |    +------+   |                                          |
>        |
>   |     v       |       |                                          |
>        |
>   |  [get auth  |       |                                          |
>        |
>   |    code]    |       |                                          |
>        |
>   |<user action>|       |                                          |
>        |
>   |     |       |       |                                          |
>        |
>   |     +       |       |                                          |
>        |
>   | PQconnect > | +--------+                     +--------------+  |
>        |
>   |             | | iddawc | <-- [ Auth code ]-> | Issuer/      |  |          |
>   |             | |        | <-- Access Token -- | Authz Server |  |          |
>   |             | +--------+                     +--------------+  |          |
>   |             |       |                                          |
> +-----------+
>   |             |       | -------------- Access Token -----------> | >
> | Validator |
>   |             |       | <---- Authorization Success/Failure ---- | <
> |   Hook    |
>   |          +------+   |                                          |
> +-----------+
>   |      +-< | Hook |   |                                          |
>        |
>   |      v   +------+   |                                          |
>        |
>   |[store       +-------+                                          |
>        |
>   |  refresh_token]                                                +----------+
>   +----------------------+
>
> 2. Device code grant
>   +----------------------+                                         +----------+
>   |             +-------+                                          |
>        |
>   | PQconnect   |       |                                          |
>        |
>   | [auth_code] |       |                                          |
> +-----------+
>   |          -> |       | -------------- Empty Token ------------> | >
> |           |
>   |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth  |
>   |             |       |                                          |
> |  Hook     |
>   |             |       |                                          |
> +-----------+
>   |             |       |                        +--------------+  |          |
>   |             |       | -------[GET]---------> | OIDC         |  | Postgres |
>   |          +------+   | <--Provider Metadata-- | Discovery    |  |          |
>   |     +- < | Hook | < |                        +--------------+  |
>        |
>   |     |    +------+   |                                          |
>        |
>   |     v       |       |                                          |
>        |
>   |  [device    | +---------+                     +--------------+ |
>        |
>   |    code]    | | iddawc  |                     | Issuer/      | |
>        |
>   |<user action>| |         | --[ Device code ]-> | Authz Server | |
>        |
>   |             | |<polling>| --[ Device code ]-> |              | |
>        |
>   |             | |         | --[ Device code ]-> |              | |
>        |
>   |             | |         |                     |              | |          |
>   |             | |         | <-- Access Token -- |              | |          |
>   |             | +---------+                     +--------------+ |          |
>   |             |       |                                          |
> +-----------+
>   |             |       | -------------- Access Token -----------> | >
> | Validator |
>   |             |       | <---- Authorization Success/Failure ---- | <
> |   Hook    |
>   |          +------+   |                                          |
> +-----------+
>   |      +-< | Hook |   |                                          |
>        |
>   |      v   +------+   |                                          |
>        |
>   |[store       +-------+                                          |
>        |
>   |  refresh_token]                                                +----------+
>   +----------------------+
>
> 3. Non-interactive flows (Client Secret / Refresh_Token)
>   +----------------------+                                         +----------+
>   |             +-------+                                          |
>        |
>   | PQconnect   |       |                                          |
>        |
>   | [grant_type]|       |                                          |          |
>   |          -> |       |                                          |
> +-----------+
>   |             |       | -------------- Empty Token ------------> | >
> |           |
>   |             | libpq | <----- Error(w\ Root URL + Audience ) -- | <
> | Pre-Auth  |
>   |             |       |                                          |
> |  Hook     |
>   |             |       |                                          |
> +-----------+
>   |             |       |                        +--------------+  |          |
>   |             |       | -------[GET]---------> | OIDC         |  | Postgres |
>   |             |       | <--Provider Metadata-- | Discovery    |  |          |
>   |             |       |                        +--------------+  |
>        |
>   |             |       |                                          |
>        |
>   |             | +--------+                     +--------------+  |
>        |
>   |             | | iddawc | <-- [ Secret ]----> | Issuer/      |  |          |
>   |             | |        | <-- Access Token -- | Authz Server |  |          |
>   |             | +--------+                     +--------------+  |          |
>   |             |       |                                          |
> +-----------+
>   |             |       | -------------- Access Token -----------> | >
> | Validator |
>   |             |       | <---- Authorization Success/Failure ---- | <
> |   Hook    |
>   |             |       |                                          |
> +-----------+
>   |             +-------+                                          +----------+
>   +----------------------+
>
> I think what was the most confusing in our latest patch is that
> flow_type was passed to the server.
> We are not proposing this going forward.
>
> > (For a future conversation: they need to set up authorization, too,
> > with custom scopes or some other magic. It's not enough to check who
> > the token belongs to; even if Postgres is just using the verified
> > email from OpenID as an authenticator, you have to also know that the
> > user authorized the token -- and therefore the client -- to access
> > Postgres on their behalf.)
>
> My understanding is that metadata in the tokens is provider specific,
> so server side hook would be the right place to handle that.
> Plus I can envision for some providers it can make sense to make a
> remote call to pull some information.
>
> The way we implement Azure AD auth today in PAAS PostgreSQL offering:
> - Server administrator uses special extension functions to create
> Azure AD enabled PostgreSQL roles.
> - PostgreSQL extension maps Roles to unique identity Ids (UID) in the Directory.
> - Connection flow: If the token is valid and Role => UID mapping
> matches, we authenticate as the Role.
> - Then its native PostgreSQL role based access control takes care of privileges.
>
> This is the same for both User- and System-to-system authorization.
> Though I assume different providers may treat user- and system-
> identities differently. So their extension would handle that.
>
> Thanks!
> Andrey.
>
> On Wed, Dec 7, 2022 at 11:06 AM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Mon, Dec 5, 2022 at 4:15 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> > > I think we can focus on the roles and responsibilities of the components first.
> > > Details of the patch can be elaborated. Like "flow type code" is a
> > > mistake on our side, and we will use the term "grant_type" which is
> > > defined by OIDC spec. As well as details of usage of refresh_token.
> >
> > (For the record, whether we call it "flow type" or "grant type"
> > doesn't address my concern.)
> >
> > > Basically Yes. We propose an increase of the server side hook responsibility.
> > > From just validating the token, to also return the provider root URL
> > > and required audience. And possibly provide more metadata in the
> > > future.
> >
> > I think it's okay to have the extension and HBA collaborate to provide
> > discovery information. Your proposal goes further than that, though,
> > and makes the server aware of the chosen client flow. That appears to
> > be an architectural violation: why does an OAuth resource server need
> > to know the client flow at all?
> >
> > > Which is in our opinion aligned with SASL protocol, where the server
> > > side is responsible for telling the client auth requirements based on
> > > the requested role in the startup packet.
> >
> > You've proposed an alternative SASL mechanism. There's nothing wrong
> > with that, per se, but I think it should be clear why we've chosen
> > something nonstandard.
> >
> > > Our understanding is that in the original patch that information came
> > > purely from hba, and we propose extension being able to control that
> > > metadata.
> > > As we see extension as being owned by the identity provider, compared
> > > to HBA which is owned by the server administrator or cloud provider.
> >
> > That seems reasonable, considering how tightly coupled the Issuer and
> > the token validation process are.
> >
> > > 2. Server Owners / PAAS providers (On premise admins, Cloud providers,
> > > multi-cloud PAAS providers).
> > >    - Install extensions and configure HBA to allow clients to
> > > authenticate with the identity providers of their choice.
> >
> > (For a future conversation: they need to set up authorization, too,
> > with custom scopes or some other magic. It's not enough to check who
> > the token belongs to; even if Postgres is just using the verified
> > email from OpenID as an authenticator, you have to also know that the
> > user authorized the token -- and therefore the client -- to access
> > Postgres on their behalf.)
> >
> > > 3. Client Application Developers (Data Wis, integration tools,
> > > PgAdmin, monitoring tools, e.t.c.)
> > >    - Independent from specific Identity providers or server providers.
> > > Write one code for all identity providers.
> >
> > Ideally, yes, but that only works if all identity providers implement
> > the same flows in compatible ways. We're already seeing instances
> > where that's not the case and we'll necessarily have to deal with that
> > up front.
> >
> > >    - Rely on application deployment owners to configure which OIDC
> > > provider to use across client and server setups.
> > > 4. Application Deployment Owners (End customers setting up applications)
> > >    - The only actor actually aware of which identity provider to use.
> > > Configures the stack based on the Identity and PostgreSQL deployments
> > > they have.
> >
> > (I have doubts that the roles will be as decoupled in practice as you
> > have described them, but I'd rather defer that for now.)
> >
> > > The critical piece of the vision is (3.) above is applications
> > > agnostic of the identity providers. Those applications rely on
> > > properly configured servers and rich driver logic (libpq,
> > > com.postgresql, npgsql) to allow their application to popup auth
> > > windows or do service-to-service authentication with any provider. In
> > > our view that would significantly democratize the deployment of OAUTH
> > > authentication in the community.
> >
> > That seems to be restating the goal of OAuth and OIDC. Can you explain
> > how the incompatible change allows you to accomplish this better than
> > standard implementations?
> >
> > > In order to allow this separation, we propose:
> > > 1. HBA + Extension is the single source of truth of Provider root URL
> > > + Required Audience for each role. If some backfill for missing OIDC
> > > discovery is needed, the provider-specific extension would be
> > > providing it.
> > > 2. Client Application knows which grant_type to use in which scenario.
> > > But can be coded without knowledge of a specific provider. So can't
> > > provide discovery details.
> > > 3. Driver (libpq, others) - coordinate the authentication flow based
> > > on client grant_type and identity provider metadata to allow client
> > > applications to use any flow with any provider in a unified way.
> > >
> > > Yes, this would require a little more complicated flow between
> > > components than in your original patch.
> >
> > Why? I claim that standard OAUTHBEARER can handle all of that. What
> > does your proposed architecture (the third diagram) enable that my
> > proposed hook (the second diagram) doesn't?
> >
> > > And yes, more complexity comes
> > > with more opportunity to make bugs.
> > > However, I see PG Server and Libpq as the places which can have more
> > > complexity. For the purpose of making work for the community
> > > participants easier and simplify adoption.
> > >
> > > Does this make sense to you?
> >
> > Some of it, but it hasn't really addressed the questions from my last mail.
> >
> > Thanks,
> > --Jacob

  41. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-12-09T00:41:21Z

    On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> 
>> I think it's okay to have the extension and HBA collaborate to
>> provide discovery information. Your proposal goes further than
>> that, though, and makes the server aware of the chosen client flow.
>> That appears to be an architectural violation: why does an OAuth
>> resource server need to know the client flow at all?
> 
> Ok. It may have left there from intermediate iterations. We did 
> consider making extension drive the flow for specific grant_type,
> but decided against that idea. For the same reason you point to. Is
> it correct that your main concern about use of grant_type was that 
> it's propagated to the server? Then yes, we will remove sending it
> to the server.

Okay. Yes, that was my primary concern.

>> Ideally, yes, but that only works if all identity providers
>> implement the same flows in compatible ways. We're already seeing
>> instances where that's not the case and we'll necessarily have to
>> deal with that up front.
> 
> Yes, based on our analysis OIDC spec is detailed enough, that 
> providers implementing that one, can be supported with generic code
> in libpq / client. Github specifically won't fit there though.
> Microsoft Azure AD, Google, Okta (including Auth0) will. 
> Theoretically discovery documents can be returned from the extension 
> (server-side) which is provider specific. Though we didn't plan to 
> prioritize that.

As another example, Google's device authorization grant is incompatible
with the spec (which they co-authored). I want to say I had problems
with Azure AD not following that spec either, but I don't remember
exactly what they were. I wouldn't be surprised to find more tiny
departures once we get deeper into implementation.

>> That seems to be restating the goal of OAuth and OIDC. Can you
>> explain how the incompatible change allows you to accomplish this
>> better than standard implementations?
> 
> Do you refer to passing grant_type to the server? Which we will get 
> rid of in the next iteration. Or other incompatible changes as well?

Just the grant type, yeah.

>> Why? I claim that standard OAUTHBEARER can handle all of that.
>> What does your proposed architecture (the third diagram) enable
>> that my proposed hook (the second diagram) doesn't?
> 
> The hook proposed on the 2nd diagram effectively delegates all Oauth 
> flows implementations to the client. We propose libpq takes care of
> pulling OpenId discovery and coordination. Which is effectively
> Diagram 1 + more flows + server hook providing root url/audience.
> 
> Created the diagrams with all components for 3 flows: [snip]

(I'll skip ahead to your later mail on this.)

>> (For a future conversation: they need to set up authorization,
>> too, with custom scopes or some other magic. It's not enough to
>> check who the token belongs to; even if Postgres is just using the
>> verified email from OpenID as an authenticator, you have to also
>> know that the user authorized the token -- and therefore the client
>> -- to access Postgres on their behalf.)
> 
> My understanding is that metadata in the tokens is provider
> specific, so server side hook would be the right place to handle
> that. Plus I can envision for some providers it can make sense to
> make a remote call to pull some information.

The server hook is the right place to check the scopes, yes, but I think
the DBA should be able to specify what those scopes are to begin with.
The provider of the extension shouldn't be expected by the architecture
to hardcode those decisions, even if Azure AD chooses to short-circuit
that choice and provide magic instead.

On 12/7/22 20:25, Andrey Chudnovsky wrote:
> That being said, the Diagram 2 would look like this with our proposal:
> [snip]
> 
> With the application taking care of all Token acquisition logic. While
> the server-side hook is participating in the pre-authentication reply.
> 
> That is definitely a required scenario for the long term and the
> easiest to implement in the client core.> And if we can do at least that flow in PG16 it will be a strong
> foundation to provide more support for specific grants in libpq going
> forward.

Agreed.
> Does the diagram above look good to you? We can then start cleaning up
> the patch to get that in first.

I maintain that the hook doesn't need to hand back artifacts to the
client for a second PQconnect call. It can just use those artifacts to
obtain the access token and hand that right back to libpq. (I think any
requirement that clients be rewritten to call PQconnect twice will
probably be a sticking point for adoption of an OAuth patch.)

That said, now that your proposal is also compatible with OAUTHBEARER, I
can pony up some code to hopefully prove my point. (I don't know if I'll
be able to do that by the holidays though.)

Thanks!
--Jacob

  42. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2022-12-13T05:06:20Z

    > The server hook is the right place to check the scopes, yes, but I think
> the DBA should be able to specify what those scopes are to begin with.
> The provider of the extension shouldn't be expected by the architecture
> to hardcode those decisions, even if Azure AD chooses to short-circuit
> that choice and provide magic instead.

Hardcode is definitely not expected, but customization for identity
provider specific, I think, should be allowed.
I can provide a couple of advanced use cases which happen in the cloud
deployments world, and require per-role management:
- Multi-tenant deployments, when root provider URL would be different
for different roles, based on which tenant they come from.
- Federation to multiple providers. Solutions like Amazon Cognito
which offer a layer of abstraction with several providers
transparently supported.

If your concern is extension not honoring the DBA configured values:
Would a server-side logic to prefer HBA value over extension-provided
resolve this concern?
We are definitely biased towards the cloud deployment scenarios, where
direct access to .hba files is usually not offered at all.
Let's find the middle ground here.

A separate reason for creating this pre-authentication hook is further
extensibility to support more metadata.
Specifically when we add support for OAUTH flows to libpq, server-side
extensions can help bridge the gap between the identity provider
implementation and OAUTH/OIDC specs.
For example, that could allow the Github extension to provide an OIDC
discovery document.

I definitely see identity providers as institutional actors here which
can be given some power through the extension hooks to customize the
behavior within the framework.

> I maintain that the hook doesn't need to hand back artifacts to the
> client for a second PQconnect call. It can just use those artifacts to
> obtain the access token and hand that right back to libpq. (I think any
> requirement that clients be rewritten to call PQconnect twice will
> probably be a sticking point for adoption of an OAuth patch.)

Obtaining a token is an asynchronous process with a human in the loop.
Not sure if expecting a hook function to return a token synchronously
is the best option here.
Can that be an optional return value of the hook in cases when a token
can be obtained synchronously?

On Thu, Dec 8, 2022 at 4:41 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Wed, Dec 7, 2022 at 3:22 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> >
> >> I think it's okay to have the extension and HBA collaborate to
> >> provide discovery information. Your proposal goes further than
> >> that, though, and makes the server aware of the chosen client flow.
> >> That appears to be an architectural violation: why does an OAuth
> >> resource server need to know the client flow at all?
> >
> > Ok. It may have left there from intermediate iterations. We did
> > consider making extension drive the flow for specific grant_type,
> > but decided against that idea. For the same reason you point to. Is
> > it correct that your main concern about use of grant_type was that
> > it's propagated to the server? Then yes, we will remove sending it
> > to the server.
>
> Okay. Yes, that was my primary concern.
>
> >> Ideally, yes, but that only works if all identity providers
> >> implement the same flows in compatible ways. We're already seeing
> >> instances where that's not the case and we'll necessarily have to
> >> deal with that up front.
> >
> > Yes, based on our analysis OIDC spec is detailed enough, that
> > providers implementing that one, can be supported with generic code
> > in libpq / client. Github specifically won't fit there though.
> > Microsoft Azure AD, Google, Okta (including Auth0) will.
> > Theoretically discovery documents can be returned from the extension
> > (server-side) which is provider specific. Though we didn't plan to
> > prioritize that.
>
> As another example, Google's device authorization grant is incompatible
> with the spec (which they co-authored). I want to say I had problems
> with Azure AD not following that spec either, but I don't remember
> exactly what they were. I wouldn't be surprised to find more tiny
> departures once we get deeper into implementation.
>
> >> That seems to be restating the goal of OAuth and OIDC. Can you
> >> explain how the incompatible change allows you to accomplish this
> >> better than standard implementations?
> >
> > Do you refer to passing grant_type to the server? Which we will get
> > rid of in the next iteration. Or other incompatible changes as well?
>
> Just the grant type, yeah.
>
> >> Why? I claim that standard OAUTHBEARER can handle all of that.
> >> What does your proposed architecture (the third diagram) enable
> >> that my proposed hook (the second diagram) doesn't?
> >
> > The hook proposed on the 2nd diagram effectively delegates all Oauth
> > flows implementations to the client. We propose libpq takes care of
> > pulling OpenId discovery and coordination. Which is effectively
> > Diagram 1 + more flows + server hook providing root url/audience.
> >
> > Created the diagrams with all components for 3 flows: [snip]
>
> (I'll skip ahead to your later mail on this.)
>
> >> (For a future conversation: they need to set up authorization,
> >> too, with custom scopes or some other magic. It's not enough to
> >> check who the token belongs to; even if Postgres is just using the
> >> verified email from OpenID as an authenticator, you have to also
> >> know that the user authorized the token -- and therefore the client
> >> -- to access Postgres on their behalf.)
> >
> > My understanding is that metadata in the tokens is provider
> > specific, so server side hook would be the right place to handle
> > that. Plus I can envision for some providers it can make sense to
> > make a remote call to pull some information.
>
> The server hook is the right place to check the scopes, yes, but I think
> the DBA should be able to specify what those scopes are to begin with.
> The provider of the extension shouldn't be expected by the architecture
> to hardcode those decisions, even if Azure AD chooses to short-circuit
> that choice and provide magic instead.
>
> On 12/7/22 20:25, Andrey Chudnovsky wrote:
> > That being said, the Diagram 2 would look like this with our proposal:
> > [snip]
> >
> > With the application taking care of all Token acquisition logic. While
> > the server-side hook is participating in the pre-authentication reply.
> >
> > That is definitely a required scenario for the long term and the
> > easiest to implement in the client core.> And if we can do at least that flow in PG16 it will be a strong
> > foundation to provide more support for specific grants in libpq going
> > forward.
>
> Agreed.
> > Does the diagram above look good to you? We can then start cleaning up
> > the patch to get that in first.
>
> I maintain that the hook doesn't need to hand back artifacts to the
> client for a second PQconnect call. It can just use those artifacts to
> obtain the access token and hand that right back to libpq. (I think any
> requirement that clients be rewritten to call PQconnect twice will
> probably be a sticking point for adoption of an OAuth patch.)
>
> That said, now that your proposal is also compatible with OAUTHBEARER, I
> can pony up some code to hopefully prove my point. (I don't know if I'll
> be able to do that by the holidays though.)
>
> Thanks!
> --Jacob

  43. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2022-12-16T23:18:38Z

    On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> If your concern is extension not honoring the DBA configured values:
> Would a server-side logic to prefer HBA value over extension-provided
> resolve this concern?

Yeah. It also seals the role of the extension here as "optional".

> We are definitely biased towards the cloud deployment scenarios, where
> direct access to .hba files is usually not offered at all.
> Let's find the middle ground here.

Sure. I don't want to make this difficult in cloud scenarios --
obviously I'd like for Timescale Cloud to be able to make use of this
too. But if we make this easy for a lone DBA (who doesn't have any
institutional power with the providers) to use correctly and securely,
then it should follow that the providers who _do_ have power and
resources will have an easy time of it as well. The reverse isn't
necessarily true. So I'm definitely planning to focus on the DBA case
first.

> A separate reason for creating this pre-authentication hook is further
> extensibility to support more metadata.
> Specifically when we add support for OAUTH flows to libpq, server-side
> extensions can help bridge the gap between the identity provider
> implementation and OAUTH/OIDC specs.
> For example, that could allow the Github extension to provide an OIDC
> discovery document.
>
> I definitely see identity providers as institutional actors here which
> can be given some power through the extension hooks to customize the
> behavior within the framework.

We'll probably have to make some compromises in this area, but I think
they should be carefully considered exceptions and not a core feature
of the mechanism. The gaps you point out are just fragmentation, and
adding custom extensions to deal with it leads to further
fragmentation instead of providing pressure on providers to just
implement the specs. Worst case, we open up new exciting security
flaws, and then no one can analyze them independently because no one
other than the provider knows how the two sides work together anymore.

Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
spec were perfect, because it's clearly not. But if we need to make
extensions to it, we can participate in IETF discussions and make our
case publicly for review, rather than enshrining MS/GitHub/Google/etc.
versions of the RFC and enabling that proliferation as a Postgres core
feature.

> Obtaining a token is an asynchronous process with a human in the loop.
> Not sure if expecting a hook function to return a token synchronously
> is the best option here.
> Can that be an optional return value of the hook in cases when a token
> can be obtained synchronously?

I don't think the hook is generally going to be able to return a token
synchronously, and I expect the final design to be async-first. As far
as I know, this will need to be solved for the builtin flows as well
(you don't want a synchronous HTTP call to block your PQconnectPoll
architecture), so the hook should be able to make use of whatever
solution we land on for that.

This is hand-wavy, and I don't expect it to be easy to solve. I just
don't think we have to solve it twice.

Have a good end to the year!
--Jacob

  44. Re: [PoC] Federated Authn/z with OAUTHBEARER

    mahendrakar s <mahendrakarforpg@gmail.com> — 2023-01-12T19:08:35Z

    Hi All,

Changes added to Jacob's patch(v2) as per the discussion in the thread.

The changes allow the customer to send the OAUTH BEARER token through psql
connection string.

Example:
psql  -U user@example.com -d 'dbname=postgres oauth_bearer_token=abc'

To configure OAUTH, the pg_hba.conf line look like:
local   all             all                                     oauth
 provider=oauth_provider issuer="https://example.com" scope="openid email"

We also added hook to libpq to pass on the metadata about the issuer.

Thanks,
Mahendrakar.


On Sat, 17 Dec 2022 at 04:48, Jacob Champion <jchampion@timescale.com>
wrote:
>
> On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > If your concern is extension not honoring the DBA configured values:
> > Would a server-side logic to prefer HBA value over extension-provided
> > resolve this concern?
>
> Yeah. It also seals the role of the extension here as "optional".
>
> > We are definitely biased towards the cloud deployment scenarios, where
> > direct access to .hba files is usually not offered at all.
> > Let's find the middle ground here.
>
> Sure. I don't want to make this difficult in cloud scenarios --
> obviously I'd like for Timescale Cloud to be able to make use of this
> too. But if we make this easy for a lone DBA (who doesn't have any
> institutional power with the providers) to use correctly and securely,
> then it should follow that the providers who _do_ have power and
> resources will have an easy time of it as well. The reverse isn't
> necessarily true. So I'm definitely planning to focus on the DBA case
> first.
>
> > A separate reason for creating this pre-authentication hook is further
> > extensibility to support more metadata.
> > Specifically when we add support for OAUTH flows to libpq, server-side
> > extensions can help bridge the gap between the identity provider
> > implementation and OAUTH/OIDC specs.
> > For example, that could allow the Github extension to provide an OIDC
> > discovery document.
> >
> > I definitely see identity providers as institutional actors here which
> > can be given some power through the extension hooks to customize the
> > behavior within the framework.
>
> We'll probably have to make some compromises in this area, but I think
> they should be carefully considered exceptions and not a core feature
> of the mechanism. The gaps you point out are just fragmentation, and
> adding custom extensions to deal with it leads to further
> fragmentation instead of providing pressure on providers to just
> implement the specs. Worst case, we open up new exciting security
> flaws, and then no one can analyze them independently because no one
> other than the provider knows how the two sides work together anymore.
>
> Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
> spec were perfect, because it's clearly not. But if we need to make
> extensions to it, we can participate in IETF discussions and make our
> case publicly for review, rather than enshrining MS/GitHub/Google/etc.
> versions of the RFC and enabling that proliferation as a Postgres core
> feature.
>
> > Obtaining a token is an asynchronous process with a human in the loop.
> > Not sure if expecting a hook function to return a token synchronously
> > is the best option here.
> > Can that be an optional return value of the hook in cases when a token
> > can be obtained synchronously?
>
> I don't think the hook is generally going to be able to return a token
> synchronously, and I expect the final design to be async-first. As far
> as I know, this will need to be solved for the builtin flows as well
> (you don't want a synchronous HTTP call to block your PQconnectPoll
> architecture), so the hook should be able to make use of whatever
> solution we land on for that.
>
> This is hand-wavy, and I don't expect it to be easy to solve. I just
> don't think we have to solve it twice.
>
> Have a good end to the year!
> --Jacob

  45. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-01-15T20:03:32Z

    More information on the latest patch.

1. We aligned the implementation with the barebone SASL for OAUTH
described here - https://www.rfc-editor.org/rfc/rfc7628
The flow can be explained in the diagram below:

  +----------------------+                                 +----------+
  |             +-------+                                  | Postgres |
  | PQconnect ->|       |                                  |          |
  |             |       |                                  |   +-----------+
  |             |       | ---------- Empty Token---------> | > |           |
  |             | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth  |
  |          +------+   |                                  |   |  Hook     |
  |     +- < | Hook |   |                                  |   +-----------+
  |     |    +------+   |                                  |          |
  |     v       |       |                                  |          |
  |  [get token]|       |                                  |          |
  |     |       |       |                                  |          |
  |     +       |       |                                  |   +-----------+
  | PQconnect > |       | --------- Access Token --------> | > | Validator |
  |             |       | <---------- Auth Result -------- | < |   Hook    |
  |             |       |                                  |   +-----------+
  |             +-------+                                  |          |
  +----------------------+                                 +----------+

2. Removed Device Code implementation in libpq. Several reasons:
   - Reduce scope and focus on the protocol first.
   - Device code implementation uses iddawc dependency. Taking this
dependency is a controversial step which requires broader discussion.
   - Device code implementation without iddaws would significantly
increase the scope of the patch, as libpq needs to poll the token
endpoint, setup different API calls, e.t.c.
   - That flow should canonically only be used for clients which can't
invoke browsers. If it is the only flow to be implemented, it can be
used in the context when it's not expected by the OAUTH protocol.

3. Temporarily removed test suite. We are actively working on aligning
the tests with the latest changes. Will add a patch with tests soon.

We will change the "V3" prefix to make it the next after the previous
iterations.

Thanks!
Andrey.

On Thu, Jan 12, 2023 at 11:08 AM mahendrakar s
<mahendrakarforpg@gmail.com> wrote:
>
> Hi All,
>
> Changes added to Jacob's patch(v2) as per the discussion in the thread.
>
> The changes allow the customer to send the OAUTH BEARER token through psql connection string.
>
> Example:
> psql  -U user@example.com -d 'dbname=postgres oauth_bearer_token=abc'
>
> To configure OAUTH, the pg_hba.conf line look like:
> local   all             all                                     oauth   provider=oauth_provider issuer="https://example.com" scope="openid email"
>
> We also added hook to libpq to pass on the metadata about the issuer.
>
> Thanks,
> Mahendrakar.
>
>
> On Sat, 17 Dec 2022 at 04:48, Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Mon, Dec 12, 2022 at 9:06 PM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > If your concern is extension not honoring the DBA configured values:
> > > Would a server-side logic to prefer HBA value over extension-provided
> > > resolve this concern?
> >
> > Yeah. It also seals the role of the extension here as "optional".
> >
> > > We are definitely biased towards the cloud deployment scenarios, where
> > > direct access to .hba files is usually not offered at all.
> > > Let's find the middle ground here.
> >
> > Sure. I don't want to make this difficult in cloud scenarios --
> > obviously I'd like for Timescale Cloud to be able to make use of this
> > too. But if we make this easy for a lone DBA (who doesn't have any
> > institutional power with the providers) to use correctly and securely,
> > then it should follow that the providers who _do_ have power and
> > resources will have an easy time of it as well. The reverse isn't
> > necessarily true. So I'm definitely planning to focus on the DBA case
> > first.
> >
> > > A separate reason for creating this pre-authentication hook is further
> > > extensibility to support more metadata.
> > > Specifically when we add support for OAUTH flows to libpq, server-side
> > > extensions can help bridge the gap between the identity provider
> > > implementation and OAUTH/OIDC specs.
> > > For example, that could allow the Github extension to provide an OIDC
> > > discovery document.
> > >
> > > I definitely see identity providers as institutional actors here which
> > > can be given some power through the extension hooks to customize the
> > > behavior within the framework.
> >
> > We'll probably have to make some compromises in this area, but I think
> > they should be carefully considered exceptions and not a core feature
> > of the mechanism. The gaps you point out are just fragmentation, and
> > adding custom extensions to deal with it leads to further
> > fragmentation instead of providing pressure on providers to just
> > implement the specs. Worst case, we open up new exciting security
> > flaws, and then no one can analyze them independently because no one
> > other than the provider knows how the two sides work together anymore.
> >
> > Don't get me wrong; it would be naive to proceed as if the OAUTHBEARER
> > spec were perfect, because it's clearly not. But if we need to make
> > extensions to it, we can participate in IETF discussions and make our
> > case publicly for review, rather than enshrining MS/GitHub/Google/etc.
> > versions of the RFC and enabling that proliferation as a Postgres core
> > feature.
> >
> > > Obtaining a token is an asynchronous process with a human in the loop.
> > > Not sure if expecting a hook function to return a token synchronously
> > > is the best option here.
> > > Can that be an optional return value of the hook in cases when a token
> > > can be obtained synchronously?
> >
> > I don't think the hook is generally going to be able to return a token
> > synchronously, and I expect the final design to be async-first. As far
> > as I know, this will need to be solved for the builtin flows as well
> > (you don't want a synchronous HTTP call to block your PQconnectPoll
> > architecture), so the hook should be able to make use of whatever
> > solution we land on for that.
> >
> > This is hand-wavy, and I don't expect it to be easy to solve. I just
> > don't think we have to solve it twice.
> >
> > Have a good end to the year!
> > --Jacob

  46. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-01-17T22:43:59Z

    On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
<achudnovskij@gmail.com> wrote:
> 2. Removed Device Code implementation in libpq. Several reasons:
>    - Reduce scope and focus on the protocol first.
>    - Device code implementation uses iddawc dependency. Taking this
> dependency is a controversial step which requires broader discussion.
>    - Device code implementation without iddaws would significantly
> increase the scope of the patch, as libpq needs to poll the token
> endpoint, setup different API calls, e.t.c.
>    - That flow should canonically only be used for clients which can't
> invoke browsers. If it is the only flow to be implemented, it can be
> used in the context when it's not expected by the OAUTH protocol.

I'm not understanding the concern in the final point -- providers
generally require you to opt into device authorization, at least as far
as I can tell. So if you decide that it's not appropriate for your use
case... don't enable it. (And I haven't seen any claims that opting into
device authorization weakens the other flows in any way. So if we're
going to implement a flow in libpq, I still think device authorization
is the best choice, since it works on headless machines as well as those
with browsers.)

All of this points at a bigger question to the community: if we choose
not to provide a flow implementation in libpq, is adding OAUTHBEARER
worth the additional maintenance cost?

My personal vote would be "no". I think the hook-only approach proposed
here would ensure that only larger providers would implement it in
practice, and in that case I'd rather spend cycles on generic SASL.

> 3. Temporarily removed test suite. We are actively working on aligning
> the tests with the latest changes. Will add a patch with tests soon.

Okay. Case in point, the following change to the patch appears to be
invalid JSON:

> +   appendStringInfo(&buf,
> +       "{ "
> +           "\"status\": \"invalid_token\", "
> +           "\"openid-configuration\": \"%s\","
> +           "\"scope\": \"%s\" ",
> +           "\"issuer\": \"%s\" ",
> +       "}",

Additionally, the "issuer" field added here is not part of the RFC. I've
written my thoughts about unofficial extensions upthread but haven't
received a response, so I'm going to start being more strident: Please,
for the sake of reviewers, call out changes you've made to the spec, and
why they're justified.

The patches seem to be out of order now (and the documentation in the
commit messages has been removed).

Thanks,
--Jacob

  47. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-01-18T01:53:56Z

    > All of this points at a bigger question to the community: if we choose
> not to provide a flow implementation in libpq, is adding OAUTHBEARER
> worth the additional maintenance cost?

> My personal vote would be "no". I think the hook-only approach proposed
> here would ensure that only larger providers would implement it in
> practice

Flow implementations in libpq are definitely a long term plan, and I
agree that it would democratise the adoption.
In the previous posts in this conversation I outlined the ones I think
we should support.

However, I don't see why it's strictly necessary to couple those.
As long as the SASL exchange for OAUTHBEARER mechanism is supported by
the protocol, the Client side can evolve at its own pace.

At the same time, the current implementation allows clients to start
building provider-agnostic OAUTH support. By using iddawc or OAUTH
client implementations in the respective platforms.
So I wouldn't refer to "larger providers", but rather "more motivated
clients" here. Which definitely overlaps, but keeps the system open.

> I'm not understanding the concern in the final point -- providers
> generally require you to opt into device authorization, at least as far
> as I can tell. So if you decide that it's not appropriate for your use
> case... don't enable it. (And I haven't seen any claims that opting into
> device authorization weakens the other flows in any way. So if we're
> going to implement a flow in libpq, I still think device authorization
> is the best choice, since it works on headless machines as well as those
> with browsers.)
I agree with the statement that Device code is the best first choice
if we absolutely have to pick one.
Though I don't think we have to.

While device flow can be used for all kinds of user-facing
applications, it's specifically designed for input-constrained
scenarios. As clearly stated in the Abstract here -
https://www.rfc-editor.org/rfc/rfc8628
The authorization code with pkce flow is recommended by the RFSc and
major providers for cases when it's feasible.
The long term goal is to provide both, though I don't see why the
backbone protocol implementation first wouldn't add value.

Another point is user authentication is one side of the whole story
and the other critical one is system-to-system authentication. Where
we have Client Credentials and Certificates.
With the latter it is much harder to get generically implemented, as
provider-specific tokens need to be signed.

Adding the other reasoning, I think libpq support for specific flows
can get in the further iterations, after the protocol support.

> in that case I'd rather spend cycles on generic SASL.
I see 2 approaches to generic SASL:
(a). Generic SASL is a framework used in the protocol, with the
mechanisms implemented on top and exposed to the DBAs as auth types to
configure in hba.
This is the direction we're going here, which is well aligned with the
existing hba-based auth configuration.
(b). Generic SASL exposed to developers on the server- and client-
side to extend on. It seems to be a much longer shot.
The specific points of large ambiguity are libpq distribution model
(which you pointed to) and potential pluggability of insecure
mechanisms.

I do see (a) as a sweet spot with a lot of value for various
participants with much less ambiguity.

> Additionally, the "issuer" field added here is not part of the RFC. I've
> written my thoughts about unofficial extensions upthread but haven't
> received a response, so I'm going to start being more strident: Please,
> for the sake of reviewers, call out changes you've made to the spec, and
> why they're justified.
Thanks for your feedback on this. We had this discussion as well, and
added that as a convenience for the client to identify the provider.
I don't see a reason why an issuer would be absolutely necessary, so
we will get your point that sticking to RFCs is a safer choice.

> The patches seem to be out of order now (and the documentation in the
> commit messages has been removed).
Feedback taken. Work in progress.

On Tue, Jan 17, 2023 at 2:44 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
> <achudnovskij@gmail.com> wrote:
> > 2. Removed Device Code implementation in libpq. Several reasons:
> >    - Reduce scope and focus on the protocol first.
> >    - Device code implementation uses iddawc dependency. Taking this
> > dependency is a controversial step which requires broader discussion.
> >    - Device code implementation without iddaws would significantly
> > increase the scope of the patch, as libpq needs to poll the token
> > endpoint, setup different API calls, e.t.c.
> >    - That flow should canonically only be used for clients which can't
> > invoke browsers. If it is the only flow to be implemented, it can be
> > used in the context when it's not expected by the OAUTH protocol.
>
> I'm not understanding the concern in the final point -- providers
> generally require you to opt into device authorization, at least as far
> as I can tell. So if you decide that it's not appropriate for your use
> case... don't enable it. (And I haven't seen any claims that opting into
> device authorization weakens the other flows in any way. So if we're
> going to implement a flow in libpq, I still think device authorization
> is the best choice, since it works on headless machines as well as those
> with browsers.)
>
> All of this points at a bigger question to the community: if we choose
> not to provide a flow implementation in libpq, is adding OAUTHBEARER
> worth the additional maintenance cost?
>
> My personal vote would be "no". I think the hook-only approach proposed
> here would ensure that only larger providers would implement it in
> practice, and in that case I'd rather spend cycles on generic SASL.
>
> > 3. Temporarily removed test suite. We are actively working on aligning
> > the tests with the latest changes. Will add a patch with tests soon.
>
> Okay. Case in point, the following change to the patch appears to be
> invalid JSON:
>
> > +   appendStringInfo(&buf,
> > +       "{ "
> > +           "\"status\": \"invalid_token\", "
> > +           "\"openid-configuration\": \"%s\","
> > +           "\"scope\": \"%s\" ",
> > +           "\"issuer\": \"%s\" ",
> > +       "}",
>
> Additionally, the "issuer" field added here is not part of the RFC. I've
> written my thoughts about unofficial extensions upthread but haven't
> received a response, so I'm going to start being more strident: Please,
> for the sake of reviewers, call out changes you've made to the spec, and
> why they're justified.
>
> The patches seem to be out of order now (and the documentation in the
> commit messages has been removed).
>
> Thanks,
> --Jacob

  48. Re: [PoC] Federated Authn/z with OAUTHBEARER

    mahendrakar s <mahendrakarforpg@gmail.com> — 2023-01-25T04:46:15Z

    Hi All,

The "issuer" field has been removed to align  with the RFC
implementation - https://www.rfc-editor.org/rfc/rfc7628.
This patch "v6" is a single patch to support the OAUTH BEARER token
through psql connection string.
Below flow is supported. Added the documentation in the commit messages.

 +----------------------+                                 +----------+
  |             +-------+                                  | Postgres |
  | PQconnect ->|       |                                  |          |
  |             |       |                                  |   +-----------+
  |             |       | ---------- Empty Token---------> | > |           |
  |             | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth  |
  |          +------+   |                                  |   |  Hook     |
  |     +- < | Hook |   |                                  |   +-----------+
  |     |    +------+   |                                  |          |
  |     v       |       |                                  |          |
  |  [get token]|       |                                  |          |
  |     |       |       |                                  |          |
  |     +       |       |                                  |   +-----------+
  | PQconnect > |       | --------- Access Token --------> | > | Validator |
  |             |       | <---------- Auth Result -------- | < |   Hook    |
  |             |       |                                  |   +-----------+
  |             +-------+                                  |          |
  +----------------------+                                 +----------+

Please note that we are working on modifying/adding new tests (from
Jacob's Patch) with the latest changes. Will add a patch with tests
soon.

Thanks,
Mahendrakar.

On Wed, 18 Jan 2023 at 07:24, Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
>
> > All of this points at a bigger question to the community: if we choose
> > not to provide a flow implementation in libpq, is adding OAUTHBEARER
> > worth the additional maintenance cost?
>
> > My personal vote would be "no". I think the hook-only approach proposed
> > here would ensure that only larger providers would implement it in
> > practice
>
> Flow implementations in libpq are definitely a long term plan, and I
> agree that it would democratise the adoption.
> In the previous posts in this conversation I outlined the ones I think
> we should support.
>
> However, I don't see why it's strictly necessary to couple those.
> As long as the SASL exchange for OAUTHBEARER mechanism is supported by
> the protocol, the Client side can evolve at its own pace.
>
> At the same time, the current implementation allows clients to start
> building provider-agnostic OAUTH support. By using iddawc or OAUTH
> client implementations in the respective platforms.
> So I wouldn't refer to "larger providers", but rather "more motivated
> clients" here. Which definitely overlaps, but keeps the system open.
>
> > I'm not understanding the concern in the final point -- providers
> > generally require you to opt into device authorization, at least as far
> > as I can tell. So if you decide that it's not appropriate for your use
> > case... don't enable it. (And I haven't seen any claims that opting into
> > device authorization weakens the other flows in any way. So if we're
> > going to implement a flow in libpq, I still think device authorization
> > is the best choice, since it works on headless machines as well as those
> > with browsers.)
> I agree with the statement that Device code is the best first choice
> if we absolutely have to pick one.
> Though I don't think we have to.
>
> While device flow can be used for all kinds of user-facing
> applications, it's specifically designed for input-constrained
> scenarios. As clearly stated in the Abstract here -
> https://www.rfc-editor.org/rfc/rfc8628
> The authorization code with pkce flow is recommended by the RFSc and
> major providers for cases when it's feasible.
> The long term goal is to provide both, though I don't see why the
> backbone protocol implementation first wouldn't add value.
>
> Another point is user authentication is one side of the whole story
> and the other critical one is system-to-system authentication. Where
> we have Client Credentials and Certificates.
> With the latter it is much harder to get generically implemented, as
> provider-specific tokens need to be signed.
>
> Adding the other reasoning, I think libpq support for specific flows
> can get in the further iterations, after the protocol support.
>
> > in that case I'd rather spend cycles on generic SASL.
> I see 2 approaches to generic SASL:
> (a). Generic SASL is a framework used in the protocol, with the
> mechanisms implemented on top and exposed to the DBAs as auth types to
> configure in hba.
> This is the direction we're going here, which is well aligned with the
> existing hba-based auth configuration.
> (b). Generic SASL exposed to developers on the server- and client-
> side to extend on. It seems to be a much longer shot.
> The specific points of large ambiguity are libpq distribution model
> (which you pointed to) and potential pluggability of insecure
> mechanisms.
>
> I do see (a) as a sweet spot with a lot of value for various
> participants with much less ambiguity.
>
> > Additionally, the "issuer" field added here is not part of the RFC. I've
> > written my thoughts about unofficial extensions upthread but haven't
> > received a response, so I'm going to start being more strident: Please,
> > for the sake of reviewers, call out changes you've made to the spec, and
> > why they're justified.
> Thanks for your feedback on this. We had this discussion as well, and
> added that as a convenience for the client to identify the provider.
> I don't see a reason why an issuer would be absolutely necessary, so
> we will get your point that sticking to RFCs is a safer choice.
>
> > The patches seem to be out of order now (and the documentation in the
> > commit messages has been removed).
> Feedback taken. Work in progress.
>
> On Tue, Jan 17, 2023 at 2:44 PM Jacob Champion <jchampion@timescale.com> wrote:
> >
> > On Sun, Jan 15, 2023 at 12:03 PM Andrey Chudnovsky
> > <achudnovskij@gmail.com> wrote:
> > > 2. Removed Device Code implementation in libpq. Several reasons:
> > >    - Reduce scope and focus on the protocol first.
> > >    - Device code implementation uses iddawc dependency. Taking this
> > > dependency is a controversial step which requires broader discussion.
> > >    - Device code implementation without iddaws would significantly
> > > increase the scope of the patch, as libpq needs to poll the token
> > > endpoint, setup different API calls, e.t.c.
> > >    - That flow should canonically only be used for clients which can't
> > > invoke browsers. If it is the only flow to be implemented, it can be
> > > used in the context when it's not expected by the OAUTH protocol.
> >
> > I'm not understanding the concern in the final point -- providers
> > generally require you to opt into device authorization, at least as far
> > as I can tell. So if you decide that it's not appropriate for your use
> > case... don't enable it. (And I haven't seen any claims that opting into
> > device authorization weakens the other flows in any way. So if we're
> > going to implement a flow in libpq, I still think device authorization
> > is the best choice, since it works on headless machines as well as those
> > with browsers.)
> >
> > All of this points at a bigger question to the community: if we choose
> > not to provide a flow implementation in libpq, is adding OAUTHBEARER
> > worth the additional maintenance cost?
> >
> > My personal vote would be "no". I think the hook-only approach proposed
> > here would ensure that only larger providers would implement it in
> > practice, and in that case I'd rather spend cycles on generic SASL.
> >
> > > 3. Temporarily removed test suite. We are actively working on aligning
> > > the tests with the latest changes. Will add a patch with tests soon.
> >
> > Okay. Case in point, the following change to the patch appears to be
> > invalid JSON:
> >
> > > +   appendStringInfo(&buf,
> > > +       "{ "
> > > +           "\"status\": \"invalid_token\", "
> > > +           "\"openid-configuration\": \"%s\","
> > > +           "\"scope\": \"%s\" ",
> > > +           "\"issuer\": \"%s\" ",
> > > +       "}",
> >
> > Additionally, the "issuer" field added here is not part of the RFC. I've
> > written my thoughts about unofficial extensions upthread but haven't
> > received a response, so I'm going to start being more strident: Please,
> > for the sake of reviewers, call out changes you've made to the spec, and
> > why they're justified.
> >
> > The patches seem to be out of order now (and the documentation in the
> > commit messages has been removed).
> >
> > Thanks,
> > --Jacob

  49. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Stephen Frost <sfrost@snowman.net> — 2023-02-20T22:35:36Z

    Greetings,

* mahendrakar s (mahendrakarforpg@gmail.com) wrote:
> The "issuer" field has been removed to align  with the RFC
> implementation - https://www.rfc-editor.org/rfc/rfc7628.
> This patch "v6" is a single patch to support the OAUTH BEARER token
> through psql connection string.
> Below flow is supported. Added the documentation in the commit messages.
> 
>  +----------------------+                                 +----------+
>   |             +-------+                                  | Postgres |
>   | PQconnect ->|       |                                  |          |
>   |             |       |                                  |   +-----------+
>   |             |       | ---------- Empty Token---------> | > |           |
>   |             | libpq | <-- Error(Discovery + Scope ) -- | < | Pre-Auth  |
>   |          +------+   |                                  |   |  Hook     |
>   |     +- < | Hook |   |                                  |   +-----------+
>   |     |    +------+   |                                  |          |
>   |     v       |       |                                  |          |
>   |  [get token]|       |                                  |          |
>   |     |       |       |                                  |          |
>   |     +       |       |                                  |   +-----------+
>   | PQconnect > |       | --------- Access Token --------> | > | Validator |
>   |             |       | <---------- Auth Result -------- | < |   Hook    |
>   |             |       |                                  |   +-----------+
>   |             +-------+                                  |          |
>   +----------------------+                                 +----------+
> 
> Please note that we are working on modifying/adding new tests (from
> Jacob's Patch) with the latest changes. Will add a patch with tests
> soon.

Having skimmed back through this thread again, I still feel that the
direction that was originally being taken (actually support something in
libpq and the backend, be it with libiddawc or something else or even
our own code, and not just throw hooks in various places) makes a lot
more sense and is a lot closer to how Kerberos and client-side certs and
even LDAP auth work today.  That also seems like a much better answer
for our users when it comes to new authentication methods than having
extensions and making libpq developers have to write their own custom
code, not to mention that we'd still need to implement something in psql
to provide such a hook if we are to have psql actually usefully exercise
this, no?

In the Kerberos test suite we have today, we actually bring up a proper
Kerberos server, set things up, and then test end-to-end installing a
keytab for the server, getting a TGT, getting a service ticket, testing
authentication and encryption, etc.  Looking around, it seems like the
equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
our own code to really be able to test this and show that it works and
that we're doing it correctly, and to let us know if we break something.

Thanks,

Stephen

  50. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-02-21T22:24:12Z

    On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> Having skimmed back through this thread again, I still feel that the
> direction that was originally being taken (actually support something in
> libpq and the backend, be it with libiddawc or something else or even
> our own code, and not just throw hooks in various places) makes a lot
> more sense and is a lot closer to how Kerberos and client-side certs and
> even LDAP auth work today.

Cool, that helps focus the effort. Thanks!

> That also seems like a much better answer
> for our users when it comes to new authentication methods than having
> extensions and making libpq developers have to write their own custom
> code, not to mention that we'd still need to implement something in psql
> to provide such a hook if we are to have psql actually usefully exercise
> this, no?

I don't mind letting clients implement their own flows... as long as
it's optional. So even if we did use a hook in the end, I agree that
we've got to exercise it ourselves.

> In the Kerberos test suite we have today, we actually bring up a proper
> Kerberos server, set things up, and then test end-to-end installing a
> keytab for the server, getting a TGT, getting a service ticket, testing
> authentication and encryption, etc.  Looking around, it seems like the
> equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> our own code to really be able to test this and show that it works and
> that we're doing it correctly, and to let us know if we break something.

The original patchset includes a test server in Python -- a major
advantage being that you can test the client and server independently
of each other, since the implementation is so asymmetric. Additionally
testing against something like Glewlwyd would be a great way to stack
coverage. (If we *only* test against a packaged server, though, it'll
be harder to test our stuff in the presence of malfunctions and other
corner cases.)

Thanks,
--Jacob

  51. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-02-22T07:00:46Z

    Thanks for the feedback,

> Having skimmed back through this thread again, I still feel that the
> direction that was originally being taken (actually support something in
> libpq and the backend, be it with libiddawc or something else or even
> our own code, and not just throw hooks in various places) makes a lot
> more sense and is a lot closer to how Kerberos and client-side certs and
> even LDAP auth work today.  That also seems like a much better answer
> for our users when it comes to new authentication methods than having
> extensions and making libpq developers have to write their own custom
> code, not to mention that we'd still need to implement something in psql
> to provide such a hook if we are to have psql actually usefully exercise
> this, no?

libpq implementation is the long term plan. However, our intention is
to start with the protocol implementation which allows us to build on
top of.

While device code is the right solution for psql, having that as the
only one can result in incentive to use it in the cases it's not
intended to.
Reasonably good implementation should support all of the following:
(1.) authorization code with pkce (for GUI applications)
(2.) device code (for console user logins)
(3.) client secret
(4.) some support for client certificate flow

(1.) and (4.) require more work to get implemented, though necessary
for encouraging the most secure grant types.
As we didn't have those pieces, we're proposing starting with the
protocol, which can be used by the ecosystem to build token flow
implementations.
Then add the libpq support for individual grant types.

We originally looked at starting with bare bone protocol for PG16 and
adding libpq support in PG17.
That plan won't happen, though still splitting the work into separate
stages would make more sense in my opinion.

Several questions to follow up:
(a.) Would you support committing the protocol first? or you see libpq
implementation for grants as the prerequisite to consider the auth
type?
(b.) As of today, the server side core does not validate that the
token is actually a valid jwt token. Instead relies on the extensions
to do the validation.
Do you think server core should do the basic validation before passing
to extensions to prevent the auth type being used for anything other
than OAUTH flows?

Tests are the plan for the commit-ready implementation.

Thanks!
Andrey.

On Tue, Feb 21, 2023 at 2:24 PM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > Having skimmed back through this thread again, I still feel that the
> > direction that was originally being taken (actually support something in
> > libpq and the backend, be it with libiddawc or something else or even
> > our own code, and not just throw hooks in various places) makes a lot
> > more sense and is a lot closer to how Kerberos and client-side certs and
> > even LDAP auth work today.
>
> Cool, that helps focus the effort. Thanks!
>
> > That also seems like a much better answer
> > for our users when it comes to new authentication methods than having
> > extensions and making libpq developers have to write their own custom
> > code, not to mention that we'd still need to implement something in psql
> > to provide such a hook if we are to have psql actually usefully exercise
> > this, no?
>
> I don't mind letting clients implement their own flows... as long as
> it's optional. So even if we did use a hook in the end, I agree that
> we've got to exercise it ourselves.
>
> > In the Kerberos test suite we have today, we actually bring up a proper
> > Kerberos server, set things up, and then test end-to-end installing a
> > keytab for the server, getting a TGT, getting a service ticket, testing
> > authentication and encryption, etc.  Looking around, it seems like the
> > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > our own code to really be able to test this and show that it works and
> > that we're doing it correctly, and to let us know if we break something.
>
> The original patchset includes a test server in Python -- a major
> advantage being that you can test the client and server independently
> of each other, since the implementation is so asymmetric. Additionally
> testing against something like Glewlwyd would be a great way to stack
> coverage. (If we *only* test against a packaged server, though, it'll
> be harder to test our stuff in the presence of malfunctions and other
> corner cases.)
>
> Thanks,
> --Jacob

  52. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Stephen Frost <sfrost@snowman.net> — 2023-02-23T18:47:55Z

    Greetings,

* Jacob Champion (jchampion@timescale.com) wrote:
> On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > Having skimmed back through this thread again, I still feel that the
> > direction that was originally being taken (actually support something in
> > libpq and the backend, be it with libiddawc or something else or even
> > our own code, and not just throw hooks in various places) makes a lot
> > more sense and is a lot closer to how Kerberos and client-side certs and
> > even LDAP auth work today.
> 
> Cool, that helps focus the effort. Thanks!

Great, glad to hear that.

> > That also seems like a much better answer
> > for our users when it comes to new authentication methods than having
> > extensions and making libpq developers have to write their own custom
> > code, not to mention that we'd still need to implement something in psql
> > to provide such a hook if we are to have psql actually usefully exercise
> > this, no?
> 
> I don't mind letting clients implement their own flows... as long as
> it's optional. So even if we did use a hook in the end, I agree that
> we've got to exercise it ourselves.

This really doesn't feel like a great area to try and do hooks or
similar in, not the least because that approach has been tried and tried
again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
them has turned out great (which is why we can't just tell people "well,
install the pam_oauth2 and watch everything work!") and this strikes me
as trying to do that yet again but worse as it's not even a dedicated
project trying to solve the problem but more like a side project.  SCRAM
was good, we've come a long way thanks to that, this feels like it
should be more in line with that rather than trying to invent yet
another new "generic" set of hooks/APIs that will just cause DBAs and
our users headaches trying to make work.

> > In the Kerberos test suite we have today, we actually bring up a proper
> > Kerberos server, set things up, and then test end-to-end installing a
> > keytab for the server, getting a TGT, getting a service ticket, testing
> > authentication and encryption, etc.  Looking around, it seems like the
> > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > our own code to really be able to test this and show that it works and
> > that we're doing it correctly, and to let us know if we break something.
> 
> The original patchset includes a test server in Python -- a major
> advantage being that you can test the client and server independently
> of each other, since the implementation is so asymmetric. Additionally
> testing against something like Glewlwyd would be a great way to stack
> coverage. (If we *only* test against a packaged server, though, it'll
> be harder to test our stuff in the presence of malfunctions and other
> corner cases.)

Oh, that's even better- I agree entirely that having test code that can
be instructed to return specific errors so that we can test that our
code responds properly is great (and is why pgbackrest has things like
a stub'd out libpq, fake s3, GCS, and Azure servers, and more) and would
certainly want to keep that, even if we also build out a test that uses
a real server to provide integration testing with not-our-code too.

Thanks!

Stephen

  53. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-02-23T23:04:22Z

    > This really doesn't feel like a great area to try and do hooks or
> similar in, not the least because that approach has been tried and tried
> again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> them has turned out great (which is why we can't just tell people "well,
> install the pam_oauth2 and watch everything work!") and this strikes me
> as trying to do that yet again but worse as it's not even a dedicated
> project trying to solve the problem but more like a side project.

In this case it's not intended to be an open-ended hook, but rather an
implementation of a specific rfc (rfc-7628) which defines a
client-server communication for the authentication flow.
The rfc itself does leave a lot of flexibility on specific parts of
the implementation. Which do require hooks:
(1.) Server side hook to validate the token, which is specific to the
OAUTH provider.
(2.) Client side hook to request the client to obtain the token.

On (1.), we would need a hook for the OAUTH provider extension to do
validation. We can though do some basic check that the credential is
indeed a JWT token signed by the requested issuer.

Specifically (2.) is where we can provide a layer in libpq to simplify
the integration. i.e. implement some OAUTH flows.
Though we would need some flexibility for the clients to bring their own token:
For example there are cases where the credential to obtain the token
is stored in a separate secure location and the token is returned from
a separate service or pushed from a more secure environment.

> another new "generic" set of hooks/APIs that will just cause DBAs and
> our users headaches trying to make work.
As I mentioned above, it's an rfc implementation, rather than our invention.
When it comes to DBAs and the users.
Builtin libpq implementations which allows psql and pgadmin to
seamlessly connect should suffice those needs.
While extensibility would allow the ecosystem to be open for OAUTH
providers, SAAS developers, PAAS providers and other institutional
players.

Thanks!
Andrey.

On Thu, Feb 23, 2023 at 10:47 AM Stephen Frost <sfrost@snowman.net> wrote:
>
> Greetings,
>
> * Jacob Champion (jchampion@timescale.com) wrote:
> > On Mon, Feb 20, 2023 at 2:35 PM Stephen Frost <sfrost@snowman.net> wrote:
> > > Having skimmed back through this thread again, I still feel that the
> > > direction that was originally being taken (actually support something in
> > > libpq and the backend, be it with libiddawc or something else or even
> > > our own code, and not just throw hooks in various places) makes a lot
> > > more sense and is a lot closer to how Kerberos and client-side certs and
> > > even LDAP auth work today.
> >
> > Cool, that helps focus the effort. Thanks!
>
> Great, glad to hear that.
>
> > > That also seems like a much better answer
> > > for our users when it comes to new authentication methods than having
> > > extensions and making libpq developers have to write their own custom
> > > code, not to mention that we'd still need to implement something in psql
> > > to provide such a hook if we are to have psql actually usefully exercise
> > > this, no?
> >
> > I don't mind letting clients implement their own flows... as long as
> > it's optional. So even if we did use a hook in the end, I agree that
> > we've got to exercise it ourselves.
>
> This really doesn't feel like a great area to try and do hooks or
> similar in, not the least because that approach has been tried and tried
> again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> them has turned out great (which is why we can't just tell people "well,
> install the pam_oauth2 and watch everything work!") and this strikes me
> as trying to do that yet again but worse as it's not even a dedicated
> project trying to solve the problem but more like a side project.  SCRAM
> was good, we've come a long way thanks to that, this feels like it
> should be more in line with that rather than trying to invent yet
> another new "generic" set of hooks/APIs that will just cause DBAs and
> our users headaches trying to make work.
>
> > > In the Kerberos test suite we have today, we actually bring up a proper
> > > Kerberos server, set things up, and then test end-to-end installing a
> > > keytab for the server, getting a TGT, getting a service ticket, testing
> > > authentication and encryption, etc.  Looking around, it seems like the
> > > equivilant would perhaps be to use Glewlwyd and libiddawc or libcurl and
> > > our own code to really be able to test this and show that it works and
> > > that we're doing it correctly, and to let us know if we break something.
> >
> > The original patchset includes a test server in Python -- a major
> > advantage being that you can test the client and server independently
> > of each other, since the implementation is so asymmetric. Additionally
> > testing against something like Glewlwyd would be a great way to stack
> > coverage. (If we *only* test against a packaged server, though, it'll
> > be harder to test our stuff in the presence of malfunctions and other
> > corner cases.)
>
> Oh, that's even better- I agree entirely that having test code that can
> be instructed to return specific errors so that we can test that our
> code responds properly is great (and is why pgbackrest has things like
> a stub'd out libpq, fake s3, GCS, and Azure servers, and more) and would
> certainly want to keep that, even if we also build out a test that uses
> a real server to provide integration testing with not-our-code too.
>
> Thanks!
>
> Stephen

  54. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Stephen Frost <sfrost@snowman.net> — 2023-02-27T20:31:04Z

    Greetings,

* Andrey Chudnovsky (achudnovskij@gmail.com) wrote:
> > This really doesn't feel like a great area to try and do hooks or
> > similar in, not the least because that approach has been tried and tried
> > again (PAM, GSSAPI, SASL would all be examples..) and frankly none of
> > them has turned out great (which is why we can't just tell people "well,
> > install the pam_oauth2 and watch everything work!") and this strikes me
> > as trying to do that yet again but worse as it's not even a dedicated
> > project trying to solve the problem but more like a side project.
> 
> In this case it's not intended to be an open-ended hook, but rather an
> implementation of a specific rfc (rfc-7628) which defines a
> client-server communication for the authentication flow.
> The rfc itself does leave a lot of flexibility on specific parts of
> the implementation. Which do require hooks:

Color me skeptical on an RFC that requires hooks.

> (1.) Server side hook to validate the token, which is specific to the
> OAUTH provider.
> (2.) Client side hook to request the client to obtain the token.

Perhaps I'm missing it... but weren't these handled with what the
original patch that Jacob had was doing?

> On (1.), we would need a hook for the OAUTH provider extension to do
> validation. We can though do some basic check that the credential is
> indeed a JWT token signed by the requested issuer.
> 
> Specifically (2.) is where we can provide a layer in libpq to simplify
> the integration. i.e. implement some OAUTH flows.
> Though we would need some flexibility for the clients to bring their own token:
> For example there are cases where the credential to obtain the token
> is stored in a separate secure location and the token is returned from
> a separate service or pushed from a more secure environment.

In those cases... we could, if we wanted, simply implement the code to
actually pull the token, no?  We don't *have* to have a hook here for
this, we could just make it work.

> > another new "generic" set of hooks/APIs that will just cause DBAs and
> > our users headaches trying to make work.
> As I mentioned above, it's an rfc implementation, rather than our invention.

While I only took a quick look, I didn't see anything in that RFC that
explicitly says that hooks or a plugin or a library or such is required
to meet the RFC.  Sure, there are places which say that the
implementation is specific to a particular server or client but that's
not the same thing.

> When it comes to DBAs and the users.
> Builtin libpq implementations which allows psql and pgadmin to
> seamlessly connect should suffice those needs.
> While extensibility would allow the ecosystem to be open for OAUTH
> providers, SAAS developers, PAAS providers and other institutional
> players.

Each to end up writing their own code to do largely the same thing
without the benefit of the larger community to be able to review and
ensure that it's done properly?

That doesn't sound like a great approach to me.

Thanks,

Stephen

  55. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-04-27T17:35:20Z

    On Fri, Sep 23, 2022 at 3:39 PM Jacob Champion <jchampion@timescale.com> wrote:
> Here's a newly rebased v5. (They're all zipped now, which I probably
> should have done a while back, sorry.)

To keep this current, v7 is rebased over latest, without the pluggable
authentication patches. This doesn't yet address the architectural
feedback that was discussed previously, so if you're primarily
interested in that, you can safely ignore this version of the
patchset.

The key changes here include
- Meson support, for both the build and the pytest suite
- Cirrus support (and unsurprisingly, Mac and Windows builds fail due
to the Linux-oriented draft code)
- A small tweak to support iddawc down to 0.9.8 (shipped with e.g.
Debian Bullseye)
- Removal of the authn_id test extension in favor of SYSTEM_USER

The meson+pytest support was big enough that I split it into its own
patch. It's not very polished yet, but it mostly works, and when
running tests via Meson it'll now spin up a test server for you. My
virtualenv approach apparently interacts poorly with the multiarch
Cirrus setup (64-bit tests pass, 32-bit tests fail).

Moving forward, the first thing I plan to tackle is asynchronous
operation, so that polling clients can still operate sanely. If I can
find a good solution there, the conversations about possible extension
points should get a lot easier.

Thanks,
--Jacob

  56. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-05-19T22:01:11Z

    On 4/27/23 10:35, Jacob Champion wrote:
> Moving forward, the first thing I plan to tackle is asynchronous
> operation, so that polling clients can still operate sanely. If I can
> find a good solution there, the conversations about possible extension
> points should get a lot easier.

Attached is patchset v8, now with concurrency and 300% more cURL! And
many more questions to answer.

This is a full reimplementation of the client-side OAuth flow. It's an
async-first engine built on top of cURL's multi handles. All pending
operations are multiplexed into a single epoll set (the "altsock"),
which is exposed through PQsocket() for the duration of the OAuth flow.
Clients return to the flow on their next call to PQconnectPoll().

Andrey and Mahendrakar: you'll probably be interested in the
conn->async_auth() callback, conn->altsock, and the pg_fe_run_oauth_flow
entry point. This is intended to be the foundation for alternative flows.

I've kept the blocking iddawc implementation for comparison, but if
you're running the tests against it, be aware that the asynchronous
tests will, predictably, hang. Skip them with `py.test -k 'not
asynchronous'`.

= The Good =

- PQconnectPoll() is no longer indefinitely blocked on a single
connection's OAuth handshake. (iddawc doesn't appear to have any
asynchronous primitives in its API, unless I've missed something crucial.)

- We now have a swappable entry point. Alternative flows could be
implemented by applications without forcing clients to redesign their
polling loops (PQconnect* should just work as expected).

- We have full control over corner cases in our default flow. Debugging
failures is much nicer, with explanations of exactly what has gone wrong
and where, compared to iddawc's "I_ERROR" messages.

- cURL is not a lightweight library by any means, but we're no longer
bundling things like web servers that we're not going to use.

= The Bad =

- Unsurprisingly, there's a lot more code now that we're implementing
the flow ourselves. The client patch has tripled in size, and we'd be on
the hook for implementing and staying current with the RFCs.

- The client implementation is currently epoll-/Linux-specific. I think
kqueue shouldn't be too much trouble for the BSDs, but it's even more
code to maintain.

- Some clients in the wild (psycopg2/psycopg) suppress all notifications
during PQconnectPoll(). To accommodate them, I no longer use the
noticeHooks for communicating the user code, but that means we have to
come up with some other way to let applications override the printing to
stderr. Something like the OpenSSL decryption callback, maybe?

= The Ugly =

- Unless someone is aware of some amazing Winsock magic, I'm pretty sure
the multiplexed-socket approach is dead in the water on Windows. I think
the strategy there probably has to be a background thread plus a fake
"self-pipe" (loopback socket) for polling... which may be controversial?

- We have to figure out how to initialize cURL in a thread-safe manner.
Newer versions of libcurl and OpenSSL improve upon this situation, but I
don't think there's a way to check at compile time whether the
initialization strategy is safe or not (and even at runtime, I think
there may be a chicken-and-egg problem with the API, where it's not safe
to check for thread-safe initialization until after you've safely
initialized).

= Next Steps =

There are so many TODOs in the cURL implementation: it's been a while
since I've done any libcurl programming, it all needs to be hardened,
and I need to comb through the relevant specs again. But I don't want to
gold-plate it if this overall approach is unacceptable. So, questions
for the gallery:

1) Would starting up a background thread (pooled or not) be acceptable
on Windows? Alternatively, does anyone know enough Winsock deep magic to
combine multiple pending events into one (selectable!) socket?

2) If a background thread is acceptable on one platform, does it make
more sense to use one on every platform and just have synchronous code
everywhere? Or should we use a threadless async implementation when we can?

3) Is the current conn->async_auth() entry point sufficient for an
application to implement the Microsoft flows discussed upthread?

4) Would we want to try to require a new enough cURL/OpenSSL to avoid
thread safety problems during initialization, or do we need to introduce
some API equivalent to PQinitOpenSSL?

5) Does this maintenance tradeoff (full control over the client vs. a
large amount of RFC-governed code) seem like it could be okay?

Thanks,
--Jacob

  57. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniele Varrazzo <daniele.varrazzo@gmail.com> — 2023-05-23T11:22:20Z

    On Sat, 20 May 2023 at 00:01, Jacob Champion <jchampion@timescale.com> wrote:

> - Some clients in the wild (psycopg2/psycopg) suppress all notifications
> during PQconnectPoll().

If there is anything we can improve in psycopg please reach out.

-- Daniele

  58. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-05-23T15:56:47Z

    On Tue, May 23, 2023 at 4:22 AM Daniele Varrazzo
<daniele.varrazzo@gmail.com> wrote:
> On Sat, 20 May 2023 at 00:01, Jacob Champion <jchampion@timescale.com> wrote:
> > - Some clients in the wild (psycopg2/psycopg) suppress all notifications
> > during PQconnectPoll().
>
> If there is anything we can improve in psycopg please reach out.

Will do, thank you! But in this case, I think there's nothing to
improve in psycopg -- in fact, it highlighted the problem with my
initial design, and now I think the notice processor will never be an
appropriate avenue for communication of the user code.

The biggest issue is that there's a chicken-and-egg situation: if
you're using the synchronous PQconnect* API, you can't override the
notice hooks while the handshake is in progress, because you don't
have a connection handle yet. The second problem is that there are a
bunch of parameters coming back from the server (user code,
verification URI, expiration time) that the application may choose to
display or use, and communicating those pieces in a (probably already
translated) flat text string is a pretty hostile API.

So I think we'll probably need to provide a global handler API,
similar to the passphrase hook we currently provide, that can receive
these pieces separately and assemble them however the application
desires. The hard part will be to avoid painting ourselves into a
corner, because this particular information is specific to the device
authorization flow, and if we ever want to add other flows into libpq,
we'll probably not want to add even more hooks.

Thanks,
--Jacob

  59. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-06-29T16:28:24Z

    On 5/19/23 15:01, Jacob Champion wrote:
> But I don't want to
> gold-plate it if this overall approach is unacceptable. So, questions
> for the gallery:
> 
> 1) Would starting up a background thread (pooled or not) be acceptable
> on Windows? Alternatively, does anyone know enough Winsock deep magic to
> combine multiple pending events into one (selectable!) socket?
> 
> 2) If a background thread is acceptable on one platform, does it make
> more sense to use one on every platform and just have synchronous code
> everywhere? Or should we use a threadless async implementation when we can?
> 
> 3) Is the current conn->async_auth() entry point sufficient for an
> application to implement the Microsoft flows discussed upthread?
> 
> 4) Would we want to try to require a new enough cURL/OpenSSL to avoid
> thread safety problems during initialization, or do we need to introduce
> some API equivalent to PQinitOpenSSL?
> 
> 5) Does this maintenance tradeoff (full control over the client vs. a
> large amount of RFC-governed code) seem like it could be okay?

There was additional interest at PGCon, so I've registered this in the
commitfest.

Potential reviewers should be aware that the current implementation
requires Linux (or, more specifically, epoll), as the cfbot shows. But
if you have any opinions on the above questions, those will help me
tackle the other platforms. :D

Thanks!
--Jacob

  60. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-01T04:28:52Z

    On Sat, May 20, 2023 at 10:01 AM Jacob Champion <jchampion@timescale.com> wrote:
> - The client implementation is currently epoll-/Linux-specific. I think
> kqueue shouldn't be too much trouble for the BSDs, but it's even more
> code to maintain.

I guess you also need a fallback that uses plain old POSIX poll()?  I
see you're not just using epoll but also timerfd.  Could that be
converted to plain old timeout bookkeeping?  That should be enough to
get every other Unix and *possibly* also Windows to work with the same
code path.

> - Unless someone is aware of some amazing Winsock magic, I'm pretty sure
> the multiplexed-socket approach is dead in the water on Windows. I think
> the strategy there probably has to be a background thread plus a fake
> "self-pipe" (loopback socket) for polling... which may be controversial?

I am not a Windows user or hacker, but there are certainly several
ways to multiplex sockets.  First there is the WSAEventSelect() +
WaitForMultipleObjects() approach that latch.c uses.  It has the
advantage that it allows socket readiness to be multiplexed with
various other things that use Windows "events".  But if you don't need
that, ie you *only* need readiness-based wakeup for a bunch of sockets
and no other kinds of fd or object, you can use winsock's plain old
select() or its fairly faithful poll() clone called WSAPoll().  It
looks a bit like that'd be true here if you could kill the timerfd?

It's a shame to write modern code using select(), but you can find
lots of shouting all over the internet about WSAPoll()'s defects, most
famously the cURL guys[1] whose blog is widely cited, so people still
do it.  Possibly some good news on that front:  by my reading of the
docs, it looks like that problem was fixed in Windows 10 2004[2] which
itself is by now EOL, so all systems should have the fix?  I suspect
that means that, finally, you could probably just use the same poll()
code path for Unix (when epoll is not available) *and* Windows these
days, making porting a lot easier.  But I've never tried it, so I
don't know what other problems there might be.  Another thing people
complain about is the lack of socketpair() or similar in winsock which
means you unfortunately can't easily make anonymous
select/poll-compatible local sockets, but that doesn't seem to be
needed here.

[1] https://daniel.haxx.se/blog/2012/10/10/wsapoll-is-broken/
[2] https://learn.microsoft.com/en-us/windows/win32/api/winsock2/nf-winsock2-wsapoll

  61. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-05T21:00:27Z

    On Fri, Jun 30, 2023 at 9:29 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Sat, May 20, 2023 at 10:01 AM Jacob Champion <jchampion@timescale.com> wrote:
> > - The client implementation is currently epoll-/Linux-specific. I think
> > kqueue shouldn't be too much trouble for the BSDs, but it's even more
> > code to maintain.
>
> I guess you also need a fallback that uses plain old POSIX poll()?

The use of the epoll API here is to combine several sockets into one,
not to actually call epoll_wait() itself. kqueue descriptors should
let us do the same, IIUC.

> I see you're not just using epoll but also timerfd.  Could that be
> converted to plain old timeout bookkeeping?  That should be enough to
> get every other Unix and *possibly* also Windows to work with the same
> code path.

I might be misunderstanding your suggestion, but I think our internal
bookkeeping is orthogonal to that. The use of timerfd here allows us
to forward libcurl's timeout requirements up to the top-level
PQsocket(). As an example, libcurl is free to tell us to call it again
in ten milliseconds, and we have to make sure a nonblocking client
calls us again after that elapses; otherwise they might hang waiting
for data that's not coming.

> > - Unless someone is aware of some amazing Winsock magic, I'm pretty sure
> > the multiplexed-socket approach is dead in the water on Windows. I think
> > the strategy there probably has to be a background thread plus a fake
> > "self-pipe" (loopback socket) for polling... which may be controversial?
>
> I am not a Windows user or hacker, but there are certainly several
> ways to multiplex sockets.  First there is the WSAEventSelect() +
> WaitForMultipleObjects() approach that latch.c uses.

I don't think that strategy plays well with select() clients, though
-- it requires a handle array, and we've just got the one socket.

My goal is to maintain compatibility with existing PQconnectPoll()
applications, where the only way we get to communicate with the client
is through the PQsocket() for the connection. Ideally, you shouldn't
have to completely rewrite your application loop just to make use of
OAuth. (I assume a requirement like that would be a major roadblock to
committing this -- and if that's not a correct assumption, then I
guess my job gets a lot easier?)

> It's a shame to write modern code using select(), but you can find
> lots of shouting all over the internet about WSAPoll()'s defects, most
> famously the cURL guys[1] whose blog is widely cited, so people still
> do it.

Right -- that's basically the root of my concern. I can't guarantee
that existing Windows clients out there are all using
WaitForMultipleObjects(). From what I can tell, whatever we hand up
through PQsocket() has to be fully Winsock-/select-compatible.

> Another thing people
> complain about is the lack of socketpair() or similar in winsock which
> means you unfortunately can't easily make anonymous
> select/poll-compatible local sockets, but that doesn't seem to be
> needed here.

For the background-thread implementation, it probably would be. I've
been looking at libevent (BSD-licensed) and its socketpair hack for
Windows...

Thanks!
--Jacob

  62. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-05T22:07:17Z

    On Thu, Jul 6, 2023 at 9:00 AM Jacob Champion <jchampion@timescale.com> wrote:
> My goal is to maintain compatibility with existing PQconnectPoll()
> applications, where the only way we get to communicate with the client
> is through the PQsocket() for the connection. Ideally, you shouldn't
> have to completely rewrite your application loop just to make use of
> OAuth. (I assume a requirement like that would be a major roadblock to
> committing this -- and if that's not a correct assumption, then I
> guess my job gets a lot easier?)

Ah, right, I get it.

I guess there are a couple of ways to do it if we give up the goal of
no-code-change-for-the-client:

1.  Generalised PQsocket(), that so that a client can call something like:

int PQpollset(const PGConn *conn, struct pollfd fds[], int fds_size,
int *nfds, int *timeout_ms);

That way, libpq could tell you about which events it would like to
wait for on which fds, and when it would like you to call it back due
to timeout, and you can either pass that information directly to
poll() or WSAPoll() or some equivalent interface (we don't care, we
just gave you the info you need), or combine it in obvious ways with
whatever else you want to multiplex with in your client program.

2.  Convert those events into new libpq events like 'I want you to
call me back in 100ms', and 'call me back when socket #42 has data',
and let clients handle that by managing their own poll set etc.  (This
is something I've speculated about to support more efficient
postgres_fdw shard query multiplexing; gotta figure out how to get
multiple connections' events into one WaitEventSet...)

I guess there is a practical middle ground where client code on
systems that have epoll/kqueue can use OAUTHBEARER without any code
change, and the feature is available on other systems too but you'll
have to change your client code to use one of those interfaces or else
you get an error 'coz we just can't do it.  Or, more likely in the
first version, you just can't do it at all...  Doesn't seem that bad
to me.

BTW I will happily do the epoll->kqueue port work if necessary.

  63. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-06T16:56:49Z

    On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I guess there are a couple of ways to do it if we give up the goal of
> no-code-change-for-the-client:
>
> 1.  Generalised PQsocket(), that so that a client can call something like:
>
> int PQpollset(const PGConn *conn, struct pollfd fds[], int fds_size,
> int *nfds, int *timeout_ms);
>
> That way, libpq could tell you about which events it would like to
> wait for on which fds, and when it would like you to call it back due
> to timeout, and you can either pass that information directly to
> poll() or WSAPoll() or some equivalent interface (we don't care, we
> just gave you the info you need), or combine it in obvious ways with
> whatever else you want to multiplex with in your client program.

I absolutely wanted something like this while I was writing the code
(it would have made things much easier), but I'd feel bad adding that
much complexity to the API if the vast majority of connections use
exactly one socket. Are there other use cases in libpq where you think
this expanded API could be useful? Maybe to lift some of the existing
restrictions for PQconnectPoll(), add async DNS resolution, or
something?

Couple complications I can think of at the moment:
1. Clients using persistent pollsets will have to remove old
descriptors, presumably by tracking the delta since the last call,
which might make for a rough transition. Bookkeeping bugs probably
wouldn't show up unless they used OAuth in their test suites. With the
current model, that's more hidden and libpq takes responsibility for
getting it right.
2. In the future, we might need to think carefully around situations
where we want multiple PGConn handles to share descriptors (e.g.
multiplexed backend connections). I avoid tricky questions at the
moment by assigning only one connection per multi pool.

> 2.  Convert those events into new libpq events like 'I want you to
> call me back in 100ms', and 'call me back when socket #42 has data',
> and let clients handle that by managing their own poll set etc.  (This
> is something I've speculated about to support more efficient
> postgres_fdw shard query multiplexing; gotta figure out how to get
> multiple connections' events into one WaitEventSet...)

Something analogous to libcurl's socket and timeout callbacks [1],
then? Or is there an existing libpq API you were thinking about using?

> I guess there is a practical middle ground where client code on
> systems that have epoll/kqueue can use OAUTHBEARER without any code
> change, and the feature is available on other systems too but you'll
> have to change your client code to use one of those interfaces or else
> you get an error 'coz we just can't do it.

That's a possibility -- if your platform is able to do it nicely,
might as well use it. (In a similar vein, I'd personally vote against
having every platform use a background thread, even if we decided to
implement it for Windows.)

> Or, more likely in the
> first version, you just can't do it at all...  Doesn't seem that bad
> to me.

Any initial opinions on whether it's worse or better than a worker thread?

> BTW I will happily do the epoll->kqueue port work if necessary.

And I will happily take you up on that; thanks!

--Jacob

[1] https://curl.se/libcurl/c/CURLMOPT_SOCKETFUNCTION.html

  64. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-06T20:47:47Z

    On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > 2.  Convert those events into new libpq events like 'I want you to
> > call me back in 100ms', and 'call me back when socket #42 has data',
> > and let clients handle that by managing their own poll set etc.  (This
> > is something I've speculated about to support more efficient
> > postgres_fdw shard query multiplexing; gotta figure out how to get
> > multiple connections' events into one WaitEventSet...)
>
> Something analogous to libcurl's socket and timeout callbacks [1],
> then? Or is there an existing libpq API you were thinking about using?

Yeah.  Libpq already has an event concept.  I did some work on getting
long-lived WaitEventSet objects to be used in various places, some of
which got committed[1], but not yet the parts related to postgres_fdw
(which uses libpq connections to talk to other PostgreSQL servers, and
runs into the limitations of PQsocket()).  Horiguchi-san had the good
idea of extending the event system to cover socket changes, but I
haven't actually tried it yet.  One day.

> > Or, more likely in the
> > first version, you just can't do it at all...  Doesn't seem that bad
> > to me.
>
> Any initial opinions on whether it's worse or better than a worker thread?

My vote is that it's perfectly fine to make a new feature that only
works on some OSes.  If/when someone wants to work on getting it going
on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
OSes we target), they can write the patch.

[1] https://www.postgresql.org/message-id/flat/CA%2BhUKGJAC4Oqao%3DqforhNey20J8CiG2R%3DoBPqvfR0vOJrFysGw%40mail.gmail.com

  65. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-07T18:48:26Z

    On Thu, Jul 6, 2023 at 1:48 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > Something analogous to libcurl's socket and timeout callbacks [1],
> > then? Or is there an existing libpq API you were thinking about using?
>
> Yeah.  Libpq already has an event concept.

Thanks -- I don't know how I never noticed libpq-events.h before.

Per-connection events (or callbacks) might bring up the same
chicken-and-egg situation discussed above, with the notice hook. We'll
be fine as long as PQconnectStart is guaranteed to return before the
PQconnectPoll engine gets to authentication, and it looks like that's
true with today's implementation, which returns pessimistically at
several points instead of just trying to continue the exchange. But I
don't know if that's intended as a guarantee for the future. At the
very least we would have to pin that implementation detail.

> > > Or, more likely in the
> > > first version, you just can't do it at all...  Doesn't seem that bad
> > > to me.
> >
> > Any initial opinions on whether it's worse or better than a worker thread?
>
> My vote is that it's perfectly fine to make a new feature that only
> works on some OSes.  If/when someone wants to work on getting it going
> on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
> OSes we target), they can write the patch.

Okay. I'm curious to hear others' thoughts on that, too, if anyone's lurking.

Thanks!
--Jacob

  66. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-07-07T21:16:05Z

    Thanks Jacob for making progress on this.

> 3) Is the current conn->async_auth() entry point sufficient for an
> application to implement the Microsoft flows discussed upthread?

Please confirm my understanding of the flow is correct:
1. Client calls PQconnectStart.
  - The client doesn't know yet what is the issuer and the scope.
  - Parameters are strings, so callback is not provided yet.
2. Client gets PgConn from PQconnectStart return value and updates
conn->async_auth to its own callback.
3. Client polls PQconnectPoll and checks conn->sasl_state until the
value is SASL_ASYNC
4. Client accesses conn->oauth_issuer and conn->oauth_scope and uses
those info to trigger the token flow.
5. Expectations on async_auth:
    a. It returns PGRES_POLLING_READING while token acquisition is going on
    b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
when token acquisition succeeds.
6. Is the client supposed to do anything with the altsock parameter?

Is the above accurate understanding?

If yes, it looks workable with a couple of improvements I think would be nice:
1. Currently, oauth_exchange function sets conn->async_auth =
pg_fe_run_oauth_flow and starts Device Code flow automatically when
receiving challenge and metadata from the server.
    There probably should be a way for the client to prevent default
Device Code flow from triggering.
2. The current signature and expectations from async_auth function
seems to be tightly coupled with the internal implementation:
    - Pieces of information need to be picked and updated in different
places in the PgConn structure.
    - Function is expected to return PostgresPollingStatusType which
is used to communicate internal state to the client.
   Would it make sense to separate the internal callback used to
communicate with Device Code flow from client facing API?
   I.e. introduce a new client facing structure and enum to facilitate
callback and its return value.

-----------
On a separate note:
The backend code currently spawns an external command for token validation.
As we discussed before, an extension hook would be a more efficient
extensibility option.
We see clients make 10k+ connections using OAuth tokens per minute to
our service, and stating external processes would be too much overhead
here.

-----------

> 5) Does this maintenance tradeoff (full control over the client vs. a
> large amount of RFC-governed code) seem like it could be okay?

It's nice for psql to have Device Code flow. Can be made even more
convenient with refresh tokens support.
And for clients on resource constrained devices to be able to
authenticate with Client Credentials (app secret) without bringing
more dependencies.

In most other cases, upstream PostgreSQL drivers written in higher
level languages have libraries / abstractions to implement OAUTH flows
for the platforms they support.

On Fri, Jul 7, 2023 at 11:48 AM Jacob Champion <jchampion@timescale.com> wrote:
>
> On Thu, Jul 6, 2023 at 1:48 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > > Something analogous to libcurl's socket and timeout callbacks [1],
> > > then? Or is there an existing libpq API you were thinking about using?
> >
> > Yeah.  Libpq already has an event concept.
>
> Thanks -- I don't know how I never noticed libpq-events.h before.
>
> Per-connection events (or callbacks) might bring up the same
> chicken-and-egg situation discussed above, with the notice hook. We'll
> be fine as long as PQconnectStart is guaranteed to return before the
> PQconnectPoll engine gets to authentication, and it looks like that's
> true with today's implementation, which returns pessimistically at
> several points instead of just trying to continue the exchange. But I
> don't know if that's intended as a guarantee for the future. At the
> very least we would have to pin that implementation detail.
>
> > > > Or, more likely in the
> > > > first version, you just can't do it at all...  Doesn't seem that bad
> > > > to me.
> > >
> > > Any initial opinions on whether it's worse or better than a worker thread?
> >
> > My vote is that it's perfectly fine to make a new feature that only
> > works on some OSes.  If/when someone wants to work on getting it going
> > on Windows/AIX/Solaris (that's the complete set of no-epoll, no-kqueue
> > OSes we target), they can write the patch.
>
> Okay. I'm curious to hear others' thoughts on that, too, if anyone's lurking.
>
> Thanks!
> --Jacob

  67. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-08T01:00:51Z

    On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > BTW I will happily do the epoll->kqueue port work if necessary.
>
> And I will happily take you up on that; thanks!

Some initial hacking, about 2 coffees' worth:
https://github.com/macdice/postgres/commits/oauth-kqueue

This compiles on FreeBSD and macOS, but I didn't have time to figure
out all your Python testing magic so I don't know if it works yet and
it's still red on CI...  one thing I wondered about is the *altsock =
timerfd part which I couldn't do.

The situation on macOS is a little odd: the man page says EVFILT_TIMER
is not implemented.  But clearly it is, we can read the source code as
I had to do to find out which unit of time it defaults to[1] (huh,
Apple's github repo for Darwin appears to have been archived recently
-- no more source code updates?  that'd be a shame!), and it works
exactly as expected in simple programs.  So I would just assume it
works until we see evidence otherwise.  (We already use a couple of
other things on macOS more or less by accident because configure finds
them, where they are undocumented or undeclared.)

[1] https://github.com/apple/darwin-xnu/blob/main/bsd/kern/kern_event.c#L1345

  68. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-10T23:21:58Z

    On Fri, Jul 7, 2023 at 2:16 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> Please confirm my understanding of the flow is correct:
> 1. Client calls PQconnectStart.
>   - The client doesn't know yet what is the issuer and the scope.

Right. (Strictly speaking it doesn't even know that OAuth will be used
for the connection, yet, though at some point we'll be able to force
the issue with e.g. `require_auth=oauth`. That's not currently
implemented.)

>   - Parameters are strings, so callback is not provided yet.
> 2. Client gets PgConn from PQconnectStart return value and updates
> conn->async_auth to its own callback.

This is where some sort of official authn callback registration (see
above reply to Daniele) would probably come in handy.

> 3. Client polls PQconnectPoll and checks conn->sasl_state until the
> value is SASL_ASYNC

In my head, the client's custom callback would always be invoked
during the call to PQconnectPoll, rather than making the client do
work in between calls. That way, a client can use custom flows even
with a synchronous PQconnectdb().

> 4. Client accesses conn->oauth_issuer and conn->oauth_scope and uses
> those info to trigger the token flow.

Right.

> 5. Expectations on async_auth:
>     a. It returns PGRES_POLLING_READING while token acquisition is going on
>     b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
> when token acquisition succeeds.

Yes. Though the token should probably be returned through some
explicit part of the callback, now that you mention it...

> 6. Is the client supposed to do anything with the altsock parameter?

The callback needs to set the altsock up with a select()able
descriptor, which wakes up the client when more work is ready to be
done. Without that, you can't handle multiple connections on a single
thread.

> If yes, it looks workable with a couple of improvements I think would be nice:
> 1. Currently, oauth_exchange function sets conn->async_auth =
> pg_fe_run_oauth_flow and starts Device Code flow automatically when
> receiving challenge and metadata from the server.
>     There probably should be a way for the client to prevent default
> Device Code flow from triggering.

Agreed. I'd like the client to be able to override this directly.

> 2. The current signature and expectations from async_auth function
> seems to be tightly coupled with the internal implementation:
>     - Pieces of information need to be picked and updated in different
> places in the PgConn structure.
>     - Function is expected to return PostgresPollingStatusType which
> is used to communicate internal state to the client.
>    Would it make sense to separate the internal callback used to
> communicate with Device Code flow from client facing API?
>    I.e. introduce a new client facing structure and enum to facilitate
> callback and its return value.

Yep, exactly right! I just wanted to check that the architecture
*looked* sufficient before pulling it up into an API.

> On a separate note:
> The backend code currently spawns an external command for token validation.
> As we discussed before, an extension hook would be a more efficient
> extensibility option.
> We see clients make 10k+ connections using OAuth tokens per minute to
> our service, and stating external processes would be too much overhead
> here.

+1. I'm curious, though -- what language do you expect to use to write
a production validator hook? Surely not low-level C...?

> > 5) Does this maintenance tradeoff (full control over the client vs. a
> > large amount of RFC-governed code) seem like it could be okay?
>
> It's nice for psql to have Device Code flow. Can be made even more
> convenient with refresh tokens support.
> And for clients on resource constrained devices to be able to
> authenticate with Client Credentials (app secret) without bringing
> more dependencies.
>
> In most other cases, upstream PostgreSQL drivers written in higher
> level languages have libraries / abstractions to implement OAUTH flows
> for the platforms they support.

Yeah, I'm really interested in seeing which existing high-level flows
can be mixed in through a driver. Trying not to get too far ahead of
myself :D

Thanks for the review!

--Jacob

  69. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-10T23:50:22Z

    On Fri, Jul 7, 2023 at 6:01 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>
> On Fri, Jul 7, 2023 at 4:57 AM Jacob Champion <jchampion@timescale.com> wrote:
> > On Wed, Jul 5, 2023 at 3:07 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > > BTW I will happily do the epoll->kqueue port work if necessary.
> >
> > And I will happily take you up on that; thanks!
>
> Some initial hacking, about 2 coffees' worth:
> https://github.com/macdice/postgres/commits/oauth-kqueue
>
> This compiles on FreeBSD and macOS, but I didn't have time to figure
> out all your Python testing magic so I don't know if it works yet and
> it's still red on CI...

This is awesome, thank you!

I need to look into the CI more, but it looks like the client tests
are passing, which is a good sign. (I don't understand why the
server-side tests are failing on FreeBSD, but they shouldn't be using
the libpq code at all, so I think your kqueue implementation is in the
clear. Cirrus doesn't have the logs from the server-side test failures
anywhere -- probably a bug in my Meson patch.)

> one thing I wondered about is the *altsock =
> timerfd part which I couldn't do.

I did that because I'm not entirely sure that libcurl is guaranteed to
have cleared out all its sockets from the mux, and I didn't want to
invite spurious wakeups. I should probably verify whether or not
that's possible. If so, we could just make that code resilient to
early wakeup, so that it matters less, or set up a second kqueue that
only holds the timer if that turns out to be unacceptable?

> The situation on macOS is a little odd: the man page says EVFILT_TIMER
> is not implemented.  But clearly it is, we can read the source code as
> I had to do to find out which unit of time it defaults to[1] (huh,
> Apple's github repo for Darwin appears to have been archived recently
> -- no more source code updates?  that'd be a shame!), and it works
> exactly as expected in simple programs.  So I would just assume it
> works until we see evidence otherwise.  (We already use a couple of
> other things on macOS more or less by accident because configure finds
> them, where they are undocumented or undeclared.)

Huh. Something to keep an eye on... might be a problem with older versions?

Thanks!
--Jacob

  70. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-11T17:50:28Z

    On Mon, Jul 10, 2023 at 4:50 PM Jacob Champion <jchampion@timescale.com> wrote:
> I don't understand why the
> server-side tests are failing on FreeBSD, but they shouldn't be using
> the libpq code at all, so I think your kqueue implementation is in the
> clear.

Oh, whoops, it's just the missed CLOEXEC flag in the final patch. (If
the write side of the pipe gets copied around, it hangs open and the
validator never sees the "end" of the token.) I'll switch the logic
around to set the flag on the write side instead of unsetting it on
the read side.

I have a WIP patch that passes tests on FreeBSD, which I'll clean up
and post Sometime Soon. macOS builds now but still fails before it
runs the test; looks like it's having trouble finding OpenSSL during
`pip install` of the test modules...

Thanks!
--Jacob

  71. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-12T01:37:22Z

    On Wed, Jul 12, 2023 at 5:50 AM Jacob Champion <jchampion@timescale.com> wrote:
> Oh, whoops, it's just the missed CLOEXEC flag in the final patch. (If
> the write side of the pipe gets copied around, it hangs open and the
> validator never sees the "end" of the token.) I'll switch the logic
> around to set the flag on the write side instead of unsetting it on
> the read side.

Oops, sorry about that.  Glad to hear it's all working!

(FTR my parenthetical note about macOS/XNU sources on Github was a
false alarm: the "apple" account has stopped publishing a redundant
copy of that, but "apple-oss-distributions" is the account I should
have been looking at and it is live.  I guess it migrated at some
point, or something.  Phew.)

  72. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-07-13T04:51:48Z

    > >   - Parameters are strings, so callback is not provided yet.
> > 2. Client gets PgConn from PQconnectStart return value and updates
> > conn->async_auth to its own callback.
>
> This is where some sort of official authn callback registration (see
> above reply to Daniele) would probably come in handy.
+1

> > 3. Client polls PQconnectPoll and checks conn->sasl_state until the
> > value is SASL_ASYNC
>
> In my head, the client's custom callback would always be invoked
> during the call to PQconnectPoll, rather than making the client do
> work in between calls. That way, a client can use custom flows even
> with a synchronous PQconnectdb().
The way I see this API working is the asynchronous client needs at least 2
PQConnectPoll calls:
1. To be notified of what the authentication requirements are and get
parameters.
2. When it acquires the token, the callback is used to inform libpq of the
token and return PGRES_POLLING_OK.

For the synchronous client, the callback implementation would need to be
aware of the fact that synchronous implementation invokes callback
frequently and be implemented accordingly.

Bottom lime, I don't see much problem with the current proposal. Just the
way of callback to know that OAUTH token is requested and get parameters
relies on PQconnectPoll being invoked after corresponding parameters of
conn object are populated.

> > > 5. Expectations on async_auth:
> > >     a. It returns PGRES_POLLING_READING while token acquisition is
going on
> > >     b. It returns PGRES_POLLING_OK and sets conn->sasl_state->token
> > > when token acquisition succeeds.
> >
> > Yes. Though the token should probably be returned through some
> > explicit part of the callback, now that you mention it...
>
> > 6. Is the client supposed to do anything with the altsock parameter?
>
> The callback needs to set the altsock up with a select()able
> descriptor, which wakes up the client when more work is ready to be
> done. Without that, you can't handle multiple connections on a single
> thread.

Ok, thanks for clarification.

> > On a separate note:
> > The backend code currently spawns an external command for token
validation.
> > As we discussed before, an extension hook would be a more efficient
> > extensibility option.
> > We see clients make 10k+ connections using OAuth tokens per minute to
> > our service, and stating external processes would be too much overhead
> > here.
>
> +1. I'm curious, though -- what language do you expect to use to write
> a production validator hook? Surely not low-level C...?

For the server side code, it would likely be identity providers publishing
extensions to validate their tokens.
Those can do that in C too. Or extensions now can be implemented in Rust
using pgrx. Which is developer friendly enough in my opinion.

> Yeah, I'm really interested in seeing which existing high-level flows
> can be mixed in through a driver. Trying not to get too far ahead of
> myself :D

I can think of the following as the most common:
1. Authorization code with PKCE. This is by far the most common for the
user login flows. Requires to spin up a browser and listen to redirect
URL/Port. Most high level platforms have libraries to do both.
2. Client Certificates. This requires an identity provider specific library
to construct and sign the token. The providers publish SDKs to do that for
most common app development platforms.

  73. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-17T23:55:06Z

    On Tue, Jul 11, 2023 at 10:50 AM Jacob Champion
<jchampion@timescale.com> wrote:
> I have a WIP patch that passes tests on FreeBSD, which I'll clean up
> and post Sometime Soon. macOS builds now but still fails before it
> runs the test; looks like it's having trouble finding OpenSSL during
> `pip install` of the test modules...

Hi Thomas,

v9 folds in your kqueue implementation (thanks again!) and I have a
quick question to check my understanding:

> +       case CURL_POLL_REMOVE:
> +           /*
> +            * We don't know which of these is currently registered, perhaps
> +            * both, so we try to remove both.  This means we need to tolerate
> +            * ENOENT below.
> +            */
> +           EV_SET(&ev[nev], socket, EVFILT_READ, EV_DELETE, 0, 0, 0);
> +           nev++;
> +           EV_SET(&ev[nev], socket, EVFILT_WRITE, EV_DELETE, 0, 0, 0);
> +           nev++;
> +           break;

We're not setting EV_RECEIPT for these -- is that because none of the
filters we're using are EV_CLEAR, and so it doesn't matter if we
accidentally pull pending events off the queue during the kevent() call?

v9 also improves the Cirrus debugging experience and fixes more issues
on macOS, so the tests should be green there now. The final patch in the
series works around what I think is a build bug in psycopg2 2.9 [1] for
the BSDs+meson.

Thanks,
--Jacob

[1] https://github.com/psycopg/psycopg2/issues/1599

  74. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2023-07-18T23:03:53Z

    On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion <jchampion@timescale.com> wrote:
> We're not setting EV_RECEIPT for these -- is that because none of the
> filters we're using are EV_CLEAR, and so it doesn't matter if we
> accidentally pull pending events off the queue during the kevent() call?

+1 for EV_RECEIPT ("just tell me about errors, don't drain any
events").  I had a vague memory that it caused portability problems.
Just checked... it was OpenBSD I was thinking of, but they finally
added that flag in 6.2 (2017).  Our older-than-that BF OpenBSD animal
recently retired so that should be fine.  (Yes, without EV_CLEAR it's
"level triggered" not "edge triggered" in epoll terminology, so the
way I had it was not broken, but the way you're suggesting would be
nicer.)  Note that you'll have to skip data == 0 (no error) too.

+ #ifdef HAVE_SYS_EVENT_H
+ /* macOS doesn't define the time unit macros, but uses milliseconds
by default. */
+ #ifndef NOTE_MSECONDS
+ #define NOTE_MSECONDS 0
+ #endif
+ #endif

While comparing the cousin OSs' man pages just now, I noticed that
it's not only macOS that lacks NOTE_MSECONDS, it's also OpenBSD and
NetBSD < 10.  Maybe just delete that cruft ^^^ and use literal 0 in
fflags directly.  FreeBSD, and recently also NetBSD, decided to get
fancy with high resolution timers, but 0 gets the traditional unit of
milliseconds on all platforms (I just wrote it like that because I
started from FreeBSD and didn't know the history/portability story).

  75. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-07-26T16:43:14Z

    On Tue, Jul 18, 2023 at 4:04 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Tue, Jul 18, 2023 at 11:55 AM Jacob Champion <jchampion@timescale.com> wrote:
> +1 for EV_RECEIPT ("just tell me about errors, don't drain any
> events").

Sounds good.

> While comparing the cousin OSs' man pages just now, I noticed that
> it's not only macOS that lacks NOTE_MSECONDS, it's also OpenBSD and
> NetBSD < 10.  Maybe just delete that cruft ^^^ and use literal 0 in
> fflags directly.

So I don't lose track of it, here's a v10 with those two changes.

Thanks!
--Jacob

  76. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-08-30T22:57:39Z

    v11 is a quick rebase over the recent Cirrus changes, and I've dropped
0006 now that psycopg2 can build against BSD/Meson setups (thanks Daniele!).

--Jacob

  77. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jchampion@timescale.com> — 2023-09-06T22:11:23Z

    v12 implements a first draft of a client hook, so applications can
replace either the device prompt or the entire OAuth flow. (Andrey and
Mahendrakar: hopefully this is close to what you need.) It also cleans
up some of the JSON tech debt.

Since (IMO) we don't want to introduce new hooks every time we make
improvements to the internal flows, the new hook is designed to
retrieve multiple pieces of data from the application. Clients either
declare their ability to get that data, or delegate the job to the
next link in the chain, which by default is a no-op. That lets us add
new data types to the end, and older clients will ignore them until
they're taught otherwise. (I'm trying hard not to over-engineer this,
but it seems like the concept of "give me some piece of data to
continue authenticating" could pretty easily subsume things like the
PQsslKeyPassHook if we wanted.)

The PQAUTHDATA_OAUTH_BEARER_TOKEN case is the one that replaces the
flow entirely, as discussed upthread. Your application gets the
discovery URI and the requested scope for the connection. It can then
either delegate back to libpq (e.g. if the issuer isn't one it can
help with), immediately return a token (e.g. if one is already cached
for the current user), or install a nonblocking callback to implement
a custom async flow. When the connection is closed (or fails), the
hook provides a cleanup function to free any resources it may have
allocated.

Thanks,
--Jacob

  78. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Shlok Kyal <shlok.kyal.oss@gmail.com> — 2023-11-03T12:28:28Z

    Hi,

On Fri, 3 Nov 2023 at 17:14, Jacob Champion <jchampion@timescale.com> wrote:
>
> v12 implements a first draft of a client hook, so applications can
> replace either the device prompt or the entire OAuth flow. (Andrey and
> Mahendrakar: hopefully this is close to what you need.) It also cleans
> up some of the JSON tech debt.

I went through CFbot and found that build is failing, links:

https://cirrus-ci.com/task/6061898244816896
https://cirrus-ci.com/task/6624848198238208
https://cirrus-ci.com/task/5217473314684928
https://cirrus-ci.com/task/6343373221527552

Just want to make sure you are aware of these failures.

Thanks,
Shlok Kumar Kyal

  79. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <champion.p@gmail.com> — 2023-11-03T23:48:29Z

    On Fri, Nov 3, 2023 at 5:28 AM Shlok Kyal <shlok.kyal.oss@gmail.com> wrote:
> Just want to make sure you are aware of these failures.

Thanks for the nudge! Looks like I need to reconcile with the changes
to JsonLexContext in 1c99cde2. I should be able to get to that next
week; in the meantime I'll mark it Waiting on Author.

--Jacob

  80. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <champion.p@gmail.com> — 2023-11-08T19:00:18Z

    On Fri, Nov 3, 2023 at 4:48 PM Jacob Champion <champion.p@gmail.com> wrote:
> Thanks for the nudge! Looks like I need to reconcile with the changes
> to JsonLexContext in 1c99cde2. I should be able to get to that next
> week; in the meantime I'll mark it Waiting on Author.

v13 rebases over latest. The JsonLexContext changes have simplified
0001 quite a bit, and there's probably a bit more minimization that
could be done.

Unfortunately the configure/Makefile build of libpq now seems to be
pulling in an `exit()` dependency in a way that Meson does not. (Or
maybe Meson isn't checking?) I still need to investigate that
difference and fix it, so I recommend Meson if you're looking to
test-drive a build.

Thanks,
--Jacob

  81. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrey Chudnovsky <achudnovskij@gmail.com> — 2023-11-10T01:42:52Z

    Hi Jacob,

Wanted to follow up on one of the topics discussed here in the past:
Do you plan to support adding an extension hook to validate the token?

It would allow a more efficient integration, then spinning a separate
process.

Thanks!
Andrey.

On Wed, Nov 8, 2023 at 11:00 AM Jacob Champion <champion.p@gmail.com> wrote:

> On Fri, Nov 3, 2023 at 4:48 PM Jacob Champion <champion.p@gmail.com>
> wrote:
> > Thanks for the nudge! Looks like I need to reconcile with the changes
> > to JsonLexContext in 1c99cde2. I should be able to get to that next
> > week; in the meantime I'll mark it Waiting on Author.
>
> v13 rebases over latest. The JsonLexContext changes have simplified
> 0001 quite a bit, and there's probably a bit more minimization that
> could be done.
>
> Unfortunately the configure/Makefile build of libpq now seems to be
> pulling in an `exit()` dependency in a way that Meson does not. (Or
> maybe Meson isn't checking?) I still need to investigate that
> difference and fix it, so I recommend Meson if you're looking to
> test-drive a build.
>
> Thanks,
> --Jacob
>

  82. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <champion.p@gmail.com> — 2023-11-15T20:20:56Z

    On Thu, Nov 9, 2023 at 5:43 PM Andrey Chudnovsky <achudnovskij@gmail.com> wrote:
> Do you plan to support adding an extension hook to validate the token?
>
> It would allow a more efficient integration, then spinning a separate process.

I think an API in the style of archive modules might probably be a
good way to go, yeah.

It's probably not very high on the list of priorities, though, since
the inputs and outputs are going to "look" the same whether you're
inside or outside of the server process. The client side is going to
need the bulk of the work/testing/validation. Speaking of which -- how
is the current PQauthDataHook design doing when paired with MS AAD
(er, Entra now I guess)? I haven't had an Azure test bed available for
a while.

Thanks,
--Jacob

  83. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2023-12-05T09:43:48Z

    > On 8 Nov 2023, at 20:00, Jacob Champion <champion.p@gmail.com> wrote:

> Unfortunately the configure/Makefile build of libpq now seems to be
> pulling in an `exit()` dependency in a way that Meson does not.

I believe this comes from the libcurl and specifically the ntlm_wb support
which is often enabled in system and package manager provided installations.
There isn't really a fix here apart from requiring a libcurl not compiled with
ntlm_wb support, or adding an exception to the exit() check in the Makefile.

Bringing this up with other curl developers to see if it could be fixed we
instead decided to deprecate the whole module as it's quirky and not used much.
This won't help with existing installations but at least it will be deprecated
and removed by the time v17 ships, so gating on a version shipped after its
removal will avoid it.

https://github.com/curl/curl/commit/04540f69cfd4bf16e80e7c190b645f1baf505a84

> (Or maybe Meson isn't checking?) I still need to investigate that
> difference and fix it, so I recommend Meson if you're looking to
> test-drive a build.

There is no corresponding check in the Meson build, which seems like a TODO.

--
Daniel Gustafsson

  84. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <champion.p@gmail.com> — 2024-01-09T18:48:55Z

    On Tue, Dec 5, 2023 at 1:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 8 Nov 2023, at 20:00, Jacob Champion <champion.p@gmail.com> wrote:
>
> > Unfortunately the configure/Makefile build of libpq now seems to be
> > pulling in an `exit()` dependency in a way that Meson does not.
>
> I believe this comes from the libcurl and specifically the ntlm_wb support
> which is often enabled in system and package manager provided installations.
> There isn't really a fix here apart from requiring a libcurl not compiled with
> ntlm_wb support, or adding an exception to the exit() check in the Makefile.
>
> Bringing this up with other curl developers to see if it could be fixed we
> instead decided to deprecate the whole module as it's quirky and not used much.
> This won't help with existing installations but at least it will be deprecated
> and removed by the time v17 ships, so gating on a version shipped after its
> removal will avoid it.
>
> https://github.com/curl/curl/commit/04540f69cfd4bf16e80e7c190b645f1baf505a84

Ooh, thank you for looking into that and fixing it!

> > (Or maybe Meson isn't checking?) I still need to investigate that
> > difference and fix it, so I recommend Meson if you're looking to
> > test-drive a build.
>
> There is no corresponding check in the Meson build, which seems like a TODO.

Okay, I'll look into that too when I get time.

Thanks,
--Jacob

  85. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-21T01:00:28Z

    Hi all,

v14 rebases over latest and fixes a warning when assertions are
disabled. 0006 is temporary and hacks past a couple of issues I have
not yet root caused -- one of which makes me wonder if 0001 needs to
be considered alongside the recent pg_combinebackup and incremental
JSON work...?

--Jacob

  86. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-22T14:08:41Z

    On Tue, Feb 20, 2024 at 5:00 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v14 rebases over latest and fixes a warning when assertions are
> disabled.

v15 is a housekeeping update that adds typedefs.list entries and runs
pgindent. It also includes a temporary patch from Daniel to get the
cfbot a bit farther (see above discussion on libcurl/exit).

--Jacob

  87. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-24T01:01:28Z

    On Thu, Feb 22, 2024 at 6:08 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v15 is a housekeeping update that adds typedefs.list entries and runs
> pgindent.

v16 is more transformational!

Daniel contributed 0004, which completely replaces the
validator_command architecture with a C module API. This solves a
bunch of problems as discussed upthread and vastly simplifies the test
framework for the server side. 0004 also adds a set of Perl tests,
which will begin to subsume some of the Python server-side tests as I
get around to porting them. (@Daniel: 0005 is my diff against your
original patch, for review.)

0008 has been modified to quickfix the pgcommon linkage on the
Makefile side; my previous attempt at this only fixed Meson. The
patchset is now carrying a lot of squash-cruft, and I plan to flatten
it in the next version.

Thanks,
--Jacob

  88. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-27T19:33:55Z

    On Tue, Feb 27, 2024 at 11:20 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1].

It looks like my patchset has been eaten by a malware scanner:

    550 Message content failed content scanning
(Sanesecurity.Foxhole.Mail_gz.UNOFFICIAL)

Was there a recent change to the lists? Is anyone able to see what the
actual error was so I don't do it again?

Thanks,
--Jacob

  89. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-28T14:05:52Z

    [Trying again, with all patches unzipped and the CC list temporarily
removed to avoid flooding people's inboxes. Original message follows.]

On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> The
> patchset is now carrying a lot of squash-cruft, and I plan to flatten
> it in the next version.

This is done in v17, which is also now based on the two patches pulled
out by Daniel in [1]. Besides the squashes, which make up most of the
range-diff, I've fixed a call to strncasecmp() which is not available
on Windows.

Daniel and I discussed trying a Python version of the test server,
since the standard library there should give us more goodies to work
with. A proof of concept is in 0009. I think the big question I have
for it is, how would we communicate what we want the server to do for
the test? (We could perhaps switch on magic values of the client ID?)
In the end I'd like to be testing close to 100% of the failure modes,
and that's likely to mean a lot of back-and-forth if the server
implementation isn't in the Perl process.

--Jacob

[1] https://postgr.es/m/flat/F51F8777-FAF5-49F2-BC5E-8F9EB423ECE0%40yesql.se

  90. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-02-28T17:40:23Z

    > On 28 Feb 2024, at 15:05, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> [Trying again, with all patches unzipped and the CC list temporarily
> removed to avoid flooding people's inboxes. Original message follows.]
> 
> On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> The
>> patchset is now carrying a lot of squash-cruft, and I plan to flatten
>> it in the next version.
> 
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1]. Besides the squashes, which make up most of the
> range-diff, I've fixed a call to strncasecmp() which is not available
> on Windows.
> 
> Daniel and I discussed trying a Python version of the test server,
> since the standard library there should give us more goodies to work
> with. A proof of concept is in 0009. I think the big question I have
> for it is, how would we communicate what we want the server to do for
> the test? (We could perhaps switch on magic values of the client ID?)
> In the end I'd like to be testing close to 100% of the failure modes,
> and that's likely to mean a lot of back-and-forth if the server
> implementation isn't in the Perl process.

Thanks for the new version, I'm digesting the test patches but for now I have a
few smaller comments:


+#define ALLOC(size) malloc(size)
I wonder if we should use pg_malloc_extended(size, MCXT_ALLOC_NO_OOM) instead
to self document the code.  We clearly don't want feature-parity with server-
side palloc here.  I know we use malloc in similar ALLOC macros so it's not
unique in that regard, but maybe?


+#ifdef FRONTEND
+               destroyPQExpBuffer(lex->errormsg);
+#else
+               pfree(lex->errormsg->data);
+               pfree(lex->errormsg);
+#endif
Wouldn't it be nicer if we abstracted this into a destroyStrVal function to a)
avoid the ifdefs and b) make it more like the rest of the new API?  While it's
only used in two places (close to each other) it's a shame to let the
underlying API bleed through the abstraction.


+   CURLM      *curlm;      /* top-level multi handle for cURL operations */
Nitpick, but curl is not capitalized cURL anymore (for some value of "anymore"
since it changed in 2016 [0]).  I do wonder if we should consistently write
"libcurl" as well since we don't use curl but libcurl.


+   PQExpBufferData     work_data;  /* scratch buffer for general use (remember
+                                      to clear out prior contents first!) */
This seems like asking for subtle bugs due to uncleared buffers bleeding into
another operation (especially since we are writing this data across the wire).
How about having an array the size of OAuthStep of unallocated buffers where
each step use it's own?  Storing the content of each step could also be useful
for debugging.  Looking at the statemachine here it's not an obvious change but
also not impossible.


+ * TODO: This disables DNS resolution timeouts unless libcurl has been
+ * compiled against alternative resolution support. We should check that.
curl_version_info() can be used to check for c-ares support.


+ * so you don't have to write out the error handling every time. They assume
+ * that they're embedded in a function returning bool, however.
It feels a bit iffy to encode the returntype in the macro, we can use the same
trick that DISABLE_SIGPIPE employs where a failaction is passed in.


+  if (!strcmp(name, field->name))
Project style is to test for (strcmp(x,y) == 0) rather than (!strcmp()) to
improve readability.


+  libpq_append_conn_error(conn, "out of memory");
While not introduced in this patch, it's not an ideal pattern to report "out of
memory" errors via a function which may allocate memory.


+  appendPQExpBufferStr(&conn->errorMessage,
+           libpq_gettext("server's error message contained an embedded NULL"));
We should maybe add ", discarding" or something similar after this string to
indicate that there was an actual error which has been thrown away, the error
wasn't that the server passed an embedded NULL.


+#ifdef USE_OAUTH
+       else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
+               !selected_mechanism)
I wonder if we instead should move the guards inside the statement and error
out with "not built with OAuth support" or something similar like how we do
with TLS and other optional components?


+   errdetail("Comma expected, but found character %s.",
+             sanitize_char(*p))));
The %s formatter should be wrapped like '%s' to indicate that the message part
is the character in question (and we can then reuse the translation since the
error message already exist for SCRAM).


+       temp = curl_slist_append(temp, "authorization_code");
+       if (!temp)
+           oom = true;
+
+       temp = curl_slist_append(temp, "implicit");
While not a bug per se, it reads a bit odd to call another operation that can
allocate memory when the oom flag has been set.  I think we can move some
things around a little to make it clearer.

The attached diff contains some (most?) of the above as a patch on top of your
v17, but as a .txt to keep the CFBot from munging on it.

--
Daniel Gustafsson

  91. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrew Dunstan <andrew@dunslane.net> — 2024-02-28T21:50:23Z

    On 2024-02-28 We 09:05, Jacob Champion wrote:
>
> Daniel and I discussed trying a Python version of the test server,
> since the standard library there should give us more goodies to work
> with. A proof of concept is in 0009. I think the big question I have
> for it is, how would we communicate what we want the server to do for
> the test? (We could perhaps switch on magic values of the client ID?)
> In the end I'd like to be testing close to 100% of the failure modes,
> and that's likely to mean a lot of back-and-forth if the server
> implementation isn't in the Perl process.



Can you give some more details about what this python gadget would buy 
us? I note that there are a couple of CPAN modules that provide OAuth2 
servers, not sure if they would be of any use.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  92. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-02-28T21:52:40Z

    > On 28 Feb 2024, at 22:50, Andrew Dunstan <andrew@dunslane.net> wrote:
> 
> On 2024-02-28 We 09:05, Jacob Champion wrote:
>> 
>> Daniel and I discussed trying a Python version of the test server,
>> since the standard library there should give us more goodies to work
>> with. A proof of concept is in 0009. I think the big question I have
>> for it is, how would we communicate what we want the server to do for
>> the test? (We could perhaps switch on magic values of the client ID?)
>> In the end I'd like to be testing close to 100% of the failure modes,
>> and that's likely to mean a lot of back-and-forth if the server
>> implementation isn't in the Perl process.
> 
> Can you give some more details about what this python gadget would buy us? I note that there are a couple of CPAN modules that provide OAuth2 servers, not sure if they would be of any use.

The main benefit would be to be able to provide a full testharness without
adding any additional dependencies over what we already have (Python being
required by meson).  That should ideally make it easy to get good coverage from
BF animals as no installation is needed.

--
Daniel Gustafsson

  93. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-02-29T14:49:02Z

    [re-adding the CC list I dropped earlier]

On Wed, Feb 28, 2024 at 1:52 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 28 Feb 2024, at 22:50, Andrew Dunstan <andrew@dunslane.net> wrote:
> > Can you give some more details about what this python gadget would buy us? I note that there are a couple of CPAN modules that provide OAuth2 servers, not sure if they would be of any use.
>
> The main benefit would be to be able to provide a full testharness without
> adding any additional dependencies over what we already have (Python being
> required by meson).  That should ideally make it easy to get good coverage from
> BF animals as no installation is needed.

As an additional note, the test suite ideally needs to be able to
exercise failure modes where the provider itself is malfunctioning. So
we hand-roll responses rather than deferring to an external
OAuth/OpenID implementation, which adds HTTP and JSON dependencies at
minimum, and Python includes both. See also the discussion with
Stephen upthread [1].

(I do think it'd be nice to eventually include a prepackaged OAuth
server in the test suite, to stack coverage for the happy path and
further test interoperability.)

Thanks,
--Jacob

[1] https://postgr.es/m/CAAWbhmh%2B6q4t3P%2BwDmS%3DJuHBpcgF-VM2cXNft8XV02yk-cHCpQ%40mail.gmail.com

  94. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-02-29T21:08:44Z

    > On 27 Feb 2024, at 20:20, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Fri, Feb 23, 2024 at 5:01 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> The
>> patchset is now carrying a lot of squash-cruft, and I plan to flatten
>> it in the next version.
> 
> This is done in v17, which is also now based on the two patches pulled
> out by Daniel in [1]. Besides the squashes, which make up most of the
> range-diff, I've fixed a call to strncasecmp() which is not available
> on Windows.

Two quick questions:

+   /* TODO */
+   CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
I might be missing something, but what this is intended for in
setup_curl_handles()?


--- /dev/null
+++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
As discussed off-list I think we should leave iddawc support for later and
focus on getting one library properly supported to start with.  If you agree,
let's drop this from the patchset to make it easier to digest.  We should make
sure we keep pluggability such that another library can be supported though,
much like the libpq TLS support.

--
Daniel Gustafsson

  95. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-01T00:04:05Z

    On Wed, Feb 28, 2024 at 9:40 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> +#define ALLOC(size) malloc(size)
> I wonder if we should use pg_malloc_extended(size, MCXT_ALLOC_NO_OOM) instead
> to self document the code.  We clearly don't want feature-parity with server-
> side palloc here.  I know we use malloc in similar ALLOC macros so it's not
> unique in that regard, but maybe?

I have a vague recollection that linking fe_memutils into libpq
tripped the exit() checks, but I can try again and see.

> +#ifdef FRONTEND
> +               destroyPQExpBuffer(lex->errormsg);
> +#else
> +               pfree(lex->errormsg->data);
> +               pfree(lex->errormsg);
> +#endif
> Wouldn't it be nicer if we abstracted this into a destroyStrVal function to a)
> avoid the ifdefs and b) make it more like the rest of the new API?  While it's
> only used in two places (close to each other) it's a shame to let the
> underlying API bleed through the abstraction.

Good idea. I'll fold this from your patch into the next set (and do
the same for the ones I've marked +1 below).

> +   CURLM      *curlm;      /* top-level multi handle for cURL operations */
> Nitpick, but curl is not capitalized cURL anymore (for some value of "anymore"
> since it changed in 2016 [0]).  I do wonder if we should consistently write
> "libcurl" as well since we don't use curl but libcurl.

Huh, I missed that memo. Thanks -- that makes it much easier to type!

> +   PQExpBufferData     work_data;  /* scratch buffer for general use (remember
> +                                      to clear out prior contents first!) */
> This seems like asking for subtle bugs due to uncleared buffers bleeding into
> another operation (especially since we are writing this data across the wire).
> How about having an array the size of OAuthStep of unallocated buffers where
> each step use it's own?  Storing the content of each step could also be useful
> for debugging.  Looking at the statemachine here it's not an obvious change but
> also not impossible.

I like that idea; I'll give it a look.

> + * TODO: This disables DNS resolution timeouts unless libcurl has been
> + * compiled against alternative resolution support. We should check that.
> curl_version_info() can be used to check for c-ares support.

+1

> + * so you don't have to write out the error handling every time. They assume
> + * that they're embedded in a function returning bool, however.
> It feels a bit iffy to encode the returntype in the macro, we can use the same
> trick that DISABLE_SIGPIPE employs where a failaction is passed in.

+1

> +  if (!strcmp(name, field->name))
> Project style is to test for (strcmp(x,y) == 0) rather than (!strcmp()) to
> improve readability.

+1

> +  libpq_append_conn_error(conn, "out of memory");
> While not introduced in this patch, it's not an ideal pattern to report "out of
> memory" errors via a function which may allocate memory.

Does trying (and failing) to allocate more memory cause any harm? Best
case, we still have enough room in the errorMessage to fit the whole
error; worst case, we mark the errorMessage broken and then
PQerrorMessage() can handle it correctly.

> +  appendPQExpBufferStr(&conn->errorMessage,
> +           libpq_gettext("server's error message contained an embedded NULL"));
> We should maybe add ", discarding" or something similar after this string to
> indicate that there was an actual error which has been thrown away, the error
> wasn't that the server passed an embedded NULL.

+1

> +#ifdef USE_OAUTH
> +       else if (strcmp(mechanism_buf.data, OAUTHBEARER_NAME) == 0 &&
> +               !selected_mechanism)
> I wonder if we instead should move the guards inside the statement and error
> out with "not built with OAuth support" or something similar like how we do
> with TLS and other optional components?

This one seems like a step backwards. IIUC, the client can currently
handle a situation where the server returns multiple mechanisms
(though the server doesn't support that yet), and I'd really like to
make use of that property without making users upgrade libpq.

That said, it'd be good to have a more specific error message in the
case where we don't have a match...

> +   errdetail("Comma expected, but found character %s.",
> +             sanitize_char(*p))));
> The %s formatter should be wrapped like '%s' to indicate that the message part
> is the character in question (and we can then reuse the translation since the
> error message already exist for SCRAM).

+1

> +       temp = curl_slist_append(temp, "authorization_code");
> +       if (!temp)
> +           oom = true;
> +
> +       temp = curl_slist_append(temp, "implicit");
> While not a bug per se, it reads a bit odd to call another operation that can
> allocate memory when the oom flag has been set.  I think we can move some
> things around a little to make it clearer.

I'm not a huge fan of nested happy paths/pyramids of doom, but in this
case it's small enough that I'm not opposed. :D

> The attached diff contains some (most?) of the above as a patch on top of your
> v17, but as a .txt to keep the CFBot from munging on it.

Thanks very much! I plan to apply all but the USE_OAUTH guard change
(but let me know if you feel strongly about it).

--Jacob

  96. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-01T00:11:49Z

    On Thu, Feb 29, 2024 at 1:08 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> +   /* TODO */
> +   CHECK_SETOPT(actx, CURLOPT_WRITEDATA, stderr);
> I might be missing something, but what this is intended for in
> setup_curl_handles()?

Ah, that's cruft left over from early debugging, just so that I could
see what was going on. I'll remove it.

> --- /dev/null
> +++ b/src/interfaces/libpq/fe-auth-oauth-iddawc.c
> As discussed off-list I think we should leave iddawc support for later and
> focus on getting one library properly supported to start with.  If you agree,
> let's drop this from the patchset to make it easier to digest.  We should make
> sure we keep pluggability such that another library can be supported though,
> much like the libpq TLS support.

Agreed. The number of changes being folded into the next set is
already pretty big so I think this will wait until next+1.

--Jacob

  97. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-01T01:08:01Z

    On Thu, Feb 29, 2024 at 4:04 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Wed, Feb 28, 2024 at 9:40 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > +       temp = curl_slist_append(temp, "authorization_code");
> > +       if (!temp)
> > +           oom = true;
> > +
> > +       temp = curl_slist_append(temp, "implicit");
> > While not a bug per se, it reads a bit odd to call another operation that can
> > allocate memory when the oom flag has been set.  I think we can move some
> > things around a little to make it clearer.
>
> I'm not a huge fan of nested happy paths/pyramids of doom, but in this
> case it's small enough that I'm not opposed. :D

I ended up rewriting this patch hunk a bit to handle earlier OOM
failures; let me know what you think.

--

v18 is the result of plenty of yak shaving now that the Windows build
is working. In addition to Daniel's changes as discussed upthread,
- I have rebased over v2 of the SASL-refactoring patches
- the last CompilerWarnings failure has been fixed
- the py.test suite now runs on Windows (but does not yet completely pass)
- py.test has been completely disabled for the 32-bit Debian test in
Cirrus; I don't know if there's a way to install 32-bit Python
side-by-side with 64-bit

We are now very, very close to green.

The new oauth_validator tests can't work on Windows, since the client
doesn't support OAuth there. The python/server tests can handle this
case, since they emulate the client behavior; do we want to try
something similar in Perl?

--Jacob

  98. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-01T17:46:27Z

    On Thu, Feb 29, 2024 at 5:08 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> We are now very, very close to green.

v19 gets us a bit closer by adding a missed import for Windows. I've
also removed iddawc support, so the client patch is lighter.

> The new oauth_validator tests can't work on Windows, since the client
> doesn't support OAuth there. The python/server tests can handle this
> case, since they emulate the client behavior; do we want to try
> something similar in Perl?

In addition to this question, I'm starting to notice intermittent
failures of the form

    error: ... failed to fetch OpenID discovery document: failed to
queue HTTP request

This corresponds to a TODO in the libcurl implementation -- if the
initial call to curl_multi_socket_action() reports that no handles are
running, I treated that as an error. But it looks like it's possible
for libcurl to finish a request synchronously if the remote responds
quickly enough, so that needs to change.

--Jacob

  99. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-11T22:51:24Z

    On Fri, Mar 1, 2024 at 9:46 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> v19 gets us a bit closer by adding a missed import for Windows. I've
> also removed iddawc support, so the client patch is lighter.

v20 fixes a bunch more TODOs:
1) the client initial response is validated more closely
2) the server's invalid_token parameters are properly escaped into the
containing JSON (though, eventually, we probably want to just reject
invalid HBA settings instead of passing them through to the client)
3) Windows-specific responses have been recorded in the test suite

While poking at item 2, I was reminded that there's an alternative way
to get OAuth parameters from the server, and it's subtly incompatible
with the OpenID spec because OpenID didn't follow the rules for
.well-known URI construction [1]. :( Some sort of knob will be
required to switch the behaviors.

I renamed the API for the validator module from res->authenticated to
res->authorized. Authentication is optional, but a validator *must*
check that the client it's talking to was authorized by the user to
access the server, whether or not the user is authenticated. (It may
additionally verify that the user is authorized to access the
database, or it may simply authenticate the user and defer to the
usermap.) Documenting that particular subtlety is going to be
interesting...

The tests now exercise different issuers for different users, which
will also be a good way to signal the server to respond in different
ways during the validator tests. It does raise the question: if a
third party provides an issuer-specific module, how do we switch
between that and some other module for a different user?

Andrew asked over at [2] if we could perhaps get 0001 in as well. I
think the main thing to figure out there is, is requiring linkage
against libpq (see 0008) going to be okay for the frontend binaries
that need JSON support? Or do we need to do something like moving
PQExpBuffer into src/common to simplify the dependency tree?

--Jacob

[1] https://www.rfc-editor.org/rfc/rfc8414.html#section-5
[2] https://www.postgresql.org/message-id/682c8fff-355c-a04f-57ac-81055c4ccda8%40dunslane.net

  100. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-03-22T18:21:19Z

    v21 is a quick rebase over HEAD, which has adopted a few pieces of
v20. I've also fixed a race condition in the tests.

On Mon, Mar 11, 2024 at 3:51 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Andrew asked over at [2] if we could perhaps get 0001 in as well. I
> think the main thing to figure out there is, is requiring linkage
> against libpq (see 0008) going to be okay for the frontend binaries
> that need JSON support? Or do we need to do something like moving
> PQExpBuffer into src/common to simplify the dependency tree?

0001 has been pared down to the part that teaches jsonapi.c to use
PQExpBuffer and track out-of-memory conditions; the linkage questions
remain.

Thanks,
--Jacob

  101. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-03-28T22:34:02Z

    > On 22 Mar 2024, at 19:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> v21 is a quick rebase over HEAD, which has adopted a few pieces of
> v20. I've also fixed a race condition in the tests.

Thanks for the rebase, I have a few comments from working with it a bit:

In jsonapi.c, makeJsonLexContextCstringLen initialize a JsonLexContext with
palloc0 which would need to be ported over to use ALLOC for frontend code.  On
that note, the errorhandling in parse_oauth_json() for content-type checks
attempts to free the JsonLexContext even before it has been created.  Here we
can just return false.


-  echo 'libpq must not be calling any function which invokes exit'; exit 1; \
+  echo 'libpq must not be calling any function which invokes exit'; \
The offending codepath in libcurl was in the NTLM_WB module, a very old and
obscure form of NTLM support which was replaced (yet remained in the tree) a
long time ago by a full NTLM implementatin.  Based on the findings in this
thread it was deprecated with a removal date set to April 2024 [0].  A bug in
the 8.4.0 release however disconnected NTLM_WB from the build and given the
lack of complaints it was decided to leave as is, so we can base our libcurl
requirements on 8.4.0 while keeping the exit() check intact.


+  else if (strcasecmp(content_type, "application/json") != 0)
This needs to handle parameters as well since it will now fail if the charset
parameter is appended (which undoubtedly will be pretty common).  The easiest
way is probably to just verify the mediatype and skip the parameters since we
know it can only be charset?


+  /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
+  CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
+  CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
CURLOPT_ERRORBUFFER is the old and finicky way of extracting error messages, we
should absolutely move to using CURLOPT_DEBUGFUNCTION instead.


+  /* && response_code != 401 TODO */ )
Why is this marked with a TODO, do you remember?


+  print("# OAuth provider (PID $pid) is listening on port $port\n");
Code running under Test::More need to use diag() for printing non-test output
like this.


Another issue I have is the sheer size and the fact that so much code is
replaced by subsequent commits, so I took the liberty to squash some of this
down into something less daunting.  The attached v22 retains the 0001 and then
condenses the rest into two commits for frontent and backend parts.  I did drop
the Python pytest patch since I feel that it's unlikely to go in from this
thread (adding pytest seems worthy of its own thread and discussion), and the
weight of it makes this seem scarier than it is.  For using it, it can be
easily applied from the v21 patchset independently.  I did tweak the commit
message to match reality a bit better, but there is a lot of work left there.

The final patch contains fixes for all of the above review comments as well as
a some refactoring, smaller clean-ups and TODO fixing.  If these fixes are
accepted I'll incorporate them into the two commits.

Next I intend to work on writing documentation for this.

--
Daniel Gustafsson

[0] https://curl.se/dev/deprecate.html
[1] https://github.com/curl/curl/pull/12479

  102. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-04-01T22:07:45Z

    On Thu, Mar 28, 2024 at 3:34 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> In jsonapi.c, makeJsonLexContextCstringLen initialize a JsonLexContext with
> palloc0 which would need to be ported over to use ALLOC for frontend code.

Seems reasonable (but see below, too).

> On
> that note, the errorhandling in parse_oauth_json() for content-type checks
> attempts to free the JsonLexContext even before it has been created.  Here we
> can just return false.

Agreed. They're zero-initialized, so freeJsonLexContext() is safe
IIUC, but it's clearer not to call the free function at all.

But for these additions:

> -   makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true);
> +   if (!makeJsonLexContextCstringLen(&lex, resp->data, resp->len, PG_UTF8, true))
> +   {
> +       actx_error(actx, "out of memory");
> +       return false;
> +   }

...since we're using the stack-based API as opposed to the heap-based
API, they shouldn't be possible to hit. Any failures in createStrVal()
are deferred to parse time on purpose.

> -  echo 'libpq must not be calling any function which invokes exit'; exit 1; \
> +  echo 'libpq must not be calling any function which invokes exit'; \
> The offending codepath in libcurl was in the NTLM_WB module, a very old and
> obscure form of NTLM support which was replaced (yet remained in the tree) a
> long time ago by a full NTLM implementatin.  Based on the findings in this
> thread it was deprecated with a removal date set to April 2024 [0].  A bug in
> the 8.4.0 release however disconnected NTLM_WB from the build and given the
> lack of complaints it was decided to leave as is, so we can base our libcurl
> requirements on 8.4.0 while keeping the exit() check intact.

Of the Cirrus machines, it looks like only FreeBSD has a new enough
libcurl for that. Ubuntu won't until 24.04, Debian Bookworm doesn't
have it unless you're running backports, RHEL 9 is still on 7.x... I
think requiring libcurl 8 is effectively saying no one will be able to
use this for a long time. Is there an alternative?

> +  else if (strcasecmp(content_type, "application/json") != 0)
> This needs to handle parameters as well since it will now fail if the charset
> parameter is appended (which undoubtedly will be pretty common).  The easiest
> way is probably to just verify the mediatype and skip the parameters since we
> know it can only be charset?

Good catch. application/json no longer defines charsets officially
[1], so I think we should be able to just ignore them. The new
strncasecmp needs to handle a spurious prefix, too; I have that on my
TODO list.

> +  /* TODO investigate using conn->Pfdebug and CURLOPT_DEBUGFUNCTION here */
> +  CHECK_SETOPT(actx, CURLOPT_VERBOSE, 1L, return false);
> +  CHECK_SETOPT(actx, CURLOPT_ERRORBUFFER, actx->curl_err, return false);
> CURLOPT_ERRORBUFFER is the old and finicky way of extracting error messages, we
> should absolutely move to using CURLOPT_DEBUGFUNCTION instead.

This new way doesn't do the same thing. Here's a sample error:

    connection to server at "127.0.0.1", port 56619 failed: failed to
fetch OpenID discovery document: Weird server reply (  Trying
127.0.0.1:36647...
    Connected to localhost (127.0.0.1) port 36647 (#0)
    Mark bundle as not supporting multiuse
    HTTP 1.0, assume close after body
    Invalid Content-Length: value
    Closing connection 0
    )

IMO that's too much noise. Prior to the change, the same error would have been

    connection to server at "127.0.0.1", port 56619 failed: failed to
fetch OpenID discovery document: Weird server reply (Invalid
Content-Length: value)

The error buffer is finicky for sure, but it's also a generic one-line
explanation of what went wrong... Is there an alternative API for that
I'm missing?

> +  /* && response_code != 401 TODO */ )
> Why is this marked with a TODO, do you remember?

Yeah -- I have a feeling that 401s coming back are going to need more
helpful hints to the user, since it implies that libpq itself hasn't
authenticated correctly as opposed to some user-related auth failure.
I was hoping to find some sample behaviors in the wild and record
those into the suite.

> +  print("# OAuth provider (PID $pid) is listening on port $port\n");
> Code running under Test::More need to use diag() for printing non-test output
> like this.

Ah, thanks.

> +#if LIBCURL_VERSION_MAJOR <= 8 && LIBCURL_VERSION_MINOR < 4

I don't think this catches versions like 7.76, does it? Maybe
`LIBCURL_VERSION_MAJOR < 8 || (LIBCURL_VERSION_MAJOR == 8 &&
LIBCURL_VERSION_MINOR < 4)`, or else `LIBCURL_VERSION_NUM < 0x080400`?

>     my $pid = open(my $read_fh, "-|", $ENV{PYTHON}, "t/oauth_server.py")
> -       // die "failed to start OAuth server: $!";
> +       or die "failed to start OAuth server: $!";
>
> -   read($read_fh, $port, 7) // die "failed to read port number: $!";
> +   read($read_fh, $port, 7) or die "failed to read port number: $!";

The first hunk here looks good (thanks for the catch!) but I think the
second is not correct behavior. $! doesn't get set unless undef is
returned, if I'm reading the docs correctly. Yay Perl.

> +   /* Sanity check the previous operation */
> +   if (actx->running != 1)
> +   {
> +       actx_error(actx, "failed to queue HTTP request");
> +       return false;
> +   }

`running` can be set to zero on success, too. I'm having trouble
forcing that code path in a test so far, but we're going to have to do
something special in that case.

> Another issue I have is the sheer size and the fact that so much code is
> replaced by subsequent commits, so I took the liberty to squash some of this
> down into something less daunting.  The attached v22 retains the 0001 and then
> condenses the rest into two commits for frontent and backend parts.

Looks good.

> I did drop
> the Python pytest patch since I feel that it's unlikely to go in from this
> thread (adding pytest seems worthy of its own thread and discussion), and the
> weight of it makes this seem scarier than it is.

Until its coverage gets ported over, can we keep it as a `DO NOT
MERGE` patch? Otherwise there's not much to run in Cirrus.

> The final patch contains fixes for all of the above review comments as well as
> a some refactoring, smaller clean-ups and TODO fixing.  If these fixes are
> accepted I'll incorporate them into the two commits.
>
> Next I intend to work on writing documentation for this.

Awesome, thank you! I will start adding coverage to the new code paths.

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc7159#section-11

  103. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-07-03T17:02:01Z

    On Mon, Apr 1, 2024 at 3:07 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>
> Awesome, thank you! I will start adding coverage to the new code paths.

This patchset rotted more than I thought it would with the new
incremental JSON, and I got stuck in rebase hell. Rather than chip
away at that while the cfbot is red, here's a rebase of v22 to get the
CI up again, and I will port what I've been working on over that. (So,
for prior reviewers: recent upthread and offline feedback is not yet
incorporated, sorry, come back later.)

The big change in v23 is that I've removed fe_memutils.c from
libpgcommon_shlib completely, to try to reduce my own hair-pulling
when it comes to keeping exit() out of libpq. (It snuck in several
ways with incremental JSON.)

As far as I can tell, removing fe_memutils causes only one problem,
which is that Informix ECPG is relying on pnstrdup(). And I think that
may be a bug in itself? There's code in deccvasc() right after the
pnstrdup() call that takes care of a failed allocation, but the
frontend pnstrdup() is going to call exit() on failure. So my 0001
patch reverts that change, which was made in 0b9466fce. If that can go
in, and I'm not missing something that makes that call okay, maybe
0002 can be peeled off as well.

Thanks,
--Jacob

  104. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-07-10T00:05:18Z

    Hi Daniel,

On Mon, Apr 1, 2024 at 3:07 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Of the Cirrus machines, it looks like only FreeBSD has a new enough
> libcurl for that. Ubuntu won't until 24.04, Debian Bookworm doesn't
> have it unless you're running backports, RHEL 9 is still on 7.x... I
> think requiring libcurl 8 is effectively saying no one will be able to
> use this for a long time. Is there an alternative?

Since the exit() checks appear to be happy now that fe_memutils is
out, I've lowered the requirement to the version of libcurl that seems
to be shipped in RHEL 8 (7.61.0). This also happens to be when TLS 1.3
ciphersuite control was added to Curl, which seems like something we
may want in the very near future, so I'm taking that as a good sign
for what is otherwise a very arbitrary cutoff point. Counterproposals
welcome :D

> Good catch. application/json no longer defines charsets officially
> [1], so I think we should be able to just ignore them. The new
> strncasecmp needs to handle a spurious prefix, too; I have that on my
> TODO list.

I've expanded this handling in v24, attached.

> This new way doesn't do the same thing. Here's a sample error:
>
>     connection to server at "127.0.0.1", port 56619 failed: failed to
> fetch OpenID discovery document: Weird server reply (  Trying
> 127.0.0.1:36647...
>     Connected to localhost (127.0.0.1) port 36647 (#0)
>     Mark bundle as not supporting multiuse
>     HTTP 1.0, assume close after body
>     Invalid Content-Length: value
>     Closing connection 0
>     )
>
> IMO that's too much noise. Prior to the change, the same error would have been
>
>     connection to server at "127.0.0.1", port 56619 failed: failed to
> fetch OpenID discovery document: Weird server reply (Invalid
> Content-Length: value)

I have reverted this change for now, but I'm still hoping there's an
alternative that can help us clean up?

> `running` can be set to zero on success, too. I'm having trouble
> forcing that code path in a test so far, but we're going to have to do
> something special in that case.

For whatever reason, the magic timing for this is popping up more and
more often on Cirrus, leading to really annoying test failures. So I
may have to abandon the search for a perfect test case and just fix
it.

> > I did drop
> > the Python pytest patch since I feel that it's unlikely to go in from this
> > thread (adding pytest seems worthy of its own thread and discussion), and the
> > weight of it makes this seem scarier than it is.
>
> Until its coverage gets ported over, can we keep it as a `DO NOT
> MERGE` patch? Otherwise there's not much to run in Cirrus.

I have added this back (marked loudly as don't-merge) so that we keep
the test coverage for now. The Perl suite (plus Python server) has
been tricked out a lot more in v24, so it shouldn't be too bad to get
things ported.

> > Next I intend to work on writing documentation for this.
>
> Awesome, thank you! I will start adding coverage to the new code paths.

Peter E asked for some documentation stubs to ease review, which I've
added. Hopefully that doesn't step on your toes any.

A large portion of your "Review comments" patch has been pulled
backwards into the previous commits; the remaining pieces are things
I'm still peering at and/or writing tests for. I also owe this thread
an updated roadmap and summary, to make it a little less daunting for
new reviewers. Soon (tm).

Thanks!
--Jacob

  105. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-07-29T12:01:58Z

    I have some comments about the first three patches, that deal with 
memory management.

v24-0001-Revert-ECPG-s-use-of-pnstrdup.patch

This looks right.

I suppose another approach would be to put a full replacement for 
strndup() into src/port/.  But since there is currently only one user, 
and most other users should be using pnstrdup(), the presented approach 
seems ok.

We should take the check for exit() calls from libpq and expand it to 
cover the other libraries as well.  Maybe there are other problems like 
this?


v24-0002-Remove-fe_memutils-from-libpgcommon_shlib.patch

I don't quite understand how this problem can arise.  The description says

"""
libpq appears to have no need for this, and the exit() references cause
our libpq-refs-stamp test to fail if the linker doesn't strip out the
unused code.
"""

But under what circumstances does "the linker doesn't strip out" happen? 
  If this happens accidentally, then we should have seen some buildfarm 
failures or something?

Also, one could look further and notice that restricted_token.c and 
sprompt.c both a) are not needed by libpq and b) can trigger exit() 
calls.  Then it's not clear why those are not affected.


v24-0003-common-jsonapi-support-libpq-as-a-client.patch

I'm reminded of thread [0].  I think there is quite a bit of confusion 
about the pqexpbuffer vs. stringinfo APIs, and they are probably used 
incorrectly quite a bit.  There are now also programs that use both of 
them!  This patch now introduces another layer on top of them.  I fear, 
at the end, nobody is going to understand any of this anymore.  Also, 
changing all the programs to link in libpq for pqexpbuffer seems like 
the opposite direction from what was suggested in [0].

I think we need to do some deeper thinking here about how we want the 
memory management on the client side to work.  Maybe we could just use 
one API but have some flags or callbacks to control the out-of-memory 
behavior.

[0]: 
https://www.postgresql.org/message-id/flat/16d0beac-a141-e5d3-60e9-323da75f49bf%40eisentraut.org

  106. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-07-29T20:51:20Z

    Thanks for working on this patchset, I'm looking over 0004 and 0005 but came
across a thing I wanted to bring up one thing sooner than waiting for the
review. In parse_device_authz we have this:

  {"user_code", JSON_TOKEN_STRING, {&authz->user_code}, REQUIRED},
  {"verification_uri", JSON_TOKEN_STRING, {&authz->verification_uri}, REQUIRED},

  /*
   * The following fields are technically REQUIRED, but we don't use
   * them anywhere yet:
   *
   * - expires_in
   */

  {"interval", JSON_TOKEN_NUMBER, {&authz->interval_str}, OPTIONAL},

Together with a colleage we found the Azure provider use "verification_url"
rather than xxx_uri.  Another discrepancy is that it uses a string for the
interval (ie: "interval":"5").  One can of course argue that Azure is wrong and
should feel bad, but I fear that virtually all (major) providers will have
differences like this, so we will have to deal with it in an extensible fashion
(compile time, not runtime configurable).

I was toying with making the name json_field name member an array, to allow
variations.  That won't help with the fieldtype differences though, so another
train of thought was to have some form of REQUIRED_XOR where fields can tied
together.  What do you think about something along these lines?

Another thing, shouldn't we really parse and interpret *all* REQUIRED fields
even if we don't use them to ensure that the JSON is wellformed?  If the JSON
we get is malformed in any way it seems like the safe/conservative option to
error out.

--
Daniel Gustafsson

  107. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-07-29T22:30:21Z

    On Mon, Jul 29, 2024 at 5:02 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> We should take the check for exit() calls from libpq and expand it to
> cover the other libraries as well.  Maybe there are other problems like
> this?

Seems reasonable, yeah.

> But under what circumstances does "the linker doesn't strip out" happen?
>   If this happens accidentally, then we should have seen some buildfarm
> failures or something?

On my machine, for example, I see differences with optimization
levels. Say you inadvertently call pfree() in a _shlib build, as I did
multiple times upthread. By itself, that shouldn't actually be a
problem (it eventually redirects to free()), so it should be legal to
call pfree(), and with -O2 the build succeeds. But with -Og, the
exit() check trips, and when I disassemble I see that pg_malloc() et
all have infected the shared object. After all, we did tell the linker
to put that object file in, and we don't ask it to garbage-collect
sections.

> Also, one could look further and notice that restricted_token.c and
> sprompt.c both a) are not needed by libpq and b) can trigger exit()
> calls.  Then it's not clear why those are not affected.

I think it's easier for the linker to omit whole object files rather
than partial ones. If libpq doesn't use any of those APIs there's not
really a reason to trip over it.

(Maybe the _shlib variants should just contain the minimum objects
required to compile.)

> I'm reminded of thread [0].  I think there is quite a bit of confusion
> about the pqexpbuffer vs. stringinfo APIs, and they are probably used
> incorrectly quite a bit.  There are now also programs that use both of
> them!  This patch now introduces another layer on top of them.  I fear,
> at the end, nobody is going to understand any of this anymore.

"anymore"? :)

In all seriousness -- I agree that this isn't sustainable. At the
moment the worst pain (the new layer) is isolated to jsonapi.c, which
seems like an okay place to try something new, since there aren't that
many clients. But to be honest I'm not excited about deciding the Best
Way Forward based on a sample size of JSON.

> Also,
> changing all the programs to link in libpq for pqexpbuffer seems like
> the opposite direction from what was suggested in [0].

(I don't really want to keep that new libpq dependency. We'd just have
to decide where PQExpBuffer is going to go if we're not okay with it.)

> I think we need to do some deeper thinking here about how we want the
> memory management on the client side to work.  Maybe we could just use
> one API but have some flags or callbacks to control the out-of-memory
> behavior.

Any src/common code that needs to handle both in-band and out-of-band
failure modes will still have to decide whether it's going to 1)
duplicate code paths or 2) just act as if in-band failures can always
happen. I think that's probably essential complexity; an ideal API
might make it nicer to deal with but it can't abstract it away.

Thanks!
--Jacob

  108. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-07-29T23:15:33Z

    On Mon, Jul 29, 2024 at 1:51 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Together with a colleage we found the Azure provider use "verification_url"
> rather than xxx_uri.

Yeah, I think that's originally a Google-ism. (As far as I can tell
they helped author the spec for this and then didn't follow it. :/ ) I
didn't recall Azure having used it back when I was testing against it,
though, so that's good to know.

> Another discrepancy is that it uses a string for the
> interval (ie: "interval":"5").

Oh, that's a new one. I don't remember needing to hack around that
either; maybe iddawc handled it silently?

> One can of course argue that Azure is wrong and
> should feel bad, but I fear that virtually all (major) providers will have
> differences like this, so we will have to deal with it in an extensible fashion
> (compile time, not runtime configurable).

Such is life... verification_url we will just have to deal with by
default, I think, since Google does/did it too. Not sure about
interval -- but do we want to make our distribution maintainers deal
with a compile-time setting for libpq, just to support various OAuth
flavors? To me it seems like we should just hold our noses and support
known (large) departures in the core.

> I was toying with making the name json_field name member an array, to allow
> variations.  That won't help with the fieldtype differences though, so another
> train of thought was to have some form of REQUIRED_XOR where fields can tied
> together.  What do you think about something along these lines?

If I designed it right, just adding alternative spellings directly to
the fields list should work. (The "required" check is by struct
member, not name, so both spellings can point to the same
destination.) The alternative typing on the other hand might require
something like a new sentinel "type" that will accept both... I hadn't
expected that.

> Another thing, shouldn't we really parse and interpret *all* REQUIRED fields
> even if we don't use them to ensure that the JSON is wellformed?  If the JSON
> we get is malformed in any way it seems like the safe/conservative option to
> error out.

Good, I was hoping to have a conversation about that. I am fine with
either option in principle. In practice I expect to add code to use
`expires_in` (so that we can pass it to custom OAuth hook
implementations) and `scope` (to check if the server has changed it on
us).

That leaves the provider... Forcing the provider itself to implement
unused stuff in order to interoperate seems like it could backfire on
us, especially since IETF standardized an alternate .well-known URI
[1] that changes some of these REQUIRED things into OPTIONAL. (One way
for us to interpret this: those fields may be required for OpenID, but
your OAuth provider might not be an OpenID provider, and our code
doesn't require OpenID.) I think we should probably tread lightly in
that particular case. Thoughts on that?

Thanks!
--Jacob

[1] https://www.rfc-editor.org/rfc/rfc8414.html

  109. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-02T17:13:40Z

    On 30.07.24 00:30, Jacob Champion wrote:
>> But under what circumstances does "the linker doesn't strip out" happen?
>>    If this happens accidentally, then we should have seen some buildfarm
>> failures or something?
> On my machine, for example, I see differences with optimization
> levels. Say you inadvertently call pfree() in a _shlib build, as I did
> multiple times upthread. By itself, that shouldn't actually be a
> problem (it eventually redirects to free()), so it should be legal to
> call pfree(), and with -O2 the build succeeds. But with -Og, the
> exit() check trips, and when I disassemble I see that pg_malloc() et
> all have infected the shared object. After all, we did tell the linker
> to put that object file in, and we don't ask it to garbage-collect
> sections.

I'm tempted to say, this is working as intended.

libpgcommon is built as a static library.  So we can put all the object 
files in the library, and its users only use the object files they 
really need.  So this garbage collection you allude to actually does 
happen, on an object-file level.

You shouldn't use pfree() interchangeably with free(), even if that is 
not enforced because it's the same thing underneath.  First, it just 
makes sense to keep the alloc and free pairs matched up.  And second, on 
Windows there is some additional restriction (vague knowledge) that the 
allocate and free functions must be in the same library, so mixing them 
freely might not even work.

  110. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-08-02T17:51:59Z

    On Fri, Aug 2, 2024 at 10:13 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> You shouldn't use pfree() interchangeably with free(), even if that is
> not enforced because it's the same thing underneath.  First, it just
> makes sense to keep the alloc and free pairs matched up.  And second, on
> Windows there is some additional restriction (vague knowledge) that the
> allocate and free functions must be in the same library, so mixing them
> freely might not even work.

Ah, I forgot about the CRT problems on Windows. So my statement of
"the linker might not garbage collect" is pretty much irrelevant.

But it sounds like we agree that we shouldn't be using fe_memutils at
all in shlib builds. (If you can't use palloc -- it calls exit -- then
you can't use pfree either.) Is 0002 still worth pursuing, once I've
correctly wordsmithed the commit? Or did I misunderstand your point?

Thanks!
--Jacob

  111. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-02T18:48:40Z

    On 02.08.24 19:51, Jacob Champion wrote:
> But it sounds like we agree that we shouldn't be using fe_memutils at
> all in shlib builds. (If you can't use palloc -- it calls exit -- then
> you can't use pfree either.) Is 0002 still worth pursuing, once I've
> correctly wordsmithed the commit? Or did I misunderstand your point?

Yes, I think with an adjusted comment and commit message, the actual 
change makes sense.

  112. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-08-05T17:53:24Z

    On Fri, Aug 2, 2024 at 11:48 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Yes, I think with an adjusted comment and commit message, the actual
> change makes sense.

Done in v25.

...along with a bunch of other stuff:

1. All the debug-mode things that we want for testing but not in
production have now been hidden behind a PGOAUTHDEBUG environment
variable, instead of being enabled by default. At the moment, that
means 1) sensitive HTTP traffic gets printed on stderr, 2) plaintext
HTTP is allowed, and 3) servers may DoS the client by sending a
zero-second retry interval (which speeds up testing a lot). I've
resurrected some of Daniel's CURLOPT_DEBUGFUNCTION implementation for
this.

I think this feature needs more thought, but I'm not sure how much. In
particular I don't think a connection string option would be
appropriate (imagine the "fun" a proxy solution would have with a
spray-my-password-to-stderr switch). But maybe it makes sense to
further divide the dangerous behavior up, so that for example you can
debug the HTTP stream without also allowing plaintext connections, or
something. And maybe stricter maintainers would like to compile the
feature out entirely?

2. The verification_url variant from Azure and Google is now directly supported.

@Daniel: I figured out why I wasn't seeing the string-based-interval
issue in my testing. I've been using Azure's v2.0 OpenID endpoint,
which seems to be much more compliant than the original. Since this is
a new feature, would it be okay to just push new users to that
endpoint rather than supporting the previous weirdness in our code?
(Either way, I think we should support verification_url.)

Along those lines, with Azure I'm now seeing that device_code is not
advertised in grant_types_supported... is that new behavior? Or did
iddawc just not care?

3. I've restructured the libcurl calls to allow
curl_multi_socket_action() to synchronously succeed on its first call,
which we've been seeing a lot in the CI as mentioned upthread. This
led to a bunch of refactoring of the top-level state machine, which
had gotten too complex. I'm much happier with the code organization
now, but it's a big diff.

4. I've changed things around to get rid of two modern libcurl
deprecation warnings. I need to ask curl-library about my use of
curl_multi_socket_all(), which seems like it's exactly what our use
case needs.

Thanks,
--Jacob

  113. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-07T07:34:14Z

    On 05.08.24 19:53, Jacob Champion wrote:
> On Fri, Aug 2, 2024 at 11:48 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> Yes, I think with an adjusted comment and commit message, the actual
>> change makes sense.
> 
> Done in v25.
> 
> ...along with a bunch of other stuff:

I have committed 0001, and I plan to backpatch it once the release 
freeze lifts.

I'll work on 0002 next.

  114. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-12T06:37:01Z

    On 07.08.24 09:34, Peter Eisentraut wrote:
> On 05.08.24 19:53, Jacob Champion wrote:
>> On Fri, Aug 2, 2024 at 11:48 AM Peter Eisentraut 
>> <peter@eisentraut.org> wrote:
>>> Yes, I think with an adjusted comment and commit message, the actual
>>> change makes sense.
>>
>> Done in v25.
>>
>> ...along with a bunch of other stuff:
> 
> I have committed 0001, and I plan to backpatch it once the release 
> freeze lifts.
> 
> I'll work on 0002 next.

I have committed 0002 now.

  115. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-08-13T21:11:56Z

    On Sun, Aug 11, 2024 at 11:37 PM Peter Eisentraut <peter@eisentraut.org> wrote:
> I have committed 0002 now.

Thanks Peter! Rebased over both in v26.

--Jacob

  116. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-26T08:18:10Z

    On 13.08.24 23:11, Jacob Champion wrote:
> On Sun, Aug 11, 2024 at 11:37 PM Peter Eisentraut <peter@eisentraut.org> wrote:
>> I have committed 0002 now.
> 
> Thanks Peter! Rebased over both in v26.

I have looked again at the jsonapi memory management patch (v26-0001).
As previously mentioned, I think adding a third or fourth (depending
on how you count) memory management API is maybe something we should
avoid.  Also, the weird layering where src/common/ now (sometimes)
depends on libpq seems not great.

I'm thinking, maybe we leave the use of StringInfo at the source code
level, but #define the symbols to use PQExpBuffer.  Something like

#ifdef JSONAPI_USE_PQEXPBUFFER

#define StringInfo PQExpBuffer
#define appendStringInfo appendPQExpBuffer
#define appendBinaryStringInfo appendBinaryPQExpBuffer
#define palloc malloc
//etc.

#endif

(simplified, the argument lists might differ)

Or, if people find that too scary, something like

#ifdef JSONAPI_USE_PQEXPBUFFER

#define jsonapi_StringInfo PQExpBuffer
#define jsonapi_appendStringInfo appendPQExpBuffer
#define jsonapi_appendBinaryStringInfo appendBinaryPQExpBuffer
#define jsonapi_palloc malloc
//etc.

#else

#define jsonapi_StringInfo StringInfo
#define jsonapi_appendStringInfo appendStringInfo
#define jsonapi_appendBinaryStringInfo appendBinaryStringInfo
#define jsonapi_palloc palloc
//etc.

#endif

That way, it's at least more easy to follow the source code because
you see a mostly-familiar API.

Also, we should make this PQExpBuffer-using mode only used by libpq,
not by frontend programs.  So libpq takes its own copy of jsonapi.c
and compiles it using JSONAPI_USE_PQEXPBUFFER.  That will make the
libpq build descriptions a bit more complicated, but everyone who is
not libpq doesn't need to change.

Once you get past all the function renaming, the logic changes in
jsonapi.c all look pretty reasonable.  Refactoring like
allocate_incremental_state() makes sense.

You could add pg_nodiscard attributes to
makeJsonLexContextCstringLen() and makeJsonLexContextIncremental() so
that callers who are using the libpq mode are forced to check for
errors.  Or maybe there is a clever way to avoid even that: Create a
fixed JsonLexContext like

     static const JsonLexContext failed_oom;

and on OOM you return that one from makeJsonLexContext*().  And then
in pg_parse_json(), when you get handed that context, you return
JSON_OUT_OF_MEMORY immediately.

Other than that detail and the need to use freeJsonLexContext(), it
looks like this new mode doesn't impose any additional burden on
callers, since during parsing they need to check for errors anyway,
and this just adds one more error type for out of memory.  That's a good 
outcome.

  117. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-08-26T23:23:06Z

    On Mon, Aug 26, 2024 at 1:18 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Or, if people find that too scary, something like
>
> #ifdef JSONAPI_USE_PQEXPBUFFER
>
> #define jsonapi_StringInfo PQExpBuffer
> [...]
>
> That way, it's at least more easy to follow the source code because
> you see a mostly-familiar API.

I was having trouble reasoning about the palloc-that-isn't-palloc code
during the first few drafts, so I will try a round with the jsonapi_
prefix.

> Also, we should make this PQExpBuffer-using mode only used by libpq,
> not by frontend programs.  So libpq takes its own copy of jsonapi.c
> and compiles it using JSONAPI_USE_PQEXPBUFFER.  That will make the
> libpq build descriptions a bit more complicated, but everyone who is
> not libpq doesn't need to change.

Sounds reasonable. It complicates the test coverage situation a little
bit, but I think my current patch was maybe insufficient there anyway,
since the coverage for the backend flavor silently dropped...

> Or maybe there is a clever way to avoid even that: Create a
> fixed JsonLexContext like
>
>      static const JsonLexContext failed_oom;
>
> and on OOM you return that one from makeJsonLexContext*().  And then
> in pg_parse_json(), when you get handed that context, you return
> JSON_OUT_OF_MEMORY immediately.

I like this idea.

Thanks!
--Jacob

  118. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-08-28T16:31:07Z

    On Mon, Aug 26, 2024 at 4:23 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I was having trouble reasoning about the palloc-that-isn't-palloc code
> during the first few drafts, so I will try a round with the jsonapi_
> prefix.

v27 takes a stab at that. I have kept the ALLOC/FREE naming to match
the strategy in other src/common source files.

The name of the variable JSONAPI_USE_PQEXPBUFFER leads to sections of
code that look like this:

+#ifdef JSONAPI_USE_PQEXPBUFFER
+    if (!new_prediction || !new_fnames || !new_fnull)
+        return false;
+#endif

To me it wouldn't be immediately obvious why "using PQExpBuffer" has
anything to do with this code; the key idea is that we expect any
allocations to be able to fail. Maybe a name like JSONAPI_ALLOW_OOM or
JSONAPI_SHLIB_ALLOCATIONS or...?

> It complicates the test coverage situation a little
> bit, but I think my current patch was maybe insufficient there anyway,
> since the coverage for the backend flavor silently dropped...

To do this without too much pain, I split the "forbidden" objects into
their own shared library, used only by the JSON tests which needed
them. I tried not to wrap too much ceremony around them, since they're
only needed in one place, so they don't have an associated Meson
dependency object.

> > Or maybe there is a clever way to avoid even that: Create a
> > fixed JsonLexContext like
> >
> >      static const JsonLexContext failed_oom;

I think this turned out nicely. Two slight deviations from this are
that we can't return a pointer-to-const, and we also need an OOM
sentinel for the JsonIncrementalState, since it's possible to
initialize incremental parsing into a JsonLexContext that's on the
stack.

--Jacob

  119. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-08-30T09:49:43Z

    On 28.08.24 18:31, Jacob Champion wrote:
> On Mon, Aug 26, 2024 at 4:23 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> I was having trouble reasoning about the palloc-that-isn't-palloc code
>> during the first few drafts, so I will try a round with the jsonapi_
>> prefix.
> 
> v27 takes a stab at that. I have kept the ALLOC/FREE naming to match
> the strategy in other src/common source files.

This looks pretty good to me.  Maybe on the naming side, this seems like 
a gratuitous divergence:

+#define jsonapi_createStringInfo           makeStringInfo

> The name of the variable JSONAPI_USE_PQEXPBUFFER leads to sections of
> code that look like this:
> 
> +#ifdef JSONAPI_USE_PQEXPBUFFER
> +    if (!new_prediction || !new_fnames || !new_fnull)
> +        return false;
> +#endif
> 
> To me it wouldn't be immediately obvious why "using PQExpBuffer" has
> anything to do with this code; the key idea is that we expect any
> allocations to be able to fail. Maybe a name like JSONAPI_ALLOW_OOM or
> JSONAPI_SHLIB_ALLOCATIONS or...?

Seems ok to me as is.  I think the purpose of JSONAPI_USE_PQEXPBUFFER is 
adequately explained by this comment

+/*
+ * By default, we will use palloc/pfree along with StringInfo.  In libpq,
+ * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on 
out-of-memory.
+ */
+#ifdef JSONAPI_USE_PQEXPBUFFER

For some of the other proposed names, I'd be afraid that someone might 
think you are free to mix and match APIs, OOM behavior, and compilation 
options.


Some comments on src/include/common/jsonapi.h:

-#include "lib/stringinfo.h"

I suspect this will fail headerscheck?  Probably needs an exception 
added there.

+#ifdef JSONAPI_USE_PQEXPBUFFER
+#define StrValType PQExpBufferData
+#else
+#define StrValType StringInfoData
+#endif

Maybe use jsonapi_StrValType here.

+typedef struct StrValType StrValType;

I don't think that is needed.  It would just duplicate typedefs that 
already exist elsewhere, depending on what StrValType is set to.

+       bool            parse_strval;
+       StrValType *strval;                     /* only used if 
parse_strval == true */

The parse_strval field could use a better explanation.

I actually don't understand the need for this field.  AFAICT, this is
just used to record whether strval is valid.  But in the cases where
it's not valid, why do we need to record that?  Couldn't you just return
failed_oom in those cases?

  120. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-09-03T20:56:07Z

    On Fri, Aug 30, 2024 at 2:49 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> This looks pretty good to me.  Maybe on the naming side, this seems like
> a gratuitous divergence:
>
> +#define jsonapi_createStringInfo           makeStringInfo

Whoops, fixed.

> Seems ok to me as is.  I think the purpose of JSONAPI_USE_PQEXPBUFFER is
> adequately explained by this comment
>
> +/*
> + * By default, we will use palloc/pfree along with StringInfo.  In libpq,
> + * use malloc and PQExpBuffer, and return JSON_OUT_OF_MEMORY on
> out-of-memory.
> + */
> +#ifdef JSONAPI_USE_PQEXPBUFFER
>
> For some of the other proposed names, I'd be afraid that someone might
> think you are free to mix and match APIs, OOM behavior, and compilation
> options.

Yeah, that's fair.

> Some comments on src/include/common/jsonapi.h:
>
> -#include "lib/stringinfo.h"
>
> I suspect this will fail headerscheck?  Probably needs an exception
> added there.

Currently it passes on my machine and the cfbot. The
forward-declaration of the struct should be enough to make clients
happy. Or was there a different way to break it?

> +#ifdef JSONAPI_USE_PQEXPBUFFER
> +#define StrValType PQExpBufferData
> +#else
> +#define StrValType StringInfoData
> +#endif
>
> Maybe use jsonapi_StrValType here.

Done.

> +typedef struct StrValType StrValType;
>
> I don't think that is needed.  It would just duplicate typedefs that
> already exist elsewhere, depending on what StrValType is set to.

Okay, removed.

> The parse_strval field could use a better explanation.
>
> I actually don't understand the need for this field.  AFAICT, this is
> just used to record whether strval is valid.

No, it's meant to track the value of the need_escapes argument to the
constructor. I've renamed it and moved the assignment to hopefully
make that a little more obvious. WDYT?

> But in the cases where
> it's not valid, why do we need to record that?  Couldn't you just return
> failed_oom in those cases?

We can do that if you'd like. I was just worried about using a valid
(broken) value of PQExpBuffer as a sentinel instead of a separate
flag. It would work as long as reviewers stay vigilant, but if we go
that direction and someone adds an unchecked

    lex->strval = jsonapi_makeStringInfo();
    // should check for NULL now, but we forgot

into a future patch, an allocation failure in _shlib builds would
silently disable string escaping instead of resulting in a
JSON_OUT_OF_MEMORY later.

Thanks,
--Jacob

  121. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-09-04T09:28:24Z

    On 03.09.24 22:56, Jacob Champion wrote:
>> The parse_strval field could use a better explanation.
>>
>> I actually don't understand the need for this field.  AFAICT, this is
>> just used to record whether strval is valid.
> No, it's meant to track the value of the need_escapes argument to the
> constructor. I've renamed it and moved the assignment to hopefully
> make that a little more obvious. WDYT?

Yes, this is clearer.

This patch (v28-0001) looks good to me now.

  122. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-09-11T07:37:34Z

    On 04.09.24 11:28, Peter Eisentraut wrote:
> On 03.09.24 22:56, Jacob Champion wrote:
>>> The parse_strval field could use a better explanation.
>>>
>>> I actually don't understand the need for this field.  AFAICT, this is
>>> just used to record whether strval is valid.
>> No, it's meant to track the value of the need_escapes argument to the
>> constructor. I've renamed it and moved the assignment to hopefully
>> make that a little more obvious. WDYT?
> 
> Yes, this is clearer.
> 
> This patch (v28-0001) looks good to me now.

This has been committed.

About the subsequent patches:

Is there any sense in dealing with the libpq and backend patches 
separately in sequence, or is this split just for ease of handling?

(I suppose the 0004 "review comments" patch should be folded into the 
respective other patches?)

What could be the next steps to keep this moving along, other than stare 
at the remaining patches until we're content with them? ;-)

  123. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-09-11T13:44:37Z

    > On 11 Sep 2024, at 09:37, Peter Eisentraut <peter@eisentraut.org> wrote:

> Is there any sense in dealing with the libpq and backend patches separately in sequence, or is this split just for ease of handling?

I think it's just make reviewing a bit easier.  At this point I think they can
be merged together, it's mostly out of historic reasons IIUC since the patchset
earlier on supported more than one library.

> (I suppose the 0004 "review comments" patch should be folded into the respective other patches?)

Yes (0003 now), along with the 0004 in the attached version (I bumped to v29 as
one commit is now committed, but the attached doesn't change Jacobs commits but
rather add to them) which contains more review comments.  More on that below:

I added a warning to autconf in case --with-oauth is used without --with-python
since this combination will error out in running the tests.  Might be
superfluous but I had an embarrassingly long headscratcher myself as to why the
tests kept failing =)

CURL_IGNORE_DEPRECATION(x;) broke pgindent, it needs to keep the semicolon on
the outside like CURL_IGNORE_DEPRECATION(x);.  This doesn't really work well
with how the macro is defined, not sure how we should handle that best (the
attached makes the style as per how pgindent want's it with the semicolon
returned).

The oauth_validator test module need to load Makefile.global before exporting
the symbols from there.  I also removed the placeholder regress test which did
nothing and turned diag() calls into note() calls to keep the output from
cluttering.

There is a first stab at documenting the validator module API, more to come (it
doesn't compile right now).

It contains a pgindent and pgperltidy run to keep things as close to in final
sync as we can to catch things like the curl deprecation macro mentioned above
early.

> What could be the next steps to keep this moving along, other than stare at the remaining patches until we're content with them? ;-)

I'm in the "stare at things" stage now to try and get this into the tree =)

To further pick away at this huge patch I propose to merge the SASL message
length hunk which can be extracted separately.  The attached .txt (to keep the
CFBot from poking at it) contains a diff which can be committed ahead of the
rest of this patch to make it a tad smaller and to keep the history of that
change a bit clearer.

--
Daniel Gustafsson

  124. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-09-11T22:54:18Z

    (Thanks for the commit, Peter!)

On Wed, Sep 11, 2024 at 6:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 11 Sep 2024, at 09:37, Peter Eisentraut <peter@eisentraut.org> wrote:
>
> > Is there any sense in dealing with the libpq and backend patches separately in sequence, or is this split just for ease of handling?
>
> I think it's just make reviewing a bit easier.  At this point I think they can
> be merged together, it's mostly out of historic reasons IIUC since the patchset
> earlier on supported more than one library.

I can definitely do that (and yeah, it was to make the review slightly
less daunting). The server side could potentially be committed
independently, if you want to parallelize a bit, but it'd have to be
torn back out if the libpq stuff didn't land in time.

> > (I suppose the 0004 "review comments" patch should be folded into the respective other patches?)

Yes. I'm using that patch as a holding area while I write tests for
the hunks, and then moving them backwards.

> I added a warning to autconf in case --with-oauth is used without --with-python
> since this combination will error out in running the tests.  Might be
> superfluous but I had an embarrassingly long headscratcher myself as to why the
> tests kept failing =)

Whoops, sorry. I guess we should just skip them if Python isn't there?

> CURL_IGNORE_DEPRECATION(x;) broke pgindent, it needs to keep the semicolon on
> the outside like CURL_IGNORE_DEPRECATION(x);.  This doesn't really work well
> with how the macro is defined, not sure how we should handle that best (the
> attached makes the style as per how pgindent want's it with the semicolon
> returned).

Ugh... maybe a case for a pre_indent rule in pgindent?

> The oauth_validator test module need to load Makefile.global before exporting
> the symbols from there.

Hm. Why was that passing the CI, though...?

> There is a first stab at documenting the validator module API, more to come (it
> doesn't compile right now).
>
> It contains a pgindent and pgperltidy run to keep things as close to in final
> sync as we can to catch things like the curl deprecation macro mentioned above
> early.

Thanks!

> > What could be the next steps to keep this moving along, other than stare at the remaining patches until we're content with them? ;-)
>
> I'm in the "stare at things" stage now to try and get this into the tree =)

Yeah, and I still owe you all an updated roadmap.

While I fix up the tests, I've also been picking away at the JSON
encoding problem that was mentioned in [1]; the recent SASLprep fix
was fallout from that, since I'm planning to pull in pieces of its
UTF-8 validation. I will eventually want to fuzz the heck out of this.

> To further pick away at this huge patch I propose to merge the SASL message
> length hunk which can be extracted separately.  The attached .txt (to keep the
> CFBot from poking at it) contains a diff which can be committed ahead of the
> rest of this patch to make it a tad smaller and to keep the history of that
> change a bit clearer.

LGTM!

--

Peter asked me if there were plans to provide a "standard" validator
module, say as part of contrib. The tricky thing is that Bearer
validation is issuer-specific, and many providers give you an opaque
token that you're not supposed to introspect at all.

We could use token introspection (RFC 7662) for online verification,
but last I looked at it, no one had actually implemented those
endpoints. For offline verification, I think the best we could do
would be to provide a generic JWT Profile (RFC 9068) validator, but
again I don't know if anyone is actually providing those token formats
in practice. I'm inclined to push that out into the future.

Thanks,
--Jacob

[1] https://www.postgresql.org/message-id/ZjxQnOD1OoCkEeMN%40paquier.xyz

  125. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-09-16T19:13:28Z

    On Wed, Sep 11, 2024 at 3:54 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Yeah, and I still owe you all an updated roadmap.

Okay, here goes. New reviewers: start here!

== What is This? ==

OAuth 2.0 is a way for a trusted third party (a "provider") to tell a
server whether a client on the other end of the line is allowed to do
something. This patchset adds OAuth support to libpq with libcurl,
provides a server-side API so that extension modules can add support
for specific OAuth providers, and extends our SASL support to carry
the OAuth access tokens over the OAUTHBEARER mechanism.

Most OAuth clients use a web browser to perform the third-party
handshake. (These are your "Okta logins", "sign in with XXX", etc.)
But there are plenty of people who use psql without a local browser,
and invoking a browser safely across all supported platforms is
actually surprisingly fraught. So this patchset implements something
called device authorization, where the client will display a link and
a code, and then you can log in on whatever device is convenient for
you. Once you've told your provider that you trust libpq to connect to
Postgres on your behalf, it'll give libpq an access token, and libpq
will forward that on to the server.

== How This Fits, or: The Sales Pitch ==

The most popular third-party auth methods we have today are probably
the Kerberos family (AD/GSS/SSPI) and LDAP. If you're not already in
an MS ecosystem, it's unlikely that you're using the former. And users
of the latter are, in my experience, more-or-less resigned to its use,
in spite of LDAP's architectural security problems and the fact that
you have to run weird synchronization scripts to tell Postgres what
certain users are allowed to do.

OAuth provides a decently mature and widely-deployed third option. You
don't have to be running the infrastructure yourself, as long as you
have a provider you trust. If you are running your own infrastructure
(or if your provider is configurable), the tokens being passed around
can carry org-specific user privileges, so that Postgres can figure
out who's allowed to do what without out-of-band synchronization
scripts. And those access tokens are a straight upgrade over
passwords: even if they're somehow stolen, they are time-limited, they
are optionally revocable, and they can be scoped to specific actions.

== Extension Points ==

This patchset provides several points of customization:

Server-side validation is farmed out entirely to an extension, which
we do not provide. (Each OAuth provider is free to come up with its
own proprietary method of verifying its access tokens, and so far the
big players have absolutely not standardized.) Depending on the
provider, the extension may need to contact an external server to see
what the token has been authorized to do, or it may be able to do that
offline using signing keys and an agreed-upon token format.

The client driver using libpq may replace the device authorization
prompt (which by default is done on standard error), for example to
move it into an existing GUI, display a scannable QR code instead of a
link, and so on.

The driver may also replace the entire OAuth flow. For example, a
client that already interacts with browsers may be able to use one of
the more standard web-based methods to get an access token. And
clients attached to a service rather than an end user could use a more
straightforward server-to-server flow, with pre-established
credentials.

== Architecture ==

The client needs to speak HTTP, which is implemented entirely with
libcurl. Originally, I used another OAuth library for rapid
prototyping, but the quality just wasn't there and I ported the
implementation. An internal abstraction layer remains in the libpq
code, so if a better client library comes along, switching to it
shouldn't be too painful.

The client-side hooks all go through a single extension point, so that
we don't continually add entry points in the API for each new piece of
authentication data that a driver may be able to provide. If we wanted
to, we could potentially move the existing SSL passphrase hook into
that, or even handle password retries within libpq itself, but I don't
see any burning reason to do that now.

I wanted to make sure that OAuth could be dropped into existing
deployments without driver changes. (Drivers will probably *want* to
look at the extension hooks for better UX, but they shouldn't
necessarily *have* to.) That has driven several parts of the design.

Drivers using the async APIs should continue to work without blocking,
even during the long HTTP handshakes. So the new client code is
structured as a typical event-driven state machine (similar to
PQconnectPoll). The protocol machine hands off control to the OAuth
machine during authentication, without really needing to know how it
works, because the OAuth machine replaces the PQsocket with a
general-purpose multiplexer that handles all of the HTTP sockets and
events. Once that's completed, the OAuth machine hands control right
back and we return to the Postgres protocol on the wire.

This decision led to a major compromise: Windows client support is
nonexistent. Multiplexer handles exist in Windows (for example with
WSAEventSelect, IIUC), but last I checked they were completely
incompatible with Winsock select(), which means existing async-aware
drivers would fail. We could compromise by providing synchronous-only
support, or by cobbling together a socketpair plus thread pool (or
IOCP?), or simply by saying that existing Windows clients need a new
API other than PQsocket() to be able to work properly. None of those
approaches have been attempted yet, though.

== Areas of Concern ==

Here are the iffy things that a committer is signing up for:

The client implementation is roughly 3k lines, requiring domain
knowledge of Curl, HTTP, JSON, and OAuth, the specifications of which
are spread across several separate standards bodies. (And some big
providers ignore those anyway.)

The OAUTHBEARER mechanism is extensible, but not in the same way as
HTTP. So sometimes, it looks like people design new OAuth features
that rely heavily on HTTP and forget to "port" them over to SASL. That
may be a point of future frustration.

C is not really anyone's preferred language for implementing an
extensible authn/z protocol running on top of HTTP, and constant
vigilance is going to be required to maintain safety. What's more, we
don't really "trust" the endpoints we're talking to in the same way
that we normally trust our servers. It's a fairly hostile environment
for maintainers.

Along the same lines, our JSON implementation assumes some level of
trust in the JSON data -- which is true for the backend, and can be
assumed for a DBA running our utilities, but is absolutely not the
case for a libpq client downloading data from Some Server on the
Internet. I've been working to fuzz the implementation and there are a
few known problems registered in the CF already.

Curl is not a lightweight dependency by any means. Typically, libcurl
is configured with a wide variety of nice options, a tiny subset of
which we're actually going to use, but all that code (and its
transitive dependencies!) is going to arrive in our process anyway.
That might not be a lot of fun if you're not using OAuth.

It's possible that the application embedding libpq is also a direct
client of libcurl. We need to make sure we're not stomping on their
toes at any point.

== TODOs/Known Issues ==

The client does not deal with verification failure well at the moment;
it just keeps retrying with a new OAuth handshake.

Some people are not going to be okay with just contacting any web
server that Postgres tells them to. There's a more paranoid mode
sketched out that lets the connection string specify the trusted
issuer, but it's not complete.

The new code still needs to play well with orthogonal connection
options, like connect_timeout and require_auth.

The server does not deal well with multi-issuer setups yet. And you
only get one oauth_validator_library...

Harden, harden, harden. There are still a handful of inline TODOs
around double-checking certain pieces of the response before
continuing with the handshake. Servers should not be able to run our
recursive descent parser out of stack. And my JSON code is using
assertions too liberally, which will turn bugs into DoS vectors. I've
been working to fit a fuzzer into more and more places, and I'm hoping
to eventually drive it directly from the socket.

Documentation still needs to be filled in. (Thanks Daniel for your work here!)

== Future Features ==

There is no support for token caching (refresh or otherwise). Each new
connection needs a new approval, and the only way to change that for
v1 is to replace the entire flow. I think that's eventually going to
annoy someone. The question is, where do you persist it? Does that
need to be another extensibility point?

We already have pretty good support for client certificates, and it'd
be great if we could bind our tokens to those. That way, even if you
somehow steal the tokens, you can't do anything with them without the
private key! But the state of proof-of-possession in OAuth is an
absolute mess, involving at least three competing standards (Token
Binding, mTLS, DPoP). I don't know what's going to win.

--

Hope this helps! Next I'll be working to fold the patches together, as
discussed upthread.

Thanks,
--Jacob

  126. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-09-27T17:58:19Z

    Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> Peter asked me if there were plans to provide a "standard" validator
> module, say as part of contrib. The tricky thing is that Bearer
> validation is issuer-specific, and many providers give you an opaque
> token that you're not supposed to introspect at all.
> 
> We could use token introspection (RFC 7662) for online verification,
> but last I looked at it, no one had actually implemented those
> endpoints. For offline verification, I think the best we could do
> would be to provide a generic JWT Profile (RFC 9068) validator, but
> again I don't know if anyone is actually providing those token formats
> in practice. I'm inclined to push that out into the future.

Have you considered sending the token for validation to the server, like this

curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo" -H "Authorization: Bearer $TOKEN"

and getting the userid (e.g. email address) from the response, as described in
[1]? ISTM that this is what pgadmin4 does - in paricular, see the
get_user_profile() function in web/pgadmin/authenticate/oauth2.py.

[1] https://www.oauth.com/oauth2-servers/signing-in-with-google/verifying-the-user-info/

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  127. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-09-27T20:45:45Z

    On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> Have you considered sending the token for validation to the server, like this
>
> curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo" -H "Authorization: Bearer $TOKEN"

In short, no, but I'm glad you asked. I think it's going to be a
common request, and I need to get better at explaining why it's not
safe, so we can document it clearly. Or else someone can point out
that I'm misunderstanding, which honestly would make all this much
easier and less complicated. I would love to be able to do it that
way.

We cannot, for the same reason libpq must send the server an access
token instead of an ID token. The /userinfo endpoint tells you who the
end user is, but it doesn't tell you whether the Bearer is actually
allowed to access the database. That difference is critical: it's
entirely possible for an end user to be authorized to access the
database, *and yet* the Bearer token may not actually carry that
authorization on their behalf. (In fact, the user may have actively
refused to give the Bearer that permission.) That's why people are so
pedantic about saying that OAuth is an authorization framework and not
an authentication framework.

To illustrate, think about all the third-party web services out there
that ask you to Sign In with Google. They ask Google for permission to
access your personal ID, and Google asks you if you're okay with that,
and you either allow or deny it. Now imagine that I ran one of those
services, and I decided to become evil. I could take my legitimately
acquired Bearer token -- which should only give me permission to query
your Google ID -- and send it to a Postgres database you're authorized
to access.

The server is supposed to introspect it, say, "hey, this token doesn't
give the bearer access to the database at all," and shut everything
down. For extra credit, the server could notice that the client ID
tied to the access token isn't even one that it recognizes! But if all
the server does is ask Google, "what's the email address associated
with this token's end user?", then it's about to make some very bad
decisions. The email address it gets back doesn't belong to Jacob the
Evil Bearer; it belongs to you.

Now, the token introspection endpoint I mentioned upthread should give
us the required information (scopes, etc.). But Google doesn't
implement that one. In fact they don't seem to have implemented custom
scopes at all in the years since I started work on this feature, which
makes me think that people are probably not going to be able to safely
log into Postgres using Google tokens. Hopefully there's some feature
buried somewhere that I haven't seen.

Let me know if that makes sense. (And again: I'd love to be proven
wrong. It would improve the reach of the feature considerably if I
am.)

Thanks,
--Jacob

  128. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-09-30T13:38:41Z

    Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> > Have you considered sending the token for validation to the server, like this
> >
> > curl -X GET "https://www.googleapis.com/oauth2/v3/userinfo" -H "Authorization: Bearer $TOKEN"
> 
> In short, no, but I'm glad you asked. I think it's going to be a
> common request, and I need to get better at explaining why it's not
> safe, so we can document it clearly. Or else someone can point out
> that I'm misunderstanding, which honestly would make all this much
> easier and less complicated. I would love to be able to do it that
> way.
> 
> We cannot, for the same reason libpq must send the server an access
> token instead of an ID token. The /userinfo endpoint tells you who the
> end user is, but it doesn't tell you whether the Bearer is actually
> allowed to access the database. That difference is critical: it's
> entirely possible for an end user to be authorized to access the
> database, *and yet* the Bearer token may not actually carry that
> authorization on their behalf. (In fact, the user may have actively
> refused to give the Bearer that permission.)

> That's why people are so pedantic about saying that OAuth is an
> authorization framework and not an authentication framework.

This statement alone sounds as if you missed *authentication*, but you seem to
admit above that the /userinfo endpoint provides it ("tells you who the end
user is"). I agree that it does. My understanding is that this endpoint, as
well as the concept of "claims" and "scopes", is introduced by OpenID, which
is an *authentication* framework, although it's built on top of OAuth.

Regarding *authorization*, I agree that the bearer token may not contain
enough information to determine whether the owner of the token is allowed to
access the database. However, I consider database a special kind of
"application", which can handle authorization on its own. In this case, the
authorization can be controlled by (not) assigning the user the LOGIN
attribute, as well as by (not) granting it privileges on particular database
objects. In short, I think that *authentication* is all we need.

> To illustrate, think about all the third-party web services out there
> that ask you to Sign In with Google. They ask Google for permission to
> access your personal ID, and Google asks you if you're okay with that,
> and you either allow or deny it. Now imagine that I ran one of those
> services, and I decided to become evil. I could take my legitimately
> acquired Bearer token -- which should only give me permission to query
> your Google ID -- and send it to a Postgres database you're authorized
> to access.
> 
> The server is supposed to introspect it, say, "hey, this token doesn't
> give the bearer access to the database at all," and shut everything
> down. For extra credit, the server could notice that the client ID
> tied to the access token isn't even one that it recognizes! But if all
> the server does is ask Google, "what's the email address associated
> with this token's end user?", then it's about to make some very bad
> decisions. The email address it gets back doesn't belong to Jacob the
> Evil Bearer; it belongs to you.

Are you sure you can legitimately acquire the bearer token containing my email
address? I think the email address returned by the /userinfo endpoint is one
of the standard claims [1]. Thus by returning the particular value of "email"
from the endpoint the identity provider asserts that the token owner does have
this address. (And that, if "email_verified" claim is "true", it spent some
effort to verify that the email address is controlled by that user.)

> Now, the token introspection endpoint I mentioned upthread

Can you please point me to the particular message?

> should give us the required information (scopes, etc.). But Google doesn't
> implement that one. In fact they don't seem to have implemented custom
> scopes at all in the years since I started work on this feature, which makes
> me think that people are probably not going to be able to safely log into
> Postgres using Google tokens. Hopefully there's some feature buried
> somewhere that I haven't seen.
> 
> Let me know if that makes sense. (And again: I'd love to be proven
> wrong. It would improve the reach of the feature considerably if I
> am.)

Another question, assuming the token verification is resolved somehow:
wouldn't it be sufficient for the initial implementation if the client could
pass the bearer token to libpq in the connection string?

Obviously, one use case is than an application / web server which needs the
token to authenticate the user could eventually pass the token to the database
server. Thus, if users could authenticate to the database using their
individual ids, it would no longer be necessary to store a separate userid /
password for the application in a configuration file.

Also, if libpq accepted the bearer token via the connection string, it would
be possible to implement the authorization as a separate front-end application
(e.g. pg_oauth_login) rather than adding more complexity to libpq itself.

(I'm learning this stuff on-the-fly, so there might be something naive in my
comments.)

[1] https://openid.net/specs/openid-connect-core-1_0.html#StandardClaims

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  129. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-09-30T16:54:28Z

    Antonin Houska <ah@cybertec.at> wrote:

> Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > Now, the token introspection endpoint I mentioned upthread
> 
> Can you please point me to the particular message?

Please ignore this dumb question. You probably referred to the email I was
responding to.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  130. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-01T00:05:06Z

    On Mon, Sep 30, 2024 at 6:38 AM Antonin Houska <ah@cybertec.at> wrote:
>
> Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> > On Fri, Sep 27, 2024 at 10:58 AM Antonin Houska <ah@cybertec.at> wrote:
> > That's why people are so pedantic about saying that OAuth is an
> > authorization framework and not an authentication framework.
>
> This statement alone sounds as if you missed *authentication*, but you seem to
> admit above that the /userinfo endpoint provides it ("tells you who the end
> user is"). I agree that it does. My understanding is that this endpoint, as
> well as the concept of "claims" and "scopes", is introduced by OpenID, which
> is an *authentication* framework, although it's built on top of OAuth.

OpenID is an authentication framework, but it's generally focused on a
type of client known as a Relying Party. In the architecture of this
patchset, the Relying Party would be libpq, which has the option of
retrieving authentication claims from the provider. Unfortunately for
us, libpq has no use for those claims. It's not trying to authenticate
the user for its own purposes.

The Postgres server, on the other hand, is not a Relying Party. (It's
an OAuth resource server, in this architecture.) It's not performing
any of the OIDC flows, it's not talking to the end user and the
provider at the same time, and it is very restricted in its ability to
influence the client exchange via the SASL mechanism.

> Regarding *authorization*, I agree that the bearer token may not contain
> enough information to determine whether the owner of the token is allowed to
> access the database. However, I consider database a special kind of
> "application", which can handle authorization on its own. In this case, the
> authorization can be controlled by (not) assigning the user the LOGIN
> attribute, as well as by (not) granting it privileges on particular database
> objects. In short, I think that *authentication* is all we need.

Authorizing the *end user's* access to the database using scopes is
optional. Authorizing the *bearer's* ability to connect on behalf of
the end user, however, is mandatory. Hopefully the below clarifies.

(I agree that most people probably want to use authentication, so that
the database can then make decisions based on HBA settings. OIDC is a
fine way to do that.)

> Are you sure you can legitimately acquire the bearer token containing my email
> address?

Yes. In general that's how OpenID-based "Sign in with <Service>"
works. All those third-party services are running around with tokens
that identify you, but unless they've asked for more abilities and
you've granted them the associated scopes, identifying you is all they
can do.

> I think the email address returned by the /userinfo endpoint is one
> of the standard claims [1]. Thus by returning the particular value of "email"
> from the endpoint the identity provider asserts that the token owner does have
> this address.

We agree that /userinfo gives authentication claims for the end user.
It's just insufficient for our use case.

For example, there are enterprise applications out there that will ask
for read access to your Google Calendar. If you're willing to grant
that, then you probably won't mind if those applications also know
your email address, but you probably do mind if they're suddenly able
to access your production databases just because you gave them your
email.

Put another way: if you log into Postgres using OAuth, and your
provider doesn't show you a big message saying "this application is
about to access *your* prod database using *your* identity; do you
want to allow that?", then your DBA has deployed a really dangerous
configuration. That's a critical protection feature you get from your
OAuth provider. Otherwise, what's stopping somebody else from setting
up their own malicious service to farm access tokens? All they'd have
to do is ask for your email.

> Another question, assuming the token verification is resolved somehow:
> wouldn't it be sufficient for the initial implementation if the client could
> pass the bearer token to libpq in the connection string?

It was discussed wayyy upthread:

    https://postgr.es/m/CAAWbhmhmBe9v3aCffz5j8Sg4HMWWkB5FvTDCSZ_Vh8E1fX91Gw%40mail.gmail.com

Basically, at that point the entire implementation becomes an exercise
for the reader. I want to avoid that if possible. I'm not adamantly
opposed to it, but I think the client-side hook implementation is
going to be better for the use cases that have been discussed so far.

> Also, if libpq accepted the bearer token via the connection string, it would
> be possible to implement the authorization as a separate front-end application
> (e.g. pg_oauth_login) rather than adding more complexity to libpq itself.

The application would still need to parse the server error response.
There was (a small) consensus at the time [1] that parsing error
messages for that purpose would be really unpleasant; hence the hook
architecture.

> (I'm learning this stuff on-the-fly, so there might be something naive in my
> comments.)

No worries! Please keep the questions coming; this OAuth architecture
is unintuitive, and I need to be able to defend it.

Thanks,
--Jacob

[1] https://postgr.es/m/CACrwV54_euYe%2Bv7bcLrxnje-JuM%3DKRX5azOcmmrXJ5qrffVZfg%40mail.gmail.com

  131. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-10-08T10:46:49Z

    Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> On Mon, Sep 30, 2024 at 6:38 AM Antonin Houska <ah@cybertec.at> wrote:
> >
> > Are you sure you can legitimately acquire the bearer token containing my email
> > address?
> 
> Yes. In general that's how OpenID-based "Sign in with <Service>"
> works. All those third-party services are running around with tokens
> that identify you, but unless they've asked for more abilities and
> you've granted them the associated scopes, identifying you is all they
> can do.
> 
> > I think the email address returned by the /userinfo endpoint is one
> > of the standard claims [1]. Thus by returning the particular value of "email"
> > from the endpoint the identity provider asserts that the token owner does have
> > this address.
> 
> We agree that /userinfo gives authentication claims for the end user.
> It's just insufficient for our use case.
> 
> For example, there are enterprise applications out there that will ask
> for read access to your Google Calendar. If you're willing to grant
> that, then you probably won't mind if those applications also know
> your email address, but you probably do mind if they're suddenly able
> to access your production databases just because you gave them your
> email.
> 
> Put another way: if you log into Postgres using OAuth, and your
> provider doesn't show you a big message saying "this application is
> about to access *your* prod database using *your* identity; do you
> want to allow that?", then your DBA has deployed a really dangerous
> configuration. That's a critical protection feature you get from your
> OAuth provider. Otherwise, what's stopping somebody else from setting
> up their own malicious service to farm access tokens? All they'd have
> to do is ask for your email.

Perhaps I understand now. I use getmail [2] to retrieve email messages from my
Google account. What made me confused is that the getmail application,
although installed on my workstation (and thus the bearer token it eventually
gets contains my email address), it's "someone else" (in particular the
"Relying Party") from the perspective of the OpenID protocol. And the same
applies to "psql" in the context of your patch.

Thus, in addition to the email, we'd need special claims which authorize the
RPs to access the database and only the database. Does this sound correct?

> > (I'm learning this stuff on-the-fly, so there might be something naive in my
> > comments.)
> 
> No worries! Please keep the questions coming; this OAuth architecture
> is unintuitive, and I need to be able to defend it.

I'd like to play with the code a bit and provide some review before or during
the next CF. That will probably generate some more questions.

> 
> [1] https://postgr.es/m/CACrwV54_euYe%2Bv7bcLrxnje-JuM%3DKRX5azOcmmrXJ5qrffVZfg%40mail.gmail.com

[2] https://github.com/getmail6/getmail6/

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  132. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-08T15:19:46Z

    On Tue, Oct 8, 2024 at 3:46 AM Antonin Houska <ah@cybertec.at> wrote:
> Perhaps I understand now. I use getmail [2] to retrieve email messages from my
> Google account. What made me confused is that the getmail application,
> although installed on my workstation (and thus the bearer token it eventually
> gets contains my email address), it's "someone else" (in particular the
> "Relying Party") from the perspective of the OpenID protocol. And the same
> applies to "psql" in the context of your patch.
>
> Thus, in addition to the email, we'd need special claims which authorize the
> RPs to access the database and only the database. Does this sound correct?

Yes. (One nitpick: the "special claims" in this case are not OpenID
claims at all, but OAuth scopes. The HBA will be configured with the
list of scopes that the server requires, and it requests those from
the client during the SASL handshake.)

> I'd like to play with the code a bit and provide some review before or during
> the next CF. That will probably generate some more questions.

Thanks very much for the review!

--Jacob

  133. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-10T23:08:50Z

    Hi all,

Here's v30, which aims to fix the infinite-retry problem: you get a
maximum of two OAuth connections to the server, and we won't prompt
the user more than once per transport. (I still need to wrap my head
around the retry behavior during transport negotiation.)

On Wed, Sep 11, 2024 at 3:54 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Wed, Sep 11, 2024 at 6:44 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> >
> > I think it's just make reviewing a bit easier.  At this point I think they can
> > be merged together, it's mostly out of historic reasons IIUC since the patchset
> > earlier on supported more than one library.
>
> I can definitely do that (and yeah, it was to make the review slightly
> less daunting). The server side could potentially be committed
> independently, if you want to parallelize a bit, but it'd have to be
> torn back out if the libpq stuff didn't land in time.

I plan to do the combination in the near future, when I'm not making a
bunch of other changes at the same time.

> > > (I suppose the 0004 "review comments" patch should be folded into the respective other patches?)
>
> Yes. I'm using that patch as a holding area while I write tests for
> the hunks, and then moving them backwards.

This is almost gone; just one piece remaining as of v30. Everything
else has been folded in with new tests.

> > CURL_IGNORE_DEPRECATION(x;) broke pgindent, it needs to keep the semicolon on
> > the outside like CURL_IGNORE_DEPRECATION(x);.  This doesn't really work well
> > with how the macro is defined, not sure how we should handle that best (the
> > attached makes the style as per how pgindent want's it with the semicolon
> > returned).
>
> Ugh... maybe a case for a pre_indent rule in pgindent?

I've taken a stab at a pre_indent rule that seems to work well enough
(though the regex itself is write-once-read-never).

> > There is a first stab at documenting the validator module API, more to come (it
> > doesn't compile right now).
> >
> > It contains a pgindent and pgperltidy run to keep things as close to in final
> > sync as we can to catch things like the curl deprecation macro mentioned above
> > early.

The rest of your second comments patch has been incorporated now, with
the exception of the following hunk:

    - read($read_fh, $port, 7) // die "failed to read port number: $!";
    + read($read_fh, $port, 7) or die "failed to read port number: $!";

read() doesn't set $! unless it returns undef, according to the docs
[1]. A zero read (i.e. immediate EOF) will be handled further down.

> > To further pick away at this huge patch I propose to merge the SASL message
> > length hunk which can be extracted separately.

I've pulled this out into 0001.

Thanks,
--Jacob

[1] https://perldoc.perl.org/functions/read

  134. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Alexander Lakhin <exclusion@gmail.com> — 2024-10-15T18:00:00Z

    Hello Peter,

11.09.2024 10:37, Peter Eisentraut wrote:
>
> This has been committed.
>

I've discovered that starting from 0785d1b8b,
make check -C src/bin/pg_combinebackup
fails under Valgrind, with the following diagnostics:
2024-10-15 14:29:52.883 UTC [3338981] 002_compare_backups.pl STATEMENT:  UPLOAD_MANIFEST
==00:00:00:20.028 3338981== Conditional jump or move depends on uninitialised value(s)
==00:00:00:20.028 3338981==    at 0xA3E68F: json_lex (jsonapi.c:1496)
==00:00:00:20.028 3338981==    by 0xA3ED13: json_lex (jsonapi.c:1666)
==00:00:00:20.028 3338981==    by 0xA3D5AF: pg_parse_json_incremental (jsonapi.c:822)
==00:00:00:20.028 3338981==    by 0xA40ECF: json_parse_manifest_incremental_chunk (parse_manifest.c:194)
==00:00:00:20.028 3338981==    by 0x31656B: FinalizeIncrementalManifest (basebackup_incremental.c:237)
==00:00:00:20.028 3338981==    by 0x73B4A4: UploadManifest (walsender.c:709)
==00:00:00:20.028 3338981==    by 0x73DF4A: exec_replication_command (walsender.c:2185)
==00:00:00:20.028 3338981==    by 0x7C58C3: PostgresMain (postgres.c:4762)
==00:00:00:20.028 3338981==    by 0x7BBDA7: BackendMain (backend_startup.c:107)
==00:00:00:20.028 3338981==    by 0x6CF60F: postmaster_child_launch (launch_backend.c:274)
==00:00:00:20.028 3338981==    by 0x6D546F: BackendStartup (postmaster.c:3415)
==00:00:00:20.028 3338981==    by 0x6D2B21: ServerLoop (postmaster.c:1648)
==00:00:00:20.028 3338981==

(Initializing
         dummy_lex.inc_state = NULL;
before
         partial_result = json_lex(&dummy_lex);
makes these TAP tests pass for me.)

Best regards,
Alexander

  135. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-15T18:10:21Z

    On Tue, Oct 15, 2024 at 11:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
> I've discovered that starting from 0785d1b8b,
> make check -C src/bin/pg_combinebackup
> fails under Valgrind, with the following diagnostics:

Yep, sorry for that (and thanks for the report!). It's currently
tracked over at [1], but I should have mentioned it here. The patch I
used is attached, renamed to not stress out the cfbot.

--Jacob

[1] https://www.postgresql.org/message-id/CAOYmi%2BkiiM83%3DH6YQ77NSSCtkGAzAnZfC0vZS%3DaLM9QZx%3DRn_A%40mail.gmail.com

  136. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-16T17:21:54Z

    On Thu, Oct 10, 2024 at 4:08 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Here's v30, which aims to fix the infinite-retry problem: you get a
> maximum of two OAuth connections to the server, and we won't prompt
> the user more than once per transport. (I still need to wrap my head
> around the retry behavior during transport negotiation.)

v31 folds in the remainder of the review patch (hooray!) and makes a
change to the 401 handling. If the server doesn't tell the client why
a token request failed, but we see that we got a 401 Unauthorized, the
error message now suggests that the oauth_client_secret setting is
unacceptable.

(If anyone out there happens to be testing this against real-world
implementations, I would love to get your feedback on this failure
mode. I can no longer get Entra ID to require a client secret during a
device flow, the way I used to be able to with Azure AD.)

I've also made the default device prompt translatable.

--Jacob

  137. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-10-17T06:28:36Z

    On 15.10.24 20:10, Jacob Champion wrote:
> On Tue, Oct 15, 2024 at 11:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
>> I've discovered that starting from 0785d1b8b,
>> make check -C src/bin/pg_combinebackup
>> fails under Valgrind, with the following diagnostics:
> 
> Yep, sorry for that (and thanks for the report!). It's currently
> tracked over at [1], but I should have mentioned it here. The patch I
> used is attached, renamed to not stress out the cfbot.

I have committed this fix.

  138. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-10-18T05:51:21Z

    Antonin Houska <ah@cybertec.at> wrote:

> I'd like to play with the code a bit and provide some review before or during
> the next CF. That will probably generate some more questions.

This is the 1st round, based on reading the code. I'll continue paying
attention to the project and possibly post some more comments in the future.


* Information on the new method should be added to pg_hba.conf.sample.method.


* Is it important that fe_oauth_state.token also contains the "Bearer"
  keyword? I'd expect only the actual token value here. The keyword can be
  added to the authentication message w/o storing it.

  The same applies to the 'token' structure in fe-auth-oauth-curl.c.


* Does PQdefaultAuthDataHook() have to be declared extern and exported via
  libpq/exports.txt ? Even if the user was interested in it, he can use
  PQgetAuthDataHook() to get the pointer (unless he already installed his
  custom hook).


* I wonder if the hooks (PQauthDataHook) can be implemented in a separate
  diff. Couldn't the first version of the feature be commitable without these
  hooks?


* Instead of allocating an instance of PQoauthBearerRequest, assigning it to
  fe_oauth_state.async_ctx, and eventually having to all its cleanup()
  function, wouldn't it be simpler to embed PQoauthBearerRequest as a member
  in fe_oauth_state ?


* oauth_validator_library is defined as PGC_SIGHUP - is that intentional?

  And regardless, the library appears to be loaded by every backend during
  authentication. Why isn't it loaded by postmaster like libraries listed in
  shared_preload_libraries? fork() would then ensure that the backends do have
  the library in their address space.


* pg_fe_run_oauth_flow()

  When first time here
			case OAUTH_STEP_TOKEN_REQUEST:
				if (!handle_token_response(actx, &state->token))
					goto error_return;

  the user hasn't been prompted yet so ISTM that the first token request must
  always fail. It seems more logical if the prompt is set to the user before
  sending the token request to the server. (Although the user probably won't
  be that fast to make the first request succeed, so consider this just a
  hint.)


* As long as I understand, the following comment would make sense:

diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
index f943a31cc08..97259fb5654 100644
--- a/src/interfaces/libpq/fe-auth-oauth.c
+++ b/src/interfaces/libpq/fe-auth-oauth.c
@@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
        switch (state->state)
        {
                case FE_OAUTH_INIT:
+                       /* Initial Client Response */
                        Assert(inputlen == -1);
 
                        if (!derive_discovery_uri(conn))

  Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
  better into oauth_init()? A side-effect of that might be (I only judge from
  reading the code, haven't tried to implement this suggestion) that
  oauth_exchange() would no longer return the SASL_ASYNC status. Furthermore,
  I'm not sure if pg_SASL_continue() can receive the SASL_ASYNC at all. So I
  wonder if moving that part from oauth_exchange() to oauth_init() would make
  the SASL_ASYNC state unnecessary.


* Finally, the user documentation is almost missing. I say that just for the
  sake of completeness, you obviously know it. (On the other hand, I think
  that the lack of user information might discourage some people from running
  the code and testing it.)

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  139. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-10-18T11:38:29Z

    > On 16 Oct 2024, at 19:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> v31 folds in the remainder of the review patch (hooray!) and makes a
> change to the 401 handling. If the server doesn't tell the client why
> a token request failed, but we see that we got a 401 Unauthorized, the
> error message now suggests that the oauth_client_secret setting is
> unacceptable.

+1

The attached diff expands the documentation a little to chip away at the
non-trivial task of documenting this feature.  I will keep working on this part
as well reviewing the rest of the patchset.  A few more comments are added as
well but nothing groundbreaking.  I've attached your v31 vanilla as well to
keep the CFBot happy.

A few small comments on things in the attached review 0005 comment:

-    (errcode(ERRCODE_PROTOCOL_VIOLATION),
-     errmsg("client selected an invalid SASL authentication mechanism")));
+    errcode(ERRCODE_PROTOCOL_VIOLATION),
+    errmsg("client selected an invalid SASL authentication mechanism"));
Might be a nitpick but in new code I think we should use the new style of
ereport() calls without extra parens around the aux functions.

In validate() it seems to me we should clear out ret->authn_id on failure to
pair belts with suspenders. Fixed by calling explicit_bzero on it in the error
path.

parse_hba_line() didn't enforce mandatory parameters, and one could configure a
usermap and skipping of the usermap at the same time which seems nonsensical.

src/test/modules/oauth_validator didn't check for PG_TEST_EXTRA which is should
since it opens a server.  We should also not test capabilities with if-elsif
since they are really separate things.

A few smaller bits and pieces of cleanup is also included.

--
Daniel Gustafsson

  140. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-10-23T15:46:14Z

    I have pushed the 0001 patch to make the max SASL message length configurable
per mechanism, so re-sending v31 as v32 without that patch to keep CFbot et.al
happy. There are no other changes over v31.
 
--
Daniel Gustafsson

  141. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-23T16:46:23Z

    On Wed, Oct 23, 2024 at 8:46 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> I have pushed the 0001 patch to make the max SASL message length configurable
> per mechanism, so re-sending v31 as v32 without that patch to keep CFbot et.al
> happy. There are no other changes over v31.

Awesome, thanks! I'm still working on the feedback from you and
Antonin upthread, so let's take this opportunity to squash together
the two main patches as one, horrifyingly large, v33-0001.

I've also rearranged the patch order, to tweak a recorded failure
message in the Python tests. No other changes have been made to the
OAuth implementation.

--Jacob

  142. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-23T22:40:57Z

    On Thu, Oct 17, 2024 at 10:51 PM Antonin Houska <ah@cybertec.at> wrote:
> This is the 1st round, based on reading the code. I'll continue paying
> attention to the project and possibly post some more comments in the future.

Thanks again for the reviews!

> * Information on the new method should be added to pg_hba.conf.sample.method.

Whoops, this will be fixed in v34.

> * Is it important that fe_oauth_state.token also contains the "Bearer"
>   keyword? I'd expect only the actual token value here. The keyword can be
>   added to the authentication message w/o storing it.
>
>   The same applies to the 'token' structure in fe-auth-oauth-curl.c.

Excellent question; I've waffled a bit on that myself. I think you're
probably right, but here's some background on why I originally made
that decision.

RFC 7628 defines not only OAUTHBEARER but also a generic template for
future OAuth-based SASL methods, and as part of that, the definition
of the "auth" key is incredibly vague:

      auth (REQUIRED):  The payload that would be in the HTTP
         Authorization header if this OAuth exchange was being carried
         out over HTTP.

I was worried that forcing a specific format would prevent future
extensibility, if say the Bearer scheme were updated to add additional
auth-params. I was also wondering if maybe a future specification
would allow OAUTHBEARER to carry a different scheme altogether, such
as DPoP [1].

However:
- auth-param support for Bearer was considered at the draft stage and
explicitly removed, with the old drafts stating "If additional
parameters are needed in the future, a different scheme would need to
be defined."
- I think the intent of RFC 7628 is that a new SASL mechanism will be
named for each new scheme (even if the new scheme shares all of the
bones of the old one). So DPoP tokens wouldn't piggyback on
OAUTHBEARER, and instead something like an OAUTHDPOP mech would need
to be defined.

So: the additional complexity in the current API is probably a YAGNI
violation, and I should just hardcode the Bearer format as you
suggest. Any future OAuth SASL mechanisms we support will have to go
through a different PQAUTHDATA type, e.g. PQAUTHDATA_OAUTH_DPOP_TOKEN.
And I'll need to make sure that I'm not improperly coupling the
concepts elsewhere in the API.

> * Does PQdefaultAuthDataHook() have to be declared extern and exported via
>   libpq/exports.txt ? Even if the user was interested in it, he can use
>   PQgetAuthDataHook() to get the pointer (unless he already installed his
>   custom hook).

I guess I don't have a strongly held opinion, but is there a good
reason not to? Exposing it means that a client application may answer
questions like "is the current hook set to the default?" and so on.
IME, hook-chain maintenance is not a lot of fun in general, and having
more visibility can be nice for third-party developers.

> * I wonder if the hooks (PQauthDataHook) can be implemented in a separate
>   diff. Couldn't the first version of the feature be commitable without these
>   hooks?

I am more than happy to split things up as needed! But in the end, I
think this is a question that can only be answered by the first brave
committer to take a bite. :)

(The original patchset didn't have these hooks; they were added as a
compromise, to prevent the builtin implementation from having to be
all things for all people.)

> * Instead of allocating an instance of PQoauthBearerRequest, assigning it to
>   fe_oauth_state.async_ctx, and eventually having to all its cleanup()
>   function, wouldn't it be simpler to embed PQoauthBearerRequest as a member
>   in fe_oauth_state ?

Hmm, that would maybe be simpler. But you'd still have to call
cleanup() and set the async_ctx, right? The primary gain would be in
reducing the number of malloc calls.

> * oauth_validator_library is defined as PGC_SIGHUP - is that intentional?

Yes, I think it's going to be important to let DBAs migrate their
authentication modules without a full restart. That probably deserves
more explicit testing, now that you mention it. Is there a specific
concern that you have with that?

>   And regardless, the library appears to be loaded by every backend during
>   authentication. Why isn't it loaded by postmaster like libraries listed in
>   shared_preload_libraries? fork() would then ensure that the backends do have
>   the library in their address space.

It _can_ be, if you want -- there's nothing that I know of preventing
the validator from also being preloaded with its own _PG_init(), is
there? But I don't think it's a good idea to force that, for the same
reason we want to allow SIGHUP.

> * pg_fe_run_oauth_flow()
>
>   When first time here
>                         case OAUTH_STEP_TOKEN_REQUEST:
>                                 if (!handle_token_response(actx, &state->token))
>                                         goto error_return;
>
>   the user hasn't been prompted yet so ISTM that the first token request must
>   always fail. It seems more logical if the prompt is set to the user before
>   sending the token request to the server. (Although the user probably won't
>   be that fast to make the first request succeed, so consider this just a
>   hint.)

That's also intentional -- if the first token response fails for a
reason _other_ than "we're waiting for the user", then we want to
immediately fail hard instead of making them dig out their phone and
go on a two-minute trip, because they're going to come back and find
that it was all for nothing.

There's a comment immediately below the part you quoted that mentions
this briefly; maybe I should move it up a bit?

> * As long as I understand, the following comment would make sense:
>
> diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
> index f943a31cc08..97259fb5654 100644
> --- a/src/interfaces/libpq/fe-auth-oauth.c
> +++ b/src/interfaces/libpq/fe-auth-oauth.c
> @@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
>         switch (state->state)
>         {
>                 case FE_OAUTH_INIT:
> +                       /* Initial Client Response */
>                         Assert(inputlen == -1);
>
>                         if (!derive_discovery_uri(conn))

There are multiple "initial client response" cases, though. What
questions are you hoping to clarify with the comment? Maybe we can
find a more direct answer.

>   Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
>   better into oauth_init()?

oauth_init() is the mechanism initialization for the SASL framework
itself, which is shared with SCRAM. In the current architecture, the
init callback doesn't take the initial client response into
consideration at all.

Generating the client response is up to the exchange callback -- and
even if we moved the SASL_ASYNC processing elsewhere, I don't think we
can get rid of its added complexity. Something has to signal upwards
that it's time to transfer control to an async engine. And we can't
make the asynchronicity a static attribute of the mechanism itself,
because we can skip the flow if something gives us a cached token.

> * Finally, the user documentation is almost missing. I say that just for the
>   sake of completeness, you obviously know it. (On the other hand, I think
>   that the lack of user information might discourage some people from running
>   the code and testing it.)

Yeah, the catch-22 of writing huge features... By the way, if anyone's
reading along and dissuaded by the lack of docs, please say so!
(Daniel has been helping me out so much with the docs; thanks again,
Daniel.)

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc9449

  143. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-24T22:28:01Z

    On Fri, Oct 18, 2024 at 4:38 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> In validate() it seems to me we should clear out ret->authn_id on failure to
> pair belts with suspenders. Fixed by calling explicit_bzero on it in the error
> path.

The new hunk says:

> cleanup:
>     /*
>      * Clear and free the validation result from the validator module once
>      * we're done with it to avoid accidental re-use.
>      */
>     if (ret->authn_id != NULL)
>     {
>         explicit_bzero(ret->authn_id, strlen(ret->authn_id));
>         pfree(ret->authn_id);
>     }
>     pfree(ret);

But I'm not clear on what's being protected against. Which code would
reuse this result?

Thanks,
--Jacob

  144. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-25T18:22:07Z

    On Wed, Oct 23, 2024 at 3:40 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> > * Information on the new method should be added to pg_hba.conf.sample.method.
>
> Whoops, this will be fixed in v34.

...which is now attached. This should also fix the build failure for
the docs themselves.

I have combed almost all of Daniel's feedback backwards into the main
patch (just the new bzero code remains, with the open question
upthread), and I've made further edits to flesh out more of the
documentation. A diff is provided so you don't have to go looking for
the doc changes. Feedback on the wording and level of detail is very
welcome!

Next up is, hopefully, url-encoding. I hadn't realized what an
absolute mess that would be [1].

Thanks,
--Jacob

[1] https://github.com/oauth-wg/oauth-v2-1/issues/128#issuecomment-1879632883

  145. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-10-28T13:24:00Z

    > On 25 Oct 2024, at 20:22, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> I have combed almost all of Daniel's feedback backwards into the main
> patch (just the new bzero code remains, with the open question
> upthread),

Re-reading I can't see a vector there, I guess I am just scarred from what
seemed to be harmless leaks in auth codepaths and treat every bit as
potentially important.  Feel free to drop from the patchset for now.

> Next up is, hopefully, url-encoding. I hadn't realized what an
> absolute mess that would be [1].

Everything and anything involving urls is a hot mess =/

Looking more at the patchset I think we need to apply conditional compilation
of the backend for oauth like how we do with other opt-in schemes in configure
and meson.  The attached .txt has a diff for making --with-oauth a requirement
for compiling support into backend libpq.

--
Daniel Gustafsson

  146. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-28T16:09:13Z

    On Mon, Oct 28, 2024 at 6:24 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On 25 Oct 2024, at 20:22, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> > I have combed almost all of Daniel's feedback backwards into the main
> > patch (just the new bzero code remains, with the open question
> > upthread),
>
> Re-reading I can't see a vector there, I guess I am just scarred from what
> seemed to be harmless leaks in auth codepaths and treat every bit as
> potentially important.  Feel free to drop from the patchset for now.

Okay. For authn_id specifically, which isn't secret and doesn't have
any power unless it's somehow copied into the ClientConnectionInfo,
I'm not sure that the bzero() gives us much. But I do see value in
clearing out, say, the Bearer token once we're finished with it.

Also in this validate() code path, I'm taking a look at the added
memory management with the pfree():
1. Should we add any more ceremony to the returned struct, to try to
ensure that the ABI matches? Or is it good enough to declare that
modules need to be compiled against a specific server version?
2. Should we split off a separate memory context to contain
allocations made by the validator?

> Looking more at the patchset I think we need to apply conditional compilation
> of the backend for oauth like how we do with other opt-in schemes in configure
> and meson.  The attached .txt has a diff for making --with-oauth a requirement
> for compiling support into backend libpq.

Do we get the flexibility we need with that approach? With other
opt-in schemes, the backend and the frontend both need some sort of
third-party dependency, but that's not true for OAuth. I could see
some people wanting to support an offline token validator on the
server side but not wanting to build the HTTP dependency into their
clients.

I was considering going in the opposite direction: With the client
hooks, a user could plug in their own implementation without ever
having to touch the built-in flow, and I'm wondering if --with-oauth
should really just be --with-builtin-oauth or similar. Then if the
server sends OAUTHBEARER, the client only complains if it doesn't have
a flow available to use, rather than checking USE_OAUTH. This kind of
ties into the other big open question of "what do we do about users
that don't want the additional overhead of something they're not
using?"

--Jacob

  147. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-10-29T10:52:23Z

    > On 28 Oct 2024, at 17:09, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Mon, Oct 28, 2024 at 6:24 AM Daniel Gustafsson <daniel@yesql.se> wrote:

>> Looking more at the patchset I think we need to apply conditional compilation
>> of the backend for oauth like how we do with other opt-in schemes in configure
>> and meson.  The attached .txt has a diff for making --with-oauth a requirement
>> for compiling support into backend libpq.
> 
> Do we get the flexibility we need with that approach? With other
> opt-in schemes, the backend and the frontend both need some sort of
> third-party dependency, but that's not true for OAuth. I could see
> some people wanting to support an offline token validator on the
> server side but not wanting to build the HTTP dependency into their
> clients.

Currently we don't support any conditional compilation which only affects
backend or frontend, all --without-XXX flags turn it off for both.  Maybe this
is something which should change but I'm not sure that property should be
altered as part of a patch rather than discussed on its own merit.

> I was considering going in the opposite direction: With the client
> hooks, a user could plug in their own implementation without ever
> having to touch the built-in flow, and I'm wondering if --with-oauth
> should really just be --with-builtin-oauth or similar. Then if the
> server sends OAUTHBEARER, the client only complains if it doesn't have
> a flow available to use, rather than checking USE_OAUTH. This kind of
> ties into the other big open question of "what do we do about users
> that don't want the additional overhead of something they're not
> using?"

We already know that GSS cause measurable performance impact on connections
even when compiled but not in use [0], so I think we should be careful about
piling on more.

--
Daniel Gustafsson

[0] 20240610181212.auytluwmbfl7lb5n@awork3.anarazel.de

  148. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-29T16:40:21Z

    On Tue, Oct 29, 2024 at 3:52 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Currently we don't support any conditional compilation which only affects
> backend or frontend, all --without-XXX flags turn it off for both.

I don't think that's strictly true; see --with-pam which affects only
server-side code, since the hard part is in the server. Similarly,
--with-oauth currently affects only client-side code.

But in any case, that confusion is why I'm proposing a change to the
option name. I chose --with-oauth way before the architecture
solidified, and it doesn't reflect reality anymore. OAuth support on
the server side doesn't require Curl, and likely never will. So if you
want to support that on a Windows server, it's going to be strange if
we also force you to build the client with a libcurl dependency that
we won't even make use of on that platform.

> We already know that GSS cause measurable performance impact on connections
> even when compiled but not in use [0], so I think we should be careful about
> piling on more.

I agree, but if the server asks for OAUTHBEARER, that's the end of it.
Either the client supports OAuth and initiates a token flow, or it
doesn't and the connection fails. That's very different from the
client-initiated transport negotiation.

On the other hand, if we're concerned about the link-time overhead
(time and/or RAM) of the new dependency, I think that's going to need
something different from a build-time switch. My guess is that
maintainers are only going to want to ship one libpq.

Thanks,
--Jacob

  149. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-10-29T17:41:27Z

    > On 29 Oct 2024, at 17:40, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Tue, Oct 29, 2024 at 3:52 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>> Currently we don't support any conditional compilation which only affects
>> backend or frontend, all --without-XXX flags turn it off for both.
> 
> I don't think that's strictly true; see --with-pam which affects only
> server-side code, since the hard part is in the server. Similarly,
> --with-oauth currently affects only client-side code.

Fair, maybe it's an unwarranted concern.  Question is though, if we added PAM
today would we have done the same?

> But in any case, that confusion is why I'm proposing a change to the
> option name.

+1

--
Daniel Gustafsson

  150. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-29T20:34:00Z

    On Fri, Oct 25, 2024 at 11:22 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Next up is, hopefully, url-encoding. I hadn't realized what an
> absolute mess that would be [1].

Here is v35, which attempts to perform URL encoding by almost entirely
deferring to Curl, in the naive hope that provider incompatibilities
with libcurl will be taken more seriously than incompatibilities with
a brand-new Postgres feature. I'm not thrilled that the IETF chose to
defer this part of the spec to WHATWG.

Additionally,
- the rest of the feedback patch has been incorporated, with
modifications to the bzero portion (which now focuses on clearing
`token` rather than `authn_id`)
- documentation for the validate_cb callback has been updated to
match, plus additional expansion
- markPQExpBufferBroken() has been promoted to the pqexpbuffer.h API,
because it happens to be useful for the encoding patch
- some duplication of the Authorization code has been refactored away
- "empty" (which is to say, default) scopes are now explicitly tested

Next up will be Antonin's suggested change to the Bearer handling, as
well as previously-discussed changes to the --with-oauth build option.

Thanks,
--Jacob

  151. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-29T20:39:36Z

    On Tue, Oct 29, 2024 at 10:41 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Question is though, if we added PAM
> today would we have done the same?

I assume so; the client can't tell PAM apart from LDAP or any other
plaintext method. (In the same vein, the server can't tell if the
client uses libcurl to grab a token, or something entirely different.)

--Jacob

  152. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Antonin Houska <ah@cybertec.at> — 2024-10-31T11:05:46Z

    Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> On Thu, Oct 17, 2024 at 10:51 PM Antonin Houska <ah@cybertec.at> wrote:
> > * oauth_validator_library is defined as PGC_SIGHUP - is that intentional?
> 
> Yes, I think it's going to be important to let DBAs migrate their
> authentication modules without a full restart. That probably deserves
> more explicit testing, now that you mention it. Is there a specific
> concern that you have with that?

No concern. I was just trying to imagine when the module needs to be changed.

> >   And regardless, the library appears to be loaded by every backend during
> >   authentication. Why isn't it loaded by postmaster like libraries listed in
> >   shared_preload_libraries? fork() would then ensure that the backends do have
> >   the library in their address space.
> 
> It _can_ be, if you want -- there's nothing that I know of preventing
> the validator from also being preloaded with its own _PG_init(), is
> there? But I don't think it's a good idea to force that, for the same
> reason we want to allow SIGHUP.

Loading the library by postmaster does not prevent the backends from reloading
it on SIGHUP later. I was simply concerned about performance. (I proposed
loading the library at another stage of backend initialization rather than
adding _PG_init() to it.)

> > * pg_fe_run_oauth_flow()
> >
> >   When first time here
> >                         case OAUTH_STEP_TOKEN_REQUEST:
> >                                 if (!handle_token_response(actx, &state->token))
> >                                         goto error_return;
> >
> >   the user hasn't been prompted yet so ISTM that the first token request must
> >   always fail. It seems more logical if the prompt is set to the user before
> >   sending the token request to the server. (Although the user probably won't
> >   be that fast to make the first request succeed, so consider this just a
> >   hint.)
> 
> That's also intentional -- if the first token response fails for a
> reason _other_ than "we're waiting for the user", then we want to
> immediately fail hard instead of making them dig out their phone and
> go on a two-minute trip, because they're going to come back and find
> that it was all for nothing.
> 
> There's a comment immediately below the part you quoted that mentions
> this briefly; maybe I should move it up a bit?

That's fine, I understand now.

> > * As long as I understand, the following comment would make sense:
> >
> > diff --git a/src/interfaces/libpq/fe-auth-oauth.c b/src/interfaces/libpq/fe-auth-oauth.c
> > index f943a31cc08..97259fb5654 100644
> > --- a/src/interfaces/libpq/fe-auth-oauth.c
> > +++ b/src/interfaces/libpq/fe-auth-oauth.c
> > @@ -518,6 +518,7 @@ oauth_exchange(void *opaq, bool final,
> >         switch (state->state)
> >         {
> >                 case FE_OAUTH_INIT:
> > +                       /* Initial Client Response */
> >                         Assert(inputlen == -1);
> >
> >                         if (!derive_discovery_uri(conn))
> 
> There are multiple "initial client response" cases, though. What
> questions are you hoping to clarify with the comment? Maybe we can
> find a more direct answer.

Easiness of reading is the only "question" here :-) It's might not always be
obvious why a variable should have some particular value. In general, the
Assert() statements are almost always preceded with a comment in the PG
source.

> >   Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
> >   better into oauth_init()?
> 
> oauth_init() is the mechanism initialization for the SASL framework
> itself, which is shared with SCRAM. In the current architecture, the
> init callback doesn't take the initial client response into
> consideration at all.

Sure. The FE_OAUTH_INIT branch in oauth_exchange() (FE) also does not generate
the initial client response.

Based on reading the SCRAM implementation, I concluded that the init()
callback can do authentication method specific things, but unlike exchange()
it does not generate any output.

> Generating the client response is up to the exchange callback -- and
> even if we moved the SASL_ASYNC processing elsewhere, I don't think we
> can get rid of its added complexity. Something has to signal upwards
> that it's time to transfer control to an async engine. And we can't
> make the asynchronicity a static attribute of the mechanism itself,
> because we can skip the flow if something gives us a cached token.

I didn't want to skip the flow. I thought that the init() callback could be
made responsible for getting the token, but forgot that it still needs some
way to signal to the caller that the async flow is needed.

Anyway, are you sure that pg_SASL_continue() can also receive the SASL_ASYNC
value from oauth_exchange()? My understanding is that pg_SASL_init() receives
it if there is no token, but after that, oauth_exchange() is not called util
the token is available, and thus it should not return SASL_ASYNC anymore.

-- 
Antonin Houska
Web: https://www.cybertec-postgresql.com

  153. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-10-31T16:24:11Z

    On Thu, Oct 31, 2024 at 4:05 AM Antonin Houska <ah@cybertec.at> wrote:
> > >   And regardless, the library appears to be loaded by every backend during
> > >   authentication. Why isn't it loaded by postmaster like libraries listed in
> > >   shared_preload_libraries? fork() would then ensure that the backends do have
> > >   the library in their address space.
> >
> > It _can_ be, if you want -- there's nothing that I know of preventing
> > the validator from also being preloaded with its own _PG_init(), is
> > there? But I don't think it's a good idea to force that, for the same
> > reason we want to allow SIGHUP.
>
> Loading the library by postmaster does not prevent the backends from reloading
> it on SIGHUP later. I was simply concerned about performance. (I proposed
> loading the library at another stage of backend initialization rather than
> adding _PG_init() to it.)

Okay. I think this is going to be one of the slower authentication
methods by necessity: the builtin flow in libpq requires a human in
the loop, and an online validator is going to be making several HTTP
calls from the backend. So if it turns out later that we need to
optimize the backend logic, I'd prefer to have a case study in hand;
otherwise I think we're likely to optimize the wrong things.

> Easiness of reading is the only "question" here :-) It's might not always be
> obvious why a variable should have some particular value. In general, the
> Assert() statements are almost always preceded with a comment in the PG
> source.

Oh, an assertion label! I can absolutely add one; I originally thought
you were proposing a label for the case itself.

> > >   Or, doesn't the FE_OAUTH_INIT branch of the switch statement actually fit
> > >   better into oauth_init()?
> >
> > oauth_init() is the mechanism initialization for the SASL framework
> > itself, which is shared with SCRAM. In the current architecture, the
> > init callback doesn't take the initial client response into
> > consideration at all.
>
> Sure. The FE_OAUTH_INIT branch in oauth_exchange() (FE) also does not generate
> the initial client response.

It might, if it ends up falling through to FE_OAUTH_REQUESTING_TOKEN.
There are two paths that can do that: the case where we have no
discovery URI, and the case where a custom user flow returns a token
synchronously (it was probably cached).

> Anyway, are you sure that pg_SASL_continue() can also receive the SASL_ASYNC
> value from oauth_exchange()? My understanding is that pg_SASL_init() receives
> it if there is no token, but after that, oauth_exchange() is not called util
> the token is available, and thus it should not return SASL_ASYNC anymore.

Correct -- the only way for the current implementation of the
OAUTHBEARER mechanism to return SASL_ASYNC is during the very first
call. That's not an assumption I want to put into the higher levels,
though; I think Michael will be unhappy with me if I introduce
additional SASL coupling after the decoupling work that's been done
over the last few releases. :D

Thanks again,
--Jacob

  154. Re: [PoC] Federated Authn/z with OAUTHBEARER

    jian he <jian.universality@gmail.com> — 2024-11-04T05:00:00Z

    Hi there.
zero knowledge of Oath, just reading through the v35-0001.
forgive me if my comments are naive.

+static int
+parse_interval(struct async_ctx *actx, const char *interval_str)
+{
+ double parsed;
+ int cnt;
+
+ /*
+ * The JSON lexer has already validated the number, which is stricter than
+ * the %f format, so we should be good to use sscanf().
+ */
+ cnt = sscanf(interval_str, "%lf", &parsed);
+
+ if (cnt != 1)
+ {
+ /*
+ * Either the lexer screwed up or our assumption above isn't true, and
+ * either way a developer needs to take a look.
+ */
+ Assert(cnt == 1);
+ return 1; /* don't fall through in release builds */
+ }
+
+ parsed = ceil(parsed);
+
+ if (parsed < 1)
+ return actx->debugging ? 0 : 1;
+
+ else if (INT_MAX <= parsed)
+ return INT_MAX;
+
+ return parsed;
+}
The above Assert looks very wrong to me.

we can also use PG_INT32_MAX, instead of INT_MAX
(generally i think PG_INT32_MAX looks more intuitive to me)


+/*
+ * The Device Authorization response, described by RFC 8628:
+ *
+ *     https://www.rfc-editor.org/rfc/rfc8628#section-3.2
+ */
+struct device_authz
+{
+ char   *device_code;
+ char   *user_code;
+ char   *verification_uri;
+ char   *interval_str;
+
+ /* Fields below are parsed from the corresponding string above. */
+ int interval;
+};

click through the link https://www.rfc-editor.org/rfc/rfc8628#section-3.2
it says
"
   expires_in
      REQUIRED.  The lifetime in seconds of the "device_code" and
      "user_code".
   interval
      OPTIONAL.  The minimum amount of time in seconds that the client
      SHOULD wait between polling requests to the token endpoint.  If no
      value is provided, clients MUST use 5 as the default.
"
these two fields seem to differ from struct device_authz.

  155. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-11-04T09:11:35Z

    > On 4 Nov 2024, at 06:00, jian he <jian.universality@gmail.com> wrote:

> + if (cnt != 1)
> + {
> + /*
> + * Either the lexer screwed up or our assumption above isn't true, and
> + * either way a developer needs to take a look.
> + */
> + Assert(cnt == 1);
> + return 1; /* don't fall through in release builds */
> + }

> The above Assert looks very wrong to me.

I think the point is to fail hard in development builds to ensure whatever
caused the disconnect between the json lexer and sscanf parsing is looked at.
It should probably be changed to Assert(false); which is the common pattern for
erroring out like this.

--
Daniel Gustafsson

  156. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-05T23:07:31Z

    On Sun, Nov 3, 2024 at 9:00 PM jian he <jian.universality@gmail.com> wrote:
> The above Assert looks very wrong to me.

I can switch to Assert(false) if that's preferred, but it makes part
of the libc assert() report useless. (I wish we had more fluent ways
to say "this shouldn't happen, but if it does, we still need to get
out safely.")

> we can also use PG_INT32_MAX, instead of INT_MAX
> (generally i think PG_INT32_MAX looks more intuitive to me)

That's a fixed-width max; we want the maximum for the `int` type here.

>    expires_in
>       REQUIRED.  The lifetime in seconds of the "device_code" and
>       "user_code".
>    interval
>       OPTIONAL.  The minimum amount of time in seconds that the client
>       SHOULD wait between polling requests to the token endpoint.  If no
>       value is provided, clients MUST use 5 as the default.
> "
> these two fields seem to differ from struct device_authz.

Yeah, Daniel and I had talked about being stricter about REQUIRED
fields that are not currently used. There's a comment making note of
this in parse_device_authz(). The v1 code will need to make expires_in
REQUIRED, so that future developers can develop features that depend
on it without worrying about breaking
currently-working-but-noncompliant deployments. (And if there are any
noncompliant deployments out there now, we need to know about them so
we can have that explicit discussion.)

Thanks,
--Jacob

  157. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-05T23:33:28Z

    On Tue, Oct 29, 2024 at 1:34 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Next up will be Antonin's suggested change to the Bearer handling, as
> well as previously-discussed changes to the --with-oauth build option.

Done in v36, attached. The Bearer scheme is enforced in all cases
except initial discovery.

--with-builtin-oauth=curl is now the way to build with the
libcurl-based Device Authorization flow. If you don't want to build
with that option, you can still use the callback hooks for your own
flow. (libpq will give you a new error message if the server requires
OAUTHBEARER but you have no flows installed.) There is also a related
002_client.pl, which provides a basic framework for testing the hooks
without an authorization server. I expect to flesh that out more.

This also means that fe-auth-oauth.c is now always incorporated into
the client build. So I had to fix the SOCKET/int confusion on Windows
in libpq-fe.h: it now has a dependency on <winsock2.h>. I don't really
like that, and if anyone has a way to decouple those safely, I'm all
ears.

Thanks,
--Jacob

  158. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-05T23:39:43Z

    On Tue, Nov 5, 2024 at 3:33 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Done in v36, attached.

Forgot to draw attention to this part:

>     +# XXX libcurl must link after libgssapi_krb5 on FreeBSD to avoid segfaults
>     +# during gss_acquire_cred(). This is possibly related to Curl's Heimdal
>     +# dependency on that platform?

Best I can tell, libpq for FreeBSD has a dependency diamond for GSS
symbols: libpq links against MIT krb5, libcurl links against Heimdal,
libpq links against libcurl. Link order becomes critical to avoid
nasty segfaults, but I have not dug deeply into the root cause.

--Jacob

  159. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-11-08T09:21:07Z

    On 06.11.24 00:33, Jacob Champion wrote:
> Done in v36, attached.

Assorted review comments from me:

Everything in the commit message between

     = Debug Mode =

and

     Several TODOs:

should be moved to the documentation.  In some cases, it already is,
but it doesn't always have the same level of detail.

(You could point from the commit message to .sgml files if you want to
highlight usage instructions, but I don't think this is generally
necessary.)

* config/programs.m4

Can we do the libcurl detection using pkg-config only?  Seems simpler,
and maintains consistency to meson.

* doc/src/sgml/client-auth.sgml

In the list of terms (this could be a <variablelist>), state how these
terms map to a PostgreSQL installation.  You already explain what the
client and the resource server are, but not who the resource owner is
and what the authorization server is.  It would also be good to be
explicit and upfront that the authorization server is a third-party
component that needs to be obtained separately.

trust_validator_authz: Personally, I'm not a fan of the "authz" and
"authn" abbreviations.  I know this is security jargon.  But are
regular users going to understand this?  Can we just spell it out?

* doc/src/sgml/config.sgml

Also here maybe state that these OAuth libraries have to be obtained
separately.

* doc/src/sgml/installation.sgml

I find the way the installation options are structured a bit odd.  I
would have expected --with-libcurl and -Dlibcurl (or --with-curl and
-Dcurl).  These build options usually just say, use this library.  We
don't spell out what, for example, libldap is used for, we just use it
and enable all the features that require it.

* doc/src/sgml/libpq.sgml

Maybe oauth_issuer should be oauth_issuer_url?  Otherwise one might
expect to just write "google" here or something.  Or there might be
other ways to contact an issuer in the future?  Just a thought.

* doc/src/sgml/oauth-validators.sgml

This chapter says "libpq" several times, but I think this is a server
side plugin, so libpq does not participate.  Check please.

* src/backend/libpq/auth-oauth.c

I'm confused by the use of PG_MAX_AUTH_TOKEN_LENGTH in the
pg_be_oauth_mech definition.  What does that mean?

+#define KVSEP 0x01
+#define AUTH_KEY "auth"
+#define BEARER_SCHEME "Bearer "

Add comments to these.

Also, add comments to all functions defined here that don't have one
yet.

* src/backend/utils/misc/guc_tables.c

Why is oauth_validator_library GUC_NOT_IN_SAMPLE?

Also, shouldn't this be an hba option instead?  What if you want to
use different validators for different connections?

* src/interfaces/libpq/fe-auth-oauth-curl.c

The CURL_IGNORE_DEPRECATION thing needs clarification.  Is that in
progress?

+#define MAX_OAUTH_RESPONSE_SIZE (1024 * 1024)

Add a comment about why this value.

+   union
+   {
+       char      **scalar;     /* for all scalar types */
+       struct curl_slist **array;  /* for type == JSON_TOKEN_ARRAY_START */
+   };

This is an anonymous union, which requires C11.  Strangely, I cannot
get clang to warn about this with -Wc11-extensions.  Probably better
to fix anyway.  (The trailing supported MSVC versions don't support
C11 yet.)

* src/interfaces/libpq/fe-auth.h

+extern const pg_fe_sasl_mech pg_oauth_mech;

Should this rather be in fe-auth-oauth.h?

* src/interfaces/libpq/libpq-fe.h

The naming scheme of types and functions in this file is clearly
obscure and has grown randomly over time.  But at least my intuition
is that the preferred way is

types start with PG
function start with PQ

and the next letter is usually lower case. (PQconnectdb, PQhost,
PGconn, PQresult)

Maybe check your additions against that.

* src/interfaces/libpq/pqexpbuffer.c
* src/interfaces/libpq/pqexpbuffer.h

Let's try to do this without opening up additional APIs here.

This is only used once, in append_urlencoded(), and there are other
ways to communicate errors, for example returning a bool.

* src/test/modules/oauth_validator/

Everything in this directory needs more comments, at least on a file
level.

Add a README in this directory.  Also update the README in the upper
directory.

* src/test/modules/oauth_validator/t/001_server.pl

On Cirrus CI Windows task, this test reports SKIP.  Can't tell why,
because the log is not kept.  I suppose you expect this to work on
Windows (but see my comment below), so it would be good to get this
test running.

* src/test/modules/oauth_validator/t/002_client.pl

+my $issuer = "https://127.0.0.1:54321";

Use PostgreSQL::Test::Cluster::get_free_port() instead of hardcoding
port numbers.

Or is this a real port?  I don't see it used anywhere else.

+   diag "running '" . join("' '", @cmd) . "'";

This should be "note" instead.  Otherwise it garbles the output.

* src/test/perl/PostgreSQL/Test/OAuthServer.pm

Add some comments to this file, what it's for.

Is this meant to work on Windows?  Just thinking, things like

kill(15, $self->{'pid'});

pgperlcritic complains:

src/test/perl/PostgreSQL/Test/OAuthServer.pm: Return value of flagged 
function ignored - read at line 39, column 2.

* src/tools/pgindent/typedefs.list

We don't need to typedef every locally used enum or similar into a
full typedef.  I suggest the following might be unnecessary:

AsyncAuthFunc
OAuthStep
fe_oauth_state_enum

  160. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-12T21:47:07Z

    On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Assorted review comments from me:

Thank you! I will cherry-pick some responses here and plan to address
the rest in a future patchset.

> trust_validator_authz: Personally, I'm not a fan of the "authz" and
> "authn" abbreviations.  I know this is security jargon.  But are
> regular users going to understand this?  Can we just spell it out?

Yes. That name's a holdover from the very first draft, actually.

Is "trust_validator_authorization" a great name in the first place?
The key concept is that user mapping is being delegated to the OAuth
system itself, so you'd better make sure that the validator has been
built to do that. (Anyone have any suggestions?)

> I find the way the installation options are structured a bit odd.  I
> would have expected --with-libcurl and -Dlibcurl (or --with-curl and
> -Dcurl).  These build options usually just say, use this library.

It's patterned directly off of -Dssl/--with-ssl (which I liberally
borrowed from) because the builtin client implementation used to have
multiple options for the library in use. I can change it if needed,
but I thought it'd be helpful for future devs if I didn't undo the
generalization.

> Maybe oauth_issuer should be oauth_issuer_url?  Otherwise one might
> expect to just write "google" here or something.  Or there might be
> other ways to contact an issuer in the future?  Just a thought.

More specifically this is an "issuer identifier", as defined by the
OAuth/OpenID discovery specs. It's a subset of a URL, and I want to
make sure users know how to differentiate between an "issuer" they
trust and the "discovery URI" that's in use for that issuer. They may
want to set one or the other -- a discovery URI is associated with
exactly one issuer, but unfortunately an issuer may have multiple
discovery URIs, which I'm actively working on. (There is also some
relation to the multiple-issuers problem mentioned below.)

> I'm confused by the use of PG_MAX_AUTH_TOKEN_LENGTH in the
> pg_be_oauth_mech definition.  What does that mean?

Just that Bearer tokens can be pretty long, so we don't want to limit
them to 1k like SCRAM does. 64k is probably overkill, but I've seen
anecdotal reports of tens of KBs and it seemed reasonable to match
what we're doing for GSS tokens.

> Also, shouldn't [oauth_validator_library] be an hba option instead?  What if you want to
> use different validators for different connections?

Yes. This is again the multiple-issuers problem; I will split that off
into its own email since this one's getting long. It has security
implications.

> The CURL_IGNORE_DEPRECATION thing needs clarification.  Is that in
> progress?

Thanks for the nudge, I've started a thread:

    https://curl.se/mail/lib-2024-11/0028.html

> This is an anonymous union, which requires C11.  Strangely, I cannot
> get clang to warn about this with -Wc11-extensions.  Probably better
> to fix anyway.  (The trailing supported MSVC versions don't support
> C11 yet.)

Oh, that's not going to be fun.

> This is only used once, in append_urlencoded(), and there are other
> ways to communicate errors, for example returning a bool.

I'd rather not introduce two parallel error indicators for the caller
to have to check for that particular part. But I can change over to
using the (identical!) termPQExpBuffer. I felt like the other API
signaled the intent a little better, though.

> On Cirrus CI Windows task, this test reports SKIP.  Can't tell why,
> because the log is not kept.  I suppose you expect this to work on
> Windows (but see my comment below)

No, builtin client support does not exist on Windows. If/when it's
added, the 001_server tests will need to be ported.

> +my $issuer = "https://127.0.0.1:54321";
>
> Use PostgreSQL::Test::Cluster::get_free_port() instead of hardcoding
> port numbers.
>
> Or is this a real port?  I don't see it used anywhere else.

It's not real; 002_client.pl doesn't start an authorization server at
all. I can make that more explicit.

> src/test/perl/PostgreSQL/Test/OAuthServer.pm: Return value of flagged
> function ignored - read at line 39, column 2.

So perlcritic recognizes "or" but not the "//" operator... Lovely.

Thanks!
--Jacob

  161. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-14T17:45:32Z

    On Tue, Nov 12, 2024 at 1:47 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> > Also, shouldn't [oauth_validator_library] be an hba option instead?  What if you want to
> > use different validators for different connections?
>
> Yes. This is again the multiple-issuers problem; I will split that off
> into its own email since this one's getting long. It has security
> implications.

Okay, so, how to use multiple issuers/providers. Here's my current
plan, with justification below:

1. libpq connection strings must specify exactly one issuer
2. the discovery document coming from the server must belong to that
libpq issuer
3. the HBA should allow a choice of discovery document and validator

= Current Bug =

First, I should point out a critical mistake I've made on the client
side: I treat oauth_issuer and oauth_client_id as if they can be
arbitrarily mixed and matched. Some of the providers I've been testing
do allow you to use one registered client across multiple issuers, but
that's the exception rather than the norm. Even if you have multiple
issuers available, you still expect your registered client to be
talking to only the provider you registered it with.

And you don't want the Postgres server to switch providers for you.
Imagine that you've registered a client application for use with a big
provider, and that provider has given you a client secret. You expect
to share that secret only with them, but with the current setup, if a
DBA wants to steal that secret from you, all they have to do is stand
up a provider of their own, and libpq will send the secret straight to
it instead. Great.

There's actually a worse scenario that's pointed out in the spec for
the Device Authorization flow [1]:

    Note that if an authorization server used with this flow is
    malicious, then it could perform a man-in-the-middle attack on the
    backchannel flow to another authorization server. [...] For this to
    be possible, the device manufacturer must either be the attacker and
    shipping a device intended to perform the man-in-the-middle attack,
    or be using an authorization server that is controlled by an
    attacker, possibly because the attacker compromised the
    authorization server used by the device.

Back when I implemented this, that paragraph seemed pointlessly
obvious: of course you must trust your authorization server. What I
missed was, the Postgres server MUST NOT be able to control the entry
point into the device flow, because that means a malicious DBA can
trivially start a device prompt with a different provider, forward you
all the details through the endpoint they control, and hope you're too
fatigued to notice the difference before clicking through. (This is
easier if that provider is one of the big ones that you're already
used to trusting.) Then they have a token with which to attack you on
a completely different platform.

So in my opinion, my patchset must be changed to require a trusted
issuer in the libpq connection string. The server can tell you which
discovery document to get from that issuer, and it can tell you which
scopes are required (as long as the user hasn't hardcoded those too),
but it shouldn't be able to force the client to talk to an arbitrary
provider or swap out issuers.

= Multiple Issuers =

Okay, with that out of the way, let's talk about multiple issuer support.

First, server-side. If a server wants different groups of
users/databases/etc. to go through different issuers, then it stands
to reason that a validator should be selectable in the HBA settings,
since a validator for Provider A may not have any clue how to validate
Provider B. I don't like the idea of pg_hba being used to load
arbitrary libraries, though; I think the superuser should have to
designate a pool of "blessed" validator libraries to load through a
GUC. As a UX improvement for the common case, maybe we don't require
the HBA to have an explicit validator parameter if the conf contains
exactly one blessed library.

In case someone does want to develop a multi-issuer validator (say, to
deal with the providers that have multiple issuers underneath their
umbrella), we need to make sure that the configured issuer in use is
available to the validator, so that they aren't susceptible to a
mix-up attack of their own.

As for the client side, I think v1 should allow only one expected
issuer per connection. There are OAuth features [2] that help clients
handle more safely, but as far as I can tell they are not widely
deployed yet, and I don't know if any of them apply to the device
flow. (With the device flow, if the client allows multiple providers,
those providers can attack each other as described above.)

If a more complicated client application associates a single end user
with multiple Postgres connections, and each connection needs its own
issuer, then that application needs to be encouraged to use a flow
which has been hardened for that use case. (Setting aside the security
problems with mix-ups, the device flow won't be particularly pleasant
for that anyway. "Here's a bunch of URLs and codes, go to all of them
before they time out, good luck!")

= Discovery Documents =

There are two flavors of discovery document, OAuth and OpenID. And
OIDC Discovery and RFC 8414 disagree on the rules, so for the issuer
"https://example.com/abcd", you have two discovery document locations
using postfix or infix styles for the path:

- OpenID: https://example.com/abcd/.well-known/openid-configuration
- OAuth:  https://example.com/.well-known/oauth-authorization-server/abcd

Some providers publish different information at each [3], so the
difference may be important for some deployments. RFC 8414 claims the
OpenID flavor should transition to the infix style at some point (a
transition that is not happening as far as I can see), so now there
are three standards. And Okta uses the construction
"https://example.com/abcd/.well-known/oauth-authorization-server",
which you may notice matches _neither_ of the two options above, so
now there are four standards.

To deal with all of this, I plan to better separate the difference
between the issuer and the discovery URL in the code, as well as allow
DBAs and clients to specify the discovery URL explicitly to override
the default OpenID flavor. For now I plan to support only
"openid-configuration" and "oauth-authorization-server" in both
postfix and infix notation (four options total, as seen in the wild).

How's all that sound?

--Jacob

[1] https://datatracker.ietf.org/doc/html/rfc8628#section-5.3
[2] https://datatracker.ietf.org/doc/html/rfc9207
[3] https://devforum.okta.com/t/is-userinfo-endpoint-available-in-oauth-authorization-server/24284

  162. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2024-11-19T11:05:29Z

    On 12.11.24 22:47, Jacob Champion wrote:
> On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> I find the way the installation options are structured a bit odd.  I
>> would have expected --with-libcurl and -Dlibcurl (or --with-curl and
>> -Dcurl).  These build options usually just say, use this library.
> 
> It's patterned directly off of -Dssl/--with-ssl (which I liberally
> borrowed from) because the builtin client implementation used to have
> multiple options for the library in use. I can change it if needed,
> but I thought it'd be helpful for future devs if I didn't undo the
> generalization.

Personally, I'm not even a fan of the -Dssl/--with-ssl system.  I'm more 
attached to --with-openssl.  But if you want to stick with that, a more 
suitable naming would be something like, say, --with-httplib=curl, which 
means, use curl for all your http needs.  Because if we later add other 
functionality that can use some http, I don't think we want to enable or 
disable them all individually, or even mix different http libraries for 
different features.  In practice, curl is a widely available and 
respected library, so I'd expect packagers to be just turn it all on 
without much further consideration.


>> I'm confused by the use of PG_MAX_AUTH_TOKEN_LENGTH in the
>> pg_be_oauth_mech definition.  What does that mean?
> 
> Just that Bearer tokens can be pretty long, so we don't want to limit
> them to 1k like SCRAM does. 64k is probably overkill, but I've seen
> anecdotal reports of tens of KBs and it seemed reasonable to match
> what we're doing for GSS tokens.

Ah, ok, I totally misread that code.  Could you maybe write this definition

+/* Mechanism declaration */
+const pg_be_sasl_mech pg_be_oauth_mech = {
+   oauth_get_mechanisms,
+   oauth_init,
+   oauth_exchange,
+
+   PG_MAX_AUTH_TOKEN_LENGTH,
+};

with designated initializers:

const pg_be_sasl_mech pg_be_oauth_mech = {
     .get_mechanisms = oauth_get_mechanisms,
     .init = oauth_init,
     .exchange = oauth_exchange,
     .max_message_length = PG_MAX_AUTH_TOKEN_LENGTH,
};


>> The CURL_IGNORE_DEPRECATION thing needs clarification.  Is that in
>> progress?
> 
> Thanks for the nudge, I've started a thread:
> 
>      https://curl.se/mail/lib-2024-11/0028.html

It looks like this has been clarified, so let's put that URL into a code 
comment.


>> This is only used once, in append_urlencoded(), and there are other
>> ways to communicate errors, for example returning a bool.
> 
> I'd rather not introduce two parallel error indicators for the caller
> to have to check for that particular part. But I can change over to
> using the (identical!) termPQExpBuffer. I felt like the other API
> signaled the intent a little better, though.

I think it's better to not drill a new hole into an established API for 
such a limited use.  So termPQExpBuffer() seems better for now.  If it 
later turns out, many callers are using termPQExpBuffer() for fake error 
handling purposes, then that can be considered independently.


>> On Cirrus CI Windows task, this test reports SKIP.  Can't tell why,
>> because the log is not kept.  I suppose you expect this to work on
>> Windows (but see my comment below)
> 
> No, builtin client support does not exist on Windows. If/when it's
> added, the 001_server tests will need to be ported.

Could you put some kind of explicit conditional or a comment in there. 
Right now, it's not possible to tell that Windows is not supported.

  163. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-21T18:51:12Z

    On Tue, Nov 19, 2024 at 3:05 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Personally, I'm not even a fan of the -Dssl/--with-ssl system.  I'm more
> attached to --with-openssl.  But if you want to stick with that, a more
> suitable naming would be something like, say, --with-httplib=curl, which
> means, use curl for all your http needs.  Because if we later add other
> functionality that can use some http, I don't think we want to enable or
> disable them all individually, or even mix different http libraries for
> different features.  In practice, curl is a widely available and
> respected library, so I'd expect packagers to be just turn it all on
> without much further consideration.

Okay, I can see that. I'll work on replacing --with-builtin-oauth. Any
votes from the gallery on --with-httplib vs. --with-libcurl?

The other suggestions look good and I've added them to my personal
TODO list. Thanks again for all the feedback!

--Jacob

  164. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-11-26T13:51:20Z

    On Thu, Nov 14, 2024 at 9:45 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> 1. libpq connection strings must specify exactly one issuer

This is done in v37. The oauth_issuer connection parameter is now
required if the server requests OAUTHBEARER.

> 2. the discovery document coming from the server must belong to that
> libpq issuer

This is also complete. If, for example, the client uses
"oauth_issuer=https://example.com/issuer", then the server is only
allowed to send it to one of four places -- either the three standard
URIs

- https://example.com/issuer/.well-known/openid-configuration
- https://example.com/.well-known/oauth-authorization-server/issuer
- https://example.com/.well-known/openid-configuration/issuer

or the decidedly nonstandard

- https://example.com/issuer/.well-known/oauth-authorization-server

The query for server parameters may be skipped, whether for
performance or paranoia reasons, by providing one of the above
well-known URIs directly. For example,
"oauth_issuer=https://example.com/issuer/.well-known/openid-configuration".
You need to have worked out your oauth_scope setting in advance,
though, if you're not willing to ask the server for it.

> 3. the HBA should allow a choice of discovery document and validator

oauth_validator_library is now oauth_validator_libraries, and the HBA
supports a `validator` option.

The `issuer` option now works the same way as `oauth_issuer` on the
client: you either provide an issuer identifier or a well-known
discovery URI. If you only provide the issuer, the server will choose
OIDC-style discovery, which seems to be far and away the most popular
choice. You can override that with whatever well-known URI you like,
but libpq still only supports the four options above for its own
safety.

I'm not sure I like the way this option works in the HBA. The client
can immediately complain if you provide a useless URI, because it has
to perform all those checks anyway, but the server has no such
guardrails. I'm wondering if I should maybe split the HBA options into
two -- "issuer" and an optional "discovery" option, perhaps? -- and
try to help out the DBAs a little more.

> I don't like the idea of pg_hba being used to load
> arbitrary libraries, though; I think the superuser should have to
> designate a pool of "blessed" validator libraries to load through a
> GUC. As a UX improvement for the common case, maybe we don't require
> the HBA to have an explicit validator parameter if the conf contains
> exactly one blessed library.

I've implemented both of these. For example, if you have

    oauth_validator_libraries = 'provider1'

then your oauth HBA lines can omit `validator`, and provider1 will be
used for all connections. Once you add a second, though:

    oauth_validator_libraries = 'provider1, provider2'

then HBA parsing will complain about an ambiguous configuration until
you provide `validator` settings for each:

    LOG:  authentication method "oauth" requires argument "validator"
to be set when oauth_validator_libraries contains multiple options

> In case someone does want to develop a multi-issuer validator (say, to
> deal with the providers that have multiple issuers underneath their
> umbrella), we need to make sure that the configured issuer in use is
> available to the validator, so that they aren't susceptible to a
> mix-up attack of their own.

This is already provided via MyProcPort, which the library is free to
examine (and our tests currently make use of it, which I'd forgotten).

--

v37 also chips away at some of the upthread feedback:
- tests that expect no issuer connections have been changed to use an
invalid IP address
- the curl_multi_socket_all deprecation history has been recorded
- oauth_validator_libraries has been added to the config sample
- pgperltidy has been run on the new TAP tests

I also added an envvar (PGOAUTHCAFILE), which is itself buried
underneath PGOAUTHDEBUG=UNSAFE, to change the CA file in use by Curl.
The Python tests use that to test HTTPS issuers. I'm not convinced yet
whether that should be a fully fleshed out/documented feature for v1;
the whole idea of the OAuth system is that your browser should be able
to connect, and if you need to tweak the X.509 tree for your provider
then you're probably going to be doing it at the system level, not at
the application level. But it's a possibility.

Thanks,
--Jacob

  165. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-11-27T16:51:04Z

    > On 21 Nov 2024, at 19:51, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Tue, Nov 19, 2024 at 3:05 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> Personally, I'm not even a fan of the -Dssl/--with-ssl system.  I'm more
>> attached to --with-openssl.

In hindsight, --with-ssl was prematurely pulled out from the various TLS
backend patchsets that were proposed a while back.  I wonder if we should
reword "Obsolete equivalent of --with-ssl=openssl" in the docs with plain
"Equivalent of ..". (which is really for another thread.)

>>  But if you want to stick with that, a more
>> suitable naming would be something like, say, --with-httplib=curl, which
>> means, use curl for all your http needs.  Because if we later add other
>> functionality that can use some http, I don't think we want to enable or
>> disable them all individually, or even mix different http libraries for
>> different features.  In practice, curl is a widely available and
>> respected library, so I'd expect packagers to be just turn it all on
>> without much further consideration.
> 
> Okay, I can see that. I'll work on replacing --with-builtin-oauth. Any
> votes from the gallery on --with-httplib vs. --with-libcurl?

I think I would vote for --with-libcurl.

--
Daniel Gustafsson

  166. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-11-27T17:27:13Z

    Thanks for the updated version, it's really starting to take good shape.  A few
small comments on v37 from a a first quick skim-through:

+	if (!strcmp(key, AUTH_KEY))
+	if (*expected_bearer && !strcmp(token, expected_bearer))
Nitpickery but these should be (strcmp(xxx) == 0) to match project style.
(ironically, the only !strcmp in the code was my mistake in ebc8b7d4416).

+	foreach(l, elemlist)
This one seems like a good candidate for a foreach_ptr construction.

+	*output = strdup(kvsep);
There is no check to ensure strdup worked AFAICT, and even though it's quite
unlikely to fail we definitely don't want to continue if it did.

fail_validator.c seems to have the #include list copied from validator.c and
pulls in unnecessarily many headers.

+    client's dummy reponse, and issues a FATAL error to end the exchange.
s/reponse/response/

In validate() I wonder if we should doublecheck that have a a proper set of
validator callbacks loaded just to make even more sure that we don't introduce
anything terrible in this codepath.

I will keep reviewing this version to try and provide more feedback.

--
Daniel Gustafsson

  167. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-12-05T18:29:52Z

    On Wed, Nov 27, 2024 at 9:27 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Thanks for the updated version, it's really starting to take good shape.  A few
> small comments on v37 from a a first quick skim-through:

Applied in v38, thanks!

> fail_validator.c seems to have the #include list copied from validator.c and
> pulls in unnecessarily many headers.

Oops, thanks for the cleanup.

> In validate() I wonder if we should doublecheck that have a a proper set of
> validator callbacks loaded just to make even more sure that we don't introduce
> anything terrible in this codepath.

Seems good. I think this part of the API is going to need an
ABI-compatibility pass, too. For example, do we want a module to
allocate the result struct itself (which locks in the struct length)?
And should we have a MAGIC_NUMBER of some sort in the static callback
list, maybe?

--

Now that our JSON API can be put into "leakproof" mode [1], I can fuzz
the parser implementations. libfuzzer has been able to find one known
issue, plus two more that I was unaware of:

1. Duplicate fields caused the previous field values to leak. (This
was the documented FIXME; we now error out in this case.)
2. The array-of-strings parsing had a subtle logic bug: if field "a"
was expected to be an array of strings, we would also accept the
construction `"a": "1"` as if it were equivalent to `"a": ["1"]`. This
messed up the internal tracking and tripped assertions.
3. handle_oauth_sasl_error() was leaking all of its parsed fields if
they didn't get hooked into the PGconn struct before a failure.

All three are fixed in v38; I will keep working on expanding the
amount of code covered by my fuzzers.

Additionally, the following pieces of feedback has been addressed:
- We now validate the incoming JSON as UTF-8 before lexing it, to
prevent invalid multibyte sequences from sneaking through in the
strings [2]. Still need to determine how \uXXXX sequences will
interact with the more punishing client encodings in error messages.
- --with-builtin-oauth/-Dwith_builtin_oauth has been renamed
--with-libcurl/-Dlibcurl, and the Autoconf side uses PKG_CHECK_MODULES
exclusively.
- markPQExpBufferBroken has been replaced with termPQExpBuffer in
append_urlencoded()
- the anonymous union has been named
- pg_be_oauth_mech uses a designated initializer

Next up, the many-many documentation requests, now that the fuzzers
can run while I write.

Thanks,
--Jacob

[1] https://postgr.es/c/5c32c21afe6
[2] https://postgr.es/m/ZjmjPyA29dIJjmjI%40paquier.xyz

  168. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-12-12T00:18:28Z

    On Thu, Dec 5, 2024 at 10:29 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>
> Next up, the many-many documentation requests, now that the fuzzers
> can run while I write.

v39 adds a great deal of documentation for implementers of custom
client flows and validators, and addresses the following upthread
feedback:
- the trust_validator_authz HBA option has been renamed to
delegate_ident_mapping
- delegate_ident_mapping is now tested as part of the oauth_validator suite
- typedefs for AsyncAuthFunc, OAuthStep, and fe_oauth_state_enum have
been removed (and the last has been renamed `enum fe_oauth_step`)
- pg_oauth_mech has been moved to fe-auth-oauth.h
- PostgreSQL::Test::OAuthServer has been moved into the
oauth_validator folder as OAuth::Server
- pgperlcritic now passes

Of Peter's notes, I think just the Windows testing comments and a
better explanation of the MAX_OAUTH_RESPONSE_SIZE remain.

On Fri, Nov 8, 2024 at 1:21 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> * src/interfaces/libpq/libpq-fe.h
>
> The naming scheme of types and functions in this file is clearly
> obscure and has grown randomly over time.  But at least my intuition
> is that the preferred way is
>
> types start with PG
> function start with PQ
>
> and the next letter is usually lower case. (PQconnectdb, PQhost,
> PGconn, PQresult)

Okay, I think I've corrected this (`struct PQxxx` are now `struct
PGxxx`, PGAuthData is now PGauthData). To summarize the new API:
- PGauthData is an enum containing PQAUTHDATA_* constants
- PGpromptOAuthDevice and PGoauthBearerRequest are type-specific
callback structures
- the PQauthDataHook and all of its related types and API start with
PQ, to parallel the PQsetSSLKeyPassHook_OpenSSL API

Thanks,
--Jacob

  169. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-12-15T22:18:39Z

    > On 5 Dec 2024, at 19:29, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> Seems good. I think this part of the API is going to need an
> ABI-compatibility pass, too. For example, do we want a module to
> allocate the result struct itself (which locks in the struct length)?
> And should we have a MAGIC_NUMBER of some sort in the static callback
> list, maybe?

I think we should, I just now experimented with setting the server major
version (backed by PG_VERSION_NUM) in the callback struct and added a simple
test.  I'm not sure if there is a whole lot more we need, maybe an opaque
integer for the module to identify its version?

> --with-libcurl/-Dlibcurl, and the Autoconf side uses PKG_CHECK_MODULES
> exclusively.

Why only use PKG_CHECK_MODULES for this rather than treating it more like other
dependencies where we fall back on other methods if not found?  While I'm
clearly not the target audience, I build libcurl all the time and being able to
point to a directory would be nice.  There's also the curl-config utility which
should be in all packaged versions.

--
Daniel Gustafsson

  170. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-12-17T00:51:53Z

    On Sun, Dec 15, 2024 at 2:18 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> I think we should, I just now experimented with setting the server major
> version (backed by PG_VERSION_NUM) in the callback struct and added a simple
> test.  I'm not sure if there is a whole lot more we need, maybe an opaque
> integer for the module to identify its version?

I think PG_VERSION_NUM should be handled by the standard
PG_MODULE_MAGIC. But an integer for the validator version would be
good, I think.

> > --with-libcurl/-Dlibcurl, and the Autoconf side uses PKG_CHECK_MODULES
> > exclusively.
>
> Why only use PKG_CHECK_MODULES for this rather than treating it more like other
> dependencies where we fall back on other methods if not found?

That was from Peter's request up in [1]. (I don't have strong opinions
on which to support, but I am vaguely persuaded by the idea of parity
between Meson and Autoconf.)

> While I'm
> clearly not the target audience, I build libcurl all the time and being able to
> point to a directory would be nice.

Doesn't the PKG_CONFIG_PATH envvar let you do that for Autoconf? Or,
if you're using Meson, -Dpkg_config_path? I was using the latter for
my local Curl builds.

> There's also the curl-config utility which
> should be in all packaged versions.

Hmm, I wonder if Meson supports alternative names for pkg-config.
Though I guess the --version handling would be different between the
two?

--Jacob

[1] https://www.postgresql.org/message-id/6bde5f56-9e7a-4148-b81c-eb6532cb3651%40eisentraut.org

  171. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2024-12-20T01:00:03Z

    On Thu, Dec 5, 2024 at 10:29 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> 1. Duplicate fields caused the previous field values to leak. (This
> was the documented FIXME; we now error out in this case.)
> 2. The array-of-strings parsing had a subtle logic bug: if field "a"
> was expected to be an array of strings, we would also accept the
> construction `"a": "1"` as if it were equivalent to `"a": ["1"]`. This
> messed up the internal tracking and tripped assertions.

My "fix" for these in v38 included a silly mistake: the
grant_types_supported array could no longer contain more than one item
without being considered duplicated. :/ I've updated the tests to
exercise this case and fixed it in v40. Fuzzers are still happy so
far.

I also made a bad assumption about the return value of
connect_ok/fails() in the server log tests, so they weren't always
checking what they should have been. These have been rewritten
entirely (and IMO the tests are more readable as a result). Some
additional negative tests have been added to oauth_validator as well.

v40 also contains:
- explicit testing for connect_timeout compatibility
- support for require_auth=oauth, including compatibility with
require_auth=!scram-sha-256
- the ability for a validator to set authn_id even if the token is not
authorized, for auditability in the logs
- the use of pq_block_sigpipe() for additional safety in the face of
CURLOPT_NOSIGNAL

I have split out the require_auth changes temporarily (0002) for ease
of review, and I plan to ping the last thread where SASL support in
require_auth was discussed [1].

Thanks!
--Jacob

[1] https://postgr.es/m/ZB5jftra/n2TbdLx%40paquier.xyz

  172. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2024-12-20T22:20:58Z

    > On 20 Dec 2024, at 02:00, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

Thanks for the new version, I was doing a v39 review but I'll roll that over
into a v40 review now.

As I was reading I was trying to identify parts can be broken out and committed
ahead of time.  This not only to trim down size, but mostly to shape the final
commit into a coherent single commit that brings a single functionality
utilizing existing APIs.  Basically I think we should keep generic
functionality out of the final commit and keep that focused on OAuth and the
required APIs and infra.

The async auth support seemed like a candidate to go in before the rest.  While
there won't be any consumers of it, it's also not limited to OAuth.  What do
you think about slicing that off and get in ahead of time?  I took a small stab
at separating out the generic bits (it includes the PG_MAX_AUTH_TOKEN_LENGTH
move as well which is unrelated, but could also be committed ahead of time)
along with some small tweaks on it.

--
Daniel Gustafsson

  173. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-01-02T10:11:07Z

    On 20.12.24 02:00, Jacob Champion wrote:
> v40 also contains:
> - explicit testing for connect_timeout compatibility
> - support for require_auth=oauth, including compatibility with
> require_auth=!scram-sha-256
> - the ability for a validator to set authn_id even if the token is not
> authorized, for auditability in the logs
> - the use of pq_block_sigpipe() for additional safety in the face of
> CURLOPT_NOSIGNAL
> 
> I have split out the require_auth changes temporarily (0002) for ease
> of review, and I plan to ping the last thread where SASL support in
> require_auth was discussed [1].

Some review of v40:

General:

There is a mix of using "URL" and "URI" throughout the patch.  I tried
to look up in the source material (RFCs) what the correct use would
be, but even they are mixing it in nonobvious ways.  Maybe this is
just hopelessly confused, or maybe there is a system that I don't
recognize.


* .cirrus.tasks.yml

Since libcurl is an "auto" meson option, it doesn't need to be enabled
explicitly.  At least that's how most of the other feature options are
handled.  So probably better to stick to that pattern.


* config/programs.m4

Useless whitespace change.


* configure.ac

+AC_MSG_CHECKING([whether to build with libcurl support for OAuth client 
flows])
etc.

Let's just write something like 'whether to build with libcurl
support' here.  So we don't have to keep updating it if the scope of
the option changes.

* meson_options.txt

+option('libcurl', type : 'feature', value: 'auto',
+  description: 'libcurl support for OAuth client flows')

Similarly, let's just write something like 'libcurl support' here.


* src/backend/libpq/auth-oauth.c

+typedef enum
+{
+   OAUTH_STATE_INIT = 0,
+   OAUTH_STATE_ERROR,
+   OAUTH_STATE_FINISHED,
+} oauth_state;
+
+/* Mechanism callback state. */
+struct oauth_ctx
+{
+   oauth_state state;

This is the only use of that type definition.  I think you can skip
the typedef and use the enum tag directly.


* src/interfaces/libpq/libpq-fe.h

+#ifdef WIN32
+#include <winsock2.h>          /* for SOCKET */
+#endif

Including a system header like that seems a bit unpleasant.  AFAICT,
you only need it for this:

+   PostgresPollingStatusType (*async) (PGconn *conn,
+                                       struct _PGoauthBearerRequest 
*request,
+                                       SOCKTYPE * altsock);

But that function could already get the altsock handle via
conn->altsock.  So maybe that is a way to avoid the whole socket type
dance in this header.


* src/test/authentication/t/001_password.pl

I suppose this file could be a separate commit?  It just separates the
SASL/SCRAM terminology for existing functionality.


* src/test/modules/oauth_validator/fail_validator.c

+{
+   elog(FATAL, "fail_validator: sentinel error");
+   pg_unreachable();
+}

This pg_unreachable() is probably not necessary after elog(FATAL).


* .../modules/oauth_validator/oauth_hook_client.c

+#include <stdio.h>
+#include <stdlib.h>

These are generally not necessary, as they come in via c.h.

+#ifdef WIN32
+#include <winsock2.h>
+#else
+#include <sys/socket.h>
+#endif

I don't think this special Windows handling is necessary, since there
is src/include/port/win32/sys/socket.h.

+static void
+usage(char *argv[])
+{
+   fprintf(stderr, "usage: %s [flags] CONNINFO\n\n", argv[0]);

Help output should go to stdout.


With the above changes, I think this patch set is structurally okay now. 
  Now it just needs to do the right things. ;-)

  174. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-07T22:20:40Z

    On Fri, Dec 20, 2024 at 2:21 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 20 Dec 2024, at 02:00, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>
> Thanks for the new version, I was doing a v39 review but I'll roll that over
> into a v40 review now.

(Sorry for the rug pull!)

> As I was reading I was trying to identify parts can be broken out and committed
> ahead of time.  This not only to trim down size, but mostly to shape the final
> commit into a coherent single commit that brings a single functionality
> utilizing existing APIs.  Basically I think we should keep generic
> functionality out of the final commit and keep that focused on OAuth and the
> required APIs and infra.

Sounds good.

> The async auth support seemed like a candidate to go in before the rest.  While
> there won't be any consumers of it, it's also not limited to OAuth.  What do
> you think about slicing that off and get in ahead of time?  I took a small stab
> at separating out the generic bits (it includes the PG_MAX_AUTH_TOKEN_LENGTH
> move as well which is unrelated, but could also be committed ahead of time)
> along with some small tweaks on it.

+1 to separating the PG_MAX_... macro move. I will take a closer look
at the async patch in isolation; there's some work I'm doing to fix a
bug Kashif (cc'd) found recently, and it has me a bit unsure about my
chosen order of operations in the async part of fe-connect.c. That
deserves its own email, but I need to investigate more.

Thanks!
--Jacob

  175. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-07T22:24:39Z

    On Thu, Jan 2, 2025 at 2:11 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> There is a mix of using "URL" and "URI" throughout the patch.  I tried
> to look up in the source material (RFCs) what the correct use would
> be, but even they are mixing it in nonobvious ways.  Maybe this is
> just hopelessly confused, or maybe there is a system that I don't
> recognize.

Ugh, yeah. I think my "system" was whether the RFC I read most
recently had used "URL" or "URI" more in the text.

In an ideal world, I'd just switch to "URL" to avoid confusion. There
are some URNs in use as part of OAuth (e.g.
`urn:ietf:params:oauth:grant-type:device_code`) but I don't think I
refer to those as URIs anyway. And more esoteric forms of URI (data:)
are not allowed.

However... there are some phrases, like "Well-Known URI", where that's
just the Name of the Thing. Similarly, when the wire protocol itself
uses "URI" (say, in JSON field names), I'd rather be consistent to
make searching easier.

Is it enough to prefer "URL" in the user-facing documentation (at
least, when it doesn't conflict with other established naming
conventions), and accept both in the code?

> * src/interfaces/libpq/libpq-fe.h
>
> +#ifdef WIN32
> +#include <winsock2.h>          /* for SOCKET */
> +#endif
>
> Including a system header like that seems a bit unpleasant.  AFAICT,
> you only need it for this:
>
> +   PostgresPollingStatusType (*async) (PGconn *conn,
> +                                       struct _PGoauthBearerRequest
> *request,
> +                                       SOCKTYPE * altsock);
>
> But that function could already get the altsock handle via
> conn->altsock.  So maybe that is a way to avoid the whole socket type
> dance in this header.

It'd also couple clients against libpq-int.h, so they'd have to
remember to recompile every release. I'm worried that'd cause a bunch
of ABI problems...

I could cheat and use uintptr_t instead of SOCKET on Windows, but that
seems like it might bite us in Win32-adjacent environments? It seems
to pass Cirrus okay. Other ideas?

> * src/test/authentication/t/001_password.pl
>
> I suppose this file could be a separate commit?  It just separates the
> SASL/SCRAM terminology for existing functionality.

The scram-sha-256 duplication tests could, I suppose. But the only
reason that's interesting to test now is because of the change to the
internals. The "server requested SCRAM-SHA-256 authentication" error
message change comes in with the new require_auth handling, so that
should all land in the same patch.

Along those lines, though, Michael Paquier suggested that maybe I
could pull the require_auth prefactoring up to the front of the
patchset. That might look a bit odd until OAuth support lands, since
it won't be adding any new useful value, but I will give it a shot.

> * src/test/modules/oauth_validator/fail_validator.c
>
> +{
> +   elog(FATAL, "fail_validator: sentinel error");
> +   pg_unreachable();
> +}
>
> This pg_unreachable() is probably not necessary after elog(FATAL).

Cirrus completes successfully with that, but MSVC starts complaining:

    warning C4715: 'fail_token': not all control paths return a value

Is that expected?

--

All other suggestions will be addressed in the next patchset. Thanks!

--Jacob

  176. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Kashif Zeeshan <kashi.zeeshan@gmail.com> — 2025-01-08T09:07:33Z

    On Wed, Jan 8, 2025 at 3:21 AM Jacob Champion <
jacob.champion@enterprisedb.com> wrote:

> On Fri, Dec 20, 2024 at 2:21 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> >
> > > On 20 Dec 2024, at 02:00, Jacob Champion <
> jacob.champion@enterprisedb.com> wrote:
> >
> > Thanks for the new version, I was doing a v39 review but I'll roll that
> over
> > into a v40 review now.
>
> (Sorry for the rug pull!)
>
> > As I was reading I was trying to identify parts can be broken out and
> committed
> > ahead of time.  This not only to trim down size, but mostly to shape the
> final
> > commit into a coherent single commit that brings a single functionality
> > utilizing existing APIs.  Basically I think we should keep generic
> > functionality out of the final commit and keep that focused on OAuth and
> the
> > required APIs and infra.
>
> Sounds good.
>
> > The async auth support seemed like a candidate to go in before the
> rest.  While
> > there won't be any consumers of it, it's also not limited to OAuth.
> What do
> > you think about slicing that off and get in ahead of time?  I took a
> small stab
> > at separating out the generic bits (it includes the
> PG_MAX_AUTH_TOKEN_LENGTH
> > move as well which is unrelated, but could also be committed ahead of
> time)
> > along with some small tweaks on it.
>
> +1 to separating the PG_MAX_... macro move. I will take a closer look
> at the async patch in isolation; there's some work I'm doing to fix a
> bug Kashif (cc'd) found recently, and it has me a bit unsure about my
> chosen order of operations in the async part of fe-connect.c. That
> deserves its own email, but I need to investigate more.
>
Thanks Jacob
Most of the testing with psql is done and working on the remaining test
cases.


>
> Thanks!
> --Jacob
>
>
>

  177. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-08T17:13:38Z

    On Tue, Jan 7, 2025 at 2:24 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Along those lines, though, Michael Paquier suggested that maybe I
> could pull the require_auth prefactoring up to the front of the
> patchset. That might look a bit odd until OAuth support lands, since
> it won't be adding any new useful value, but I will give it a shot.

While I take a look at the async patch from upthread, here is my
attempt at pulling the require_auth change out.

Note that there's a dead branch that cannot be exercised until OAuth
lands. We're not going to process the SASL mechanism name at all if no
mechanisms are allowed to begin with, and right now SASL is synonymous
with SCRAM. I can change that by always allowing AuthenticationSASL
messages -- even if none of the allowed authentication types use SASL
-- but that approach didn't seem to generate excitement on- or
off-list the last time I proposed it [1].

Thanks,
--Jacob

[1] https://postgr.es/m/CAAWbhmg%2BGzNMK5Li182BKSbzoFVaKk_dDJ628NnuV80GqYgFFg%40mail.gmail.com

  178. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-01-08T19:37:12Z

    On 07.01.25 23:24, Jacob Champion wrote:
> On Thu, Jan 2, 2025 at 2:11 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> There is a mix of using "URL" and "URI" throughout the patch.  I tried
>> to look up in the source material (RFCs) what the correct use would
>> be, but even they are mixing it in nonobvious ways.  Maybe this is
>> just hopelessly confused, or maybe there is a system that I don't
>> recognize.
> 
> Ugh, yeah. I think my "system" was whether the RFC I read most
> recently had used "URL" or "URI" more in the text.
> 
> In an ideal world, I'd just switch to "URL" to avoid confusion. There
> are some URNs in use as part of OAuth (e.g.
> `urn:ietf:params:oauth:grant-type:device_code`) but I don't think I
> refer to those as URIs anyway. And more esoteric forms of URI (data:)
> are not allowed.
> 
> However... there are some phrases, like "Well-Known URI", where that's
> just the Name of the Thing. Similarly, when the wire protocol itself
> uses "URI" (say, in JSON field names), I'd rather be consistent to
> make searching easier.
> 
> Is it enough to prefer "URL" in the user-facing documentation (at
> least, when it doesn't conflict with other established naming
> conventions), and accept both in the code?

The above explanation makes sense to me.  I don't know what you mean by 
"accept in the code".  I would agree with "tolerate some inconsistency" 
in the code but not with, like, create alias names for all the interface 
names.

> 
>> * src/interfaces/libpq/libpq-fe.h
>>
>> +#ifdef WIN32
>> +#include <winsock2.h>          /* for SOCKET */
>> +#endif
>>
>> Including a system header like that seems a bit unpleasant.  AFAICT,
>> you only need it for this:
>>
>> +   PostgresPollingStatusType (*async) (PGconn *conn,
>> +                                       struct _PGoauthBearerRequest
>> *request,
>> +                                       SOCKTYPE * altsock);
>>
>> But that function could already get the altsock handle via
>> conn->altsock.  So maybe that is a way to avoid the whole socket type
>> dance in this header.
> 
> It'd also couple clients against libpq-int.h, so they'd have to
> remember to recompile every release. I'm worried that'd cause a bunch
> of ABI problems...

Couldn't that function use PQsocket() to get at the actual socket from 
the PGconn handle?

>> * src/test/modules/oauth_validator/fail_validator.c
>>
>> +{
>> +   elog(FATAL, "fail_validator: sentinel error");
>> +   pg_unreachable();
>> +}
>>
>> This pg_unreachable() is probably not necessary after elog(FATAL).
> 
> Cirrus completes successfully with that, but MSVC starts complaining:
> 
>      warning C4715: 'fail_token': not all control paths return a value
> 
> Is that expected?

Ah yes, because MSVC doesn't support the noreturn attribute.  (See 
<https://www.postgresql.org/message-id/flat/pxr5b3z7jmkpenssra5zroxi7qzzp6eswuggokw64axmdixpnk%40zbwxuq7gbbcw>.) 
  So ok to leave as you had it for now.

  179. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-08T20:29:35Z

    On Wed, Jan 8, 2025 at 11:37 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> I don't know what you mean by
> "accept in the code".  I would agree with "tolerate some inconsistency"
> in the code but not with, like, create alias names for all the interface
> names.

"Tolerate inconsistency" was what I had in mind. So I'll plan to do a
pass on the user documentation, but not a search-and-replace in the
code at this point.

> > It'd also couple clients against libpq-int.h, so they'd have to
> > remember to recompile every release. I'm worried that'd cause a bunch
> > of ABI problems...
>
> Couldn't that function use PQsocket() to get at the actual socket from
> the PGconn handle?

It's an output parameter (i.e. the async callback is responsible for
setting conn->altsock). Unless I've missed the point entirely, I don't
think PQsocket() helps here.

> >      warning C4715: 'fail_token': not all control paths return a value
> >
> > Is that expected?
>
> Ah yes, because MSVC doesn't support the noreturn attribute.  (See
> <https://www.postgresql.org/message-id/flat/pxr5b3z7jmkpenssra5zroxi7qzzp6eswuggokw64axmdixpnk%40zbwxuq7gbbcw>.)
>   So ok to leave as you had it for now.

Will do.

Thanks!
--Jacob

  180. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-01-09T16:17:31Z

    On 08.01.25 21:29, Jacob Champion wrote:
>>> It'd also couple clients against libpq-int.h, so they'd have to
>>> remember to recompile every release. I'm worried that'd cause a bunch
>>> of ABI problems...
>> Couldn't that function use PQsocket() to get at the actual socket from
>> the PGconn handle?
> It's an output parameter (i.e. the async callback is responsible for
> setting conn->altsock). Unless I've missed the point entirely, I don't
> think PQsocket() helps here.

Maybe it would work to just use plain "int" as the type here.  Any 
socket number must fit into int anyway in order for PQsocket() to be 
able to return it.  The way I understand Windows socket handles, this 
should work.

  181. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-09T19:18:21Z

    On Thu, Jan 9, 2025 at 8:17 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>
> Maybe it would work to just use plain "int" as the type here.  Any
> socket number must fit into int anyway in order for PQsocket() to be
> able to return it.  The way I understand Windows socket handles, this
> should work.

Looks like it should work for current Windows, yeah. This is the
approach taken by OpenSSL [1].

It'd be sad to copy-paste the API bug into a new place, though. If
we're going to disconnect this API from SOCKET, can we use uintptr_t
instead on Windows? If someone eventually adds an alternative to
PQsocket(), as Tom suggested in [2], it'd be nice not to have to
duplicate this callback too.

--Jacob

[1] https://docs.openssl.org/3.4/man3/SSL_set_fd/#notes
[2] https://www.postgresql.org/message-id/153442.1624889951%40sss.pgh.pa.us

  182. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-09T20:40:04Z

    > On 20 Dec 2024, at 02:00, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> v40 also contains:

A few small comments on v40 rather than saving up for a longer email:

+ ereport(LOG, errmsg("Internal error in OAuth validator module"));
Tiny nitpick, the errmsg() should start with lowercase 'i'.


+  libpq_append_conn_error(conn, "no custom OAuth flows are available, and libpq was not built using --with-libcurl");
Since we at some point will move away from autoconf I think we should avoid
such implementation details in error messages.  How about "was not built with
libcurl support"?


+ * Find the start of the .well-known prefix. IETF rules state this must be
+ * at the beginning of the path component, but OIDC defined it at the end
+ * instead, so we have to search for it anywhere.
I was looking for a reference for OIDC defining the WK prefix placement but I
could only find it deferring to RFC5785 like how RFC8414 does.  Can you inject
a document reference for this?


+ if (strcmp(conn->oauth_issuer_id, discovery_issuer) != 0)
Shouldn't the scheme component really be compared case-insensitive, or has it
been normalized at this point?  Not sure how much it matters in practice but if
not perhaps we should add a TODO marker there?


Support for oauth seems to be missing from pg_hba_file_rules() which should be
added in hbafuncs.c:get_hba_options().

--
Daniel Gustafsson

  183. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-09T22:35:06Z

    On Thu, Jan 9, 2025 at 12:40 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> + * Find the start of the .well-known prefix. IETF rules state this must be
> + * at the beginning of the path component, but OIDC defined it at the end
> + * instead, so we have to search for it anywhere.
> I was looking for a reference for OIDC defining the WK prefix placement but I
> could only find it deferring to RFC5785 like how RFC8414 does.  Can you inject
> a document reference for this?

I'll add a note in the comment. It's in Section 4 of OIDC Discovery
1.0: https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig

(This references RFC 5785, but only to obliquely point out that it's
not actually compliant with RFC 5785, if the issuer ID has a path
component. Section 4.1 gives an example.)

> + if (strcmp(conn->oauth_issuer_id, discovery_issuer) != 0)
> Shouldn't the scheme component really be compared case-insensitive, or has it
> been normalized at this point?  Not sure how much it matters in practice but if
> not perhaps we should add a TODO marker there?

I don't think we should. While I've read some fights about the meaning
of "identical" in OIDCD Sec 4.3, IETF seems to be pushing hard for
exact equality of issuer IDs. RFC 9207 says [1]

   This [issuer] comparison MUST use simple string comparison as
defined in Section 6.2.1 of [RFC3986].

(Simple string comparison being byte/character-wise rather than
performing a normalization step.) While RFC 9207 doesn't govern the
Device Authorization flow yet (maybe not ever?), the current OAuth 2.1
draft refers to its rules as a MUST [2], and I think we should just be
strict for the safety of future flow implementations.

I'm sure someone's going to complain at some point, but IMNSHO, the
fix for them is just to use the same formatting and capitalization as
the discovery document, and move on.

--

I'll address the other comments in the upcoming v41.

Thanks!
--Jacob

[1] https://www.rfc-editor.org/rfc/rfc9207#section-2.4
[2] https://www.ietf.org/archive/id/draft-ietf-oauth-v2-1-12.html#section-7.13.1

  184. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-09T22:44:34Z

    > On 9 Jan 2025, at 23:35, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Thu, Jan 9, 2025 at 12:40 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>> + * Find the start of the .well-known prefix. IETF rules state this must be
>> + * at the beginning of the path component, but OIDC defined it at the end
>> + * instead, so we have to search for it anywhere.
>> I was looking for a reference for OIDC defining the WK prefix placement but I
>> could only find it deferring to RFC5785 like how RFC8414 does.  Can you inject
>> a document reference for this?
> 
> I'll add a note in the comment. It's in Section 4 of OIDC Discovery
> 1.0: https://openid.net/specs/openid-connect-discovery-1_0.html#ProviderConfig
> 
> (This references RFC 5785, but only to obliquely point out that it's
> not actually compliant with RFC 5785, if the issuer ID has a path
> component. Section 4.1 gives an example.)

Thanks!

>> + if (strcmp(conn->oauth_issuer_id, discovery_issuer) != 0)
>> Shouldn't the scheme component really be compared case-insensitive, or has it
>> been normalized at this point?  Not sure how much it matters in practice but if
>> not perhaps we should add a TODO marker there?
> 
> I don't think we should. While I've read some fights about the meaning
> of "identical" in OIDCD Sec 4.3, IETF seems to be pushing hard for
> exact equality of issuer IDs. RFC 9207 says [1]
> 
>   This [issuer] comparison MUST use simple string comparison as
> defined in Section 6.2.1 of [RFC3986].
> 
> (Simple string comparison being byte/character-wise rather than
> performing a normalization step.) While RFC 9207 doesn't govern the
> Device Authorization flow yet (maybe not ever?), the current OAuth 2.1
> draft refers to its rules as a MUST [2], and I think we should just be
> strict for the safety of future flow implementations.
> 
> I'm sure someone's going to complain at some point, but IMNSHO, the
> fix for them is just to use the same formatting and capitalization as
> the discovery document, and move on.

Fair enough, I buy that. Maybe the above could be de-opinionated slightly and added as a comment to help others reading the code down the line?

--
Daniel Gustafsson

  185. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-01-10T13:27:00Z

    On 09.01.25 20:18, Jacob Champion wrote:
> It'd be sad to copy-paste the API bug into a new place, though. If
> we're going to disconnect this API from SOCKET, can we use uintptr_t
> instead on Windows? If someone eventually adds an alternative to
> PQsocket(), as Tom suggested in [2], it'd be nice not to have to
> duplicate this callback too.

Assuming that uintptr_t is the right underlying type for SOCKET, that 
seems ok.

But also note that

#ifdef WIN32

might not work because WIN32 is defined by the PostgreSQL build system, 
not by the compiler (see src/include/port/win32.h).

  186. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-13T23:21:31Z

    On Fri, Jan 10, 2025 at 5:27 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Assuming that uintptr_t is the right underlying type for SOCKET, that
> seems ok.

Best I can tell, SOCKET is a UINT_PTR (distinct from PUINT) so I think
that's correct.

> #ifdef WIN32
>
> might not work because WIN32 is defined by the PostgreSQL build system,
> not by the compiler (see src/include/port/win32.h).

Ah, thanks for catching that. Changed to _WIN32.

On Thu, Jan 9, 2025 at 2:44 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>
> > On 9 Jan 2025, at 23:35, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > I'm sure someone's going to complain at some point, but IMNSHO, the
> > fix for them is just to use the same formatting and capitalization as
> > the discovery document, and move on.
>
> Fair enough, I buy that. Maybe the above could be de-opinionated slightly and added as a comment to help others reading the code down the line?

Done!

On Thu, Jan 9, 2025 at 12:40 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Support for oauth seems to be missing from pg_hba_file_rules() which should be
> added in hbafuncs.c:get_hba_options().

Done, and added a basic test for it too.

--

v41 handles the feedback since v40, continues fleshing out
documentation, and splits out three prefactoring patches:
- 0001 moves PG_MAX_AUTH_TOKEN_LENGTH, as discussed upthread
- 0002 handles the non-OAuth-specific changes to require_auth (0005
now highlights the OAuth-specific pieces)
- 0003 adds SASL_ASYNC and its handling code

When I applied Daniel's async_auth_portion patch from earlier, custom
flows began double-freeing their socket descriptors. That made logical
sense but it wasn't immediately clear to me how to fix it, until I
realized that "async authentication state" and "SASL mechanism state"
are two different things with different lifetimes, and my previous
attempts conflated them. 0003 introduces a cleanup_async_auth()
callback, to explicitly free the altsock and its related supporting
allocations as soon as it's no longer needed. Not only does this solve
the double-free, it removes an extra layer of indirection from 0004
and neatly fixes a TODO where the Curl handles were sticking around
for the lifetime of the Postgres connection. Assertions have been
added to keep the new internal API consistent.

pqSocketCheck() was returning ready if it found buffered SSL data,
even if an altsock had been set. I separated the two paths more
completely in 0003.

The FreeBSD 13.3 image started failing to correctly resolve libcurl
package dependencies, leading to missing libssh2 symbols at runtime.
And 13.3 went EOL at the end of 2024 -- which is possibly related to
the breakage? -- so I seemingly cannot perform a `pkg update` to try
to fix things. I've added a hack around this in 0006 that can
hopefully be removed again when our Cirrus images transition to 14.2
[1].

Next email will discuss the architectural bug that Kashif found.

Thanks,
--Jacob

[1] https://github.com/anarazel/pg-vm-images/pull/109

  187. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-14T01:00:00Z

    On Mon, Jan 13, 2025 at 3:21 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Next email will discuss the architectural bug that Kashif found.

Okay, here goes. A standard OAuth connection attempt looks like this
(oh, I hope Gmail doesn't mangle it):

    Issuer    User    libpq   Backend
      |        |        |
      |        x -----> x -----> o      [1] Startup Packet
      |        |        |        |
      |        |        x <----- x      [2] OAUTHBEARER Request
      |        |        |        |
      |        |        x -----> x      [3] Parameter Discovery
      |        |        |        |
      |        |        x <----- o      [4] Parameters Stored
      |        |        |
      |        |        |
      |        |        |
      |        |        x -----> o      [5] New Startup Packet
      |        |        |        |
      |        |        x <----- x      [6] OAUTHBEARER Request
      |        |        |        |
      x <----- x <----> x        |
      x <----- x <----> x        |      [7] OAuth Handshake
      x <----- x <----> x        |
      |        |        |        |
      o        |        x -----> x      [8] Send Token
               |        |        |
               | <----- x <----- x      [9] Connection Established
               |        |        |
               x <----> x <----> x
               x <----> x <----> x      [10] Use the DB
               .        .        .
               .        .        .
               .        .        .

When the server first asks for a token via OAUTHBEARER (step 2), the
client doesn't necessarily know what the server's requirements are for
a given user. It uses the rest of the doomed OAUTHBEARER exchange to
store the issuer and scope information in the PGconn (step 3-4), then
disconnects and sets need_new_connection in PQconnectPoll() so that a
second connection is immediately opened (step 5). When the OAUTHBEARER
mechanism takes control the second time, it has everything it needs to
conduct the login flow with the issuer (step 7). It then sends the
obtained token to establish a connection (steps 8 onward).

The problem is that step 7 is consuming the authentication_timeout for
the backend. I'm very good at completing these flows quickly, but if
you can't complete the browser prompts in time, you will simply not be
able to log into the server. Which is harsh to say the least. (Imagine
the pain if the standard psql password prompt timed out.) DBAs can get
around it by increasing the timeout, obviously, but that doesn't feel
very good as a solution.

Last week I looked into a fix where libpq would simply try again with
the stored token if the backend hangs up on it during the handshake,
but I think that will end up making the UX worse. The token validation
on the server side isn't going to be instantaneous, so if the client
is able to complete the token exchange in 59 seconds and send it to
the backend, there's an excellent chance that the connection is still
going to be torn down in a way that's indistinguishable from a crash.
We don't want the two sides to fight for time.

So I think what I'm going to need to do is modify v41-0003 to allow
the mechanism to politely hang up the connection while the flow is in
progress. This further decouples the lifetimes of the mechanism and
the async auth -- the async state now has to live outside of the SASL
exchange -- but I think it's probably more architecturally sound. Yell
at me if that sounds unmaintainable or if there's a more obvious fix
I'm missing.

Huge thanks to Kashif for pointing this out!

--Jacob

  188. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-17T19:02:15Z

    On Mon, Jan 13, 2025 at 5:00 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> So I think what I'm going to need to do is modify v41-0003 to allow
> the mechanism to politely hang up the connection while the flow is in
> progress.

This is done in v42. The rough conversation now looks like this:

    Issuer    User    libpq   Backend
      |        |        |
      |        x -----> x -----> o      [1] Startup Packet
      |        |        |        |
      |        |        x <----- x      [2] OAUTHBEARER Request
      |        |        |        |
      |        |        x -----> x      [3] Parameter Discovery
      |        |        |        |
      |        |        x <----- x      [4] Parameters Stored
      |        |        |        |
      |        |        x -----> x      [5] Finish OAUTHBEARER
      x <----> x <----> x        |      [6] OAuth Handshake
      x <----> x <----> x <----- o      [7] Server Hangs Up
      x <----> x <----> x
      |        |        |
      |        |        x -----> o      [8] New Startup Packet
      |        |        |        |
      |        |        x <----- x      [9] OAUTHBEARER Request
      |        |        |        |
      o        |        x -----> x      [10] Send Token
               |        |        |
               | <----- x <----- x      [11] Connection Established
               |        |        |
               x <----> x <----> x
               x <----> x <----> x      [11] Use the DB
               .        .        .
               .        .        .
               .        .        .

The key change is that the client sends the final OAUTHBEARER response
_before_ beginning the OAuth flow, allowing the server to concurrently
close its side of the discovery connection (steps 5-7). This requires
only a single change to the proposed SASL_ASYNC feature in v41-0003:
when a mechanism returns SASL_ASYNC during exchange(), it may also
specify an output response packet to send before switching over to
async auth. (This is similar to how mechanisms may send a response
packet when returning SASL_FAILED.)

That change simplifies the description of the flow a bit, too, and
I've updated the documentation. If an OAuth flow needs to run, two
connections to the server will be made. The only way to skip the
discovery connection now is if a (custom) hook has a token cached.

> This further decouples the lifetimes of the mechanism and
> the async auth -- the async state now has to live outside of the SASL
> exchange --

The only part of the state that I had to move was the token itself,
which now lives in conn->oauth_token. This is cleaned up with a new
pqClear- function, so that it can live across connection attempts and
be proactively cleared from memory after a successful connection.

> but I think it's probably more architecturally sound.

As evidence, this change flushed out a few bugs and provided the basis
to fix every TODO in fe-auth-oauth.c, so I'm pretty happy with it:
- an AuthenticationSASLFinal message is not allowed, as OAUTHBEARER
does not specify any additional server data (this bug goes all the way
back to v1)
- a server is not allowed to switch discovery URLs on a client between
connection attempts
- a server is not allowed to override a previously determined oauth_scope
- the connection is retried only if a conn->oauth_token was not
initially set (this simplifies conn->oauth_want_retry considerably)
- require_auth=oauth will complain if a discovery connection lets the client in

The existing Perl tests were not affected by this refactoring, other
than a latent test bug that got caught with the fallout. The advisory
Python tests (which pin behavior on the wire) needed more changes.
I've also added some tests to 002_client.pl which have the custom hook
misbehave in various ways and pin the expected error messages.

v41-0005 has probably outlived its usefulness by now, and I've folded
those changes into v42-0004.

Thanks!
--Jacob

  189. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Kashif Zeeshan <kashi.zeeshan@gmail.com> — 2025-01-20T06:19:49Z

    On Tue, Jan 14, 2025 at 6:00 AM Jacob Champion <
jacob.champion@enterprisedb.com> wrote:

> On Mon, Jan 13, 2025 at 3:21 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
> > Next email will discuss the architectural bug that Kashif found.
>
> Okay, here goes. A standard OAuth connection attempt looks like this
> (oh, I hope Gmail doesn't mangle it):
>
>     Issuer    User    libpq   Backend
>       |        |        |
>       |        x -----> x -----> o      [1] Startup Packet
>       |        |        |        |
>       |        |        x <----- x      [2] OAUTHBEARER Request
>       |        |        |        |
>       |        |        x -----> x      [3] Parameter Discovery
>       |        |        |        |
>       |        |        x <----- o      [4] Parameters Stored
>       |        |        |
>       |        |        |
>       |        |        |
>       |        |        x -----> o      [5] New Startup Packet
>       |        |        |        |
>       |        |        x <----- x      [6] OAUTHBEARER Request
>       |        |        |        |
>       x <----- x <----> x        |
>       x <----- x <----> x        |      [7] OAuth Handshake
>       x <----- x <----> x        |
>       |        |        |        |
>       o        |        x -----> x      [8] Send Token
>                |        |        |
>                | <----- x <----- x      [9] Connection Established
>                |        |        |
>                x <----> x <----> x
>                x <----> x <----> x      [10] Use the DB
>                .        .        .
>                .        .        .
>                .        .        .
>
> When the server first asks for a token via OAUTHBEARER (step 2), the
> client doesn't necessarily know what the server's requirements are for
> a given user. It uses the rest of the doomed OAUTHBEARER exchange to
> store the issuer and scope information in the PGconn (step 3-4), then
> disconnects and sets need_new_connection in PQconnectPoll() so that a
> second connection is immediately opened (step 5). When the OAUTHBEARER
> mechanism takes control the second time, it has everything it needs to
> conduct the login flow with the issuer (step 7). It then sends the
> obtained token to establish a connection (steps 8 onward).
>
> The problem is that step 7 is consuming the authentication_timeout for
> the backend. I'm very good at completing these flows quickly, but if
> you can't complete the browser prompts in time, you will simply not be
> able to log into the server. Which is harsh to say the least. (Imagine
> the pain if the standard psql password prompt timed out.) DBAs can get
> around it by increasing the timeout, obviously, but that doesn't feel
> very good as a solution.
>
> Last week I looked into a fix where libpq would simply try again with
> the stored token if the backend hangs up on it during the handshake,
> but I think that will end up making the UX worse. The token validation
> on the server side isn't going to be instantaneous, so if the client
> is able to complete the token exchange in 59 seconds and send it to
> the backend, there's an excellent chance that the connection is still
> going to be torn down in a way that's indistinguishable from a crash.
> We don't want the two sides to fight for time.
>
> So I think what I'm going to need to do is modify v41-0003 to allow
> the mechanism to politely hang up the connection while the flow is in
> progress. This further decouples the lifetimes of the mechanism and
> the async auth -- the async state now has to live outside of the SASL
> exchange -- but I think it's probably more architecturally sound. Yell
> at me if that sounds unmaintainable or if there's a more obvious fix
> I'm missing.
>
> Huge thanks to Kashif for pointing this out!
>
Thanks Jacob, the latest patch fixed the issues.

>
> --Jacob
>

  190. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-20T22:10:39Z

    > On 14 Jan 2025, at 00:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> - 0001 moves PG_MAX_AUTH_TOKEN_LENGTH, as discussed upthread
> - 0002 handles the non-OAuth-specific changes to require_auth (0005
> now highlights the OAuth-specific pieces)
> - 0003 adds SASL_ASYNC and its handling code

I was reading these diffs with the aim of trying to get them in sooner rather
than later to get us closer to the full patchset committed.  Two small things
came to mind:

+  /*
+   * The mechanism should have set up the necessary callbacks; all we
+   * need to do is signal the caller.
+   */
+  *async = true;
+  return STATUS_OK;
Is it worth adding assertions here to ensure that everything has been set up
properly to help when adding a new mechanism in the future?


+   /* Done. Tear down the async implementation. */
+   conn->cleanup_async_auth(conn);
+   conn->cleanup_async_auth = NULL;
+   Assert(conn->altsock == PGINVALID_SOCKET);
In pqDropConnection() we set ->altsock to NULL just to be sure rather than
assert that cleanup has done so.  Shouldn't we be consistent in the
expectation and set to NULL here as well?

--
Daniel Gustafsson

  191. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-21T00:40:36Z

    On Mon, Jan 20, 2025 at 2:10 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> +  /*
> +   * The mechanism should have set up the necessary callbacks; all we
> +   * need to do is signal the caller.
> +   */
> +  *async = true;
> +  return STATUS_OK;
> Is it worth adding assertions here to ensure that everything has been set up
> properly to help when adding a new mechanism in the future?

Yeah, I think that'd be helpful.

> +   /* Done. Tear down the async implementation. */
> +   conn->cleanup_async_auth(conn);
> +   conn->cleanup_async_auth = NULL;
> +   Assert(conn->altsock == PGINVALID_SOCKET);
> In pqDropConnection() we set ->altsock to NULL

(I assume you mean PGINVALID_SOCKET?)

> just to be sure rather than
> assert that cleanup has done so.  Shouldn't we be consistent in the
> expectation and set to NULL here as well?

I'm not opposed; I just figured that the following code might be a bit
confusing:

    Assert(conn->altsock == PGINVALID_SOCKET);
    conn->altsock = PGINVALID_SOCKET;

But I can add a comment to the assignment to try to explain. I don't
know what the likelihood of landing code that trips that assertion is,
but an explicit assignment would at least stop problems from
cascading.

--Jacob

  192. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-21T00:43:41Z

    On Mon, Jan 20, 2025 at 4:40 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> But I can add a comment to the assignment to try to explain. I don't
> know what the likelihood of landing code that trips that assertion is,
> but an explicit assignment would at least stop problems from
> cascading.

On second thought, I can just fail the connection if this happens.

--Jacob

  193. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-21T09:29:22Z

    > On 21 Jan 2025, at 01:40, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Mon, Jan 20, 2025 at 2:10 PM Daniel Gustafsson <daniel@yesql.se> wrote:

>> +   /* Done. Tear down the async implementation. */
>> +   conn->cleanup_async_auth(conn);
>> +   conn->cleanup_async_auth = NULL;
>> +   Assert(conn->altsock == PGINVALID_SOCKET);
>> In pqDropConnection() we set ->altsock to NULL
> 
> (I assume you mean PGINVALID_SOCKET?)

Doh, yes.

>> just to be sure rather than
>> assert that cleanup has done so.  Shouldn't we be consistent in the
>> expectation and set to NULL here as well?
> 
> I'm not opposed; I just figured that the following code might be a bit
> confusing:
> 
>    Assert(conn->altsock == PGINVALID_SOCKET);
>    conn->altsock = PGINVALID_SOCKET;
> 
> But I can add a comment to the assignment to try to explain. I don't
> know what the likelihood of landing code that trips that assertion is,
> but an explicit assignment would at least stop problems from
> cascading.

It is weird, but stopping the escalation of a problem seems important.

> On 21 Jan 2025, at 01:43, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 

> On second thought, I can just fail the connection if this happens.

Yeah, I think that's the best option here.

--
Daniel Gustafsson

  194. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-21T16:46:43Z

    On Tue, Jan 21, 2025 at 1:29 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On second thought, I can just fail the connection if this happens.
>
> Yeah, I think that's the best option here.

Done that way in v43.

--Jacob

  195. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-27T22:49:50Z

    > On 21 Jan 2025, at 17:46, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> Done that way in v43.

I've spent some time staring at, and testing, 0001, 0002 and 0003 with the
intent of getting them in to pave the way for the end goal of getting 0004 in.
In general I would say they are ready, I only have a small nitpick on 0002:

+	conn->allowed_sasl_mechs[0] = &pg_scram_mech;
I'm not a huge fan of this hardcoding in fill_allowed_sasl_mechs().  It's true
that we only have one as of this patch, but we might as well plan a little for
the future maintainability. I took a quick stab in the attached.

On top of that I just re-arranged a comment to, IMHO, better match the style in
the rest of the file.

Unless there are objections I aim at committing these patches reasonably soon
to lower the barrier for getting OAuth support committed.

--
Daniel Gustafsson

  196. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-01-28T00:59:36Z

    On Mon, Jan 27, 2025 at 2:50 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> +       conn->allowed_sasl_mechs[0] = &pg_scram_mech;
> I'm not a huge fan of this hardcoding in fill_allowed_sasl_mechs().  It's true
> that we only have one as of this patch, but we might as well plan a little for
> the future maintainability. I took a quick stab in the attached.

Okay. I've folded that in and simplified it some, to remove the unused
names and just store the mechanism pointers without a wrapper struct;
see what you think.

> Unless there are objections I aim at committing these patches reasonably soon
> to lower the barrier for getting OAuth support committed.

Thanks!

--

v44 tackles threadsafety for older versions of Curl. If we can't prove
that the installed libcurl is threadsafe at configure time, we'll wrap
our one-time initialization in the pg_g_threadlock. Otherwise, we
won't bother with locking, but we will bail out loudly if our
threadsafety code has not been compiled in and libcurl has been
downgraded to a version/build that can't do that itself. Documentation
has been added for clients, to detail when they need to worry about
PQregisterThreadLock(), in the same way they already do with Kerberos.

While I was playing with that, I noticed that the Autoconf side of
things was not correctly picking up pkg-config variables, and my local
environment had masked the bug. I've added code to handle
CFLAGS/LDFLAGS in the same way that e.g. libxml is handled.

libpq no longer requires the authorization server to advertise support
for the device_code grant type. Entra ID doesn't appear to add that to
any of the openid-configurations it publishes, which was the primary
impetus for the change. Note that if a provider claims to support a
device_authorization_endpoint but then rejects a device_code grant,
we're not going to know what spec they're implementing anyway, so this
check likely doesn't give us any particular advantage. I've removed it
with an explanatory comment.

A description of the OAUTHBEARER handshake has been added to our
protocol docs, and I've added a comment to the new GUC in the sample
file. I've also added slightly nicer error messages in the case that
either OAuth endpoint isn't secured by HTTPS.

Thanks,
--Jacob

  197. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-01-31T16:23:36Z

    > On 28 Jan 2025, at 01:59, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Mon, Jan 27, 2025 at 2:50 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>> Unless there are objections I aim at committing these patches reasonably soon
>> to lower the barrier for getting OAuth support committed.

After staring at the patchset even more I committed patches 0001 and 0002 today
as a preparatory step for getting OAuth in.  I will work on the 0003 (which is
now 0001) next.

Attached is a v45 which is v44 without the now committer patches to keep the
CFBot happy.

--
Daniel Gustafsson

  198. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-06T22:02:43Z

    > On 31 Jan 2025, at 17:23, Daniel Gustafsson <daniel@yesql.se> wrote:
> 
>> On 28 Jan 2025, at 01:59, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
>> On Mon, Jan 27, 2025 at 2:50 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>>> Unless there are objections I aim at committing these patches reasonably soon
>>> to lower the barrier for getting OAuth support committed.
> 
> After staring at the patchset even more I committed patches 0001 and 0002 today
> as a preparatory step for getting OAuth in.  I will work on the 0003 (which is
> now 0001) next.

After more staring and commitmessage tweaking I pushed the v45-0001 patch and
it has so far built green in a number of BF animals (as well as in CI).

This to pave the way for the main OAUTHBEARER patch in this set, which I hope
we can reach a final version of very soon.

Attached is a v46 which is v45 minus the now committed patch. 

--
Daniel Gustafsson

  199. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-07T05:48:19Z

    On Thu, Feb 6, 2025 at 2:02 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Attached is a v46 which is v45 minus the now committed patch.

Thank you! Attached is v47, which creeps ever closer to the finish line.

For ease of review, v47-0001 is identical to v46-0001. The new changes
are split into separate fixup! commits which I'll squash in the next
round. They're ordered roughly in order of increasing complexity:

- 0002 removes and/or rewrites TODO comments that I do not plan to implement.
- 0003 makes the kqueue implementation register a one-shot timer
rather than a repeating timer, to match the epoll implementation.

- 0004 fixes a bug in backend cleanup:

I noticed that there was a "private state cookie changed" error in
some of the test logs, but none of the tests had actually failed.
Changing that to a PANIC revealed that before_shmem_exit() is too late
to run the cleanup function, since the state allocation has already
been released. I've swapped that out for a reset callback.

- 0005 warns at configure time if libcurl doesn't have a nonblocking
DNS implementation.
- 0006 augments bare Asserts during client-side JSON parsing with code
that will fail gracefully in production builds as well.
- 0007 escapes binary data during the printing of libcurl debug
output. (If you're having a bad enough day to need the debug spray,
you're probably not in the mood for the sound of a hundred BELs.)
- 0008 parses and passes through the expires_in and optional
verification_uri_complete fields from the device endpoint to any
custom user prompt. (We do not use them ourselves, at the moment. But
after seeing some nice demos of RHEL/PAM/sssd support for device flow
QR codes at FOSDEM, I think we're definitely going to want to make
those available to devs.)

- 0009 is gold-plating for the OAUTH_STEP_WAIT_INTERVAL state:

If PQconnectPoll client calls us early while we're waiting for the
ping interval to expire, we will immediately send the next request
even if we should be waiting. That bothers me a bit, because if our
implementation gets a tempban from an OAuth provider because one of
our clients accidentally implemented a busy-loop, I think we're likely
to get the blame. Ideally we should kick back up to the caller and
tell them to wait longer, instead.

Checking to see if the timer has expired is easy enough for
epoll/timerfd, but I wasn't able to find an easy way to do that with a
single kqueue. Instead, I split the kqueue in two and treat the second
one as the timer. (If it becomes readable, the timer has expired.)
There is an additional advantage in that I get to remove some `#ifdef
HAVE_SYS_EPOLL_H` sections; the two implementations are closer in
spirit now.

Thanks,
--Jacob

  200. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-07T20:12:34Z

    > On 7 Feb 2025, at 06:48, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Thu, Feb 6, 2025 at 2:02 PM Daniel Gustafsson <daniel@yesql.se> wrote:
>> Attached is a v46 which is v45 minus the now committed patch.
> 
> Thank you! Attached is v47, which creeps ever closer to the finish line.

Great, thanks!  Below are a few quick comments after a first read-through and
compile/test cycle:

> - 0005 warns at configure time if libcurl doesn't have a nonblocking
> DNS implementation.

Is it really enough to do this at build time?  A very small percentage of users
running this will also be building their own libpq so the warning is lost on
them.  That being said, I'm not entirely sure what else we could do (bleeping a
warning every time is clearly not userfriendly) so maybe this is a TODO in the
code?

> - 0006 augments bare Asserts during client-side JSON parsing with code
> that will fail gracefully in production builds as well.

+  oauth_json_set_error(ctx,       /* don't bother translating */
With the project style format for translator comments this should be:

	+  /* translator: xxx */
	+  oauth_json_set_error(ctx,

> - 0007 escapes binary data during the printing of libcurl debug
> output. (If you're having a bad enough day to need the debug spray,
> you're probably not in the mood for the sound of a hundred BELs.)

Does it make sense to prefix the printing in debug_callback() with some header
stating that the following data is debug output from curl and not postgres?  I
have a feeling I'm stockholm syndromed by knowing the internals so I'm not sure
if that would be helpful to someone not knowing the implementation details.

> - 0008 parses and passes through the expires_in and optional
> verification_uri_complete fields from the device endpoint to any
> custom user prompt. (We do not use them ourselves, at the moment. But
> after seeing some nice demos of RHEL/PAM/sssd support for device flow
> QR codes at FOSDEM, I think we're definitely going to want to make
> those available to devs.)

Aha, cool!  I was a bit surprised to not find a definition of expires_in in RFC
8628, as in what happens if -1 is passed?  8628 seems to broadly speaking fall
into the category of "just dont do the wrong thing and all will be fine" =/.
Another question that comes to mind is how the reciever should interpret the
information since it doesn't know when the device_code/user_code was generated
so it doesn't know how much of expires_in which has already passed.  (Which is
not something for us to solve, just a general observation.)

+   even if they can't use the non-textual method. Review the RFC's
+   <ulink url="https://datatracker.ietf.org/doc/html/rfc8628#section-3.3.1">notes
+   on non-textual verification</ulink>.
To align more with the rest of the documentation I think something along these
lines is better: "For more information, see section 3.3.1 in <ulink ..>RFC
8628</ulink>.

      Assert(cnt == 1);
-     return 1;                       /* don't fall through in release builds */
+     return 0;
While not introduced in this fixup patch, reading it again now I sort of think
we should make that Assert(false) to clearly indicate that we don't think the
assertion will ever pass, we're just asking to error out since we already know
the failure condition holds.

> - 0009 is gold-plating for the OAUTH_STEP_WAIT_INTERVAL state:

+  actx_error(actx, "failed to create timer kqueue: %m");
Maybe we should add a translator note explaining that kqueue should not be
translated since it's very easy to mistake it for "queue".  Doing it on the
first string including kqueue should be enough I suppose.

--
Daniel Gustafsson

  201. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-08T01:56:18Z

    On Fri, Feb 7, 2025 at 12:12 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Is it really enough to do this at build time?  A very small percentage of users
> running this will also be building their own libpq so the warning is lost on
> them.  That being said, I'm not entirely sure what else we could do (bleeping a
> warning every time is clearly not userfriendly) so maybe this is a TODO in the
> code?

I've added a TODO back. At the moment, I don't have any good ideas; if
the user isn't building libpq, they're not going to be able to take
action on the warning anyway, and for many use cases they're probably
not going to care.

> +  oauth_json_set_error(ctx,       /* don't bother translating */
> With the project style format for translator comments this should be:
>
>         +  /* translator: xxx */
>         +  oauth_json_set_error(ctx,

This comment was just meant to draw attention to the lack of
libpq_gettext(). Does it still need a translator note if we don't run
it through translation?

> Does it make sense to prefix the printing in debug_callback() with some header
> stating that the following data is debug output from curl and not postgres?  I
> have a feeling I'm stockholm syndromed by knowing the internals so I'm not sure
> if that would be helpful to someone not knowing the implementation details.

Seems reasonable; I've added "[libcurl]" to the front.

> Aha, cool!  I was a bit surprised to not find a definition of expires_in in RFC
> 8628, as in what happens if -1 is passed?  8628 seems to broadly speaking fall
> into the category of "just dont do the wrong thing and all will be fine" =/.

Yup. :(

> Another question that comes to mind is how the reciever should interpret the
> information since it doesn't know when the device_code/user_code was generated
> so it doesn't know how much of expires_in which has already passed.  (Which is
> not something for us to solve, just a general observation.)

And even if we passed the Date header value through from the server,
that'd do the wrong thing if the clocks are off. I think for now,
expires_in is likely to be a best-effort UX, in the vein of "hey,
maybe type faster if you don't want a timeout."

> To align more with the rest of the documentation I think something along these
> lines is better: "For more information, see section 3.3.1 in <ulink ..>RFC
> 8628</ulink>.

Done.

> While not introduced in this fixup patch, reading it again now I sort of think
> we should make that Assert(false) to clearly indicate that we don't think the
> assertion will ever pass, we're just asking to error out since we already know
> the failure condition holds.

Done.

> Maybe we should add a translator note explaining that kqueue should not be
> translated since it's very easy to mistake it for "queue".  Doing it on the
> first string including kqueue should be enough I suppose.

Done.

--

v48 is attached.

- 0001 contains all of the v47 fixups squashed into v47-0001.
- 0002 contains all the above feedback and rewrites two more commented TODOs.
- 0003 completes a couple of summary paragraphs in the documentation,
and makes it clear that the builtin flow is not currently supported on
Windows.
- 0004 gets a missed pgperltidy and explicitly skips unsupported tests
on Windows.
- 0005 cowardly pulls the MAX_OAUTH_RESPONSE_SIZE down to 256k.

- 0006 gives us additional levers to pull in the event that API or ABI
changes must be backported for security reasons:

Daniel and I talked at FOSDEM about wanting to have additional
guardrails on the server-side validator API. Ideally, we'd wait for
major version boundaries to change APIs, as per usual. But if any bugs
come to light that affect the security of the system, we may want to
have more control over the boundary between the server and the
validator. So I've added two features to the API.

The first is a magic number embedded in the OAuthValidatorCallbacks
struct. Should it ever be necessary to force a recompilation of
validator modules, that number can be bumped in an emergency to allow
the server to reject modules with an older ABI (or otherwise treat
them differently).

The second is state->sversion, added to the ValidatorModuleState
struct, which contains the PG_VERSION_NUM. This currently has no use,
but if there's ever a situation where the ValidatorModule* structs
need to gain new members within a stable release line, this would let
module developers make sense of the situation. It also provides an
easy way for modules to enforce a minimum minor version, for example
if there's a critical security bug in older versions that they'd
rather not deal with.

By adding these fields in addition to the existing module magic
machinery, we've probably doomed them to be unused cruft. But that
seems better than the reverse situation.

Thanks!

--Jacob

  202. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-02-12T14:55:23Z

    On 08.02.25 02:56, Jacob Champion wrote:
> On Fri, Feb 7, 2025 at 12:12 PM Daniel Gustafsson<daniel@yesql.se> wrote:
>> Is it really enough to do this at build time?  A very small percentage of users
>> running this will also be building their own libpq so the warning is lost on
>> them.  That being said, I'm not entirely sure what else we could do (bleeping a
>> warning every time is clearly not userfriendly) so maybe this is a TODO in the
>> code?
> I've added a TODO back. At the moment, I don't have any good ideas; if
> the user isn't building libpq, they're not going to be able to take
> action on the warning anyway, and for many use cases they're probably
> not going to care.

This just depends on how people have built their libcurl, right?

Do we have any information whether the async-dns-free build is a common 
configuration?

  203. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-02-12T14:59:11Z

    On 08.02.25 02:56, Jacob Champion wrote:
>> +  oauth_json_set_error(ctx,       /* don't bother translating */
>> With the project style format for translator comments this should be:
>>
>>          +  /* translator: xxx */
>>          +  oauth_json_set_error(ctx,
> This comment was just meant to draw attention to the lack of
> libpq_gettext(). Does it still need a translator note if we don't run
> it through translation?

No, that wouldn't have any effect.

I think you can just remove that comment.  It's pretty established that 
internal errors don't need translation, so it would be understood from 
looking at the code.

  204. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-13T21:23:01Z

    On Wed, Feb 12, 2025 at 6:55 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> This just depends on how people have built their libcurl, right?
>
> Do we have any information whether the async-dns-free build is a common
> configuration?

I don't think the annual Curl survey covers that, unfortunately.

On Wed, Feb 12, 2025 at 6:59 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> I think you can just remove that comment.  It's pretty established that
> internal errors don't need translation, so it would be understood from
> looking at the code.

Okay, will do.

Thanks,
--Jacob

  205. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-13T22:56:29Z

    > On 8 Feb 2025, at 02:56, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

Thanks for the new version!

> - 0004 gets a missed pgperltidy and explicitly skips unsupported tests
> on Windows.

+if ($Config{osname} eq 'MSWin32')
We can get away with nog importing Config at all since Test::Utils export a
symbol for this already in $windows_os.  Fixed in my attached.

> Daniel and I talked at FOSDEM about wanting to have additional
> guardrails on the server-side validator API. Ideally, we'd wait for
> major version boundaries to change APIs, as per usual. But if any bugs
> come to light that affect the security of the system, we may want to
> have more control over the boundary between the server and the
> validator. So I've added two features to the API.

I think what you added there is quite sufficient for handling the worst case
that ideally should never happen.  Even though I can't see us breaking this
given the code being trivial, I also don't feel like realizing after the fact
when we need it that it was subtly broken, so I added a test validator which
use the wrong ABI version.

I have now read the entire patch cover-to-cover twice to try and catch any
rough or sharp edges.  Unsurprisingly given the number of revisions this patch
has gone through, and the number of hours that have been put into it, there
isn't much to be found.  Most of my findings below are well and truly in the
nit- pickery category (and my favourite, paranoia-induced defensive
programming).  There are no architectural flaws that I can detect, and cross-
referencing with the RFC's I don't see anything mixed up in spec compliance.

To make it easier for you to see what I mean I have implemented most of the
comments and attached as a fixup patch, from which you can cherry-pick hunks
you agree with.  Those I didn't implement should be marked as such below.

As we discussed off-list I took the liberty of squashing the previous fixup
patches into a single one, and squashed your fixes for my comments against v47
into 0001.  All of my proposals are in 0004.

Some comments:

+        The system which hosts the protected resources which are
The repetition of "which ..  which" reads a bit off to me, I propose to
simplify as "The system hosting then.." instead.

+        The organization, product vendor, or other entity which develops and/or
+        administers the OAuth servers and clients for a given application.
Since we define terminology here, shouldn't this be "OAuth resource servers"?

+        <productname>PostgreSQL</productname> does not provide an authorization
+        server; it's obtained from the OAuth provider.
The "obtained from" part makes it sound like you need to get some server
software to run this with PostgreSQL.  How about "; it is the responsibility
of.."?

+         <xref linkend="libpq-connect-oauth-issuer"/> setting. No variations in
+         case or format are permitted.
While not wrong in any way, I think it would be clearer to write "formatting"
here since that's really what we are talking about no?

+         Build with libcurl support for OAuth 2.0 client flows.
+         This requires the <productname>curl</productname> package to be
+         installed.  Building with this will check for the required header files
I don't think we need to document that curl need to be installed since it's
likely already on the system of anyone reading this.  I do however think we
should state the minimum required version.

+        setting in <link linkend="auth-oauth">the server's HBA configuration.</link>
Nitpickery: I prefer to have the period outside the <link> markup.

+        the connection will fail. This is required to prevent a class of "mix-up
+        attacks" on OAuth clients.
Since mix-up attacks aren't very well documented I think we should aid the
readers by linking to the OAuth WG announcement of this class of attacks.  From
there readers can find the original paper but I think linking directly to that
is less helpful than the mailinglist post.

+   running a client via SSH. Client applications may implement their own flows
For consistency with the rest of the docs we should wrap SSH with an
<application> tag.

+   You will then log into your OAuth provider, which will ask whether you want
A think third person here reads more like the rest of the docs.

+        which <application>libpq</application> will call when when action is
"when when", I think this should really be "when an"?

+       sprays HTTP traffic (containing several critical secrets) to standard
+       error during the OAuth flow
This might not be readily understandable by non-native speakers.

+   Similarly, if you are using Curl inside your application,
Should use <productname> markup.

+   more recent versions of Curl that are built to support threadsafe
s/threadsafe/thread-safe/g for documentation consistency (ditto in other places
where used as a adjective and not code identifier).

+  itself; validator modules provide the glue between the server and the OAuth
s/glue/integration layer/ to avoid confusing readers not used to english idioms.

+   Since a misbehaving validator might let unauthorized users into the database,
+   correct implementation is critical. See
"Don't make any bugs" isn't very helpful advice =) Expanded on it slightly.

+   An OAuth validator module is loaded by dynamically loading one of the shared
The double use of load in "loaded .. loading", rewording to try and simplify.

+    The server has ensured that the token is well-formed syntactically, but no
"server" is an overloaded nomenclature here, perhaps using libpq instead to
clearly indicate that it's postgres and not an OAuth server.

+    The connection will only proceed if the module sets
+    <structfield>authorized</structfield> to <literal>true</literal>.  To
In other places we use the structure name as well and not just the member,
adding that here to be consistent.

+ * Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
Off-by-one

+ * - RFC 7628: https://tools.ietf.org/html/rfc7628
Replacing all tools.ietf.org mentions with datatracker.ietf.org to save a few
redirects.

+                       p++;
+                       if (*p != ',')
If the SASL exchange, are we certain that a rogue client cannot inject a
message which trips us past the end of string?  Should we be doublecheck when
advancing p across the message?

+sanitize_char(char c)
+{
+       static char buf[5];
With the multithreading work on the horizon we should probably avoid static
variables like these to not create work for our future selves?  The code isn't
as neat when passing in a buffer/length but it avoids the need for a static or
threadlocal variable. Or am I overthinking this?

+       initStringInfo(&issuer);
+       appendStringInfoString(&issuer, ctx->issuer);
The double StringInfoData variables in generate_error_response to be able to
JSON escape the issuer is a bit of an eye-sore, a version of
escape_json_with_len which also took an offset into the buffer on where to
start maybe?  Nothing that is urgent to address now (and I have not changed
anything here), but I'll keep it at the back of my head.

+       ereport(LOG, errmsg("internal error in OAuth validator module"));
I wonder if this should be using WARNING instead?  It's really something that
should trigger alarm bells going off.  I've also added an errcode for easier
fleet analysis.

In load_validator_library we don't explicitly verify that the required callback
is defined in the returned structure, which seems like a cheap enough belts and
suspenders level check.

+       if (parsed < 1)
+               return actx->debugging ? 0 : 1;
Is 1 second a sane lower bound on interval for all situations?  I'm starting to
wonder if we should be more conservative here, or even make it configurable in
some way? The default if not set of 5 seconds is quite a lot higher than 1.

+       if (INT_MAX <= parsed)
I think it's closer to project to style to keep the variable on the left side
in such comparisons, so changed these.

+       parsed = parse_json_number(expires_in_str);
+       parsed = round(parsed);
Shouldn't we floor() the value here to ensure we never report an expiration
time longer than the actual expiration?

+       * Some services (Google, Azure) spell verification_uri differently.
I did another round of documentation reading and couldn't find any provider
which also use "verification_url_complete".  However, since it looks so similar
to verification_url it seems worthwhile to add a comment to save readers from
the same rabbithole.

register_socket() doesn't have an error catch for the case when neither epoll
nor kqeue is supported.  Shouldn't it set actx_error() here as well?  (Not done
in my review patch.)

+       if (actx->curl_err[0])
+       {
+               size_t          len;
+
+               appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
Should this also qualify that the error comes from outside of postgres?
Something like "(libcurl:%s)" to match?  I haven't changed this in the attached
since I'm still on the fence, but I'm leanings that we probably should.
Thoughts?

-   * We only support one mechanism at the moment, so rather than deal with a
+   * We only support two mechanisms at the moment, so rather than deal with a
While there's nothing incorrect about this comment, I have a feeling we won't
support more mechanisms than we can justify having a simple array for anytime
soon =)

Sorry for the wall of text.

In general, I feel that this is getting very close to its final form wrt being
a committable patch, and assuming we don't find anything structurally unsound
in the coming days I don't see a blocker for getting this into v18 before the
final commitfest.  If anyone disagrees with this I'd love for that be brought
up.

--
Daniel Gustafsson

  206. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-13T23:01:07Z

    > On 13 Feb 2025, at 22:23, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Wed, Feb 12, 2025 at 6:55 AM Peter Eisentraut <peter@eisentraut.org> wrote:
>> This just depends on how people have built their libcurl, right?
>> 
>> Do we have any information whether the async-dns-free build is a common
>> configuration?
> 
> I don't think the annual Curl survey covers that, unfortunately.

We should be able to get a decent idea by inspecting the packaging scripts for
the major distributions I think.

--
Daniel Gustafsson

  207. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-15T01:14:16Z

    On Thu, Feb 13, 2025 at 2:56 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> To make it easier for you to see what I mean I have implemented most of the
> comments and attached as a fixup patch, from which you can cherry-pick hunks
> you agree with.  Those I didn't implement should be marked as such below.
>
> As we discussed off-list I took the liberty of squashing the previous fixup
> patches into a single one, and squashed your fixes for my comments against v47
> into 0001.  All of my proposals are in 0004.

Great! I have attached v50; 0001 has almost all of v49 squashed in,
0002 has my changes on top (includes a pgindent/pgperltidy), and 0003
holds the only part I don't like (see below). 0004 contains the
FreeBSD hack that I suspect we'll need to merge in until Cirrus images
are updated.

> +        The organization, product vendor, or other entity which develops and/or
> +        administers the OAuth servers and clients for a given application.
> Since we define terminology here, shouldn't this be "OAuth resource servers"?

The resource server is Postgres for our purposes; I've changed it to
"authorization servers".

> +   Since a misbehaving validator might let unauthorized users into the database,
> +   correct implementation is critical. See
> "Don't make any bugs" isn't very helpful advice =) Expanded on it slightly.

Hmm, I think the overloading of "validate" in the replacement text
could be confusing. I guess my point is less "don't write bugs" and
more "a bug here has extreme impact"? I've taken another shot at it;
see what you think.

> +    The server has ensured that the token is well-formed syntactically, but no
> "server" is an overloaded nomenclature here, perhaps using libpq instead to
> clearly indicate that it's postgres and not an OAuth server.

I've replaced this with "PostgreSQL" to match up with Peter's earlier
feedback (we were using "libpq" to describe the backend and he wanted
to avoid that).

> +sanitize_char(char c)
> +{
> +       static char buf[5];
> With the multithreading work on the horizon we should probably avoid static
> variables like these to not create work for our future selves?  The code isn't
> as neat when passing in a buffer/length but it avoids the need for a static or
> threadlocal variable. Or am I overthinking this?

This is the only part of the feedback patch that I'm not a fan of,
mostly because it begins to diverge heavily from the SCRAM code it
copied from. I don't disagree with the goal of getting rid of the
static buffers, but I would like to see them modified at the same time
so that we can refactor easily if/when a third SASL mechanism shows
up. (Maybe with a psprintf() rather than buffers?)

> +                       p++;
> +                       if (*p != ',')
> If the SASL exchange, are we certain that a rogue client cannot inject a
> message which trips us past the end of string?  Should we be doublecheck when
> advancing p across the message?

The existing != checks will bail out if they get to the end of the
string. It relies on byte-at-a-time advancement for safety, as well as
the SASL code higher in the stack that ensures that the input buffer
is always null terminated. (SCRAM relies on that too.) If we ever
jumped farther than a byte, we'd need stronger checks, but at the
moment I don't think this change helps us.

> In load_validator_library we don't explicitly verify that the required callback
> is defined in the returned structure, which seems like a cheap enough belts and
> suspenders level check.

Yeah, there's a later check at time of use, but it's not as
user-friendly. I've adjusted the new error message to make it a bit
closer to the logical plugin wording.

> +       if (parsed < 1)
> +               return actx->debugging ? 0 : 1;
> Is 1 second a sane lower bound on interval for all situations?  I'm starting to
> wonder if we should be more conservative here, or even make it configurable in
> some way? The default if not set of 5 seconds is quite a lot higher than 1.

Mmm, maybe it should be made configurable, but one second seems like a
long time from a CPU perspective. Maybe it would be applicable to
embedded clients? But only if some provider out there actually starts
using smaller intervals than their clients can stand... Should we wait
to hear from someone who is interested in configuring it?

> +       parsed = parse_json_number(expires_in_str);
> +       parsed = round(parsed);
> Shouldn't we floor() the value here to ensure we never report an expiration
> time longer than the actual expiration?

Sounds reasonable. Done in 0002.

> register_socket() doesn't have an error catch for the case when neither epoll
> nor kqeue is supported.  Shouldn't it set actx_error() here as well?  (Not done
> in my review patch.)

Done.

> +       if (actx->curl_err[0])
> +       {
> +               size_t          len;
> +
> +               appendPQExpBuffer(&conn->errorMessage, " (%s)", actx->curl_err);
> Should this also qualify that the error comes from outside of postgres?
> Something like "(libcurl:%s)" to match?  I haven't changed this in the attached
> since I'm still on the fence, but I'm leanings that we probably should.
> Thoughts?

Done. More context is probably better than less here.

> -   * We only support one mechanism at the moment, so rather than deal with a
> +   * We only support two mechanisms at the moment, so rather than deal with a
> While there's nothing incorrect about this comment, I have a feeling we won't
> support more mechanisms than we can justify having a simple array for anytime
> soon =)

Yeah. My goal was mostly to justify the use of the (unusual)
static-length list to future readers.

Thank you so much for the reviews!

--Jacob

  208. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-17T12:03:36Z

    > On 15 Feb 2025, at 02:14, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

>> +   Since a misbehaving validator might let unauthorized users into the database,
>> +   correct implementation is critical. See
>> "Don't make any bugs" isn't very helpful advice =) Expanded on it slightly.
> 
> Hmm, I think the overloading of "validate" in the replacement text
> could be confusing. I guess my point is less "don't write bugs" and
> more "a bug here has extreme impact"? I've taken another shot at it;
> see what you think.

I'm not sure we're at the right wording still.  I have a feeling this topic is
worth a longer paragraph describing the potential severity of various error
conditions, but I don't think that needs to go in now, we can iterate on that
over time as well.

>> +    The server has ensured that the token is well-formed syntactically, but no
>> "server" is an overloaded nomenclature here, perhaps using libpq instead to
>> clearly indicate that it's postgres and not an OAuth server.
> 
> I've replaced this with "PostgreSQL" to match up with Peter's earlier
> feedback (we were using "libpq" to describe the backend and he wanted
> to avoid that).

Ah yes, much better.

>> +sanitize_char(char c)
>> +{
>> +       static char buf[5];
>> With the multithreading work on the horizon we should probably avoid static
>> variables like these to not create work for our future selves?  The code isn't
>> as neat when passing in a buffer/length but it avoids the need for a static or
>> threadlocal variable. Or am I overthinking this?
> 
> This is the only part of the feedback patch that I'm not a fan of,
> mostly because it begins to diverge heavily from the SCRAM code it
> copied from. I don't disagree with the goal of getting rid of the
> static buffers, but I would like to see them modified at the same time
> so that we can refactor easily if/when a third SASL mechanism shows
> up. (Maybe with a psprintf() rather than buffers?)

Fair enough, I can get behind that.

>> +                       p++;
>> +                       if (*p != ',')
>> If the SASL exchange, are we certain that a rogue client cannot inject a
>> message which trips us past the end of string?  Should we be doublecheck when
>> advancing p across the message?
> 
> The existing != checks will bail out if they get to the end of the
> string. It relies on byte-at-a-time advancement for safety, as well as
> the SASL code higher in the stack that ensures that the input buffer
> is always null terminated. (SCRAM relies on that too.) If we ever
> jumped farther than a byte, we'd need stronger checks, but at the
> moment I don't think this change helps us.

Thanks for clarifying.

>> In load_validator_library we don't explicitly verify that the required callback
>> is defined in the returned structure, which seems like a cheap enough belts and
>> suspenders level check.
> 
> Yeah, there's a later check at time of use, but it's not as
> user-friendly. I've adjusted the new error message to make it a bit
> closer to the logical plugin wording.

-    errmsg("%s module \"%s\" must define the symbol %s",
+    errmsg("%s module \"%s\" must provide a %s callback",

My rationale for picking the former message was that it's same as we have
earlier in the file, so it didn't add more translator work for (ideally) rarely
used errors.

That being said, I agree that should probably align these messages with the
counterparts for archive modules and logical plugins, which currently use the
following:

    errmsg("archive modules have to define the symbol %s", "_PG_archive_module_init")
    errmsg("archive modules must register an archive callback")
    elog(ERROR, "output plugins have to declare the _PG_output_plugin_init symbol");
    elog(ERROR, "output plugins have to register a begin callback");

It's a bit surprising to me that we use elog() for output plugins, while these
errors should be rare they can be triggered by third-party code so it seems
more appropriate to use ereport() IMHO.  Given that these are so similar we
should be able to reduce translator burden by providing more or less just two
messages.

Since this will be reaching into other parts of the code, it should be its own
patch though, so for now let's go with what you proposed and we can revisit this.

>> +       if (parsed < 1)
>> +               return actx->debugging ? 0 : 1;
>> Is 1 second a sane lower bound on interval for all situations?  I'm starting to
>> wonder if we should be more conservative here, or even make it configurable in
>> some way? The default if not set of 5 seconds is quite a lot higher than 1.
> 
> Mmm, maybe it should be made configurable, but one second seems like a
> long time from a CPU perspective. Maybe it would be applicable to
> embedded clients? But only if some provider out there actually starts
> using smaller intervals than their clients can stand... Should we wait
> to hear from someone who is interested in configuring it?

I indeed think we should await feedback, making it configurable isn't exactly
free so I hesitate to do it if nobody wants it.

The attached v51 squashes your commits together, discarding the changes
discussed here, and takes a stab at a commit message for these as this is
getting very close to be able to go in.  There are no additional changes.

--
Daniel Gustafsson

  209. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-17T18:15:27Z

    On Mon, Feb 17, 2025 at 4:03 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> The attached v51 squashes your commits together, discarding the changes
> discussed here, and takes a stab at a commit message for these as this is
> getting very close to be able to go in.  There are no additional changes.

Awesome, thank you! It's been a little bit since I've re-run my
fuzzers, and a new Valgrind run would be a good idea, so I will just
keep throwing tests at it and review the new commit messages while
they run.

--Jacob

  210. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-17T23:51:23Z

    On Mon, Feb 17, 2025 at 10:15 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> It's been a little bit since I've re-run my
> fuzzers, and a new Valgrind run would be a good idea, so I will just
> keep throwing tests at it

Fuzzers are happy so far.

Valgrind did find something! A mistake I made during parameter
discovery: setup_oauth_parameters() ensures that conn->oauth_issuer_id
is always set using the "issuer" connection option, but during the
second connection, I reassigned the pointer for it (and
conn->oauth_discovery_uri) and leaked the previous allocations.

v52-0002 fixes that. I've taken the opportunity to document that those
two parameters are designed to be unchangeable for the connection once
they've been assigned.

--

Reviews for the commit message:

> postgres cannot ship with one built-in.

s/postgres/Postgres/. Maybe a softening to "does not" ship with one?

> Each pg_hba entry can
> specify one, or more, validators or be left blank for the validator
> installed as default.

Each pg_hba entry can specify only one of the DBA-blessed validators, not more.

> This adds a requirement on libucurl

s/libucurl/libcurl/

And as discussed offlist, we should note that the builtin device flow
is not currently supported on Windows.

Thanks!
--Jacob

  211. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-19T14:13:35Z

    > On 18 Feb 2025, at 00:51, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Mon, Feb 17, 2025 at 10:15 AM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> It's been a little bit since I've re-run my
>> fuzzers, and a new Valgrind run would be a good idea, so I will just
>> keep throwing tests at it
> 
> Fuzzers are happy so far.
> 
> Valgrind did find something! A mistake I made during parameter
> discovery: setup_oauth_parameters() ensures that conn->oauth_issuer_id
> is always set using the "issuer" connection option, but during the
> second connection, I reassigned the pointer for it (and
> conn->oauth_discovery_uri) and leaked the previous allocations.

Nice.

> Reviews for the commit message:

All proposed changes applied.

The attached rebased has your 0002 fix as well as some minor tweaks like a few
small whitespace changes from a pgperltidy run and a copyright date fix which
still said 2024.

Unless something shows up I plan to commit this sometime tomorrow to allow it
ample time in the tree before the freeze.

--
Daniel Gustafsson

  212. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-19T19:54:24Z

    On Wed, Feb 19, 2025 at 6:13 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> The attached rebased has your 0002 fix as well as some minor tweaks like a few
> small whitespace changes from a pgperltidy run and a copyright date fix which
> still said 2024.

LGTM.

Thanks!
--Jacob

  213. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-20T16:29:58Z

    > On 19 Feb 2025, at 15:13, Daniel Gustafsson <daniel@yesql.se> wrote:

> Unless something shows up I plan to commit this sometime tomorrow to allow it
> ample time in the tree before the freeze.

I spent a few more hours staring at this, and ran it through a number of CI and
local builds, without anything showing up.  Pushed to master with the first set
of buildfarm animals showing green builds.

--
Daniel Gustafsson

  214. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-20T17:28:36Z

    On Thu, Feb 20, 2025 at 8:30 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> I spent a few more hours staring at this, and ran it through a number of CI and
> local builds, without anything showing up.  Pushed to master with the first set
> of buildfarm animals showing green builds.

Thank you!! And _huge thanks_ to everyone who's reviewed and provided
feedback. I'm going to start working with Andrew on getting the new
tests going in the buildfarm.

If you've been reading along and would like to get started with OAuth
validators and flows, but don't know where to start, please reach out.
The proof of the new APIs will be in the using, and the best time to
tell me if you hate those APIs is now :D

--Jacob

  215. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-02-20T20:07:21Z

    Daniel Gustafsson <daniel@yesql.se> writes:
> I spent a few more hours staring at this, and ran it through a number of CI and
> local builds, without anything showing up.  Pushed to master with the first set
> of buildfarm animals showing green builds.

After doing an in-tree "make check", I see

$ git status
On branch master
Your branch is up to date with 'origin/master'.

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        src/test/modules/oauth_validator/oauth_hook_client

Looks like we're short a .gitignore entry.  (It does appear that
"make clean" cleans it up, at least.)

			regards, tom lane

  216. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-20T20:21:45Z

    On Thu, Feb 20, 2025 at 12:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Looks like we're short a .gitignore entry.  (It does appear that
> "make clean" cleans it up, at least.)

So we are! Sorry about that. The attached patch gets in-tree builds
clean for me again.

Thanks,
--Jacob

  217. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-20T20:37:13Z

    > On 20 Feb 2025, at 21:21, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Thu, Feb 20, 2025 at 12:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Looks like we're short a .gitignore entry.  (It does appear that
>> "make clean" cleans it up, at least.)
> 
> So we are! Sorry about that. The attached patch gets in-tree builds
> clean for me again.

Fixed, thanks for the report!

--
Daniel Gustafsson

  218. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-02-21T17:18:57Z

    Hi,

On 2025-02-20 09:28:36 -0800, Jacob Champion wrote:
> On Thu, Feb 20, 2025 at 8:30 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > I spent a few more hours staring at this, and ran it through a number of CI and
> > local builds, without anything showing up.  Pushed to master with the first set
> > of buildfarm animals showing green builds.
>
> Thank you!! And _huge thanks_ to everyone who's reviewed and provided
> feedback. I'm going to start working with Andrew on getting the new
> tests going in the buildfarm.

+1


One question about interruptability. The docs say:

+    <varlistentry>
+     <term>Interruptibility</term>
+     <listitem>
+      <para>
+       Modules must remain interruptible by signals so that the server can
+       correctly handle authentication timeouts and shutdown signals from
+       <application>pg_ctl</application>. For example, a module receiving
+       <symbol>EINTR</symbol>/<symbol>EAGAIN</symbol> from a blocking call
+       should call <function>CHECK_FOR_INTERRUPTS()</function> before retrying.
+       The same should be done during any long-running loops. Failure to follow
+       this guidance may result in unresponsive backend sessions.
+      </para>
+     </listitem>
+    </varlistentry>

Is EINTR checking really sufficient?

I don't think we can generally rely on all blocking system calls to be
interruptible by signals on all platforms?

And, probably worse, isn't relying on getting EINTR rather racy, due to the
chance of the signal arriving between CHECK_FOR_INTERRUPTS() and the blocking
system call?

Afaict the only real way to do safely across platforms is to never call
blocking functions, e.g. by using non-blocking sockets and waiting for
readiness using latches.



And a second one about supporting !CURL_VERSION_ASYNCHDNS:

Is it a good idea to support that? We e.g. rely on libpq connections made by
the backend to be interruptible. Admittedly that's, I think, already not
bulletproof, due to libpq's DNS lookups going through libc if connection
string contains a name that needs to be looked up, but this seems to make that
a bit worse? With the connection string the DNS lookups can at least be
avoided, not that I think we properly document that...

Greetings,

Andres

  219. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-21T21:18:01Z

    > On 21 Feb 2025, at 18:18, Andres Freund <andres@anarazel.de> wrote:

> One question about interruptability. The docs say:
> ....

> Afaict the only real way to do safely across platforms is to never call
> blocking functions, e.g. by using non-blocking sockets and waiting for
> readiness using latches.

Fair point, we'll work on a proposed new wording for this.

> And a second one about supporting !CURL_VERSION_ASYNCHDNS:
> 
> Is it a good idea to support that?

We could block building instead of the current warning, but that's at best what
we can do.  I spent some time skimming over package definitions for the major
distributions ans OS's and couldn't find any that use sync dns.

--
Daniel Gustafsson

  220. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-02-23T16:49:21Z

    Daniel Gustafsson <daniel@yesql.se> writes:
> I spent a few more hours staring at this, and ran it through a number of CI and
> local builds, without anything showing up.  Pushed to master with the first set
> of buildfarm animals showing green builds.

Coverity has a nit-pick about this:

/srv/coverity/git/pgsql-git/postgresql/src/interfaces/libpq/fe-auth-oauth.c: 784 in setup_token_request()
778     		if (!request_copy)
779     		{
780     			libpq_append_conn_error(conn, "out of memory");
781     			goto fail;
782     		}
783     
>>>     CID 1643156:  High impact quality  (WRITE_CONST_FIELD)
>>>     A write to an aggregate overwrites a const-qualified field within the aggregate.
784     		memcpy(request_copy, &request, sizeof(request));
785     
786     		conn->async_auth = run_user_oauth_flow;
787     		conn->cleanup_async_auth = cleanup_user_oauth_flow;
788     		state->async_ctx = request_copy;
789     	}

This is evidently because of the fields declared const:

	/* Hook inputs (constant across all calls) */
	const char *const openid_configuration; /* OIDC discovery URI */
	const char *const scope;	/* required scope(s), or NULL */

IMO, the set of cases where it's legitimate to mark individual struct
fields as const is negligibly small, and this doesn't seem to be one
of them.  It's not obvious to me where/how PGoauthBearerRequest
structs are supposed to be constructed, but I find it hard to believe
that they will all spring full-grown from the forehead of Zeus.
Nonetheless, this declaration requires exactly that.

(I'm kind of surprised that we're not getting similar bleats from
any buildfarm animals, but so far I don't see any.)

BTW, as another nitpicky style matter: why do PGoauthBearerRequest
etc. spell their struct tag names differently from their typedef names
(that is, with/without an underscore)?  That is not our project style
anywhere else, and I'm failing to detect a good reason to do it here.

			regards, tom lane

  221. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-23T17:08:37Z

    > On 23 Feb 2025, at 17:49, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> Daniel Gustafsson <daniel@yesql.se> writes:
>> I spent a few more hours staring at this, and ran it through a number of CI and
>> local builds, without anything showing up.  Pushed to master with the first set
>> of buildfarm animals showing green builds.
> 
> Coverity has a nit-pick about this:
> 
> /srv/coverity/git/pgsql-git/postgresql/src/interfaces/libpq/fe-auth-oauth.c: 784 in setup_token_request()
> 778      if (!request_copy)
> 779      {
> 780      libpq_append_conn_error(conn, "out of memory");
> 781      goto fail;
> 782      }
> 783     
>>>>    CID 1643156:  High impact quality  (WRITE_CONST_FIELD)
>>>>    A write to an aggregate overwrites a const-qualified field within the aggregate.
> 784      memcpy(request_copy, &request, sizeof(request));
> 785     
> 786      conn->async_auth = run_user_oauth_flow;
> 787      conn->cleanup_async_auth = cleanup_user_oauth_flow;
> 788      state->async_ctx = request_copy;
> 789      }
> 
> This is evidently because of the fields declared const:
> 
> /* Hook inputs (constant across all calls) */
> const char *const openid_configuration; /* OIDC discovery URI */
> const char *const scope; /* required scope(s), or NULL */
> 
> IMO, the set of cases where it's legitimate to mark individual struct
> fields as const is negligibly small, and this doesn't seem to be one
> of them.

Thanks for the report, will fix.

> BTW, as another nitpicky style matter: why do PGoauthBearerRequest
> etc. spell their struct tag names differently from their typedef names
> (that is, with/without an underscore)?  That is not our project style
> anywhere else, and I'm failing to detect a good reason to do it here.

Indeed it isn't, the only explanation is that I missed it. Will fix.

--
Daniel Gustafsson

  222. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-23T23:45:32Z

    >> IMO, the set of cases where it's legitimate to mark individual struct
>> fields as const is negligibly small, and this doesn't seem to be one
>> of them.
> 
> Thanks for the report, will fix.
> 
>> BTW, as another nitpicky style matter: why do PGoauthBearerRequest
>> etc. spell their struct tag names differently from their typedef names
>> (that is, with/without an underscore)?  That is not our project style
>> anywhere else, and I'm failing to detect a good reason to do it here.
> 
> Indeed it isn't, the only explanation is that I missed it. Will fix.

The attached diff passes CI and works for me, will revisit in the morning.

--
Daniel Gustafsson

  223. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-24T09:00:25Z

    > On 24 Feb 2025, at 00:45, Daniel Gustafsson <daniel@yesql.se> wrote:

> The attached diff passes CI and works for me, will revisit in the morning.

Dave reported (on Discord) that the OPTIONAL macro collided with windef.h, so
attached is a small fix for that as well (even though we don't support Windows
here right now there is little point in clashing since we don't need that
particular macro name so rename anyways).

--
Daniel Gustafsson

  224. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrew Dunstan <andrew@dunslane.net> — 2025-02-24T14:41:12Z

    On 2025-02-20 Th 11:29 AM, Daniel Gustafsson wrote:
>> On 19 Feb 2025, at 15:13, Daniel Gustafsson<daniel@yesql.se> wrote:
>> Unless something shows up I plan to commit this sometime tomorrow to allow it
>> ample time in the tree before the freeze.
> I spent a few more hours staring at this, and ran it through a number of CI and
> local builds, without anything showing up.  Pushed to master with the first set
> of buildfarm animals showing green builds.


I notice that 001_server.pl contains this:


if ($ENV{with_python} ne 'yes')
{
     plan skip_all => 'OAuth tests require --with-python to run';
}


and various other things that insist on this. But I think all we should 
need is for Python to be present, whether or not we are building plpython.



cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com

  225. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-02-24T14:48:13Z

    > On 24 Feb 2025, at 15:41, Andrew Dunstan <andrew@dunslane.net> wrote:

> I notice that 001_server.pl contains this:
> 
> if ($ENV{with_python} ne 'yes')
> {
>     plan skip_all => 'OAuth tests require --with-python to run';
> }
> 
> and various other things that insist on this. But I think all we should need is for Python to be present, whether or not we are building plpython.

Agreed.  Right now the --with-python check is what runs PGAC_PATH_PYTHON but
maybe there is value in separating this going forward into a) have python; b)
have python and build plpython?

--
Daniel Gustafsson

  226. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-02-24T15:10:14Z

    Hi,

On 2025-02-24 15:48:13 +0100, Daniel Gustafsson wrote:
> > On 24 Feb 2025, at 15:41, Andrew Dunstan <andrew@dunslane.net> wrote:
> 
> > I notice that 001_server.pl contains this:
> > 
> > if ($ENV{with_python} ne 'yes')
> > {
> >     plan skip_all => 'OAuth tests require --with-python to run';
> > }
> > 
> > and various other things that insist on this. But I think all we should need is for Python to be present, whether or not we are building plpython.
> 
> Agreed.  Right now the --with-python check is what runs PGAC_PATH_PYTHON but
> maybe there is value in separating this going forward into a) have python; b)
> have python and build plpython?

FWIW, For other things we need in tests, e.g. gzip, we don't actually use a
configure test.  I think we could do the same for the python commandline
binary.

Greetings,

Andres Freund

  227. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-24T17:39:52Z

    On Fri, Feb 21, 2025 at 9:19 AM Andres Freund <andres@anarazel.de> wrote:
> I don't think we can generally rely on all blocking system calls to be
> interruptible by signals on all platforms?

Probably not; I wasn't sure how much detail to put in here after "must
remain interruptible."

> And, probably worse, isn't relying on getting EINTR rather racy, due to the
> chance of the signal arriving between CHECK_FOR_INTERRUPTS() and the blocking
> system call?

That is worse, and to be honest I hadn't ever thought about that race
condition before your email. So wait... how do people actually rely on
EINTR in production-grade client software? pselect/ppoll exist,
clearly, but they're used so rarely IME. (Have we all just been
subconsciously trained to mash Ctrl-C until the program finally stops?
I'm honestly kind of horrified by this revelation.)

Anyway, yes, this documentation clearly needs to be strengthened.

> Is it a good idea to support that? We e.g. rely on libpq connections made by
> the backend to be interruptible. Admittedly that's, I think, already not
> bulletproof, due to libpq's DNS lookups going through libc if connection
> string contains a name that needs to be looked up, but this seems to make that
> a bit worse?

A bit. The same for Kerberos, IIRC. Is the current configure warning
not strong enough to imply that the packager is on shaky ground? (I
patterned that off of the LDAP crash warning, which seemed much worse
to me. :D)

We can always declare non-support, I suppose, but anyone who doesn't
care about that problem (say, because they just want to use
single-connection psql) is then stuck hacking around it.

--Jacob

  228. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-24T17:39:57Z

    On Sun, Feb 23, 2025 at 8:49 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> IMO, the set of cases where it's legitimate to mark individual struct
> fields as const is negligibly small, and this doesn't seem to be one
> of them.  It's not obvious to me where/how PGoauthBearerRequest
> structs are supposed to be constructed, but I find it hard to believe
> that they will all spring full-grown from the forehead of Zeus.
> Nonetheless, this declaration requires exactly that.

As read-only inputs to the client API, they're not meant to be changed
for the lifetime of the struct (and the lifetime of the client flow).
The only place to initialize such a struct is directly above this
code.

> (I'm kind of surprised that we're not getting similar bleats from
> any buildfarm animals, but so far I don't see any.)

Is there a reason for compilers to complain? memcpy's the way I know
of to put a const-member struct on the heap, but maybe there are other
ways that don't annoy Coverity?

If the cost of this warning is too high, removing the const
declarations isn't the end of the world. But we use unconstify and
other type-punning copies in so many other places that this didn't
seem all that bad for the goal of helping out the client writer.

> BTW, as another nitpicky style matter: why do PGoauthBearerRequest
> etc. spell their struct tag names differently from their typedef names
> (that is, with/without an underscore)?  That is not our project style
> anywhere else, and I'm failing to detect a good reason to do it here.

This underscore pattern was copied directly from PQconninfoOption and
PQprintOpt.

Thanks,
--Jacob

  229. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-24T17:43:41Z

    On Mon, Feb 24, 2025 at 1:00 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> Dave reported (on Discord) that the OPTIONAL macro collided with windef.h,

Ugh.

> so
> attached is a small fix for that as well (even though we don't support Windows
> here right now there is little point in clashing since we don't need that
> particular macro name so rename anyways).

No objections to those patches, from inspection. (See note to Tom,
above, but I won't die on the const hill.)

Thanks!
--Jacob

  230. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-02-24T20:30:49Z

    Hi,

On 2025-02-24 09:39:52 -0800, Jacob Champion wrote:
> On Fri, Feb 21, 2025 at 9:19 AM Andres Freund <andres@anarazel.de> wrote:
> > And, probably worse, isn't relying on getting EINTR rather racy, due to the
> > chance of the signal arriving between CHECK_FOR_INTERRUPTS() and the blocking
> > system call?
> 
> That is worse, and to be honest I hadn't ever thought about that race
> condition before your email. So wait... how do people actually rely on
> EINTR in production-grade client software? pselect/ppoll exist,
> clearly, but they're used so rarely IME. (Have we all just been
> subconsciously trained to mash Ctrl-C until the program finally stops?
> I'm honestly kind of horrified by this revelation.)

If you need to handle the race it you need to combine it with something
additional, e.g. the so called "self pipe trick".  Which e.g. the latch / wait
event set code does.


> > Is it a good idea to support that? We e.g. rely on libpq connections made by
> > the backend to be interruptible. Admittedly that's, I think, already not
> > bulletproof, due to libpq's DNS lookups going through libc if connection
> > string contains a name that needs to be looked up, but this seems to make that
> > a bit worse?
> 
> A bit. The same for Kerberos, IIRC. Is the current configure warning
> not strong enough to imply that the packager is on shaky ground?

I don't think it's strong enough.


> (I patterned that off of the LDAP crash warning, which seemed much worse to
> me. :D)

I don't think that's a comparable case, because there were in-production uses
of PG+ldap that (kind of) worked. Whereas we start on a green field here.

Greetings,

Andres Freund

  231. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-24T22:02:04Z

    On Mon, Feb 24, 2025 at 12:30 PM Andres Freund <andres@anarazel.de> wrote:
> If you need to handle the race it you need to combine it with something
> additional, e.g. the so called "self pipe trick".  Which e.g. the latch / wait
> event set code does.

Right; I'm just used to that trick being deployed in massively
parallel async event engines rather than linear synchronous code
waiting on a single descriptor. I'm still a bit in disbelief, to be
honest. I'll get over it. Thank you for the note!

> > A bit. The same for Kerberos, IIRC. Is the current configure warning
> > not strong enough to imply that the packager is on shaky ground?
>
> I don't think it's strong enough.
>
> > (I patterned that off of the LDAP crash warning, which seemed much worse to
> > me. :D)
>
> I don't think that's a comparable case, because there were in-production uses
> of PG+ldap that (kind of) worked. Whereas we start on a green field here.

Fair enough. I'll work on a patch to disallow it; best case, no one
ever complains, and we've pruned an entire configuration from the list
of things to worry about.

Thanks!
--Jacob

  232. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-25T17:22:40Z

    On Mon, Feb 24, 2025 at 2:02 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Fair enough. I'll work on a patch to disallow it; best case, no one
> ever complains, and we've pruned an entire configuration from the list
> of things to worry about.

Here goes:

- 0001 fails configuration if the AsynchDNS feature is not built into libcurl.
- 0002 removes EINTR references from the validator documentation and
instead points authors towards our internal Wait APIs.
- 0003 is an optional followup to the const changes from upthread:
there's no need to memcpy() now, and anyone reading the code without
the history might wonder why I chose such a convoluted way to copy a
struct. :D

WDYT?

--Jacob

  233. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-02-28T13:43:52Z

    Hi,

If you trigger the new optional NetBSD CI task, the oauthvalidator
tests implode[1].  Apparently that OS's kevent() doesn't like zero
relative timeouts for EVFILT_TIMER[2].  I see that you worked around
the same problem for Linux timerfd already by rounding 0 up to 1, so
we could just do the same here, and it passes with the attached.  A
cute alternative, not tested, might be to put NOTE_ABSTIME into fflag
if timeout == 0 (then it's an absolute time in the past, which should
fire immediately).

But I'm curious, how hard would it be to do this ↓ instead and not
have that problem on any OS?

     * There might be an optimization opportunity here: if timeout == 0, we
     * could signal drive_request to immediately call
     * curl_multi_socket_action, rather than returning all the way up the
     * stack only to come right back. But it's not clear that the additional
     * code complexity is worth it.

[1] https://cirrus-ci.com/task/6354435774873600
[2] https://github.com/NetBSD/src/blob/67c7c4658e77aa4534b6aac8c041d77097c5e722/sys/kern/kern_event.c#L1375

  234. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-28T17:37:29Z

    On Fri, Feb 28, 2025 at 5:44 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> If you trigger the new optional NetBSD CI task, the oauthvalidator
> tests implode[1].

Oh, thank you for reporting that. I need to pay more attention to the
BSD CI thread.

> Apparently that OS's kevent() doesn't like zero
> relative timeouts for EVFILT_TIMER[2].  I see that you worked around
> the same problem for Linux timerfd already by rounding 0 up to 1, so
> we could just do the same here, and it passes with the attached.

Just from inspection, that looks good to me. I'll look into running
the new BSD tasks on the other patches I posted above, too.

Should we maybe consider just doing that across the board, and put up
with the inefficiency? Admittedly 1ms is a lot more dead time than
1ns...

> A
> cute alternative, not tested, might be to put NOTE_ABSTIME into fflag
> if timeout == 0 (then it's an absolute time in the past, which should
> fire immediately).

That could work. I think I need to stare at the man pages more if we
go that direction, since (IIUC) NOTE_ABSTIME changes up some other
default behavior for the timer.

> But I'm curious, how hard would it be to do this ↓ instead and not
> have that problem on any OS?
>
>      * There might be an optimization opportunity here: if timeout == 0, we
>      * could signal drive_request to immediately call
>      * curl_multi_socket_action, rather than returning all the way up the
>      * stack only to come right back. But it's not clear that the additional
>      * code complexity is worth it.

I'm not sure if it's hard, so much as it is confusing. My first
attempt at it a while back gave me the feeling that I wouldn't
remember how it worked in a few months.

Here are the things that I think we would have to consider, at minimum:
1. Every call to curl_multi_socket_action/all now has to look for the
new "time out immediately" flag after returning.
2. If it is set, and if actx->running has been set to zero, we have to
decide what that means conceptually. (Is that an impossible case? Or
do we ignore the timeout and assume the request is done/failed?)
3. Otherwise, we need to clear the flag and immediately call
curl_multi_socket_action(CURL_SOCKET_TIMEOUT), and repeat until that
flag is no longer set. That feels brittle to me, because if there's
some misunderstanding in our code or some strange corner case in an
old version of Curl on some platform, and we keep getting a timeout of
zero, we'll hit an infinite loop. (The current behavior instead
returns control to the top level every time, and gives
curl_multi_socket_all() a chance to right the ship by checking the
status of all the outstanding sockets.)
4. Even if 2 and 3 are just FUD, there's another potential call to
set_timer(0) in OAUTH_STEP_TOKEN_REQUEST. The interval can only be
zero in debug mode, so if we add new code for that case alone, the
tests will be mostly exercising a non-production code path.

I prefer your patch, personally.

Thanks!
--Jacob

  235. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-02-28T21:57:03Z

    On Fri, Feb 28, 2025 at 9:37 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Just from inspection, that looks good to me. I'll look into running
> the new BSD tasks on the other patches I posted above, too.

After your patch gets us past the zero timeout bug, NetBSD next runs into

    failed to fetch OpenID discovery document: Unsupported protocol
(libcurl: Received HTTP/0.9 when not allowed)'

...but only for a single test (nonempty oauth_client_secret), which is
very strange. And it doesn't always fail during the same HTTP request.
I'll look into it.

--Jacob

  236. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-02-28T23:38:37Z

    On Sat, Mar 1, 2025 at 6:37 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Should we maybe consider just doing that across the board, and put up
> with the inefficiency? Admittedly 1ms is a lot more dead time than
> 1ns...

Last time I checked, NetBSD is still using scheduler ticks (100hz
periodic interrupt) for all this kind of stuff so it's even worse than
that :-)

> I prefer your patch, personally.

Cool, I'll commit it shortly unless someone else comes up with a better idea.

  237. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-01T00:36:29Z

    On Sat, Mar 1, 2025 at 10:57 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> After your patch gets us past the zero timeout bug, NetBSD next runs into
>
>     failed to fetch OpenID discovery document: Unsupported protocol
> (libcurl: Received HTTP/0.9 when not allowed)'
>
> ...but only for a single test (nonempty oauth_client_secret), which is
> very strange. And it doesn't always fail during the same HTTP request.
> I'll look into it.

In case it's relevant, it was green for me, but I also ran it in
combination with my 3x-go-faster patch on that other thread. . o O {
Timing/race stuff?  Normally the build farm shakes that stuff out a
bit more reliably than CI, but I doubt libcurl is set up on many
animals... }

  238. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-04T00:07:35Z

    On Fri, Feb 28, 2025 at 4:37 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> In case it's relevant, it was green for me, but I also ran it in
> combination with my 3x-go-faster patch on that other thread. . o O {
> Timing/race stuff?  Normally the build farm shakes that stuff out a
> bit more reliably than CI, but I doubt libcurl is set up on many
> animals... }

That does help, thanks. Luckily, I can still sometimes reproduce with
that patch, which should speed things up nicely.

Commenting out the failing test causes the next test to fail with
basically the same error, so there's something stateful going on.
There are some suspicious messages that occasionally show up right
before the failure:

    # [libcurl] * IPv6: ::1
    # [libcurl] * IPv4: 127.0.0.1
    # [libcurl] *   Trying [::1]:65269...
    # [libcurl] * getsockname() failed with errno 22: Invalid argument
    # [libcurl] * connect to ::1 port 65269 from ::1 port 65270
failed: Connection refused
    # [libcurl] *   Trying 127.0.0.1:65269...
    # [libcurl] * Connected to localhost (127.0.0.1) port 65269

Later, Curl reconnects via IPv6 -- this time succeeding -- but then
the response gets mangled in some way. I assume headers are being
truncated, based on Curl's complaint about "HTTP/0.9".

The NetBSD man pages say that EINVAL is returned when the socket is
already shut down, suggesting some sort of bad interaction between
Curl and the test authorization server (and/or the OS?). I wonder if
my test server doesn't handle dual-stack setups correctly. I'll see if
I can get ktruss working on either side.

Thanks,
--Jacob

  239. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-04T04:10:49Z

    On Tue, Mar 4, 2025 at 1:07 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>     # [libcurl] * getsockname() failed with errno 22: Invalid argument

Weird.

> Later, Curl reconnects via IPv6 -- this time succeeding -- but then
> the response gets mangled in some way. I assume headers are being
> truncated, based on Curl's complaint about "HTTP/0.9".

And weirder.  With no evidence, I kinda wonder if that part could be a
bug in curl, if it gets a failure in an unexpected place like that and
gets confused, but let's start with the error...

> The NetBSD man pages say that EINVAL is returned when the socket is
> already shut down, suggesting some sort of bad interaction between
> Curl and the test authorization server (and/or the OS?). I wonder if
> my test server doesn't handle dual-stack setups correctly. I'll see if
> I can get ktruss working on either side.

POSIX says that about getsockname() too, as does the macOS man page,
but not those of FreeBSD, OpenBSD or Linux, but I don't think that's
the issue.  (From a quick peek: FreeBSD (in_getsockaddr) asserts that
sotoinpcb(so) is not NULL so it's always able to cough up the address
from the protocol control block object, while NetBSD (tcp_sockaddr)
and macOS/XNU (in6_getsockaddr) check for NULL and return EINVAL, so
at a wild guess, there may some path somewhere that knows the socket
is shutdown in both directions and all data has been drained so it can
be destroyed, and POSIX doesn't require sockets in this state to be
able to tell you their address so that's all OK?)

It's also unspecified if you haven't called connect() or bind() (well,
POSIX actually says that the address it gives you is unspecified, not
that the error is unspecified...).

I tried on a NetBSD 9 Vagrant box I had lying around, and ... ahh:

 28729      1 psql     write(0x2, 0x7f7fffddf3d0, 0x24) = 36
       "[libcurl] *   Trying [::1]:64244...\n"
 28729      1 psql     setsockopt(0x9, 0x6, 0x1, 0x7f7fffde015c, 0x4) = 0
 28729      1 psql     setsockopt(0x9, 0xffff, 0x800, 0x7f7fffde015c, 0x4) = 0
 26362      1 perl     __select50                  = 1
 28729      1 psql     connect                     Err#36 EINPROGRESS
 26362      1 perl     read                        = 36
 28729      1 psql     getsockname(0x9, 0x7f7fffde0240,
0x7f7fffde023c) Err#22 EINVAL

Other times it succeeds:

 28729      1 psql     connect                     Err#36 EINPROGRESS
 28729      1 psql     getsockname                 = 0

I think that is telling us that a non-blocking socket can be in a
state that is not yet connected enough even to tell you its local
address?  That is, connect() returns without having allocated a local
address, and does that part asynchronously too?  I don't know what to
think about that yet...

https://stackoverflow.com/questions/25333547/is-it-safe-to-call-getsockname-while-a-nonblocking-stream-socket-is-connecting

  240. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-04T17:08:00Z

    On Mon, Mar 3, 2025 at 4:07 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I wonder if
> my test server doesn't handle dual-stack setups correctly.

Spoilers: it's this.

> I'll see if
> I can get ktruss working on either side.

ktruss shows absolutely no syscall activity on the authorization
server during the failing test, because Curl's talking to something
else. sockstat confirms that I completely forgot to listen on IPv6 in
the test server. Dual stack sockets only work from the IPv6
direction...

There must be some law of conservation of weirdness, where the
strangest failure modes have the most boring explanations. I'll work
on a fix.

On Mon, Mar 3, 2025 at 8:11 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I think that is telling us that a non-blocking socket can be in a
> state that is not yet connected enough even to tell you its local
> address?  That is, connect() returns without having allocated a local
> address, and does that part asynchronously too?  I don't know what to
> think about that yet...

That is also really good to know, though. So that EINVAL message
might, in the end, be completely unrelated to the bug? (Curl doesn't
worry about the error, looks like, just prints it to the debug
stream.)

Thanks!
--Jacob

  241. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-04T22:37:27Z

    On Wed, Mar 5, 2025 at 6:08 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> ktruss shows absolutely no syscall activity on the authorization
> server during the failing test, because Curl's talking to something
> else. sockstat confirms that I completely forgot to listen on IPv6 in
> the test server. Dual stack sockets only work from the IPv6
> direction...
>
> There must be some law of conservation of weirdness, where the
> strangest failure modes have the most boring explanations. I'll work
> on a fix.

Heh, wow, that was confusing :-)  Actually I'm still confused (why
passing sometimes then?) but I'm sure all will become clear with your
patch...

> On Mon, Mar 3, 2025 at 8:11 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > I think that is telling us that a non-blocking socket can be in a
> > state that is not yet connected enough even to tell you its local
> > address?  That is, connect() returns without having allocated a local
> > address, and does that part asynchronously too?  I don't know what to
> > think about that yet...
>
> That is also really good to know, though. So that EINVAL message
> might, in the end, be completely unrelated to the bug? (Curl doesn't
> worry about the error, looks like, just prints it to the debug
> stream.)

Yeah.  I wonder if it only happens if the connection is doomed to fail
already, or something.  But I don't plan to dig further if it's
harmless (maybe curl shouldn't really do that, IDK).

  242. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-04T22:44:48Z

    On Tue, Mar 4, 2025 at 2:38 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> Heh, wow, that was confusing :-)  Actually I'm still confused (why
> passing sometimes then?)

Curl doesn't mind if the IPv6 connection fails outright; it'll use the
IPv4 in that case. But if something else ephemeral pops up on IPv6 and
starts speaking something that's not HTTP, that's a problem.

> but I'm sure all will become clear with your
> patch...

Maybe. My first attempt gets all the BSDs green except macOS -- which
now fails in a completely different test, haha... -_-

I'll keep you posted.

--Jacob

  243. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-06T20:57:24Z

    On Tue, Mar 4, 2025 at 2:44 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Maybe. My first attempt gets all the BSDs green except macOS -- which
> now fails in a completely different test, haha... -_-

Small update: there is not one bug, but three that interact. ಠ_ಠ

1) The test server advertises an issuer of `https://localhost:<port>`,
but it doesn't listen on all localhost interfaces. When Curl tries to
contact the issuer on IPv6, its Happy Eyeballs handling usually falls
back to IPv4 after discovering that IPv6 is nonfunctional, but
occasionally it contacts something that was temporarily listening
there instead.

Since I don't really want to write a bunch of IPv6 fallback code for
the test server -- this should be testing OAuth, not finding all the
ways that buildfarm OSes can expose dual stack sockets -- I changed
the issuer to be IPv4-only. When I did this, the interval timing tests
immediately failed on macOS.

2) macOS's EVFILT_TIMER implementation seems to be different from the
other BSDs. On Mac, when you re-add a timer to a kqueue, any existing
timer-fired events for it are not cleared out and the kqueue might
remain readable. This breaks a postcondition of our set_timer()
function, which is that new timeouts are supposed to completely
replace previous timeouts.

With a dual stack issuer, the Happy Eyeballs timeouts would be
routinely cleared out by libcurl, setting up a clean slate for the
next call to set_timer(). But with an IPv4-only issuer, libcurl didn't
need to clear out the timeouts (they'd already fired), which meant
that our call to set the ping interval was ineffective.

3) There is a related performance bug on other platforms. If a Curl
timeout happens partway through a request (so libcurl won't clear it),
the timer-expired event will stay set and CPU will be burned to spin
pointlessly on drive_request(). This is much easier to notice after
taking Happy Eyeballs out of the picture. It doesn't cause logical
failures -- Curl basically discards the unnecessary calls -- but it's
definitely unintended.

--

Problem 1 is a simple patch. I am working on a fix for Problem 2, but
I got stuck trying to get a "perfect" solution working yesterday...
Since this is a partial(?) blocker for getting NetBSD going, I'm going
to pivot to an ugly-but-simple approach today.

I plan to defer working on Problem 3, which should just be a
performance bug, until the tests are green again. And I would like to
eventually add some stronger unit tests for the timer behavior, to
catch other potential OS-specific problems in the future.

Thanks,
--Jacob

  244. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-07T00:35:43Z

    On Thu, Mar 6, 2025 at 12:57 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Problem 1 is a simple patch. I am working on a fix for Problem 2, but
> I got stuck trying to get a "perfect" solution working yesterday...
> Since this is a partial(?) blocker for getting NetBSD going, I'm going
> to pivot to an ugly-but-simple approach today.

Attached:
- 0001 fixes IPv6 failures,
- 0002 fixes set_timer() on Mac, and
- 0003-0005 are the existing followup patches from upthread.

Thanks!
--Jacob

  245. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-07T05:12:50Z

    On Fri, Mar 7, 2025 at 9:57 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> 2) macOS's EVFILT_TIMER implementation seems to be different from the
> other BSDs. On Mac, when you re-add a timer to a kqueue, any existing
> timer-fired events for it are not cleared out and the kqueue might
> remain readable. This breaks a postcondition of our set_timer()
> function, which is that new timeouts are supposed to completely
> replace previous timeouts.

I don't see that behaviour on my Mac with a simple program, and that
seems like it couldn't possibly be intended.  Hmm...  <browses source
code painfully> I wonder if this atomic generation scheme has a hole
in it, under concurrency...

https://github.com/apple-oss-distributions/xnu/blob/8d741a5de7ff4191bf97d57b9f54c2f6d4a15585/bsd/kern/kern_event.c#L1661

The code on the other OSes just dequeues it when reprogramming the
timer, which involves a lock and no doubt a few more cycles, and is
clearly not quite as exciting but ...

  246. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-07T17:30:52Z

    On Thu, Mar 6, 2025 at 9:13 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I don't see that behaviour on my Mac with a simple program, and that
> seems like it couldn't possibly be intended.

What version of macOS?

Just to make sure I'm not chasing ghosts, I've attached my test
program. Here are my CI results for running it:

= FreeBSD =

[      6 us] timer is set
[   1039 us] kqueue is readable
[   1050 us] timer is reset
[   1052 us] kqueue is not readable

= NetBSD =

[      3 us] timer is set
[  14993 us] kqueue is readable
[  15000 us] timer is reset
[  15002 us] kqueue is not readable

= OpenBSD =

[     24 us] timer is set
[  19660 us] kqueue is readable
[  19709 us] timer is reset
[  19712 us] kqueue is not readable

= macOS Sonoma =

[      4 us] timer is set
[   1282 us] kqueue is readable
[   1286 us] timer is reset
[   1287 us] kqueue is still readable

--Jacob

  247. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-07T21:51:41Z

    On Sat, Mar 8, 2025 at 6:31 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> On Thu, Mar 6, 2025 at 9:13 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> > I don't see that behaviour on my Mac with a simple program, and that
> > seems like it couldn't possibly be intended.
>
> What version of macOS?
>
> Just to make sure I'm not chasing ghosts, I've attached my test
> program. Here are my CI results for running it:

Ah, right, yeah I see that here too.  I thought you were saying that
kevent() could report an already triggered alarm even though we'd
replaced it (it doesn') but of course you meant poll(kq) as libpq
does.

I believe I know exactly why: kqueues are considered readable (by
poll/select/other kqueues) if there are any events queued[1].  Apple's
EVFILT_TIMER implementation is doing that trick[2] where it leaves
them queued, but filt_timerprocess() filters them out if its own
private _FIRED flag isn't set, so kevent() itself won't wake up or
return them.  That trick doesn't survive nesting.  I think I would
call that a bug.  (I think I would keep the atomic CAS piece -- it
means you don't have to drain the timer callout synchronously when
reprogramming it which is a cool trick, but I think they overshot when
they left the knote queued.)

Maybe just do the delete-and-add in one call?

    EV_SET(&ev[0], 1, EVFILT_TIMER, EV_DELETE, 0, 0, 0);
    EV_SET(&ev[1], 1, EVFILT_TIMER, EV_ADD | EV_ONESHOT, 0, timeout, 0);
    if (kevent(kq, &ev[0], 2, NULL, 0, NULL) < 0)

[1] https://github.com/apple-oss-distributions/xnu/blob/8d741a5de7ff4191bf97d57b9f54c2f6d4a15585/bsd/kern/kern_event.c#L1078
[2] https://github.com/apple-oss-distributions/xnu/blob/8d741a5de7ff4191bf97d57b9f54c2f6d4a15585/bsd/kern/kern_event.c#L1661

  248. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-07T23:02:50Z

    On Fri, Mar 7, 2025 at 1:52 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> I believe I know exactly why: kqueues are considered readable (by
> poll/select/other kqueues) if there are any events queued[1].  Apple's
> EVFILT_TIMER implementation is doing that trick[2] where it leaves
> them queued, but filt_timerprocess() filters them out if its own
> private _FIRED flag isn't set, so kevent() itself won't wake up or
> return them.  That trick doesn't survive nesting.  I think I would
> call that a bug.

Bleh. Thank you for the analysis!

> Maybe just do the delete-and-add in one call?
>
>     EV_SET(&ev[0], 1, EVFILT_TIMER, EV_DELETE, 0, 0, 0);
>     EV_SET(&ev[1], 1, EVFILT_TIMER, EV_ADD | EV_ONESHOT, 0, timeout, 0);
>     if (kevent(kq, &ev[0], 2, NULL, 0, NULL) < 0)

I think that requires me to copy the EV_RECEIPT handling from
register_socket(), to make sure an ENOENT is correctly ignored on
delete but doesn't mask failures from the addition. Do you prefer that
to the separate calls? (Or, better yet, is it easier than I'm making
it?)

Thanks!
--Jacob

  249. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Nazir Bilal Yavuz <byavuz81@gmail.com> — 2025-03-17T11:36:46Z

    Hi,

I just wanted to report that the 'oauth_validator/t/001_server.pl'
test failed on FreeBSD in one of my local CI runs [1]. I looked at the
thread but could not find the same error report; if this is already
known, please excuse me.

HEAD was at 3943f5cff6 and there were no other changes. Sharing the
failure here for visibility:

[11:09:56.548] stderr:
[11:09:56.548] #   Failed test 'stress-async: stdout matches'
[11:09:56.548] #   at
/tmp/cirrus-ci-build/src/test/modules/oauth_validator/t/001_server.pl
line 409.
[11:09:56.548] #                   ''
[11:09:56.548] #     doesn't match '(?^:connection succeeded)'
[11:09:56.548] #   Failed test 'stress-async: stderr matches'
[11:09:56.548] #   at
/tmp/cirrus-ci-build/src/test/modules/oauth_validator/t/001_server.pl
line 410.
[11:09:56.548] #                   '[libcurl] * Host localhost:39251
was resolved.
[11:09:56.548] # [libcurl] * IPv6: ::1
[11:09:56.548] # [libcurl] * IPv4: 127.0.0.1
[11:09:56.548] # [libcurl] *   Trying [::1]:39251...
[11:09:56.548] # [libcurl] * Immediate connect fail for ::1: Connection refused
[11:09:56.548] # [libcurl] *   Trying 127.0.0.1:39251...
[11:09:56.548] # [libcurl] * Connected to localhost (127.0.0.1) port 39251
[11:09:56.548] # [libcurl] * using HTTP/1.x
[11:09:56.548] # [libcurl] > GET
/param/.well-known/openid-configuration HTTP/1.1
[11:09:56.548] # [libcurl] > Host: localhost:39251
[11:09:56.548] # [libcurl] >
[11:09:56.548] # [libcurl] * Request completely sent off
[11:09:56.548] # [libcurl] * HTTP 1.0, assume close after body
[11:09:56.548] # [libcurl] < HTTP/1.0 200 OK
[11:09:56.548] # [libcurl] < Server: BaseHTTP/0.6 Python/3.11.11
[11:09:56.548] # [libcurl] < Date: Mon, 17 Mar 2025 11:09:55 GMT
[11:09:56.548] # [libcurl] < Content-Type: application/json
[11:09:56.548] # [libcurl] < Content-Length: 400
[11:09:56.548] # [libcurl] <
[11:09:56.548] # [libcurl] < {"issuer":
"http://localhost:39251/param", "token_endpoint":
"http://localhost:39251/param/token", "device_authorization_endpoint":
"http://localhost:39251/param/authorize", "response_types_supported":
["token"], "subject_types_supported": ["public"],
"id_token_signing_alg_values_supported": ["RS256"],
"grant_types_supported": ["authorization_code",
"urn:ietf:params:oauth:grant-type:device_code"]}
[11:09:56.548] # [libcurl] * shutting down connection #0
[11:09:56.548] # [libcurl] * Hostname localhost was found in DNS cache
[11:09:56.548] # [libcurl] *   Trying [::1]:39251...
[11:09:56.548] # [libcurl] * Immediate connect fail for ::1: Connection refused
[11:09:56.548] # [libcurl] *   Trying 127.0.0.1:39251...
[11:09:56.548] # [libcurl] * Connected to localhost (127.0.0.1) port 39251
[11:09:56.548] # [libcurl] * using HTTP/1.x
[11:09:56.548] # [libcurl] > POST /param/authorize HTTP/1.1
[11:09:56.548] # [libcurl] > Host: localhost:39251
[11:09:56.548] # [libcurl] > Content-Length: 92
[11:09:56.548] # [libcurl] > Content-Type: application/x-www-form-urlencoded
[11:09:56.548] # [libcurl] >
[11:09:56.548] # [libcurl] >
scope=openid+postgres&client_id=eyJpbnRlcnZhbCI6MSwicmV0cmllcyI6MSwic3RhZ2UiOiJhbGwifQ%3D%3D
[11:09:56.548] # [libcurl] * upload completely sent off: 92 bytes
[11:09:56.548] # [libcurl] * HTTP 1.0, assume close after body
[11:09:56.548] # [libcurl] < HTTP/1.0 200 OK
[11:09:56.548] # [libcurl] < Server: BaseHTTP/0.6 Python/3.11.11
[11:09:56.548] # [libcurl] < Date: Mon, 17 Mar 2025 11:09:55 GMT
[11:09:56.548] # [libcurl] < Content-Type: application/json
[11:09:56.548] # [libcurl] < Content-Length: 132
[11:09:56.548] # [libcurl] <
[11:09:56.548] # [libcurl] < {"device_code": "postgres", "user_code":
"postgresuser", "verification_uri": "https://example.com/",
"expires_in": 5, "interval": 1}
[11:09:56.548] # [libcurl] * shutting down connection #1
[11:09:56.548] # [libcurl] * Hostname localhost was found in DNS cache
[11:09:56.548] # [libcurl] *   Trying [::1]:39251...
[11:09:56.548] # [libcurl] * Connected to localhost (::1) port 39251
[11:09:56.548] # [libcurl] * using HTTP/1.x
[11:09:56.548] # [libcurl] > POST /param/token HTTP/1.1
[11:09:56.548] # [libcurl] > Host: localhost:39251
[11:09:56.548] # [libcurl] > Content-Length: 157
[11:09:56.548] # [libcurl] > Content-Type: application/x-www-form-urlencoded
[11:09:56.548] # [libcurl] >
[11:09:56.548] # [libcurl] >
device_code=postgres&grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code&client_id=eyJpbnRlcnZhbCI6MSwicmV0cmllcyI6MSwic3RhZ2UiOiJhbGwifQ%3D%3D
[11:09:56.548] # [libcurl] * upload completely sent off: 157 bytes
[11:09:56.548] # [libcurl] * Received HTTP/0.9 when not allowed
[11:09:56.548] # [libcurl] * closing connection #2
[11:09:56.548] # connection to database failed: connection to server
on socket "/tmp/xZKYtq40nL/.s.PGSQL.21400" failed: failed to obtain
access token: Unsupported protocol (libcurl: Received HTTP/0.9 when
not allowed)
[11:09:56.548] # '
[11:09:56.548] #           matches '(?^:connection to database failed)'
[11:09:56.548] # Looks like you failed 2 tests of 121.
[11:09:56.548]
[11:09:56.548] (test program exited with status code 2)

[1] https://cirrus-ci.com/task/4621590844932096

-- 
Regards,
Nazir Bilal Yavuz
Microsoft

  250. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-17T15:08:30Z

    On Mon, Mar 17, 2025 at 4:37 AM Nazir Bilal Yavuz <byavuz81@gmail.com> wrote:
>
> Hi,
>
> I just wanted to report that the 'oauth_validator/t/001_server.pl'
> test failed on FreeBSD in one of my local CI runs [1]. I looked at the
> thread but could not find the same error report; if this is already
> known, please excuse me.

Thanks for the report! Yes, this looks like the issue that NetBSD was having:

> [11:09:56.548] # [libcurl] *   Trying [::1]:39251...
> [11:09:56.548] # [libcurl] * Connected to localhost (::1) port 39251

Curl should not have connected to ::1 (the test server isn't listening
on IPv6). Whatever is talking on that port doesn't understand HTTP,
and we later fail with the "HTTP/0.9" error -- a slightly confusing
way to describe a protocol violation.

0001 will fix that. I think we should get that and 0002 in, ASAP. (And
the others.) Thomas has shown me a side quest to get rid of the second
kqueue instance, but so far that is not bearing fruit and we shouldn't
wait on it.

Thanks again!
--Jacob

  251. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T04:09:21Z

    On Tue, Mar 18, 2025 at 4:08 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> 0001 will fix that. I think we should get that and 0002 in, ASAP. (And
> the others.)

All pushed (wasn't sure if Daniel was going to but once I got tangled
up in all that kqueue stuff he probably quite reasonably assumed that
I would :-)).

> Thomas has shown me a side quest to get rid of the second
> kqueue instance, but so far that is not bearing fruit and we shouldn't
> wait on it.

Cool, thanks for looking into it anyway.  (Feel free to post a
nonworking patch with a got-stuck-here-because problem statement...)

  252. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-19T04:17:15Z

    Thomas Munro <thomas.munro@gmail.com> writes:
> All pushed

You may have noticed it already, but indri reports that this
printf-like call isn't right:

fe-auth-oauth-curl.c:1392:49: error: data argument not used by format string [-Werror,-Wformat-extra-args]
 1392 |                 actx_error(actx, "deleting kqueue timer: %m", timeout);
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~  ^
fe-auth-oauth-curl.c:324:59: note: expanded from macro 'actx_error'
  324 |         appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
      |                                                          ~~~     ^

"timeout" isn't being used anymore.

			regards, tom lane

  253. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T04:34:18Z

    On Wed, Mar 19, 2025 at 5:17 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> fe-auth-oauth-curl.c:1392:49: error: data argument not used by format string [-Werror,-Wformat-extra-args]
>  1392 |                 actx_error(actx, "deleting kqueue timer: %m", timeout);
>       |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~  ^
> fe-auth-oauth-curl.c:324:59: note: expanded from macro 'actx_error'
>   324 |         appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
>       |                                                          ~~~     ^
>
> "timeout" isn't being used anymore.

Yeah.  Thanks, fixed.

Now I'm wondering about teaching CI to fail on compiler warnings, ie
not just the special warnings task but also in the Mac etc builds.
The reason it doesn't is because it's sort of annoying to stop the
main tests because of a format string snafu, but we must be able to
put a new step at the end after all tests that scans the build logs
for warning and then raises the alarm...

  254. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-19T04:57:29Z

    BTW, I was pretty seriously disheartened just now to realize that
this feature was implemented by making libpq depend on libcurl.
I'd misread the relevant commit messages to say that libcurl was
just being used as test infrastructure; but nope, it's a genuine
build and runtime dependency.  I wonder how much big-picture
thinking went into that.  I can see at least two objections:

* This represents a pretty large expansion of dependency footprint,
not just for us but for the umpteen hundred packages that depend on
libpq.  libcurl alone maybe wouldn't be so bad, but have you looked
at libcurl's dependencies?  On RHEL8,

$ ldd /usr/lib64/libcurl.so.4.5.0
        linux-vdso.so.1 (0x00007fffd3075000)
        libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f992097a000)
        libidn2.so.0 => /lib64/libidn2.so.0 (0x00007f992075c000)
        libssh.so.4 => /lib64/libssh.so.4 (0x00007f99204ec000)
        libpsl.so.5 => /lib64/libpsl.so.5 (0x00007f99202db000)
        libssl.so.1.1 => /lib64/libssl.so.1.1 (0x00007f9920046000)
        libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x00007f991fb5b000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f991f906000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f991f61b000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f991f404000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f991f200000)
        libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x00007f991efb1000)
        liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x00007f991eda1000)
        libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007f991eb94000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f991e97c000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f991e75c000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f991e386000)
        libunistring.so.2 => /lib64/libunistring.so.2 (0x00007f991e005000)
        librt.so.1 => /lib64/librt.so.1 (0x00007f991ddfd000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f9920e30000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00007f991dbf9000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f991d9e8000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f991d7e4000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f991d5cc000)
        libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f991d3ae000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f991d02c000)
        libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007f991ce0b000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f991cbe0000)
        libcrypt.so.1 => /lib64/libcrypt.so.1 (0x00007f991c9b7000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f991c733000)

* Given libcurl's very squishy portfolio:

  libcurl is a free and easy-to-use client-side URL transfer library, supporting
  FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP,
  SMTP, POP3 and RTSP. libcurl supports SSL certificates, HTTP POST, HTTP PUT,
  FTP uploading, HTTP form based upload, proxies, cookies, user+password
  authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer
  resume, http proxy tunneling and more.

it's not exactly hard to imagine them growing a desire to handle
"postgresql://" URLs, which they would surely do by invoking libpq.
Then we'll have circular build dependencies and circular runtime
dependencies, not to mention inter-library recursion at runtime.


This is not quite a hill that I wish to die on, but I will
flatly predict that we will regret this.

			regards, tom lane

  255. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-03-19T13:31:42Z

    On Wed, Mar 19, 2025 at 12:57:29AM -0400, Tom Lane wrote:
> * Given libcurl's very squishy portfolio:
> 
>   libcurl is a free and easy-to-use client-side URL transfer library, supporting
>   FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP,
>   SMTP, POP3 and RTSP. libcurl supports SSL certificates, HTTP POST, HTTP PUT,
>   FTP uploading, HTTP form based upload, proxies, cookies, user+password
>   authentication (Basic, Digest, NTLM, Negotiate, Kerberos4), file transfer
>   resume, http proxy tunneling and more.
> 
> it's not exactly hard to imagine them growing a desire to handle
> "postgresql://" URLs, which they would surely do by invoking libpq.
> Then we'll have circular build dependencies and circular runtime
> dependencies, not to mention inter-library recursion at runtime.
> 
> 
> This is not quite a hill that I wish to die on, but I will
> flatly predict that we will regret this.

I regularly see curl security fixes in my Debian updates, so there is a
security issue that any serious curl bug could also make Postgres
vulnerable.  I might be willing to die on that hill.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  256. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-03-19T13:38:08Z

    > On 19 Mar 2025, at 05:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> 
> BTW, I was pretty seriously disheartened just now to realize that
> this feature was implemented by making libpq depend on libcurl.
> I'd misread the relevant commit messages to say that libcurl was
> just being used as test infrastructure; but nope, it's a genuine
> build and runtime dependency.  I wonder how much big-picture
> thinking went into that.

A considerable amount. 

libcurl is not a dependency for OAuth support in libpq, the support was
designed to be exensible such that clients can hook in their own flow
implementations.  This part does not require libcurl.  It is however a
dependency for the RFC 8628 implementation which is included when building with
--with-libcurl, this in order to ship something which can be used out of the
box (for actual connections *and* testing) without clients being forced to
provide their own implementation.

This obviously means that the RFC8628 part could be moved to contrib/, but I
fear we wouldn't make life easier for packagers by doing that.

> * Given libcurl's very squishy portfolio:
>  ...
> it's not exactly hard to imagine them growing a desire to handle
> "postgresql://" URLs,

While there is no guarantee that such a pull request wont be submitted,
speaking as a (admittedly not very active at the moment) libcurl maintainer I
consider it highly unlikely that it would be accepted.  A postgres connnection
does not fit into what libcurl/curl is and wants to be.

--
Daniel Gustafsson

  257. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-03-19T13:46:50Z

    Hi,

On 2025-03-19 17:34:18 +1300, Thomas Munro wrote:
> On Wed, Mar 19, 2025 at 5:17 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > fe-auth-oauth-curl.c:1392:49: error: data argument not used by format string [-Werror,-Wformat-extra-args]
> >  1392 |                 actx_error(actx, "deleting kqueue timer: %m", timeout);
> >       |                                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~  ^
> > fe-auth-oauth-curl.c:324:59: note: expanded from macro 'actx_error'
> >   324 |         appendPQExpBuffer(&(ACTX)->errbuf, libpq_gettext(FMT), ##__VA_ARGS__)
> >       |                                                          ~~~     ^
> >
> > "timeout" isn't being used anymore.
> 
> Yeah.  Thanks, fixed.
> 
> Now I'm wondering about teaching CI to fail on compiler warnings, ie
> not just the special warnings task but also in the Mac etc builds.
> The reason it doesn't is because it's sort of annoying to stop the
> main tests because of a format string snafu, but we must be able to
> put a new step at the end after all tests that scans the build logs
> for warning and then raises the alarm...

The best way would probably be to tee the output of the build to a log file
and then have a script at the end to check for errors in that.

Greetings,

Andres Freund

  258. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T20:11:43Z

    On Thu, Mar 20, 2025 at 2:38 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On 19 Mar 2025, at 05:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > BTW, I was pretty seriously disheartened just now to realize that
> > this feature was implemented by making libpq depend on libcurl.
> > I'd misread the relevant commit messages to say that libcurl was
> > just being used as test infrastructure; but nope, it's a genuine
> > build and runtime dependency.  I wonder how much big-picture
> > thinking went into that.
>
> A considerable amount.
>
> libcurl is not a dependency for OAuth support in libpq, the support was
> designed to be exensible such that clients can hook in their own flow
> implementations.  This part does not require libcurl.  It is however a
> dependency for the RFC 8628 implementation which is included when building with
> --with-libcurl, this in order to ship something which can be used out of the
> box (for actual connections *and* testing) without clients being forced to
> provide their own implementation.
>
> This obviously means that the RFC8628 part could be moved to contrib/, but I
> fear we wouldn't make life easier for packagers by doing that.

How feasible/fragile/weird would it be to dlopen() it on demand?
Looks like it'd take ~20 function pointers:

                 U curl_easy_cleanup
                 U curl_easy_escape
                 U curl_easy_getinfo
                 U curl_easy_init
                 U curl_easy_setopt
                 U curl_easy_strerror
                 U curl_free
                 U curl_global_init
                 U curl_multi_add_handle
                 U curl_multi_cleanup
                 U curl_multi_info_read
                 U curl_multi_init
                 U curl_multi_remove_handle
                 U curl_multi_setopt
                 U curl_multi_socket_action
                 U curl_multi_socket_all
                 U curl_multi_strerror
                 U curl_slist_append
                 U curl_slist_free_all
                 U curl_version_info

  259. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-19T21:03:59Z

    Thomas Munro <thomas.munro@gmail.com> writes:
> On Thu, Mar 20, 2025 at 2:38 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>> On 19 Mar 2025, at 05:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>>> BTW, I was pretty seriously disheartened just now to realize that
>>> this feature was implemented by making libpq depend on libcurl.

> How feasible/fragile/weird would it be to dlopen() it on demand?

FWIW, that would not really move the needle one bit so far as
my worries are concerned.  What I'm unhappy about is the very
sizable expansion of our build dependency footprint as well
as the sizable expansion of the 'package requires' footprint.
The fact that the new dependencies are mostly indirect doesn't
soften that blow at all.

To address that (without finding some less kitchen-sink-y OAuth
implementation to depend on), we'd need to shove the whole thing
into a separately-built, separately-installable package.

What I expect is likely to happen is that packagers will try to do
that themselves to avoid the dependency bloat.  AFAICT our current
setup will make that quite painful for them, and in any case I
don't believe it's work we should make them do.  If they fail to
do that, the burden of the extra dependencies will fall on end
users.  Either way, it's not going to make us look good.

			regards, tom lane

  260. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T22:02:57Z

    On Thu, Mar 20, 2025 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > How feasible/fragile/weird would it be to dlopen() it on demand?
>
> FWIW, that would not really move the needle one bit so far as
> my worries are concerned.  What I'm unhappy about is the very
> sizable expansion of our build dependency footprint as well
> as the sizable expansion of the 'package requires' footprint.
> The fact that the new dependencies are mostly indirect doesn't
> soften that blow at all.
>
> To address that (without finding some less kitchen-sink-y OAuth
> implementation to depend on), we'd need to shove the whole thing
> into a separately-built, separately-installable package.
>
> What I expect is likely to happen is that packagers will try to do
> that themselves to avoid the dependency bloat.  AFAICT our current
> setup will make that quite painful for them, and in any case I
> don't believe it's work we should make them do.  If they fail to
> do that, the burden of the extra dependencies will fall on end
> users.  Either way, it's not going to make us look good.

It would increase the build dependencies, assuming a package
maintainer wants to enable as many features as possible, but it would
*not* increase the 'package requires' footprint, merely the 'package
suggests' footprint (as Debian calls it), and it's up to the user
whether they install suggested extra packages, no?

  261. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T22:14:29Z

    On Thu, Mar 20, 2025 at 11:02 AM Thomas Munro <thomas.munro@gmail.com> wrote:
> On Thu, Mar 20, 2025 at 10:04 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Thomas Munro <thomas.munro@gmail.com> writes:
> > > How feasible/fragile/weird would it be to dlopen() it on demand?

. o O { There may also be security reasons to reject the idea, would
need to look into that... }

  262. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-19T22:19:23Z

    Thomas Munro <thomas.munro@gmail.com> writes:
> It would increase the build dependencies, assuming a package
> maintainer wants to enable as many features as possible, but it would
> *not* increase the 'package requires' footprint, merely the 'package
> suggests' footprint (as Debian calls it), and it's up to the user
> whether they install suggested extra packages, no?

Maybe I'm confused, but what I saw was a hard dependency on libcurl,
as well as several of its dependencies:

$ ./configure --with-libcurl
...
$ make
...
$ ldd src/interfaces/libpq/libpq.so.5.18 
        linux-vdso.so.1 (0x00007ffc145fe000)
        libcurl.so.4 => /lib64/libcurl.so.4 (0x00007f2c2fa36000)
        libm.so.6 => /lib64/libm.so.6 (0x00007f2c2f95b000)
        libc.so.6 => /lib64/libc.so.6 (0x00007f2c2f600000)
        libnghttp2.so.14 => /lib64/libnghttp2.so.14 (0x00007f2c2f931000)
        libidn2.so.0 => /lib64/libidn2.so.0 (0x00007f2c2f910000)
        libssh.so.4 => /lib64/libssh.so.4 (0x00007f2c2f89b000)
        libpsl.so.5 => /lib64/libpsl.so.5 (0x00007f2c2f885000)
        libssl.so.3 => /lib64/libssl.so.3 (0x00007f2c2f51a000)
        libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f2c2f000000)
        libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x00007f2c2f82f000)
        libkrb5.so.3 => /lib64/libkrb5.so.3 (0x00007f2c2ef26000)
        libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x00007f2c2f816000)
        libcom_err.so.2 => /lib64/libcom_err.so.2 (0x00007f2c2f80d000)
        libldap.so.2 => /lib64/libldap.so.2 (0x00007f2c2eebf000)
        liblber.so.2 => /lib64/liblber.so.2 (0x00007f2c2eead000)
        libbrotlidec.so.1 => /lib64/libbrotlidec.so.1 (0x00007f2c2ee9f000)
        libz.so.1 => /lib64/libz.so.1 (0x00007f2c2ee85000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f2c2fb43000)
        libunistring.so.2 => /lib64/libunistring.so.2 (0x00007f2c2ed00000)
        libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x00007f2c2ecef000)
        libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x00007f2c2ece8000)
        libresolv.so.2 => /lib64/libresolv.so.2 (0x00007f2c2ecd4000)
        libevent-2.1.so.7 => /lib64/libevent-2.1.so.7 (0x00007f2c2ec7b000)
        libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00007f2c2ec5b000)
        libbrotlicommon.so.1 => /lib64/libbrotlicommon.so.1 (0x00007f2c2ec38000)
        libselinux.so.1 => /lib64/libselinux.so.1 (0x00007f2c2ec0b000)
        libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f2c2ebd1000)
        libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f2c2eb35000)

I don't think that will be satisfied by 'package suggests'.
Even if it somehow manages to load, the result of trying to
use OAuth would be a segfault rather than any useful message.

			regards, tom lane

  263. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Thomas Munro <thomas.munro@gmail.com> — 2025-03-19T22:28:50Z

    On Thu, Mar 20, 2025 at 11:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Thomas Munro <thomas.munro@gmail.com> writes:
> > It would increase the build dependencies, assuming a package
> > maintainer wants to enable as many features as possible, but it would
> > *not* increase the 'package requires' footprint, merely the 'package
> > suggests' footprint (as Debian calls it), and it's up to the user
> > whether they install suggested extra packages, no?
>
> Maybe I'm confused, but what I saw was a hard dependency on libcurl,
> as well as several of its dependencies:

> I don't think that will be satisfied by 'package suggests'.
> Even if it somehow manages to load, the result of trying to
> use OAuth would be a segfault rather than any useful message.

I was imagining that it would just error out if you try to use that
stuff and it fails to open libcurl.  Then it's up to end users: if
they want to use libpq + OAuth, they have to install both libpq5 and
libcurl packages, and if they don't their connections will just fail,
presumably with some error message explaining why.  Or something like
that...

  264. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-03-19T23:11:28Z

    On Thu, Mar 20, 2025 at 11:28:50AM +1300, Thomas Munro wrote:
> On Thu, Mar 20, 2025 at 11:19 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Thomas Munro <thomas.munro@gmail.com> writes:
> > > It would increase the build dependencies, assuming a package
> > > maintainer wants to enable as many features as possible, but it would
> > > *not* increase the 'package requires' footprint, merely the 'package
> > > suggests' footprint (as Debian calls it), and it's up to the user
> > > whether they install suggested extra packages, no?
> >
> > Maybe I'm confused, but what I saw was a hard dependency on libcurl,
> > as well as several of its dependencies:
> 
> > I don't think that will be satisfied by 'package suggests'.
> > Even if it somehow manages to load, the result of trying to
> > use OAuth would be a segfault rather than any useful message.
> 
> I was imagining that it would just error out if you try to use that
> stuff and it fails to open libcurl.  Then it's up to end users: if
> they want to use libpq + OAuth, they have to install both libpq5 and
> libcurl packages, and if they don't their connections will just fail,
> presumably with some error message explaining why.  Or something like
> that...

Am I understanding that curl is being used just to honor the RFC and it
is only for testing?  That seems like a small reason to add such a
dependency.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  265. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-03-19T23:59:28Z

    On Wed, Mar 19, 2025 at 02:38:08PM +0100, Daniel Gustafsson wrote:
> > On 19 Mar 2025, at 05:57, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > 
> > BTW, I was pretty seriously disheartened just now to realize that
> > this feature was implemented by making libpq depend on libcurl.
> > I'd misread the relevant commit messages to say that libcurl was
> > just being used as test infrastructure; but nope, it's a genuine
> > build and runtime dependency.  I wonder how much big-picture
> > thinking went into that.
> 
> A considerable amount. 
> 
> libcurl is not a dependency for OAuth support in libpq, the support was
> designed to be exensible such that clients can hook in their own flow
> implementations.  This part does not require libcurl.  It is however a
> dependency for the RFC 8628 implementation which is included when building with
> --with-libcurl, this in order to ship something which can be used out of the
> box (for actual connections *and* testing) without clients being forced to
> provide their own implementation.
> 
> This obviously means that the RFC8628 part could be moved to contrib/, but I
> fear we wouldn't make life easier for packagers by doing that.

I see it now ---- without having RFC 8628 built into the server, clients
have to implement it.  Do we know what percentage would need to do that?
The spec:

	https://datatracker.ietf.org/doc/html/rfc8628

Do we think packagers will use the --with-libcurl configure option?

It does kind of make sense for curl to handle OAUTH since curl has to
simulate a browser.  I assume we can't call a shell to invoke curl from
the command line.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  266. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-20T20:33:26Z

    Hi all,

With the understanding that the patchset is no longer just "my" baby...

= Dependencies =

I like seeing risk/reward discussions. I agonized over the choice of
HTTP dependency, and I transitioned from an "easier" OAuth library
over to Curl early on because of the same tradeoffs.

That said... Tom, I think the dependency list you've presented is not
quite fair, because it doesn't show what libpq's dependency list was
before adding Curl. From your email, and my local Rocky 9 install, I
think these are the net-new dependencies that we (and packagers) need
to worry about:

        libcurl.so.4
        libnghttp2.so.14
        libidn2.so.0
        libssh.so.4
        libpsl.so.5
        libunistring.so.2
        libbrotlidec.so.1
        libbrotlicommon.so.1
        libz.so.1

That's more than I'd like, to be perfectly honest. I'm least happy
about libssh, because we're not using SFTP but we have to pay for it.
And the Deb-alikes add librtmp, which I'm not thrilled about either.

The rest are, IMO, natural dependencies of a mature HTTP client: the
HTTP/1 and HTTP/2 engines, Punycode, the Public Suffix List, UTF
handling, and common response compression types. Those are kind of
part and parcel of communicating on the web. (If we find an HTTP
client that does all those things itself, awesome, but then we have to
ask how well they did it.)

So one question for the collective is -- putting Curl itself aside --
is having a basic-but-usable OAuth flow, out of the box, worth the
costs of a generic HTTP client? A non-trivial footprint *will* be
there, whether it's one library or several, whether we delay-load it
or not, whether we have the unused SFTP/RTMP dependencies or not. But
we could still find ways to reduce that cost for people who aren't
using it, if necessary.

= Asides =

I would also like to point out: End users opt into this by
preregistering a client ID with an OAuth issuer ID, then providing
that pair of IDs in the connection string. We will not just start
crawling the web because a server tells us to. I don't want to
downplay the additional risk of having it in the address space, but
the design goal is that vulnerabilities in the HTTP logic should not
affect users who have not explicitly consented to the use of OAuth.

There were some other questions/statements made upthread that I want
to clarify too:

On Wed, Mar 19, 2025 at 4:11 PM Bruce Momjian <bruce@momjian.us> wrote:
> Am I understanding that curl is being used just to honor the RFC and it
> is only for testing?

No. (I see you found it later, but to state clearly for the record:
it's not just for testing.) libcurl is used for the Device
Authorization flow implementation. You don't have to use Device
Authorization to use OAuth, but we don't provide any alternative flows
in-tree; you'd have to use the libpq API to insert your own flow.

> I see it now ---- without having RFC 8628 built into the server,

(libpq, not the server. We do not ship server-side plugins at all, yet.)

> clients
> have to implement it.  Do we know what percentage would need to do that?

For version 1 of the feature, Device Authorization is the only option
for our utilities (psql et al). I can't really speculate on
percentages; it depends on what percentage want to use OAuth and don't
like (or can't use) our builtin flow. Obviously the percentage goes up
to 100% if we don't provide one. Plus we lose significant testability,
plus no one can use it from psql.

> Do we think packagers will use the --with-libcurl configure option?

Well, hopefully, yes. The tradeoffs of the builtin flow were chosen
explicitly so that existing clients could use it with minimal-to-no
code changes.

Thanks!
--Jacob

  267. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-03-20T20:50:57Z

    On Thu, Mar 20, 2025 at 01:33:26PM -0700, Jacob Champion wrote:
> That's more than I'd like, to be perfectly honest. I'm least happy
> about libssh, because we're not using SFTP but we have to pay for it.
> And the Deb-alikes add librtmp, which I'm not thrilled about either.
> 
> The rest are, IMO, natural dependencies of a mature HTTP client: the
> HTTP/1 and HTTP/2 engines, Punycode, the Public Suffix List, UTF
> handling, and common response compression types. Those are kind of
> part and parcel of communicating on the web. (If we find an HTTP
> client that does all those things itself, awesome, but then we have to
> ask how well they did it.)
> 
> So one question for the collective is -- putting Curl itself aside --
> is having a basic-but-usable OAuth flow, out of the box, worth the
> costs of a generic HTTP client? A non-trivial footprint *will* be
> there, whether it's one library or several, whether we delay-load it
> or not, whether we have the unused SFTP/RTMP dependencies or not. But
> we could still find ways to reduce that cost for people who aren't
> using it, if necessary.

One observation is that security scanning tools are going to see the
curl dependency and look at any CSVs related to them and ask us, whether
they are using OAUTH or not.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  268. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-03-20T21:08:54Z

    Bruce Momjian <bruce@momjian.us> writes:
> On Thu, Mar 20, 2025 at 01:33:26PM -0700, Jacob Champion wrote:
>> So one question for the collective is -- putting Curl itself aside --
>> is having a basic-but-usable OAuth flow, out of the box, worth the
>> costs of a generic HTTP client?

> One observation is that security scanning tools are going to see the
> curl dependency and look at any CSVs related to them and ask us, whether
> they are using OAUTH or not.

Yes.  Also, none of this has addressed my complaint about the extent
of the build and install dependencies.  Yes, simply not selecting
--with-libcurl removes the problem ... but most packagers are under
very heavy pressure to enable all features of a package.

From what's been said here, only a small minority of users are likely
to have any interest in this feature.  So my answer to "is it worth
the cost" is no, and would be no even if I had a lower estimate of
the costs.

I don't have any problem with making a solution available to those
users who want it --- but I really do NOT want this to be part of
stock libpq nor done as part of the core Postgres build.  I do not
think that the costs of that have been fully accounted for, especially
not the fact that almost all of those costs fall on people other than
us.

I'd like to see this moved out to some separate package that has to be
explicitly linked in and then hooks into libpq's custom-provider API.

			regards, tom lane

  269. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-03-20T23:26:49Z

    Hi,

On 2025-03-20 17:08:54 -0400, Tom Lane wrote:
> Bruce Momjian <bruce@momjian.us> writes:
> > On Thu, Mar 20, 2025 at 01:33:26PM -0700, Jacob Champion wrote:
> >> So one question for the collective is -- putting Curl itself aside --
> >> is having a basic-but-usable OAuth flow, out of the box, worth the
> >> costs of a generic HTTP client?
> 
> > One observation is that security scanning tools are going to see the
> > curl dependency and look at any CSVs related to them and ask us, whether
> > they are using OAUTH or not.
> 
> Yes.  Also, none of this has addressed my complaint about the extent
> of the build and install dependencies.  Yes, simply not selecting
> --with-libcurl removes the problem ... but most packagers are under
> very heavy pressure to enable all features of a package.

How about we provide the current libpq.so without linking to curl and also a
libpq-oauth.so that has curl support? If we do it right libpq-oauth.so would
itself link to libpq.so, making libpq-oauth.so a fairly small library.

That way packagers can split libpq-oauth.so into a separate package, while
still just building once.

That'd be a bit of work on the buildsystem side, but it seems doable.


> From what's been said here, only a small minority of users are likely
> to have any interest in this feature.  So my answer to "is it worth
> the cost" is no, and would be no even if I had a lower estimate of
> the costs.

I think this is likely going to be rather widely used, way more widely than
e.g. kerberos or ldap support in libpq. My understanding is that there's a
fair bit of pressure in lots of companies to centralize authentication towards
centralized systems, even for server applications.


> I don't have any problem with making a solution available to those
> users who want it --- but I really do NOT want this to be part of
> stock libpq nor done as part of the core Postgres build.  I do not
> think that the costs of that have been fully accounted for, especially
> not the fact that almost all of those costs fall on people other than
> us.

I am on board with not having it as part of stock libpq, but I don't see what
we gain by not building it as part of postgres (if the dependencies are
available, of course).

Greetings,

Andres Freund

  270. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andrew Dunstan <andrew@dunslane.net> — 2025-03-21T12:40:45Z

    On 2025-03-20 Th 7:26 PM, Andres Freund wrote:
> Hi,
>
> On 2025-03-20 17:08:54 -0400, Tom Lane wrote:
>> Bruce Momjian <bruce@momjian.us> writes:
>>> On Thu, Mar 20, 2025 at 01:33:26PM -0700, Jacob Champion wrote:
>>>> So one question for the collective is -- putting Curl itself aside --
>>>> is having a basic-but-usable OAuth flow, out of the box, worth the
>>>> costs of a generic HTTP client?
>>> One observation is that security scanning tools are going to see the
>>> curl dependency and look at any CSVs related to them and ask us, whether
>>> they are using OAUTH or not.
>> Yes.  Also, none of this has addressed my complaint about the extent
>> of the build and install dependencies.  Yes, simply not selecting
>> --with-libcurl removes the problem ... but most packagers are under
>> very heavy pressure to enable all features of a package.
> How about we provide the current libpq.so without linking to curl and also a
> libpq-oauth.so that has curl support? If we do it right libpq-oauth.so would
> itself link to libpq.so, making libpq-oauth.so a fairly small library.
>
> That way packagers can split libpq-oauth.so into a separate package, while
> still just building once.
>
> That'd be a bit of work on the buildsystem side, but it seems doable.
>

That certainly seems worth exploring.


>>  From what's been said here, only a small minority of users are likely
>> to have any interest in this feature.  So my answer to "is it worth
>> the cost" is no, and would be no even if I had a lower estimate of
>> the costs.
> I think this is likely going to be rather widely used, way more widely than
> e.g. kerberos or ldap support in libpq. My understanding is that there's a
> fair bit of pressure in lots of companies to centralize authentication towards
> centralized systems, even for server applications.


Indeed. There is still work to do on OAUTH2 but the demand you mention 
is just going to keep increasing.


>
>
>> I don't have any problem with making a solution available to those
>> users who want it --- but I really do NOT want this to be part of
>> stock libpq nor done as part of the core Postgres build.  I do not
>> think that the costs of that have been fully accounted for, especially
>> not the fact that almost all of those costs fall on people other than
>> us.
> I am on board with not having it as part of stock libpq, but I don't see what
> we gain by not building it as part of postgres (if the dependencies are
> available, of course).
>

+1.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

  271. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-03-21T18:02:02Z

    > On 21 Mar 2025, at 13:40, Andrew Dunstan <andrew@dunslane.net> wrote:
> On 2025-03-20 Th 7:26 PM, Andres Freund wrote:

>> How about we provide the current libpq.so without linking to curl and also a
>> libpq-oauth.so that has curl support? If we do it right libpq-oauth.so would
>> itself link to libpq.so, making libpq-oauth.so a fairly small library.
>> 
>> That way packagers can split libpq-oauth.so into a separate package, while
>> still just building once.
>> 
>> That'd be a bit of work on the buildsystem side, but it seems doable.
> 
> That certainly seems worth exploring.

This is being worked on.

--
Daniel Gustafsson

  272. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-03-26T19:09:14Z

    On Fri, Mar 21, 2025 at 11:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> >> How about we provide the current libpq.so without linking to curl and also a
> >> libpq-oauth.so that has curl support? If we do it right libpq-oauth.so would
> >> itself link to libpq.so, making libpq-oauth.so a fairly small library.
> >>
> >> That way packagers can split libpq-oauth.so into a separate package, while
> >> still just building once.
> >>
> >> That'd be a bit of work on the buildsystem side, but it seems doable.
> >
> > That certainly seems worth exploring.
>
> This is being worked on.

Attached is a proof of concept, with code from Daniel and myself,
which passes the CI as a starting point.

Roughly speaking, some things to debate are
- the module API itself
- how much to duplicate from libpq vs how much to export
- is this even what you had in mind

libpq-oauth.so is dlopen'd when needed. If it's not found or it
doesn't have the right symbols, builtin OAuth will not happen. Right
now we have an SO version of 1; maybe we want to remove the SO version
entirely to better indicate that it shouldn't be linked?

Two symbols are exported for the async authentication callbacks. Since
the module understands PGconn internals, maybe we could simplify this
to a single callback that manipulates the connection directly.

To keep the diff small to start, the current patch probably exports
too much. I think appendPQExpBufferVA makes sense, considering we
export much of the PQExpBuffer API already, but I imagine we won't
want to expose the pg_g_threadlock symbol. (libpq could maybe push
that pointer into the libpq-oauth module at load time, instead of
having the module pull it.) And we could probably go either way with
the PQauthDataHook; I prefer having a getter and setter for future
flexibility, but it would be simpler to just export the hook directly.

The following functions are duplicated from libpq:
- libpq_block_sigpipe
- libpq_reset_sigpipe
- libpq_binddomain
- libpq_[n]gettext
- libpq_append_conn_error
- oauth_unsafe_debugging_enabled

Those don't seem too bad to me, though maybe there's a good way to
deduplicate. But i18n needs further work. It builds right now, but I
don't think it works yet.

WDYT?

Thanks,
--Jacob

  273. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-03-31T14:06:49Z

    Re: Andres Freund
> > Yes.  Also, none of this has addressed my complaint about the extent
> > of the build and install dependencies.  Yes, simply not selecting
> > --with-libcurl removes the problem ... but most packagers are under
> > very heavy pressure to enable all features of a package.

And this feature is kind of only useful if it's available anywhere. If
only half of your clients are able to use SSO, you'd probably stick
with passwords anyway. So it needs to be enabled by default.

> How about we provide the current libpq.so without linking to curl and also a
> libpq-oauth.so that has curl support? If we do it right libpq-oauth.so would
> itself link to libpq.so, making libpq-oauth.so a fairly small library.
> 
> That way packagers can split libpq-oauth.so into a separate package, while
> still just building once.

That's definitely a good plan. The blast radius of build dependencies
isn't really a problem, the install/run-time is.

Perhaps we could do the same with libldap and libgssapi? (Though
admittedly I have never seen any complaints or nagging questions from
security people about these.)

Christoph

  274. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-01T22:40:49Z

    On Mon, Mar 31, 2025 at 7:06 AM Christoph Berg <myon@debian.org> wrote:
> Perhaps we could do the same with libldap and libgssapi? (Though
> admittedly I have never seen any complaints or nagging questions from
> security people about these.)

If we end up happy with how the Curl indirection works, that seems
like it'd be kind of nice in theory. I'm not sure how many people
would notice, though.

On Wed, Mar 26, 2025 at 12:09 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Right
> now we have an SO version of 1; maybe we want to remove the SO version
> entirely to better indicate that it shouldn't be linked?

Maybe a better idea would be to ship an SONAME of
`libpq-oauth.so.0.<major>`, without any symlinks, so that there's
never any ambiguity about which module belongs with which libpq.

--Jacob

  275. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-03T18:02:03Z

    On Tue, Mar 18, 2025 at 9:09 PM Thomas Munro <thomas.munro@gmail.com> wrote:
> All pushed (wasn't sure if Daniel was going to but once I got tangled
> up in all that kqueue stuff he probably quite reasonably assumed that
> I would :-)).

Attached are two more followups, separate from the libcurl split:
- 0001 is a patch originally proposed at [1]. Christoph pointed out
that the build fails on a platform that tries to enable Curl without
having either epoll() or kqueue(), due to a silly mistake I made in
the #ifdefs.
- 0002 should fix some timeouts in 002_client.pl reported by Andres on
Discord. I allowed a short connect_timeout to propagate into tests
which should not have it.

(The goal of 0001 is to get things building for now. After I finish
splitting the implementation into its own module, it'll make more
sense to simply not build that module on platforms that can't
implement a useful flow.)

Thanks,
--Jacob

[1] https://postgr.es/m/CAOYmi%2B%3D4898tXuTvb2LstorRo9JsAnBcn8LE%3DqrgVPiPW8ZfCw%40mail.gmail.com

  276. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-03T19:50:16Z

    > On 3 Apr 2025, at 20:02, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Tue, Mar 18, 2025 at 9:09 PM Thomas Munro <thomas.munro@gmail.com> wrote:
>> All pushed (wasn't sure if Daniel was going to but once I got tangled
>> up in all that kqueue stuff he probably quite reasonably assumed that
>> I would :-)).
> 
> Attached are two more followups, separate from the libcurl split:
> - 0001 is a patch originally proposed at [1]. Christoph pointed out
> that the build fails on a platform that tries to enable Curl without
> having either epoll() or kqueue(), due to a silly mistake I made in
> the #ifdefs.
> - 0002 should fix some timeouts in 002_client.pl reported by Andres on
> Discord. I allowed a short connect_timeout to propagate into tests
> which should not have it.

Thanks, both LGTM so pushed.

--
Daniel Gustafsson

  277. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-05T00:27:46Z

    On Thu, Apr 3, 2025 at 12:50 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Thanks, both LGTM so pushed.

Thank you!

On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Maybe a better idea would be to ship an SONAME of
> `libpq-oauth.so.0.<major>`, without any symlinks, so that there's
> never any ambiguity about which module belongs with which libpq.

While I was looking into this I found that Debian's going to use the
existence of an SONAME to check other things, which I assume will make
Christoph's life harder. I have switched over to
'libpq-oauth-<major>.so', without any SONAME or symlinks.

v2 simplifies quite a few things and breaks out the new duplicated
code into its own file. I pared down the exports from libpq, by having
it push the pg_g_threadlock pointer directly into the module when
needed. I think a future improvement would be to combine the dlopen
with the libcurl initialization, so that everything is done exactly
once and the module doesn't need to know about threadlocks at all.

i18n is still not working correctly on my machine. I've gotten `make
init-po` to put the files into the right places now, but if I fake a
.po file and install the generated .mo, the translations still don't
seem to be found at runtime. Is anyone able to take a quick look to
see if I'm missing something obvious?

I still need to disable the module entirely on Windows (and other
platforms without support), and potentially rename the --with-libcurl
option.

Thanks,
--Jacob

  278. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-06T20:48:16Z

    > On Thu, Apr 3, 2025 at 12:50 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> > Thanks, both LGTM so pushed.

Ack, the build there worked now. (Albeit without running any tests,
but let's not care too much about this snowflake architecture.)

> On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion
> While I was looking into this I found that Debian's going to use the
> existence of an SONAME to check other things, which I assume will make
> Christoph's life harder. I have switched over to
> 'libpq-oauth-<major>.so', without any SONAME or symlinks.

Since this is a plugin for libpq and nothing external is linking
directly to it, using a formal SONAME wouldn't gain anything, right.

Thanks,
Christoph

  279. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-04-07T13:59:19Z

    On 05.04.25 02:27, Jacob Champion wrote:
> On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> Maybe a better idea would be to ship an SONAME of
>> `libpq-oauth.so.0.<major>`, without any symlinks, so that there's
>> never any ambiguity about which module belongs with which libpq.
> 
> While I was looking into this I found that Debian's going to use the
> existence of an SONAME to check other things, which I assume will make
> Christoph's life harder. I have switched over to
> 'libpq-oauth-<major>.so', without any SONAME or symlinks.

Yes, this is correct.  We want a shared module, not a shared library, in 
meson parlance.

But the installation directory of a shared module should not be directly 
libdir.  That is reserved for libraries that you can link at build-time. 
  Here are some examples I found of other packages that have a library 
that itself has plugins:

https://packages.debian.org/bookworm/amd64/libaspell15/filelist
https://packages.debian.org/bookworm/amd64/libkrb5-3/filelist
https://packages.debian.org/bookworm/amd64/libmagickcore-6.q16-6/filelist


> v2 simplifies quite a few things and breaks out the new duplicated
> code into its own file. I pared down the exports from libpq, by having
> it push the pg_g_threadlock pointer directly into the module when
> needed. I think a future improvement would be to combine the dlopen
> with the libcurl initialization, so that everything is done exactly
> once and the module doesn't need to know about threadlocks at all.

Looks mostly ok.  I need the following patch to get it to build on macOS:

diff --git a/src/interfaces/libpq-oauth/Makefile 
b/src/interfaces/libpq-oauth/Makefile
index 461c44b59c1..d5469ca0e11 100644
--- a/src/interfaces/libpq-oauth/Makefile
+++ b/src/interfaces/libpq-oauth/Makefile
@@ -28,8 +28,9 @@ OBJS = \
     oauth-utils.o

  SHLIB_LINK_INTERNAL = $(libpq_pgport_shlib)
-SHLIB_LINK = -lcurl
+SHLIB_LINK = -lcurl $(filter -lintl, $(LIBS))
  SHLIB_PREREQS = submake-libpq
+BE_DLLLIBS =

  SHLIB_EXPORTS = exports.txt

(The second change is not strictly required, but it disables the use of 
-bundle_loader postgres, since this is not a backend-loadable module.)

I don't know whether we need an exports file for libpq-oauth.  The other 
shared modules don't provide an export file, and I'm not sure whether 
this is even supported for shared modules.  I guess it doesn't hurt?

The PKG_CONFIG_REQUIRES_PRIVATE setting in libpq-oauth/Makefile is 
meaningless and can be removed.

In fe-auth-oauth.c, I note that dlerror() is not necessarily 
thread-safe.  Since there isn't really an alternative, I guess it's ok 
to leave it like that, but I figured it should be mentioned.


> i18n is still not working correctly on my machine. I've gotten `make
> init-po` to put the files into the right places now, but if I fake a
> .po file and install the generated .mo, the translations still don't
> seem to be found at runtime. Is anyone able to take a quick look to
> see if I'm missing something obvious?

Not sure, the code looks correct at first glance.  However, you could 
also just keep the libpq-oauth strings in the libpq catalog.  There 
isn't really a need to make a separate one, since the versions you end 
up installing are locked to each other.  So you could for example in 
libpq's nls.mk just add

../libpq-oauth/oauth-curl.c

etc. to the files.

Maybe it would also make sense to make libpq-oauth a subdirectory of the 
libpq directory instead of a peer.

  280. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-07T14:21:33Z

    Hi,

On 2025-04-04 17:27:46 -0700, Jacob Champion wrote:
> += Load-Time ABI =
> +
> +This module ABI is an internal implementation detail, so it's subject to change
> +without warning, even during minor releases (however unlikely). The compiled
> +version of libpq-oauth should always match the compiled version of libpq.

Shouldn't we then include the *minor* version in the soname? I think otherwise
we run into the danger of the wrong library version being loaded in some
cases. Imagine a program being told with libpq to use via rpath. But then we
load the oauth module via a dlopen without a specified path - it'll just
search the global library locations.

Which actually makes me wonder if we ought to instead load the library from a
specific location...


> +TODO
> diff --git a/src/interfaces/libpq-oauth/exports.txt b/src/interfaces/libpq-oauth/exports.txt
> new file mode 100644
> index 00000000000..3787b388e04
> --- /dev/null
> +++ b/src/interfaces/libpq-oauth/exports.txt
> @@ -0,0 +1,4 @@
> +# src/interfaces/libpq-oauth/exports.txt
> +pg_fe_run_oauth_flow      1
> +pg_fe_cleanup_oauth_flow  2
> +pg_g_threadlock           3

The pg_g_threadlock thing seems pretty ugly. Can't we just pass that to the
relevant functions?  But more fundamentally, are we actually using
pg_g_threadlock anywhere? I removed all the releant code and the tests still
seem to pass?


> diff --git a/src/interfaces/libpq-oauth/meson.build b/src/interfaces/libpq-oauth/meson.build
> new file mode 100644
> index 00000000000..1834afbf7a5
> --- /dev/null
> +++ b/src/interfaces/libpq-oauth/meson.build
> @@ -0,0 +1,32 @@
> +# Copyright (c) 2022-2025, PostgreSQL Global Development Group
> +
> +if not libcurl.found() or host_system == 'windows'
> +  subdir_done()
> +endif

Why does this not work on windows? I don't see similar code in the removed
lines?

Greetings,

Andres Freund

  281. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-07T14:43:40Z

    Hi,

On 2025-04-07 15:59:19 +0200, Peter Eisentraut wrote:
> On 05.04.25 02:27, Jacob Champion wrote:
> > On Tue, Apr 1, 2025 at 3:40 PM Jacob Champion
> > <jacob.champion@enterprisedb.com> wrote:
> > > Maybe a better idea would be to ship an SONAME of
> > > `libpq-oauth.so.0.<major>`, without any symlinks, so that there's
> > > never any ambiguity about which module belongs with which libpq.
> > 
> > While I was looking into this I found that Debian's going to use the
> > existence of an SONAME to check other things, which I assume will make
> > Christoph's life harder. I have switched over to
> > 'libpq-oauth-<major>.so', without any SONAME or symlinks.
> 
> Yes, this is correct.  We want a shared module, not a shared library, in
> meson parlance.

It's not entirely obvious to me that we do want that.

There recently was a breakage of building with PG on macos with meson, due to
the meson folks implementing a feature request to move away from using
bundles, as
1) bundles apparently aren't supported on iOS
2) there apparently aren't any restrictions left that require using bundles,
   and there haven't been for a while.

They've now reverted these changes, due to the postgres build failures that
caused as well as recognizing they probably moved too fast, but the iOS
portion seems like it could be relevant for us?


Afaict this library doesn't have unresolved symbols, due to just linking to
libpq. So I don't think we really need this to be a shared module?


> But the installation directory of a shared module should not be directly
> libdir.

Agreed. However, it seems like relocatability would be much more important for
something like libpq than server modules... Particularly on windows it'll
often just be shipped alongside the executable - which won't work if we load
from pklibdir or such.


> I don't know whether we need an exports file for libpq-oauth.  The other
> shared modules don't provide an export file, and I'm not sure whether this
> is even supported for shared modules.  I guess it doesn't hurt?

It does seem just using PGDLLEXPORT would suffice here.


> The PKG_CONFIG_REQUIRES_PRIVATE setting in libpq-oauth/Makefile is
> meaningless and can be removed.
> 
> In fe-auth-oauth.c, I note that dlerror() is not necessarily thread-safe.

I sometimes really really hate posix.

Greetings,

Andres Freund

  282. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-04-07T16:38:19Z

    On 07.04.25 16:43, Andres Freund wrote:
>>> While I was looking into this I found that Debian's going to use the
>>> existence of an SONAME to check other things, which I assume will make
>>> Christoph's life harder. I have switched over to
>>> 'libpq-oauth-<major>.so', without any SONAME or symlinks.
>> Yes, this is correct.  We want a shared module, not a shared library, in
>> meson parlance.
> It's not entirely obvious to me that we do want that.
> 
> There recently was a breakage of building with PG on macos with meson, due to
> the meson folks implementing a feature request to move away from using
> bundles, as
> 1) bundles apparently aren't supported on iOS
> 2) there apparently aren't any restrictions left that require using bundles,
>     and there haven't been for a while.
> 
> They've now reverted these changes, due to the postgres build failures that
> caused as well as recognizing they probably moved too fast, but the iOS
> portion seems like it could be relevant for us?

Um, interesting.  AFAICT, the change you mention was reverted from the 
1.7 branch because it was accidentally backpatched, but it remains in 
master.

(For those just catching up:

https://github.com/mesonbuild/meson/issues/14240
https://github.com/mesonbuild/meson/pull/14340
https://github.com/mesonbuild/meson/commit/fa3f7e10b47d1f2f438f216f6c44f56076a01bfc
)

Overall, this seems like a good idea, as it removes a historical 
platform-specific particularity.  (I found a historical analysis at 
<https://stackoverflow.com/questions/2339679/>.)

But it does break existing users that add -bundle_loader, because 
-bundle_loader only works with -bundle and is rejected with -dynamiclib.

To test, I patched the makefiles to use -dynamiclib instead of -bundle, 
which also required removing -bundle_loader, and it also required adding 
-Wl,-undefined,dynamic_lookup.  This built correctly and generally worked.

But then you also run into a new variant of this issue:

https://www.postgresql.org/message-id/E1o4HOv-001Oyi-5n@gemulon.postgresql.org

Because there is no -bundle_loader, the symbol search order appears to 
be different, and so hash_search() gets found in the OS library first.

So this is all going to be a mess at some point sooner or later. :-(


> Afaict this library doesn't have unresolved symbols, due to just linking to
> libpq. So I don't think we really need this to be a shared module?

Apart from the hard distinction on macOS, in terms of the build system, 
the distinction between "library" and "module" is mainly whether the 
resulting library gets a soname, version symlinks, and what directory it 
is installed in, so in that sense the discussion so far indicates that 
it should be a module.  I suppose on macOS we could link it like a 
library and install it like a module, but that would effectively create 
a third category, and I don't see why that would be worth it.

  283. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-07T16:41:25Z

    Hi all,

Thanks for all the feedback! I'll combine them all into one email:

On Mon, Apr 7, 2025 at 6:59 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> Looks mostly ok.  I need the following patch to get it to build on macOS:
> [...]
> (The second change is not strictly required, but it disables the use of
> -bundle_loader postgres, since this is not a backend-loadable module.)

Hm, okay. I'll take a closer look at this, thanks.

> The PKG_CONFIG_REQUIRES_PRIVATE setting in libpq-oauth/Makefile is
> meaningless and can be removed.

Ah, right. Will do.

> In fe-auth-oauth.c, I note that dlerror() is not necessarily
> thread-safe.  Since there isn't really an alternative, I guess it's ok
> to leave it like that, but I figured it should be mentioned.

Yeah. The XXX comments there are a reminder to me to lock the stderr
printing behind debug mode, since I hope most non-packaging people are
not going to be troubleshooting load-time errors. But see the
threadlock discussions below.

> Not sure, the code looks correct at first glance.  However, you could
> also just keep the libpq-oauth strings in the libpq catalog.  There
> isn't really a need to make a separate one, since the versions you end
> up installing are locked to each other.  So you could for example in
> libpq's nls.mk just add
>
> ../libpq-oauth/oauth-curl.c
>
> etc. to the files.

Oh, that's an interesting idea. Thanks, I'll give it a try.

> Maybe it would also make sense to make libpq-oauth a subdirectory of the
> libpq directory instead of a peer.

Works for me.

On Mon, Apr 7, 2025 at 7:21 AM Andres Freund <andres@anarazel.de> wrote:
> On 2025-04-04 17:27:46 -0700, Jacob Champion wrote:
> > +This module ABI is an internal implementation detail, so it's subject to change
> > +without warning, even during minor releases (however unlikely). The compiled
> > +version of libpq-oauth should always match the compiled version of libpq.
>
> Shouldn't we then include the *minor* version in the soname?

No objection here.

> I think otherwise
> we run into the danger of the wrong library version being loaded in some
> cases. Imagine a program being told with libpq to use via rpath. But then we
> load the oauth module via a dlopen without a specified path - it'll just
> search the global library locations.

Ah, you mean if the RPATH'd build doesn't have a flow, but the
globally installed one (with a different ABI) does? Yeah, that would
be a problem.

> Which actually makes me wonder if we ought to instead load the library from a
> specific location...

We could hardcode the disk location for version 1, I guess. I kind of
liked giving packagers the flexibility they're used to having from the
ld.so architecture, though. See below.

> > +# src/interfaces/libpq-oauth/exports.txt
> > +pg_fe_run_oauth_flow      1
> > +pg_fe_cleanup_oauth_flow  2
> > +pg_g_threadlock           3
>
> The pg_g_threadlock thing seems pretty ugly. Can't we just pass that to the
> relevant functions?

We can do it however we want, honestly, especially since the ABI isn't
public/stable. I chose this way just to ease review.

> But more fundamentally, are we actually using
> pg_g_threadlock anywhere? I removed all the releant code and the tests still
> seem to pass?

If you have an older Curl installation, it is used. Newer libcurls
don't need it.

A future simplification could be to pull the use of the threadlock
back into libpq, and have it perform a one-time
dlopen-plus-Curl-initialization under the lock... That would also get
rid of the dlerror() thread safety problems. But that's an awful lot
of moving parts under a mutex, which makes me a little nervous.

> > diff --git a/src/interfaces/libpq-oauth/meson.build b/src/interfaces/libpq-oauth/meson.build
> > +if not libcurl.found() or host_system == 'windows'
> > +  subdir_done()
> > +endif
>
> Why does this not work on windows? I don't see similar code in the removed
> lines?

The Device Authorization flow is not currently implemented on Windows.

On Mon, Apr 7, 2025 at 7:43 AM Andres Freund <andres@anarazel.de> wrote:
> > Yes, this is correct.  We want a shared module, not a shared library, in
> > meson parlance.
>
> It's not entirely obvious to me that we do want that.
>
> There recently was a breakage of building with PG on macos with meson, due to
> the meson folks implementing a feature request to move away from using
> bundles, as
> 1) bundles apparently aren't supported on iOS
> 2) there apparently aren't any restrictions left that require using bundles,
>    and there haven't been for a while.

Could you explain how this is related to .app bundles? I thought I was
just building a standard dylib.

> Afaict this library doesn't have unresolved symbols, due to just linking to
> libpq. So I don't think we really need this to be a shared module?

Correct, no unresolved symbols. My naive understanding was that
distros were going to impose restrictions on an SONAME'd library that
we may not want to deal with.

> > But the installation directory of a shared module should not be directly
> > libdir.
>
> Agreed. However, it seems like relocatability would be much more important for
> something like libpq than server modules... Particularly on windows it'll
> often just be shipped alongside the executable - which won't work if we load
> from pklibdir or such.

Yeah, I really like the simplicity of "use the standard runtime
loader, just on-demand." Seems more friendly to the ecosystem.

Are there technical downsides of putting it into $libdir? I understand
there are "people" downsides, since we don't really want to signal
that this is a publicly linkable thing... but surely if you go through
the process of linking our library (which has no SONAME, includes the
major/minor version in its -l option, and has no pkgconfig) and
shoving a private pointer to a threadlock into it, you can keep all
the pieces when they break?

> > I don't know whether we need an exports file for libpq-oauth.  The other
> > shared modules don't provide an export file, and I'm not sure whether this
> > is even supported for shared modules.  I guess it doesn't hurt?
>
> It does seem just using PGDLLEXPORT would suffice here.

My motivation was to strictly identify the ABI that we intend libpq to
use, to try to future-proof things for everybody. Especially since
we're duplicating functions from libpq that we'd rather not be. (The
use of RTLD_LOCAL maybe makes that more of a belt-and-suspenders
thing.)

Are there any downsides to the exports file?

Thanks,
--Jacob

  284. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-07T16:52:57Z

    Hi,

On 2025-04-07 18:38:19 +0200, Peter Eisentraut wrote:
> On 07.04.25 16:43, Andres Freund wrote:
> > There recently was a breakage of building with PG on macos with meson, due to
> > the meson folks implementing a feature request to move away from using
> > bundles, as
> > 1) bundles apparently aren't supported on iOS
> > 2) there apparently aren't any restrictions left that require using bundles,
> >     and there haven't been for a while.
> >
> > They've now reverted these changes, due to the postgres build failures that
> > caused as well as recognizing they probably moved too fast, but the iOS
> > portion seems like it could be relevant for us?
>
> Um, interesting.  AFAICT, the change you mention was reverted from the 1.7
> branch because it was accidentally backpatched, but it remains in master.

I think the plan is to either redesign it in master or to revert it.


> (For those just catching up:
>
> https://github.com/mesonbuild/meson/issues/14240
> https://github.com/mesonbuild/meson/pull/14340
> https://github.com/mesonbuild/meson/commit/fa3f7e10b47d1f2f438f216f6c44f56076a01bfc
> )
>
> Overall, this seems like a good idea, as it removes a historical
> platform-specific particularity.  (I found a historical analysis at
> <https://stackoverflow.com/questions/2339679/>.)
>
> But it does break existing users that add -bundle_loader, because
> -bundle_loader only works with -bundle and is rejected with -dynamiclib.

Seems hard to imagine that somebody would inject -bundle_loader separately
from src/makefiles/Makefile.darwin?


> To test, I patched the makefiles to use -dynamiclib instead of -bundle,
> which also required removing -bundle_loader, and it also required adding
> -Wl,-undefined,dynamic_lookup.  This built correctly and generally worked.
>
> But then you also run into a new variant of this issue:
>
> https://www.postgresql.org/message-id/E1o4HOv-001Oyi-5n@gemulon.postgresql.org
>
> Because there is no -bundle_loader, the symbol search order appears to be
> different, and so hash_search() gets found in the OS library first.
>
> So this is all going to be a mess at some point sooner or later. :-(

Yikes, that is depressing / scary.  I wonder if we ought to rename our
hash_search with some macro magic or such regardless of using -bundle or not.


> > Afaict this library doesn't have unresolved symbols, due to just linking to
> > libpq. So I don't think we really need this to be a shared module?
>
> Apart from the hard distinction on macOS, in terms of the build system, the
> distinction between "library" and "module" is mainly whether the resulting
> library gets a soname, version symlinks, and what directory it is installed
> in, so in that sense the discussion so far indicates that it should be a
> module.

I don't think that happens if you don't specify a soname etc. And we'd need to
adjust the install dir either way, I think?


> I suppose on macOS we could link it like a library and install it
> like a module, but that would effectively create a third category, and I
> don't see why that would be worth it.

I think there are postgres clients for iphone, not sure if they use
libpq. Today libpq might actually cross-build successfully for iOS [1]. But if
we use shared_module() that won't be the case for libpq-oauth.

Anyway, I don't have a strong position on this, I just wanted to bring it up
for consideration.

Greetings,

Andres Freund

[1] I couldn't immediately quickly figure out how to install additional SDKs
on the commandline on macos an then gave up before attaching a monitor to my
mac mini.

  285. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-07T17:05:43Z

    Hi,

On 2025-04-07 09:41:25 -0700, Jacob Champion wrote:
> On Mon, Apr 7, 2025 at 7:21 AM Andres Freund <andres@anarazel.de> wrote:
> > On 2025-04-04 17:27:46 -0700, Jacob Champion wrote:
> > I think otherwise
> > we run into the danger of the wrong library version being loaded in some
> > cases. Imagine a program being told with libpq to use via rpath. But then we
> > load the oauth module via a dlopen without a specified path - it'll just
> > search the global library locations.
> 
> Ah, you mean if the RPATH'd build doesn't have a flow, but the
> globally installed one (with a different ABI) does? Yeah, that would
> be a problem.

That and more: Even if the RPATH libpq does have oauth support, the
libpq-oauth won't necessarily be at the front of the global library search
path.  So afaict you'll often get a different libpq-oauth.

Except perhaps on macos, where all this stuff works differently again. But I
managed to unload the required knowledge out of my brain and don't want to
further think about that :)



> > > +# src/interfaces/libpq-oauth/exports.txt
> > > +pg_fe_run_oauth_flow      1
> > > +pg_fe_cleanup_oauth_flow  2
> > > +pg_g_threadlock           3
> >
> > The pg_g_threadlock thing seems pretty ugly. Can't we just pass that to the
> > relevant functions?
> 
> We can do it however we want, honestly, especially since the ABI isn't
> public/stable. I chose this way just to ease review.

I found it rather confusing that libpq looks up a symbol and then sets
libpq-oauth's symbol to a pointer in libpq's namespace.


> > But more fundamentally, are we actually using
> > pg_g_threadlock anywhere? I removed all the releant code and the tests still
> > seem to pass?
> 
> If you have an older Curl installation, it is used. Newer libcurls
> don't need it.

Oh, wow. We hide the references to pg_g_threadlock behind a friggin macro?
That's just ...

Not your fault, I know...


> A future simplification could be to pull the use of the threadlock
> back into libpq, and have it perform a one-time
> dlopen-plus-Curl-initialization under the lock... That would also get
> rid of the dlerror() thread safety problems. But that's an awful lot
> of moving parts under a mutex, which makes me a little nervous.

I still think we should simply reject at configure time if curl init isn't
threadsafe ;)


> On Mon, Apr 7, 2025 at 7:43 AM Andres Freund <andres@anarazel.de> wrote:
> > > Yes, this is correct.  We want a shared module, not a shared library, in
> > > meson parlance.
> >
> > It's not entirely obvious to me that we do want that.
> >
> > There recently was a breakage of building with PG on macos with meson, due to
> > the meson folks implementing a feature request to move away from using
> > bundles, as
> > 1) bundles apparently aren't supported on iOS
> > 2) there apparently aren't any restrictions left that require using bundles,
> >    and there haven't been for a while.
> 
> Could you explain how this is related to .app bundles? I thought I was
> just building a standard dylib.

The other kind of bundles (what on earth apple was thinking with the naming
here I don't know). Stuff liked with ld -bundle.




> > > I don't know whether we need an exports file for libpq-oauth.  The other
> > > shared modules don't provide an export file, and I'm not sure whether this
> > > is even supported for shared modules.  I guess it doesn't hurt?
> >
> > It does seem just using PGDLLEXPORT would suffice here.
> 
> My motivation was to strictly identify the ABI that we intend libpq to
> use, to try to future-proof things for everybody. Especially since
> we're duplicating functions from libpq that we'd rather not be. (The
> use of RTLD_LOCAL maybe makes that more of a belt-and-suspenders
> thing.)

PGDLLEXPORT serves that purpose too, fwiw.  These days we use compiler flags
that restrict function visibility of everything not annotated with
PGDLLEXPORT.

However - that's all in c.h, port/win32.h,port/cygwin.h, , which libpq headers
might not want to include.


> Are there any downsides to the exports file?

I think it's fine either way.

Greetings,

Andres Freund

  286. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-07T17:32:31Z

    On Mon, Apr 7, 2025 at 10:05 AM Andres Freund <andres@anarazel.de> wrote:
> On 2025-04-07 09:41:25 -0700, Jacob Champion wrote:
> > Ah, you mean if the RPATH'd build doesn't have a flow, but the
> > globally installed one (with a different ABI) does? Yeah, that would
> > be a problem.
>
> That and more: Even if the RPATH libpq does have oauth support, the
> libpq-oauth won't necessarily be at the front of the global library search
> path.  So afaict you'll often get a different libpq-oauth.

ldopen() should respect RPATH, though? Either way, I agree with
pushing the minor version into the name (or else deciding that we will
keep the ABI completely stable across minor version bumps; not sure I
want to guarantee that just yet).

> > We can do it however we want, honestly, especially since the ABI isn't
> > public/stable. I chose this way just to ease review.
>
> I found it rather confusing that libpq looks up a symbol and then sets
> libpq-oauth's symbol to a pointer in libpq's namespace.

Yeah, I think a one-time init call would make this nicer.

> > A future simplification could be to pull the use of the threadlock
> > back into libpq, and have it perform a one-time
> > dlopen-plus-Curl-initialization under the lock... That would also get
> > rid of the dlerror() thread safety problems. But that's an awful lot
> > of moving parts under a mutex, which makes me a little nervous.
>
> I still think we should simply reject at configure time if curl init isn't
> threadsafe ;)

Practically speaking, I don't think that's a choice we can make. For
example, RHEL won't have threadsafe Curl until 10.

> > Could you explain how this is related to .app bundles? I thought I was
> > just building a standard dylib.
>
> The other kind of bundles (what on earth apple was thinking with the naming
> here I don't know). Stuff liked with ld -bundle.

Ah, some new corner-case magic to learn...

> These days we use compiler flags
> that restrict function visibility of everything not annotated with
> PGDLLEXPORT.

Hm, I missed/forgot that. That is nice. Personally I like having a
single file document the exports, so I'll keep it that way for now
unless there are objections, but it's good to know it's not necessary.

Thanks,
--Jacob

  287. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-07T18:47:18Z

    Re: Jacob Champion
> > > +This module ABI is an internal implementation detail, so it's subject to change
> > > +without warning, even during minor releases (however unlikely). The compiled
> > > +version of libpq-oauth should always match the compiled version of libpq.
> >
> > Shouldn't we then include the *minor* version in the soname?
> 
> No objection here.

Mmmmh. Since we are currently only talking about 3 symbols, it doesn't
sound very likely that we'd have to bump this in a major branch.
Putting the minor version into the filename would make looking at
package diffs harder when upgrading. Do we really need this as opposed
to some hardcoded number like libpq.so.5.18 ?

Perhaps reusing the number from libpq.so.5.18 also for this lib would
be the way to go?

> > Which actually makes me wonder if we ought to instead load the library from a
> > specific location...
> 
> We could hardcode the disk location for version 1, I guess. I kind of
> liked giving packagers the flexibility they're used to having from the
> ld.so architecture, though. See below.

pkglibdir or a subdirectory of it might make sense.

Though for Debian, I'd actually prefer
/usr/lib/$DEB_HOST_MULTIARCH/libpq/libpq-oauth...
since the libpq packaging is independent from the major version
packaging.

> Are there technical downsides of putting it into $libdir? I understand
> there are "people" downsides, since we don't really want to signal
> that this is a publicly linkable thing... but surely if you go through
> the process of linking our library (which has no SONAME, includes the
> major/minor version in its -l option, and has no pkgconfig) and
> shoving a private pointer to a threadlock into it, you can keep all
> the pieces when they break?

The Debian Policy expectation is that everything in libdir is a proper
library that could be linked to, and that random private stuff should
be elsewhere. But if being able to use the default lib search path is
an important argument, we could put it there.

Christoph

  288. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-07T21:49:43Z

    On Mon, Apr 7, 2025 at 11:47 AM Christoph Berg <myon@debian.org> wrote:
> Mmmmh. Since we are currently only talking about 3 symbols, it doesn't
> sound very likely that we'd have to bump this in a major branch.

The ABI extends to the pointers we're using, though. This module uses
PGconn* internals and libpq-int.h. [1]

> Putting the minor version into the filename would make looking at
> package diffs harder when upgrading. Do we really need this as opposed
> to some hardcoded number like libpq.so.5.18 ?
>
> Perhaps reusing the number from libpq.so.5.18 also for this lib would
> be the way to go?

That doesn't address Andres' concern, though; if multiple
installations all use libpq.so.5.18, they still can't necessarily mix
and match.

In fact you can't mix and match across different settings of
ENABLE_SSL/GSS/SSPI, either. So I guess that nudges me towards
pkglibdir/<some subdirectory>, to avoid major pain for some unlucky
end user.

> Though for Debian, I'd actually prefer
> /usr/lib/$DEB_HOST_MULTIARCH/libpq/libpq-oauth...
> since the libpq packaging is independent from the major version
> packaging.

Not sure I understand this. Do you mean you'd patch our lookup for
Debian, to find it there instead of pkglibdir? I don't think we can
adopt that ourselves, for the same reasons as above; the two sides
have to be in lockstep.

> The Debian Policy expectation is that everything in libdir is a proper
> library that could be linked to, and that random private stuff should
> be elsewhere. But if being able to use the default lib search path is
> an important argument, we could put it there.

I was hoping the default lib search would make your life (and ours)
easier. If it doesn't, I can lock it down.

Thanks,
--Jacob

[1] Future work ideas in this area include allowing other people to
compile their own loadable flow plugin, so that the utilities can use
it. (Only Device Authorization can be used by psql et al, for 18.) At
that point, developers will need a limited API to twiddle the
connection handle, and our builtin flow(s?) could use the same API.
But that's not work we can tackle for 18.

  289. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-07T21:58:23Z

    Re: Jacob Champion
> > Putting the minor version into the filename would make looking at
> > package diffs harder when upgrading. Do we really need this as opposed
> > to some hardcoded number like libpq.so.5.18 ?
> >
> > Perhaps reusing the number from libpq.so.5.18 also for this lib would
> > be the way to go?
> 
> That doesn't address Andres' concern, though; if multiple
> installations all use libpq.so.5.18, they still can't necessarily mix
> and match.

Well the whole world is linking against libpq5.so and not breaking
either. Why is this module worse? (I guess the answer is internal data
structures... but does it have to be worse?)

> > Though for Debian, I'd actually prefer
> > /usr/lib/$DEB_HOST_MULTIARCH/libpq/libpq-oauth...
> > since the libpq packaging is independent from the major version
> > packaging.
> 
> Not sure I understand this. Do you mean you'd patch our lookup for
> Debian, to find it there instead of pkglibdir? I don't think we can
> adopt that ourselves, for the same reasons as above; the two sides
> have to be in lockstep.

Because pkglibdir would be something like /usr/lib/postgresql/17/lib,
even when there is only one libpq5 package for all major server
versions on Debian. So if you have postgresql-16 installed, you'd end
up with

/usr/lib/postgresql/16/{bin,lib}  everything from PG 16
/usr/lib/x86_64-linux-gnu/libpq*  libpq5
/usr/lib/postgresql/17/lib/libpq-oauth.so

... which is weird.

> [1] Future work ideas in this area include allowing other people to
> compile their own loadable flow plugin, so that the utilities can use
> it. (Only Device Authorization can be used by psql et al, for 18.) At
> that point, developers will need a limited API to twiddle the
> connection handle, and our builtin flow(s?) could use the same API.
> But that's not work we can tackle for 18.

Perhaps keep things simple for PG18 and choose a simple filename and
location. If future extensions need something more elaborate, we can
still switch later.

Christoph

  290. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-07T22:26:47Z

    On Mon, Apr 7, 2025 at 2:58 PM Christoph Berg <myon@debian.org> wrote:
> Why is this module worse? (I guess the answer is internal data
> structures... but does it have to be worse?)

It doesn't have to be, in general, and the coupling surface is small
enough (libpq_append_conn_error) that we have a relatively easy path
toward decoupling it in the future. But for 18, I suspect no one will
be happy with me if I try to turn that inside out right this instant.
The goal was just to turn an internal implementation detail into a
delay-loaded internal implementation detail.

> Because pkglibdir would be something like /usr/lib/postgresql/17/lib,
> even when there is only one libpq5 package for all major server
> versions on Debian. So if you have postgresql-16 installed, you'd end
> up with
>
> /usr/lib/postgresql/16/{bin,lib}  everything from PG 16
> /usr/lib/x86_64-linux-gnu/libpq*  libpq5
> /usr/lib/postgresql/17/lib/libpq-oauth.so
>
> ... which is weird.

Weird, sure -- but it's correct, right? Because you have PG17's OAuth
flow installed.

If someone comes to the list with a flow bug in three years, and I ask
them what version they have installed, and they tell me "PG16, and
it's loading /usr/lib/aarch64-linux-gnu/libpq/libpq-oauth.so." That
won't be incredibly helpful IMHO.

> > [1] Future work ideas in this area include allowing other people to
> > compile their own loadable flow plugin, so that the utilities can use
> > it. (Only Device Authorization can be used by psql et al, for 18.) At
> > that point, developers will need a limited API to twiddle the
> > connection handle, and our builtin flow(s?) could use the same API.
> > But that's not work we can tackle for 18.
>
> Perhaps keep things simple for PG18 and choose a simple filename and
> location. If future extensions need something more elaborate, we can
> still switch later.

Sounds good. Any opinions from the gallery on what a "libpq plugin
subdirectory" in pkglibdir should be called? ("client", "modules",
"plugins"...?) Is there still a good reason to put any explicit
versioning into the filename if we do that?

Thanks,
--Jacob

  291. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T02:10:10Z

    On Mon, Apr 7, 2025 at 3:26 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Sounds good. Any opinions from the gallery on what a "libpq plugin
> subdirectory" in pkglibdir should be called? ("client", "modules",
> "plugins"...?)

Hm, one immediate consequence of hardcoding pkglibdir is that we can
no longer rely on LD_LIBRARY_PATH for pre-installation testing.
(Contrast with the server, which is able to relocate extension paths
based on its executable location.)

--Jacob

  292. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-08T10:34:02Z

    > On 8 Apr 2025, at 04:10, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Mon, Apr 7, 2025 at 3:26 PM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> Sounds good. Any opinions from the gallery on what a "libpq plugin
>> subdirectory" in pkglibdir should be called? ("client", "modules",
>> "plugins"...?)
> 
> Hm, one immediate consequence of hardcoding pkglibdir is that we can
> no longer rely on LD_LIBRARY_PATH for pre-installation testing.
> (Contrast with the server, which is able to relocate extension paths
> based on its executable location.)

That strikes me as a signifant drawback.

--
Daniel Gustafsson

  293. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T14:32:21Z

    On Tue, Apr  8, 2025 at 12:34:02PM +0200, Daniel Gustafsson wrote:
> > On 8 Apr 2025, at 04:10, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > 
> > On Mon, Apr 7, 2025 at 3:26 PM Jacob Champion
> > <jacob.champion@enterprisedb.com> wrote:
> >> Sounds good. Any opinions from the gallery on what a "libpq plugin
> >> subdirectory" in pkglibdir should be called? ("client", "modules",
> >> "plugins"...?)
> > 
> > Hm, one immediate consequence of hardcoding pkglibdir is that we can
> > no longer rely on LD_LIBRARY_PATH for pre-installation testing.
> > (Contrast with the server, which is able to relocate extension paths
> > based on its executable location.)
> 
> That strikes me as a signifant drawback.

Uh, where are we on the inclusion of curl in our build?  Maybe it was
explained but I have not seen it.  I still see
src/interfaces/libpq/fe-auth-oauth-curl.c.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  294. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T15:04:22Z

    On Tue, Apr 8, 2025 at 7:32 AM Bruce Momjian <bruce@momjian.us> wrote:
> Uh, where are we on the inclusion of curl in our build?  Maybe it was
> explained but I have not seen it.

The above is discussing a patch to split this into its own loadable
module. Andres and Christoph's feedback has been shaping where we put
that module, exactly.

Thanks,
--Jacob

  295. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-08T15:13:47Z

    > On 8 Apr 2025, at 17:04, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Tue, Apr 8, 2025 at 7:32 AM Bruce Momjian <bruce@momjian.us> wrote:
>> Uh, where are we on the inclusion of curl in our build?  Maybe it was
>> explained but I have not seen it.
> 
> The above is discussing a patch to split this into its own loadable
> module. Andres and Christoph's feedback has been shaping where we put
> that module, exactly.

There is also an open item registered for this.

--
Daniel Gustafsson

  296. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T15:13:51Z

    On Tue, Apr  8, 2025 at 08:04:22AM -0700, Jacob Champion wrote:
> On Tue, Apr 8, 2025 at 7:32 AM Bruce Momjian <bruce@momjian.us> wrote:
> > Uh, where are we on the inclusion of curl in our build?  Maybe it was
> > explained but I have not seen it.
> 
> The above is discussing a patch to split this into its own loadable
> module. Andres and Christoph's feedback has been shaping where we put
> that module, exactly.

Uh, I was afraid that was the case, which is why I asked.  We have just
hit feature freeze, so it is not good that we are still "shaping" the
patch.  Should we consider reverting it?  It is true we still "adjust"
patches after feature freeze.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  297. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-08T15:20:11Z

    Hi,

On 2025-04-08 11:13:51 -0400, Bruce Momjian wrote:
> On Tue, Apr  8, 2025 at 08:04:22AM -0700, Jacob Champion wrote:
> > On Tue, Apr 8, 2025 at 7:32 AM Bruce Momjian <bruce@momjian.us> wrote:
> > > Uh, where are we on the inclusion of curl in our build?  Maybe it was
> > > explained but I have not seen it.
> > 
> > The above is discussing a patch to split this into its own loadable
> > module. Andres and Christoph's feedback has been shaping where we put
> > that module, exactly.
> 
> Uh, I was afraid that was the case, which is why I asked.  We have just
> hit feature freeze, so it is not good that we are still "shaping" the
> patch.  Should we consider reverting it?  It is true we still "adjust"
> patches after feature freeze.

You brought the dependency concern up well after the feature was merged, after
it had been in development for a *long* time. It wasn't a secret that it had a
dependency on curl.  I don't think it's fair to penalize a feature's authors
to not finish implementing a complicated and completely new requirement within
17 days.

Greetings,

Andres Freund

  298. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T15:22:21Z

    On Tue, Apr  8, 2025 at 11:20:11AM -0400, Andres Freund wrote:
> Hi,
> 
> On 2025-04-08 11:13:51 -0400, Bruce Momjian wrote:
> > On Tue, Apr  8, 2025 at 08:04:22AM -0700, Jacob Champion wrote:
> > > On Tue, Apr 8, 2025 at 7:32 AM Bruce Momjian <bruce@momjian.us> wrote:
> > > > Uh, where are we on the inclusion of curl in our build?  Maybe it was
> > > > explained but I have not seen it.
> > > 
> > > The above is discussing a patch to split this into its own loadable
> > > module. Andres and Christoph's feedback has been shaping where we put
> > > that module, exactly.
> > 
> > Uh, I was afraid that was the case, which is why I asked.  We have just
> > hit feature freeze, so it is not good that we are still "shaping" the
> > patch.  Should we consider reverting it?  It is true we still "adjust"
> > patches after feature freeze.
> 
> You brought the dependency concern up well after the feature was merged, after
> it had been in development for a *long* time. It wasn't a secret that it had a
> dependency on curl.  I don't think it's fair to penalize a feature's authors
> to not finish implementing a complicated and completely new requirement within
> 17 days.

Fair point --- I was just asking.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  299. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-08T16:01:42Z

    Jacob Champion:
> The above is discussing a patch to split this into its own loadable
> module.

Wasn't sure where to put this exactly, the thread is long and I couldn't 
find any discussion around it:

How does the proposal with a loadable module affect a static libpq.a?

I have not tried, yet, but is my assumption correct, that I could build 
a libpq.a with oauth/curl support on current HEAD?

If yes, would that still be an option after the split?

Thanks,

Wolfgang

  300. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T16:14:32Z

    On Tue, Apr  8, 2025 at 06:01:42PM +0200, Wolfgang Walther wrote:
> Jacob Champion:
> > The above is discussing a patch to split this into its own loadable
> > module.
> 
> Wasn't sure where to put this exactly, the thread is long and I couldn't
> find any discussion around it:
> 
> How does the proposal with a loadable module affect a static libpq.a?
> 
> I have not tried, yet, but is my assumption correct, that I could build a
> libpq.a with oauth/curl support on current HEAD?
> 
> If yes, would that still be an option after the split?

How does this patch help us avoid having to handle curl CVEs and its
curl's additional dependencies?  As I understand the patch, it makes
libpq _not_ have additional dependencies but moves the dependencies to a
special loadable library that libpq can use.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  301. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T16:14:54Z

    On Tue, Apr 8, 2025 at 9:02 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> How does the proposal with a loadable module affect a static libpq.a?

The currently proposed patch would have you package and install a
separate .so module implementing OAuth, which the staticlib would load
once when needed. Similarly to how you still have to somehow
dynamically link your static app against Curl.

As a staticlib user, how do you feel about that?

> I have not tried, yet, but is my assumption correct, that I could build
> a libpq.a with oauth/curl support on current HEAD?

Yes.

--Jacob

  302. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T16:17:03Z

    On Tue, Apr 8, 2025 at 9:14 AM Bruce Momjian <bruce@momjian.us> wrote:
> How does this patch help us avoid having to handle curl CVEs and its
> curl's additional dependencies?  As I understand the patch, it makes
> libpq _not_ have additional dependencies but moves the dependencies to a
> special loadable library that libpq can use.

It allows packagers to ship the OAuth library separately, so end users
that don't want the additional exposure don't have to install it at
all.

--Jacob

  303. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-08T16:32:35Z

    Jacob Champion:
> The currently proposed patch would have you package and install a
> separate .so module implementing OAuth, which the staticlib would load
> once when needed. Similarly to how you still have to somehow
> dynamically link your static app against Curl.
>
> As a staticlib user, how do you feel about that?

When linking statically, I am producing entirely statically linked 
single binaries. Those contain libpq, all other dependencies, and would 
also contain curl.

The "entirely statically linked" thing is actually enforced by the build 
system (NixOS' pkgsStatic here), so dlopen() might just not be possible. 
Not exactly sure right now, whether it's stubbed out or just not 
available at all.

This means that shipping another .so file will not happen with this 
approach. Assuming OAuth will be picked up by some of the bigger 
providers, that would... make me feel quite bad about it, actually.

I'm not seeing the overall problem, yet. When I build with 
--enable-curl... ofc, I have a dependency on cURL. That's kind of the 
point. When I don't want that, then I just disable it. And that should 
also not be a problem for distributions - they could offer a libpq and a 
libpq_oauth package, where only one of them can be installed at the same 
time, I guess? *

Best,

Wolfgang

* Currently, the two build systems don't handle the "please build only 
libpq" scenario well. If that was supported better, building a second 
package with oauth support could be much easier.

  304. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T16:33:55Z

    On Tue, Apr  8, 2025 at 09:17:03AM -0700, Jacob Champion wrote:
> On Tue, Apr 8, 2025 at 9:14 AM Bruce Momjian <bruce@momjian.us> wrote:
> > How does this patch help us avoid having to handle curl CVEs and its
> > curl's additional dependencies?  As I understand the patch, it makes
> > libpq _not_ have additional dependencies but moves the dependencies to a
> > special loadable library that libpq can use.
> 
> It allows packagers to ship the OAuth library separately, so end users
> that don't want the additional exposure don't have to install it at
> all.

Okay, so how would they do that?  I understand how that would happen if
it was an external extension, but how if it is under /src or /contrib.

FYI, I see a good number of curl CVEs:

	https://curl.se/docs/security.html

Would we have to put out minor releases for curl CVEs?  I don't think we
have to for OpenSSL so would curl be the same?

I am asking these questions now so we can save time in getting this
closed.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  305. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-08T16:35:05Z

    Jacob Champion:
> It allows packagers to ship the OAuth library separately, so end users
> that don't want the additional exposure don't have to install it at
> all.

Ah, this came in after I sent my other mail, with this foot-note:

 > Currently, the two build systems don't handle the "please build only 
libpq" scenario well. If that was supported better, building a second 
package with oauth support could be much easier.

I think we should rather improve the build systems to handle this case, 
to give packagers more flexibility.

Best,

Wolfgang

  306. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-08T16:42:19Z

    > On 8 Apr 2025, at 18:33, Bruce Momjian <bruce@momjian.us> wrote:

> Would we have to put out minor releases for curl CVEs?  I don't think we
> have to for OpenSSL so would curl be the same?

Why do you envision this being different from all other dependencies we have?
For OpenSSL we also happily build against a version (and until recently,
several versions) which is EOL and don't even recieve security fixes.

--
Daniel Gustafsson

  307. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T16:43:01Z

    On Tue, Apr 8, 2025 at 9:33 AM Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, Apr  8, 2025 at 09:17:03AM -0700, Jacob Champion wrote:
> > It allows packagers to ship the OAuth library separately, so end users
> > that don't want the additional exposure don't have to install it at
> > all.
>
> Okay, so how would they do that?  I understand how that would happen if
> it was an external extension, but how if it is under /src or /contrib.

By adding the new .so to a different package. For example, RPM specs
would just let you say "hey, this .so I just built doesn't go into the
main client package, it goes into an add-on that depends on the client
package." It's the same way separate client and server packages get
generated from the same single build of Postgres.

> Would we have to put out minor releases for curl CVEs?

In general, no.

Thanks,
--Jacob

  308. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T16:48:15Z

    On Tue, Apr  8, 2025 at 06:42:19PM +0200, Daniel Gustafsson wrote:
> > On 8 Apr 2025, at 18:33, Bruce Momjian <bruce@momjian.us> wrote:
> 
> > Would we have to put out minor releases for curl CVEs?  I don't think we
> > have to for OpenSSL so would curl be the same?
> 
> Why do you envision this being different from all other dependencies we have?
> For OpenSSL we also happily build against a version (and until recently,
> several versions) which is EOL and don't even receive security fixes.

I don't know.  I know people scan our downloads and report when the
scanners detect something, but I am unclear what those scanners are
doing.  Would they see some new risks with curl?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  309. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T16:49:58Z

    On Tue, Apr  8, 2025 at 09:43:01AM -0700, Jacob Champion wrote:
> On Tue, Apr 8, 2025 at 9:33 AM Bruce Momjian <bruce@momjian.us> wrote:
> > On Tue, Apr  8, 2025 at 09:17:03AM -0700, Jacob Champion wrote:
> > > It allows packagers to ship the OAuth library separately, so end users
> > > that don't want the additional exposure don't have to install it at
> > > all.
> >
> > Okay, so how would they do that?  I understand how that would happen if
> > it was an external extension, but how if it is under /src or /contrib.
> 
> By adding the new .so to a different package. For example, RPM specs
> would just let you say "hey, this .so I just built doesn't go into the
> main client package, it goes into an add-on that depends on the client
> package." It's the same way separate client and server packages get
> generated from the same single build of Postgres.

Do we have any idea how many packagers are interested in doing this?

> > Would we have to put out minor releases for curl CVEs?
> 
> In general, no.

Good.

FYI, I saw bug bounty dollar amounts next to each curl CVE:

	https://curl.se/docs/security.html

No wonder some people ask for bounties.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  310. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T16:50:26Z

    On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> And that should also not be a problem for distributions - they could offer a libpq and a libpq_oauth package, where only one of them can be installed at the same time, I guess? *

My outsider understanding is that maintaining this sort of thing
becomes a major headache, because of combinatorics. You don't really
want to ship a libpq and libpq-with-gss and libpq-with-oauth and
libpq-with-oauth-and-gss and ...

--Jacob

  311. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-08T16:57:18Z

    Jacob Champion:
> On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
>> And that should also not be a problem for distributions - they could offer a libpq and a libpq_oauth package, where only one of them can be installed at the same time, I guess? *
> My outsider understanding is that maintaining this sort of thing
> becomes a major headache, because of combinatorics. You don't really
> want to ship a libpq and libpq-with-gss and libpq-with-oauth and
> libpq-with-oauth-and-gss and ...

That would only be the case, if you were to consider those other 
dependencies as "dangerous" as cURL. But we already depend on them. So 
if it's really the case that cURL is that much worse, that we consider 
loading it as a module... then the combinatorics should not be a problem 
either.

However, if the other deps are considered problematic as well, then the 
ship has already sailed, and there is not point for a special case here 
anymore.

Best,

Wolfgang

  312. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:00:56Z

    On Tue, Apr 8, 2025 at 9:49 AM Bruce Momjian <bruce@momjian.us> wrote:
> On Tue, Apr  8, 2025 at 09:43:01AM -0700, Jacob Champion wrote:
> > By adding the new .so to a different package. For example, RPM specs
> > would just let you say "hey, this .so I just built doesn't go into the
> > main client package, it goes into an add-on that depends on the client
> > package." It's the same way separate client and server packages get
> > generated from the same single build of Postgres.
>
> Do we have any idea how many packagers are interested in doing this?

I'm not sure how to answer this. The primary drivers from the dev side
are you and Tom, I think. Christoph seems to be on board with a split
as long as we don't make his life harder. Wolfgang appears to be a
packager who would not make use of a split (and in fact cannot).

--Jacob

  313. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T17:02:11Z

    On Tue, Apr  8, 2025 at 06:57:18PM +0200, Wolfgang Walther wrote:
> Jacob Champion:
> > On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> > > And that should also not be a problem for distributions - they could offer a libpq and a libpq_oauth package, where only one of them can be installed at the same time, I guess? *
> > My outsider understanding is that maintaining this sort of thing
> > becomes a major headache, because of combinatorics. You don't really
> > want to ship a libpq and libpq-with-gss and libpq-with-oauth and
> > libpq-with-oauth-and-gss and ...
> 
> That would only be the case, if you were to consider those other
> dependencies as "dangerous" as cURL. But we already depend on them. So if
> it's really the case that cURL is that much worse, that we consider loading
> it as a module... then the combinatorics should not be a problem either.
> 
> However, if the other deps are considered problematic as well, then the ship
> has already sailed, and there is not point for a special case here anymore.

Yes, I think this is what I am asking too.  For me it was curl's
security reputation and whether that would taint the security reputation
of libpq. For Tom, I think it was the dependency additions.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  314. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T17:03:10Z

    On Tue, Apr  8, 2025 at 10:00:56AM -0700, Jacob Champion wrote:
> On Tue, Apr 8, 2025 at 9:49 AM Bruce Momjian <bruce@momjian.us> wrote:
> > On Tue, Apr  8, 2025 at 09:43:01AM -0700, Jacob Champion wrote:
> > > By adding the new .so to a different package. For example, RPM specs
> > > would just let you say "hey, this .so I just built doesn't go into the
> > > main client package, it goes into an add-on that depends on the client
> > > package." It's the same way separate client and server packages get
> > > generated from the same single build of Postgres.
> >
> > Do we have any idea how many packagers are interested in doing this?
> 
> I'm not sure how to answer this. The primary drivers from the dev side
> are you and Tom, I think. Christoph seems to be on board with a split
> as long as we don't make his life harder. Wolfgang appears to be a
> packager who would not make use of a split (and in fact cannot).

Okay, I have just posted a more detailed email about my security
concern, so let's look at that.  I am ready to admit I am wrong.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  315. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:04:26Z

    On Tue, Apr 8, 2025 at 9:57 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> if it's really the case that cURL is that much worse

(it is not, but I am sympathetic to the argument that if you don't use
it, you don't need it in the process space)

> However, if the other deps are considered problematic as well, then the
> ship has already sailed, and there is not point for a special case here
> anymore.

I think this line of argument is unlikely to find traction. Upthread
there were people asking if we could maybe split out other
possibly-unused dependencies in the future, like Kerberos.

--Jacob

  316. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-08T17:10:07Z

    Jacob Champion:
>> However, if the other deps are considered problematic as well, then the
>> ship has already sailed, and there is not point for a special case here
>> anymore.
> I think this line of argument is unlikely to find traction. Upthread
> there were people asking if we could maybe split out other
> possibly-unused dependencies in the future, like Kerberos.

Well, yes, that's kind of what I'm saying. There shouldn't be a special 
case for cURL, but those other deps should be handled equally as well.

And if that means making libpq modular at run-time, then this should be 
planned and built with all deps, and other use-cases (like static 
linking) in mind - and not like it is right now.

Best,

Wolfgang

  317. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:13:46Z

    On Tue, Apr 8, 2025 at 10:10 AM Wolfgang Walther
<walther@technowledgy.de> wrote:
> And if that means making libpq modular at run-time, then this should be planned and built with all deps, and other use-cases (like static linking) in mind - and not like it is right now.

I think that'd be neat in concept, but specifically this thread is
discussing a PG18 open item. For future releases, if we're happy with
how Curl gets split out, maybe that would be fuel for other
delay-loaded client dependencies. I'm not sure.

--Jacob

  318. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T17:15:47Z

    On Tue, Apr  8, 2025 at 10:13:46AM -0700, Jacob Champion wrote:
> On Tue, Apr 8, 2025 at 10:10 AM Wolfgang Walther
> <walther@technowledgy.de> wrote:
> > And if that means making libpq modular at run-time, then this should be planned and built with all deps, and other use-cases (like static linking) in mind - and not like it is right now.
> 
> I think that'd be neat in concept, but specifically this thread is
> discussing a PG18 open item. For future releases, if we're happy with
> how Curl gets split out, maybe that would be fuel for other
> delay-loaded client dependencies. I'm not sure.

Well, if we think we are going to do that, it seems we would need a
different architecture than the one being proposed for PG 18, which
could lead to a lot of user/developer API churn.

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  319. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:19:21Z

    On Tue, Apr 8, 2025 at 10:15 AM Bruce Momjian <bruce@momjian.us> wrote:
> Well, if we think we are going to do that, it seems we would need a
> different architecture than the one being proposed for PG 18, which
> could lead to a lot of user/developer API churn.

A major goal of the current patch proposal is to explicitly hide this
from the end-user and public APIs. So it can be changed without public
breakage. It can't be hidden from packagers, of course, but that's the
point of the feature request.

--Jacob

  320. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:36:08Z

    On Tue, Apr 8, 2025 at 3:34 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On 8 Apr 2025, at 04:10, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > Hm, one immediate consequence of hardcoding pkglibdir is that we can
> > no longer rely on LD_LIBRARY_PATH for pre-installation testing.
> > (Contrast with the server, which is able to relocate extension paths
> > based on its executable location.)
>
> That strikes me as a signifant drawback.

Yeah, but it's one of those things that feels like it must have been
solved by the others in the space. Once it's installed, the concern
goes away (unless you demand absolute relocatability without
recompilation). I'll take a look at how libkrb/libmagick do their
testing.

If it somehow turns out to be impossible, one option might be to shove
a more detailed ABI identifier into the name. In other words, builds
without ENABLE_SSL/GSS/SSPI or whatever get different names on disk.
That doesn't scale at all, but it's a short-term option that would put
more pressure on a medium-term stable ABI.

Thanks,
--Jacob

  321. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T17:44:58Z

    On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> This means that shipping another .so file will not happen with this approach. Assuming OAuth will be picked up by some of the bigger providers, that would... make me feel quite bad about it, actually.

It occurs to me that I didn't respond to this point explicitly. I
would like to avoid making your life harder.

Would anybody following along be opposed to a situation where
- dynamiclib builds go through the dlopen() shim
- staticlib builds always rely on statically linked symbols

Or do we need to be able to mix and match?

--Jacob

  322. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-08T18:11:19Z

    Hi,

On 2025-04-08 13:02:11 -0400, Bruce Momjian wrote:
> On Tue, Apr  8, 2025 at 06:57:18PM +0200, Wolfgang Walther wrote:
> > Jacob Champion:
> > > On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> > > > And that should also not be a problem for distributions - they could offer a libpq and a libpq_oauth package, where only one of them can be installed at the same time, I guess? *
> > > My outsider understanding is that maintaining this sort of thing
> > > becomes a major headache, because of combinatorics. You don't really
> > > want to ship a libpq and libpq-with-gss and libpq-with-oauth and
> > > libpq-with-oauth-and-gss and ...
> > 
> > That would only be the case, if you were to consider those other
> > dependencies as "dangerous" as cURL. But we already depend on them. So if
> > it's really the case that cURL is that much worse, that we consider loading
> > it as a module... then the combinatorics should not be a problem either.
> > 
> > However, if the other deps are considered problematic as well, then the ship
> > has already sailed, and there is not point for a special case here anymore.
> 
> Yes, I think this is what I am asking too.  For me it was curl's
> security reputation and whether that would taint the security reputation
> of libpq. For Tom, I think it was the dependency additions.

I'd say that curl's security reputation is higher than most of our other
dependencies. We have dependencies for libraries with regular security issues,
with those issues at times not getting addressed for prolonged amounts of
time.

I do agree that there's an issue increasing libpq's indirect footprint
substantially, but I don't think that's due to curl's reputation or
anything. It's just needing a significantly higher number of shared libraries,
which comes with runtime and distribution overhead.

Greetings,

Andres Freund

  323. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Bruce Momjian <bruce@momjian.us> — 2025-04-08T18:25:20Z

    On Tue, Apr  8, 2025 at 02:11:19PM -0400, Andres Freund wrote:
> Hi,
> 
> On 2025-04-08 13:02:11 -0400, Bruce Momjian wrote:
> > On Tue, Apr  8, 2025 at 06:57:18PM +0200, Wolfgang Walther wrote:
> > > Jacob Champion:
> > > > On Tue, Apr 8, 2025 at 9:32 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> > > > > And that should also not be a problem for distributions - they could offer a libpq and a libpq_oauth package, where only one of them can be installed at the same time, I guess? *
> > > > My outsider understanding is that maintaining this sort of thing
> > > > becomes a major headache, because of combinatorics. You don't really
> > > > want to ship a libpq and libpq-with-gss and libpq-with-oauth and
> > > > libpq-with-oauth-and-gss and ...
> > > 
> > > That would only be the case, if you were to consider those other
> > > dependencies as "dangerous" as cURL. But we already depend on them. So if
> > > it's really the case that cURL is that much worse, that we consider loading
> > > it as a module... then the combinatorics should not be a problem either.
> > > 
> > > However, if the other deps are considered problematic as well, then the ship
> > > has already sailed, and there is not point for a special case here anymore.
> > 
> > Yes, I think this is what I am asking too.  For me it was curl's
> > security reputation and whether that would taint the security reputation
> > of libpq. For Tom, I think it was the dependency additions.
> 
> I'd say that curl's security reputation is higher than most of our other
> dependencies. We have dependencies for libraries with regular security issues,
> with those issues at times not getting addressed for prolonged amounts of
> time.

I see curl CVEs regularly as part of Debian minor updates, which is why
I had concerns, but if it is similar to OpenSSL, and better than other
libraries that don't even get CVEs, I guess it okay.  However, is this
true for libpq libraries or database server libraries.  Does it matter?

-- 
  Bruce Momjian  <bruce@momjian.us>        https://momjian.us
  EDB                                      https://enterprisedb.com

  Do not let urgent matters crowd out time for investment in the future.

  324. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T19:07:51Z

    On Tue, Apr 8, 2025 at 10:36 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Yeah, but it's one of those things that feels like it must have been
> solved by the others in the space. Once it's installed, the concern
> goes away (unless you demand absolute relocatability without
> recompilation). I'll take a look at how libkrb/libmagick do their
> testing.

Perhaps unsurprisingly, they inject different lookup paths via
envvars. We could do the same (I have FUD about the security
characteristics)...

> If it somehow turns out to be impossible, one option might be to shove
> a more detailed ABI identifier into the name.

...but I wonder if I can invert the dependency on
libpq_append_conn_error entirely, and remove that part of the ABI
surface, then revisit the discussion on `-<major>.so` vs
`-<major>-<minor>.so`.

--Jacob

  325. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T19:22:55Z

    On Tue, Apr 8, 2025 at 11:25 AM Bruce Momjian <bruce@momjian.us> wrote:
> However, is this
> true for libpq libraries or database server libraries.  Does it matter?

The dependency on Curl is through libpq. We have some server-side
features that pull in libpq and would transitively depend on Curl. But
for Curl to be initialized server-side, the two peers still have to
agree on the use of OAuth.

It seems unlikely that users would opt into that for, say,
postgres_fdw in PG18, because the Device Authorization flow is the
only one we currently ship, and it's intended for end users. A flow
that prints a code to stderr is not very helpful for your proxy.

--Jacob

  326. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-08T21:32:09Z

    On Mon, Apr 7, 2025 at 9:41 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> > Not sure, the code looks correct at first glance.  However, you could
> > also just keep the libpq-oauth strings in the libpq catalog.  There
> > isn't really a need to make a separate one, since the versions you end
> > up installing are locked to each other.  So you could for example in
> > libpq's nls.mk just add
> >
> > ../libpq-oauth/oauth-curl.c
> >
> > etc. to the files.
>
> Oh, that's an interesting idea. Thanks, I'll give it a try.

A consequence of this is that our copy of libpq_binddomain isn't using
the same mutex as libpq's copy to protect the "libpq-18" message
domain. We could discuss whether or not it matters, since we don't
support Windows, but it doesn't feel architecturally sound to me. If
we want to reuse the same domain, I think the module should be using
libpq's libpq_gettext(). (Which we could do, again through the magic
of dependency injection.)

> > Maybe it would also make sense to make libpq-oauth a subdirectory of the
> > libpq directory instead of a peer.
>
> Works for me.

It does not, however, work for our $(recurse) setup in the makefiles
-- a shared library depending on a parent directory's shared library
leads to infinite recursion, with the current tools -- so I'll keep it
at the current directory level for now.

Thanks,
--Jacob

  327. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-09T02:57:47Z

    On Tue, Apr 8, 2025 at 2:32 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I think the module should be using
> libpq's libpq_gettext(). (Which we could do, again through the magic
> of dependency injection.)

To illustrate what I mean, v3 introduces an initialization function
that names the three internal dependencies (libpq_gettext,
pg_g_threadlock, and conn->errorMessage) explicitly. I decided not to
attempt injecting the variadic libpq_append_conn_error function, and
instead focus a level below it, since we must depend directly on
libpq_gettext anyway.

This is maybe overkill, if it's decided that the two halves must
always come from the same build, but I think it should decouple the
two sides enough that this is now a question of user experience rather
than ABI correctness.

Is it acceptable/desirable for a build, which has not been configured
--with-libcurl, to still pick up a compatible OAuth implementation
installed by the distro? If so, we can go with a "bare" dlopen(). If
that's not okay, I think we will probably need to use pkglibdir or
some derivative, and introduce a way for tests (and users?) to
override that directory selection. Unless someone has a good idea on
how we can split the difference.

--Jacob

  328. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-09T08:14:32Z

    Re: Jacob Champion
> Is it acceptable/desirable for a build, which has not been configured
> --with-libcurl, to still pick up a compatible OAuth implementation
> installed by the distro? If so, we can go with a "bare" dlopen(). If
> that's not okay, I think we will probably need to use pkglibdir or
> some derivative, and introduce a way for tests (and users?) to
> override that directory selection. Unless someone has a good idea on
> how we can split the difference.

One design goal could be reproducible builds-alike, that is, have
libpq configured with or without libcurl be completely identical, and
the feature being present is simply the libpq-oauth.so file existing
or not. That might make using plain dlopen() more attractive.

Christoph

  329. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2025-04-09T08:38:47Z

    On Wed, Apr 9, 2025, 10:58 Jacob Champion <jacob.champion@enterprisedb.com>
wrote:

> Is it acceptable/desirable for a build, which has not been configured
> --with-libcurl, to still pick up a compatible OAuth implementation
> installed by the distro? If so, we can go with a "bare" dlopen(). If
> that's not okay, I think we will probably need to use pkglibdir or
> some derivative, and introduce a way for tests (and users?) to
> override that directory selection. Unless someone has a good idea on
> how we can split the difference.
>

That seems like it could cause some confusing situations and also making
local testing of different compilation options difficult. How about
ifdef-ing away the dlopen call if --with-libcurl is not specified. So to
have oauth support you need to compile libpq with --with-libcurl AND the
libpq-oauth.so file needs to be present.

(resent because I failed to reply to all from my phone)

>

  330. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-09T12:50:56Z

    On Wed, Apr 9, 2025 at 1:14 AM Christoph Berg <myon@debian.org> wrote:
> One design goal could be reproducible builds-alike, that is, have
> libpq configured with or without libcurl be completely identical, and
> the feature being present is simply the libpq-oauth.so file existing
> or not. That might make using plain dlopen() more attractive.

I think that's more or less what the current v3 does, but Jelte's
point (which is upthread for me but downthread for others :D) is that
making them identical might not actually be a desirable thing in this
situation, because if you don't compile --with-libcurl, when you test
that the feature is disabled you might potentially find that it is
not.

On Wed, Apr 9, 2025 at 1:39 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> How about ifdef-ing away the dlopen call if --with-libcurl is not specified.

This sounds like a pretty decent, simple way to go. Christoph, does
that ring any alarm bells from your perspective?

Thanks,
--Jacob

  331. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-09T17:44:40Z

    Re: Jacob Champion
> > How about ifdef-ing away the dlopen call if --with-libcurl is not specified.
> 
> This sounds like a pretty decent, simple way to go. Christoph, does
> that ring any alarm bells from your perspective?

Ok for me. The opposite that I said in the other mail was just a
suggestion.

Christoph

  332. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-09T23:08:33Z

    On Wed, Apr 9, 2025 at 10:44 AM Christoph Berg <myon@debian.org> wrote:
>
> Re: Jacob Champion
> > > How about ifdef-ing away the dlopen call if --with-libcurl is not specified.
> >
> > This sounds like a pretty decent, simple way to go. Christoph, does
> > that ring any alarm bells from your perspective?
>
> Ok for me. The opposite that I said in the other mail was just a
> suggestion.

Cool, thanks! v4 does it that way. It also errors out at configure
time if you demand libpq-oauth on a platform that does not support it.

The Autoconf side was still polluting LIBS and CPPFLAGS with Curl
stuff. I have isolated them in v4, with some additional m4
boilerplate. IMO this makes the subtle difference between USE_LIBCURL
(which means the libpq-oauth module is enabled to build) and
HAVE_LIBCURL (which means you have libcurl installed locally) even
more confusing.

Christoph noted that this was also confusing from the packaging side,
earlier, and Daniel proposed -Doauth-client/--with-oauth-client as the
feature switch name instead. Any objections? Unfortunately it would
mean a buildfarm email is in order, so we should get it locked in.

Next up: staticlibs.

Thanks,
--Jacob

  333. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2025-04-09T23:42:28Z

    On Thu, Apr 10, 2025, 07:08 Jacob Champion <jacob.champion@enterprisedb.com>
wrote:

> Christoph noted that this was also confusing from the packaging side,
> earlier, and Daniel proposed -Doauth-client/--with-oauth-client as the
> feature switch name instead.
>

+1

Next up: staticlibs.
>

I think your suggestion of not using any .so files would best there (from w
user perspective). I'd be quite surprised if a static build still resulted
in me having to manage shared library files anyway.

>

  334. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-11T00:12:46Z

    On Wed, Apr 9, 2025 at 4:42 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> I think your suggestion of not using any .so files would best there (from w user perspective). I'd be quite surprised if a static build still resulted in me having to manage shared library files anyway.

Done this way in v5. I had planned to separate the implementations by
a #define, but I ran into issues with Makefile.shlib, so I split the
shared and dynamic versions into separate files. I just now realized
that we do something about this exact problem in src/common, so I'll
see if I can copy its technique for the next go round.

In the next version, I'll try to add --with-oauth-client while keeping
--with-libcurl as an alias, to let the buildfarm migrate off of it
before it's removed.

Thanks!
--Jacob

  335. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-11T16:21:14Z

    Jacob Champion:
> On Wed, Apr 9, 2025 at 4:42 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
>> I think your suggestion of not using any .so files would best there (from w user perspective). I'd be quite surprised if a static build still resulted in me having to manage shared library files anyway.
> Done this way in v5. I had planned to separate the implementations by
> a #define, but I ran into issues with Makefile.shlib, so I split the
> shared and dynamic versions into separate files. I just now realized
> that we do something about this exact problem in src/common, so I'll
> see if I can copy its technique for the next go round.

I tried to apply this patch to nixpkgs' libpq build [1]. First, I pinned 
a recent commit from master (one where the v5 patch will apply cleanly 
later) and enabled --with-libcurl [2].

At this stage, without the patch applied, I observe the following:

1. The default, dynamically linked, build succeeds and libpq.so is 
linked to libcurl.so as expected!

2. The statically linked build fails during configure:

   checking for curl_multi_init in -lcurl... no
   configure: error: library 'curl' does not provide curl_multi_init

config.log tells me that it can't link to libcurl, because of undefined 
references, for example:

   undefined reference to `psl_is_cookie_domain_acceptable'
   undefined reference to `nghttp2_session_check_request_allowed'

I assume the many libs listed in Libs.private in libcurl.pc are not 
added automatically for this check?


Next, I applied the v5 patch and:

3. Running the same build as in step 1 above (dynamically linked), I can 
see that libpq.so does have some reference to dlopen / libpq-oauth in it 
- good. But libpq-oauth.so itself is not built. The commands I am using 
to build just the libpq package are essentially like this:

   make submake-libpgport
   make submake-libpq
   make -C src/bin/pg_config install
   make -C src/common install
   make -C src/include install
   make -C src/interfaces/libpq install
   make -C src/port install

I tried adding "make submake-libpq-oauth", but that doesn't exist.

When I do "make -C src/interfaces/libpq-oauth", I get this error:

   make: *** No rule to make target 'oauth-curl.o', needed by 
'libpq-oauth-18.so'.  Stop.

Not sure how to proceed to build libpq-oauth.so.


4. The statically linked build fails with the same configure error as above.


I can only test autoconf right now, not meson - don't have a working 
setup for that, yet.

Best,

Wolfgang

[1]: 
https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/sql/postgresql/libpq.nix

  336. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-04-11T17:32:31Z

    On 08.04.25 19:44, Jacob Champion wrote:
> Would anybody following along be opposed to a situation where
> - dynamiclib builds go through the dlopen() shim
> - staticlib builds always rely on statically linked symbols

If this can be implemented in a straightforward way, that would be the 
best way, I think.

  337. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-14T16:12:53Z

    On Fri, Apr 11, 2025 at 9:21 AM Wolfgang Walther
<walther@technowledgy.de> wrote:
> I tried to apply this patch to nixpkgs' libpq build [1]. First, I pinned
> a recent commit from master (one where the v5 patch will apply cleanly
> later) and enabled --with-libcurl [2].

(The [2] link is missing, I think.)

> 2. The statically linked build fails during configure:

I'm confused by this -- the build produces staticlibs alongside the
dynamically linked ones, so that's what I've been testing against.
What different options do you pass to configure for a "statically
linked build"?

>    undefined reference to `psl_is_cookie_domain_acceptable'
>    undefined reference to `nghttp2_session_check_request_allowed'
>
> I assume the many libs listed in Libs.private in libcurl.pc are not
> added automatically for this check?

Not unless there is some magic in PKG_CHECK_MODULES I've never heard
of (which is entirely possible!). Furthermore I imagine that the
transitive dependencies of all its dependencies are not added either.

Does your build method currently work for dependency forests like
libgssapi_krb5 and libldap? (I want to make sure I'm not accidentally
doing less work than we currently support for those other deps, but
I'm also not planning to add more feature work as part of this
particular open item.)

> I tried adding "make submake-libpq-oauth", but that doesn't exist.

There is no submake for this because no other targets depend on it.
Currently I don't have any plans to add one (but -C should work).

> When I do "make -C src/interfaces/libpq-oauth", I get this error:
>
>    make: *** No rule to make target 'oauth-curl.o', needed by
> 'libpq-oauth-18.so'.  Stop.

I cannot reproduce this. The CI seems happy, too. Is this patch the
only modification you've made to our build system, or are there more
changes?

I'm about to rewrite this part somewhat, so a deep dive may not be very helpful.

Thanks,
--Jacob

  338. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-04-14T16:42:02Z

    Hi,

On 2025-04-11 18:21:14 +0200, Wolfgang Walther wrote:
> Jacob Champion:
> > On Wed, Apr 9, 2025 at 4:42 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> > > I think your suggestion of not using any .so files would best there (from w user perspective). I'd be quite surprised if a static build still resulted in me having to manage shared library files anyway.
> > Done this way in v5. I had planned to separate the implementations by
> > a #define, but I ran into issues with Makefile.shlib, so I split the
> > shared and dynamic versions into separate files. I just now realized
> > that we do something about this exact problem in src/common, so I'll
> > see if I can copy its technique for the next go round.
> 
> I tried to apply this patch to nixpkgs' libpq build [1]. First, I pinned a
> recent commit from master (one where the v5 patch will apply cleanly later)
> and enabled --with-libcurl [2].
> 
> At this stage, without the patch applied, I observe the following:
> 
> 1. The default, dynamically linked, build succeeds and libpq.so is linked to
> libcurl.so as expected!
> 
> 2. The statically linked build fails during configure:

What specifically does "statically linked build" mean? There is no such thing
in postgres, so this must be either patching upstream or injecting build flags
somehow?  The [1] link wasn't immediately elucidating.


>   checking for curl_multi_init in -lcurl... no
>   configure: error: library 'curl' does not provide curl_multi_init
> 
> config.log tells me that it can't link to libcurl, because of undefined
> references, for example:
> 
>   undefined reference to `psl_is_cookie_domain_acceptable'
>   undefined reference to `nghttp2_session_check_request_allowed'
> 
> I assume the many libs listed in Libs.private in libcurl.pc are not added
> automatically for this check?

The configure test shouldn't link statically, so this doesn't make sense to
me?

Greetings,

Andres Freund

  339. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-14T17:09:55Z

    Jacob Champion:
> (The [2] link is missing, I think.)

Ah, sry. This is the link:

https://github.com/wolfgangwalther/nixpkgs/commits/postgresql-libpq-curl/

It's the last two commits on that branch.

> I'm confused by this -- the build produces staticlibs alongside the
> dynamically linked ones, so that's what I've been testing against.
> What different options do you pass to configure for a "statically
> linked build"?

It's not so much the options, but more that for this build there are no 
shared libs available at buildtime at all. You can consider it a "fully 
static system". So in your case, you'd always do the configure test with 
shared libs, but I can't.

The build system passes --enable-static and --disable-shared to 
configure, but both of those are ignored by configure, as indicated by a 
WARNING immediately.

>> Not unless there is some magic in PKG_CHECK_MODULES I've never heard
>> of (which is entirely possible!). Furthermore I imagine that the
>> transitive dependencies of all its dependencies are not added either.

IIUC, the transitive dependencies would be part of libcurl's 
Libs.private / Requires.private (assuming that file is correctly 
created). So that would be taken care of, I guess.


> Does your build method currently work for dependency forests like
> libgssapi_krb5 and libldap? (I want to make sure I'm not accidentally
> doing less work than we currently support for those other deps, but
> I'm also not planning to add more feature work as part of this
> particular open item.)

We currently build libpq with neither libldap, nor libkrb5, at least for 
the static case. But I just tried on the bigger postgresql package and 
force-enabled libldap there for the static build - it fails in exactly 
the same way.

So yes, not related to your patch. I do understand that PostgreSQL's 
autoconf build system is not designed for "static only", I am certainly 
not expecting you to fix that.

I think meson will do better here, but I was not able to make that work, 
yet.


>> When I do "make -C src/interfaces/libpq-oauth", I get this error:
>>
>>     make: *** No rule to make target 'oauth-curl.o', needed by
>> 'libpq-oauth-18.so'.  Stop.
> I cannot reproduce this. The CI seems happy, too. Is this patch the
> only modification you've made to our build system, or are there more
> changes?

We apply another patch to change the default socket directory to /run, 
but that's certainly unrelated. All the other custom stuff only kicks in 
afterwards, in the installPhase, so unrelated as well.

I just tried the same thing on the bigger postgresql package, where the 
full build is run and not only libpq / libpq-oauth. It fails with the 
same error. No rule for oauth-curl.o.


> I'm about to rewrite this part somewhat, so a deep dive may not be very helpful.

OK. I will try to get meson running, at least enough to try this patch 
again. Maybe that gives better results.

Thanks,

Wolfgang

  340. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-14T18:27:46Z

    Wolfgang Walther:
 > So yes, not related to your patch. I do understand that PostgreSQL's
 > autoconf build system is not designed for "static only", I am certainly
 > not expecting you to fix that.
 >
 > I think meson will do better here, but I was not able to make that work,
 > yet.
I did a basic meson build. Full postgresql package, not libpq-only.

The static-only build just works. On master that is. Same as the regular 
build.

So yes, meson will handle the static stuff much better.


 > I just tried the same thing on the bigger postgresql package, where the
 > full build is run and not only libpq / libpq-oauth. It fails with the
 > same error. No rule for oauth-curl.o.
Applying the v5 patch to the above meson build, will give me a different 
error. This time for both the static-only and the regular build:

   src/interfaces/libpq-oauth/meson.build:18:22: ERROR: File 
oauth-curl.c does not exist.

This.. clears it up, because that file is indeed missing for me on disk. 
I assume that's because this file is tracked as a rename in the v5 
patch. I can apply this with git, but not directly in the nix build 
system. TIL, I need to use "fetchpatch2" instead of "fetchpatch" for 
that. Sure thing.


So, with the patch applied correctly, I get the following:

1. Meson regular build:

libpq-oauth-18.so
libpq.so
libpq.so.5
libpq.so.5.18

The libpq.so file has references to dlopen and libpq-auth-18.so, cool.

2. Meson static-only build:

libpq.a
libpq-oauth-18.a

The libpq.a file has no references to dlopen, but plenty of references 
to curl stuff.

I'm not sure what the libpq-oauth-18.a file is for.

3. Back to the lipq-only build with autoconf, from where I started. I 
only need to add the following line:

make -C src/interfaces/libpq-oauth install

and get this:

libpq-oauth-18.so
libpq.so
libpq.so.5
libpq.so.5.18

Sweet!

4. Of course the static-only build does not work with autoconf, but 
that's expected.

So, sorry for the noise before. Now, that I know how to apply patches 
with renames, I will try your next patch as well.

Best,

Wolfgang

  341. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-14T19:14:20Z

    On Mon, Apr 14, 2025 at 11:27 AM Wolfgang Walther
<walther@technowledgy.de> wrote:
>    src/interfaces/libpq-oauth/meson.build:18:22: ERROR: File
> oauth-curl.c does not exist.
>
> This.. clears it up, because that file is indeed missing for me on disk.

Aha! Okay, glad I don't need to track that down.

> libpq.a
> libpq-oauth-18.a
>
> The libpq.a file has no references to dlopen, but plenty of references
> to curl stuff.

Which references? libpq-oauth should be the only thing using Curl symbols:

    $ nm src/interfaces/libpq/libpq.a | grep --count curl
    0
    $ nm src/interfaces/libpq-oauth/libpq-oauth-18.a | grep --count curl
    116

> I'm not sure what the libpq-oauth-18.a file is for.

That implements the flow. You'll need to link that into your
application or it will complain about missing flow symbols. (I don't
think there's an easy way to combine the two libraries in our Autoconf
setup; the only ways I can think of right now would introduce a
circular dependency between libpq and libpq-oauth...)

Thanks!
--Jacob

  342. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-04-14T19:46:29Z

    Jacob Champion:
>> libpq.a
>> libpq-oauth-18.a
>>
>> The libpq.a file has no references to dlopen, but plenty of references
>> to curl stuff.
> 
> Which references? libpq-oauth should be the only thing using Curl symbols:
> 
>      $ nm src/interfaces/libpq/libpq.a | grep --count curl
>      0
>      $ nm src/interfaces/libpq-oauth/libpq-oauth-18.a | grep --count curl
>      116

Not sure what I was looking at earlier, probably too many different 
builds at the same time. Now I can't find the curl symbols in libpq.a 
either...

>> I'm not sure what the libpq-oauth-18.a file is for.
> 
> That implements the flow. You'll need to link that into your
> application or it will complain about missing flow symbols. (I don't
> think there's an easy way to combine the two libraries in our Autoconf
> setup; the only ways I can think of right now would introduce a
> circular dependency between libpq and libpq-oauth...)

... which immediately explains what the libpq-oauth-18.a file is for, yes.

But that means we'd need a -lpq-oauth-18 or something like that in 
Libs.private in libpq.pc, right?

This seems to be missing, I checked both the autoconf and meson builds.

Best,

Wolfgang

  343. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-14T20:17:13Z

    On Mon, Apr 14, 2025 at 12:46 PM Wolfgang Walther
<walther@technowledgy.de> wrote:
> But that means we'd need a -lpq-oauth-18 or something like that in
> Libs.private in libpq.pc, right?

I believe so. I'm in the middle of the .pc stuff right now; v6 should
have the fixes as long as I don't get stuck.

Thanks,
--Jacob

  344. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-15T00:13:35Z

    On Mon, Apr 14, 2025 at 1:17 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I believe so. I'm in the middle of the .pc stuff right now; v6 should
> have the fixes as long as I don't get stuck.

Done in v6-0001. I think this is now architecturally complete, so if
reviewers are happy I can work on docs and the commit message. As a
summary:

- We provide a libpq-oauth-18.so module for shared builds, and a
corresponding .a for static builds, when OAuth is enabled.
- Platforms which cannot support the builtin flow now error out if you
request OAuth at configure time.
- When OAuth is enabled and there's no custom client flow, libpq.so
loads the module via dlopen(), which respects RPATH/LD_LIBRARY_PATH et
al. If it's not installed, OAuth doesn't continue.
- Static builds must link libpq-oauth-18.a explicitly. libpq.pc now
puts -lpq-oauth-18 in Libs.private, and libcurl in Requires.private.
- Internally, we compile separate versions of fe-auth-oauth.c to
handle the different cases (disabled, dynamic, static). This is
borrowed from src/common.
- The only new export from libpq is appendPQExpBufferVA. Other
internal symbols are shared with libpq-oauth via dependency injection.

v6-0002 is a WIP rename of the --with-libcurl option to
--with-oauth-client. I'm not sure I have all of the Meson corner cases
with auto_features figured out, but maybe it doesn't matter since it's
temporary (it targets a total of seven buildfarm animals, and once
they've switched we can remove the old name). I have added a separate
open item for this.

Thanks,
--Jacob

  345. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Peter Eisentraut <peter@eisentraut.org> — 2025-04-15T12:31:50Z

    On 10.04.25 01:08, Jacob Champion wrote:
> Christoph noted that this was also confusing from the packaging side,
> earlier, and Daniel proposed -Doauth-client/--with-oauth-client as the
> feature switch name instead. Any objections? Unfortunately it would
> mean a buildfarm email is in order, so we should get it locked in.

We had that discussion early in the development, and I still think it's 
not the right choice.

The general idea, at least on the Autoconf side, is that --with-FOO 
means enable all the features that require library FOO.  For example, 
--with-ldap enables all the LDAP-related features, including 
authentication support in libpq, authentication support in the server, 
and service lookup in libpq.  --with-[open]ssl enables all the features 
that use OpenSSL, including SSL support in the client and server but 
also encryption support in pgcrypto.

The naming system you propose has problems:

First, what if we add another kind of "oauth-client" that doesn't use 
libcurl, how would you extend the set of options?

Second, what if we add some kind of oauth plugin for the server that 
uses libcurl, how would you extend the set of options?

If you used that system for options in the ldap or openssl cases, you'd 
end up with maybe six options (and packagers would forget to turn on 
half of them).  But worse, what you are hiding is the information what 
dependencies you are pulling in, which is the actual reason for the 
options.  (If there was no external dependency, there would be no option 
at all.)

This seems unnecessarily complicated and inconsistent.  Once you have 
made the choice of taking the libcurl dependency, why not build 
everything that requires it?

(Nitpick: If you go with this kind of option, it should be --enable-XXX 
on the Autoconf side.)

  346. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-15T15:34:19Z

    Re: Peter Eisentraut
> But worse, what you are hiding is the information what dependencies
> you are pulling in, which is the actual reason for the options.  (If there
> was no external dependency, there would be no option at all.)
> 
> This seems unnecessarily complicated and inconsistent.  Once you have made
> the choice of taking the libcurl dependency, why not build everything that
> requires it?

I agree with this reasoning and retract my suggestion to rename the option.

Thanks for the explanation,
Christoph

  347. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-15T17:53:07Z

    On Tue, Apr 15, 2025 at 5:31 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> On 10.04.25 01:08, Jacob Champion wrote:
> > Christoph noted that this was also confusing from the packaging side,
> > earlier,

Since Christoph has withdrawn the request, I will drop -0002.

However, I'll go ahead and put some of my opinions down on paper here:

> The general idea, at least on the Autoconf side, is that --with-FOO
> means enable all the features that require library FOO.

I don't think this is particularly user-friendly if it's not obvious
what feature is enabled by FOO.

LDAP? PAM? Sure. SSL? Eh, I think the pgcrypto coupling is a little
strange -- that's not implied by "SSL" at all! -- but it's not
problematic enough to complain loudly. --with-gssapi selects... some
dependency... which may or may not come from a particular library.
--with-bsd-auth doesn't add any library dependencies at all, instead
depending on the kernel, but it makes sense.

But there's no connection between "libcurl" and "OAuth Device
Authorization flow" in anyone's mind except the people who have worked
on that feature.

If the argument is that we'd need to switch to --enable-oauth-client
rather than --with-oauth-client, that works for me. But I don't quite
understand the desire to stick to the existing configuration
methodology for something that's very different from an end-user
perspective.

> The naming system you propose has problems:
>
> First, what if we add another kind of "oauth-client" that doesn't use
> libcurl, how would you extend the set of options?

With an extension to the values that you can provide to
--with-oauth-client, similarly to what was originally proposed for
--with-ssl.

> Second, what if we add some kind of oauth plugin for the server that
> uses libcurl, how would you extend the set of options?

With a new option.

But let me turn this around, because we currently have the opposite
problem: if someone comes in and adds a completely new feature
depending on libcurl, and you want OAuth but you do not want that new
feature -- or vice-versa -- what do you do? In other words, what if
your concern is not with libcurl, but with the feature itself?

> But worse, what you are hiding is the information what
> dependencies you are pulling in, which is the actual reason for the
> options. (If there was no external dependency, there would be no option
> at all.)

I'm not sure I agree, either practically or philosophically. I like to
see the build dependencies, definitely, but I also like to see the
features. (Meson will make both things visible separately, for that
matter.)

> This seems unnecessarily complicated and inconsistent.  Once you have
> made the choice of taking the libcurl dependency, why not build
> everything that requires it?

Simply because the end user or packager might not want to.

In any case -- I won't die on this particular hill, and I'm happy to
continue forward with 0001 alone.

Thanks!
--Jacob

  348. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-15T18:02:13Z

    On Tue, Apr 15, 2025 at 8:34 AM Christoph Berg <myon@debian.org> wrote:
> I agree with this reasoning and retract my suggestion to rename the option.

(Thank you for chiming in; having the packager feedback has been
extremely helpful.)

While I have you, may I ask whether you're okay (from the packager
perspective) with the current division of dynamic and static
behaviors?

Dynamic: --with-libcurl builds a runtime-loadable module, and if you
don't install it, OAuth isn't supported (i.e. it's optional)
Static: --with-libcurl builds an additional linkable staticlib, which
you must link into your application (i.e. not optional)

I want to make absolutely sure the existing packager requests are not
conflicting. :D

Thanks,
--Jacob

  349. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-15T18:57:32Z

    Re: Jacob Champion
> But there's no connection between "libcurl" and "OAuth Device
> Authorization flow" in anyone's mind except the people who have worked
> on that feature.

Fwiw that was exactly the reason I originally voiced the idea to
rename.

> But let me turn this around, because we currently have the opposite
> problem: if someone comes in and adds a completely new feature
> depending on libcurl, and you want OAuth but you do not want that new
> feature -- or vice-versa -- what do you do? In other words, what if
> your concern is not with libcurl, but with the feature itself?

What made me reconsider was Peter saying that what defines the blast
radius of some feature is usually the extra dependency pulled in. If
you don't like tracking OpenSSL problems, build without it. If you
don't like libcurl, build without it. That's the "we are going to be
hated by security scanner people" argument that brought up this
sub-thread.

Now if the feature itself were a problem, that might change how
configuration should be working. Is "libpq can now initiate oauth
requests" something people would like to be able to control?


Re: Jacob Champion
> Dynamic: --with-libcurl builds a runtime-loadable module, and if you
> don't install it, OAuth isn't supported (i.e. it's optional)

Ok.

> Static: --with-libcurl builds an additional linkable staticlib, which
> you must link into your application (i.e. not optional)

Debian does not care really about static libs. We are currently
shipping libpq.a, but if it breaks in any funny way, we might as well
remove it.

Christoph

  350. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Robert Haas <robertmhaas@gmail.com> — 2025-04-15T19:21:33Z

    On Tue, Apr 15, 2025 at 8:32 AM Peter Eisentraut <peter@eisentraut.org> wrote:
> On 10.04.25 01:08, Jacob Champion wrote:
> > Christoph noted that this was also confusing from the packaging side,
> > earlier, and Daniel proposed -Doauth-client/--with-oauth-client as the
> > feature switch name instead. Any objections? Unfortunately it would
> > mean a buildfarm email is in order, so we should get it locked in.
>
> We had that discussion early in the development, and I still think it's
> not the right choice.

I strongly agree. I think it will not be long before somebody
implements a second feature depending on libcurl, and there's no
precedent for the idea that we allow those features to be enabled and
disabled individually. If that turns out to be something that is
wanted, then since this will be a separate library, a packager can
choose not to ship it, or to put it in a separate package, if they
wish. If there's REALLY a lot of demand for a separate enable/disable
switch for this feature then we can consider making this an exception
to what we do for all other dependent libraries, but I bet there won't
be. I can imagine someone not wanting libcurl on their system on the
theory that it would potentially open up the ability to download data
from arbitrary URLs which might be considered bad from a security
posture -- but I don't really see why someone would be particularly
upset about one particular way in which libcurl might be used.

I also don't mind being wrong, of course. But I think it's better to
bet on making this like other things and then change strategy if that
doesn't work out, rather than starting out by making this different
from other things.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  351. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-15T19:44:48Z

    On Tue, Apr 15, 2025 at 11:57 AM Christoph Berg <myon@debian.org> wrote:
> What made me reconsider was Peter saying that what defines the blast
> radius of some feature is usually the extra dependency pulled in. If
> you don't like tracking OpenSSL problems, build without it. If you
> don't like libcurl, build without it. That's the "we are going to be
> hated by security scanner people" argument that brought up this
> sub-thread.
>
> Now if the feature itself were a problem, that might change how
> configuration should be working. Is "libpq can now initiate oauth
> requests" something people would like to be able to control?

Well... I'd sure like to live in a world where users thought about the
implications and risks of what they're using and why, rather than
farming a decision out to a static analysis tool. ("And as long as I'm
dreaming, I'd like a pony.")

But end users already control the initiation of OAuth requests (it's
opt-in via the connection string), so that's not really the goal.

> Debian does not care really about static libs. We are currently
> shipping libpq.a, but if it breaks in any funny way, we might as well
> remove it.

Awesome. I think we have a consensus.

Thanks!
--Jacob

  352. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-15T19:45:28Z

    On Tue, Apr 15, 2025 at 12:21 PM Robert Haas <robertmhaas@gmail.com> wrote:
> I also don't mind being wrong, of course. But I think it's better to
> bet on making this like other things and then change strategy if that
> doesn't work out, rather than starting out by making this different
> from other things.

Works for me. (And it's less work, too!)

Thanks,
--Jacob

  353. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2025-04-15T21:38:35Z

    On Tue, 15 Apr 2025 at 19:53, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> But let me turn this around, because we currently have the opposite
> problem: if someone comes in and adds a completely new feature
> depending on libcurl, and you want OAuth but you do not want that new
> feature -- or vice-versa -- what do you do? In other words, what if
> your concern is not with libcurl, but with the feature itself?

After reconsidering this, I now agree with Peter and Robert that
--with-libcurl is the flag that we should be relying on. Specifically
because of the situation you're describing above: Once you have
libcurl, why wouldn't you want all the features (e.g. in some other
thread there was a suggestion about fetching the PGSERVICEFILE from a
HTTP endpoint).

It's not like we add compile time flags for other user facing features
like --enable-index-scan. All the --enable-xyz options that we have
are for developer features (like debug and asserts). Starting to add
such a flag for this feature seems unnecessary.

Regarding discoverability, I think the error message that you have
already solves that:

> libpq_append_conn_error(conn, "no custom OAuth flows are available, and libpq was not built with libcurl support");

Side-note: I think it would be good to have a different error when
libpq was build with libcurl support, but the dlopen failed. Something
like:

libpq_append_conn_error(conn, "no custom OAuth flows are available,
and libpq-oauth could not be loaded library could not be loaded. Try
installing the libpq-oauth package from the same source that you
installed libpq from");

  354. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-18T00:47:42Z

    On Tue, Apr 15, 2025 at 2:38 PM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> libpq_append_conn_error(conn, "no custom OAuth flows are available,
> and libpq-oauth could not be loaded library could not be loaded. Try
> installing the libpq-oauth package from the same source that you
> installed libpq from");

Thanks! I think that's a little too prescriptive for packagers,
personally, but I agree that the current message isn't correct
anymore. I've gone with "no custom OAuth flows are available, and the
builtin flow is not installed". (I suppose packagers could patch in a
platform-specific message if they really wanted?)

--

Other changes in v7:

- The option name remains --with-libcurl.
- Daniel and I have tweaked the documentation, and a draft commit message is up
- Removed the ENABLE_NLS-mismatch assertion in oauth-utils.c; we don't
need to care anymore
- Added an initialization mutex

I was feeling paranoid about injecting dependency pointers
concurrently to their use in another thread. They're _supposed_ to be
constant... but I have no doubt that someone somewhere knows of a
platform/compiler/linker combo where that blows up anyway.
Initialization is now run once, under pthread_mutex protection.

- Fixed module load on macOS

The green CI was masking a bug with its use of DYLD_LIBRARY_PATH: we
don't make use of RPATH on macOS, so after installing libpq, it lost
the ability to find libpq-oauth. (A stale installation due to SIP
weirdness was masking this on my local machine; sorry for not catching
it before.)

I have swapped to using an absolute path on Mac only, because unlike
LD_LIBRARY_PATH on *nix, DYLD_LIBRARY_PATH can still override absolute
paths in dlopen()! Whee. I could use a sanity check from a native Mac
developer, but I believe this mirrors the expected behavior for a
"typical" runtime dependency: libraries point directly to the things
they depend on.

With those, I have no more TODOs and I believe this is ready for a
final review round.

Thanks,
--Jacob

  355. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-18T17:01:17Z

    On Thu, Apr 17, 2025 at 5:47 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> With those, I have no more TODOs and I believe this is ready for a
> final review round.

Some ABI self-review. These references to conn->errorMessage also need
the indirection treatment, which I'm working on now:

>     if (actx->errctx)
>     {
>         appendPQExpBufferStr(&conn->errorMessage,
>                              libpq_gettext(actx->errctx));
>         appendPQExpBufferStr(&conn->errorMessage, ": ");
>     ...

I was searching backwards through history to confirm that we don't
rearrange struct pg_conn in back branches; turns out that was a false
assumption. See e8f60e6fe2:

    While at it, fix some places where parameter-related infrastructure
    was added with the aid of a dartboard, or perhaps with the aid of
    the anti-pattern "add new stuff at the end".  It should be safe
    to rearrange the contents of struct pg_conn even in released
    branches, since that's private to libpq (and we'd have to move
    some fields in some builds to fix this, anyway).

So that means, I think, the name needs to go back to -<major>-<minor>,
unless anyone can think of a clever way around it. (Injecting
conn->errorMessage to avoid the messiness around ENABLE_GSS et al is
still useful, but injecting every single offset doesn't seem
maintainable to me.) Sorry, Christoph; I know that's not what you were
hoping for.

--Jacob

  356. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-19T12:03:29Z

    Re: Jacob Champion
> > libpq_append_conn_error(conn, "no custom OAuth flows are available,
> > and libpq-oauth could not be loaded library could not be loaded. Try
> > installing the libpq-oauth package from the same source that you
> > installed libpq from");
> 
> Thanks! I think that's a little too prescriptive for packagers,
> personally, but I agree that the current message isn't correct
> anymore. I've gone with "no custom OAuth flows are available, and the
> builtin flow is not installed".

This whole oauth business is highly confusing if you aren't a web
security expert. It's a pretty long way from "the builtin flow is not
installed" to "if you want this to work, you need to install an extra
library/package on the client", so I don't think this message is
helpful.

The originally suggested message was pretty good in that regard. The
distinction about custom flows could probably be dropped.

How about this:

  No libpq OAuth flows are available. (Try installing the libpq-oauth package.)

People who have custom flows will likely know that they have to do
anyway.

Devrim: Does that match the package name you'd use?

> (I suppose packagers could patch in a
> platform-specific message if they really wanted?)

We could, but I'd prefer if we didn't have to. :*)

Christoph

  357. Re: [PoC] Federated Authn/z with OAUTHBEARER

    ivan.kush@tantorlabs.com — 2025-04-20T17:12:01Z

    Hello!

I'm testing OAuth Device Flow implementation on Google. Met several 
problems.

Postgres from master branch, commit 764d501d24b
Google Device Flow API
https://developers.google.com/identity/protocols/oauth2/limited-input-device

1) In Device Authorization Request Google returns 428 code on pending
https://developers.google.com/identity/protocols/oauth2/limited-input-device#authorization-pending

Source code handles only 400/401 error codes, they are in the Section 
5.2 RFC6749
* An error response uses either 400 Bad Request or 401 Unauthorized.
* There are references online to implementations using 403 for error
* return which would violate the specification.

-----------------
I suggest to add a GUC in postgresql.conf that contains additional 
non-standard error codes for a specific service.
oauth_add_error_codes = [
   {
          issuer: google
         add_err_codes: [428],
     },
     {
         issuer: someservice
         add_err_code: [403],
     }
]
So Google can contain 400,401,428

Additionally write parsing of such json-like config-values. Will be cool 
to create serializer, that matches struct to such json-like GUC.

Or we can create a separate file oauth.conf where json-like data will 
be. And postgresql.conf may contain link to this file, name oauth_conf GUC

oauth_conf = /var/lib/postgres/data/oauth.conf


=================

2) Google requires client_secret only in the Device Access Token Request 
(Section 3.3 RFC-8628). Also note that secret is in a body of a request
https://developers.google.com/identity/protocols/oauth2/limited-input-device#step-4:-poll-googles-authorization-server

curl -d "client_id=client_id&client_secret=client_secret& \
          device_code=device_code& \
grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Adevice_code" \
          -H "Content-Type: application/x-www-form-urlencoded" \
          https://oauth2.googleapis.com/token

Not Device Authorization Request (Section 3.1 RFC-8628)
https://developers.google.com/identity/protocols/oauth2/limited-input-device#step-2:-handle-the-authorization-server-response

curl -d "client_id=client_id&scope=email%20profile" \
         https://oauth2.googleapis.com/device/code

But Postgres sends client_secret in both request, also in Device 
Authorization Request. See calls of a func add_client_identification in 
funs start_device_authz & start_token_request
Azure also use secret only in Device Access Token Request 
https://learn.microsoft.com/en-us/entra/identity-platform/v2-oauth2-device-code#device-authorization-request
-----------------

I suggest to remove send secret on Device Authorization Request.

=================
3) Additionally if secret exists PG sends it only using Basic Auth. But 
RFC contain only MAY word about Basic Auth. Section 2.3.1 RFC 6749,
if (conn->oauth_client_secret) /* Zero-length secrets are permitted! */
{
username = urlencode(conn->oauth_client_id);
password = urlencode(conn->oauth_client_secret);
...
CHECK_SETOPT(actx, CURLOPT_HTTPAUTH, CURLAUTH_BASIC, goto cleanup);
CHECK_SETOPT(actx, CURLOPT_USERNAME, username, goto cleanup);
CHECK_SETOPT(actx, CURLOPT_PASSWORD, password, goto cleanup);
actx->used_basic_auth = true;
}
Also this section contains words about body, Google use such approach

Alternatively, the authorization server MAY support including the client 
credentials in the request-body using the following parameters: 
client_id REQUIRED. The client identifier issued to the client during 
the registration process described by Section 2.2 
<https://datatracker.ietf.org/doc/html/rfc6749#section-2.2>. 
client_secret REQUIRED. The client secret. The client MAY omit the 
parameter if the client secret is an empty string.
https://developers.google.com/identity/protocols/oauth2/limited-input-device#step-2:-handle-the-authorization-server-response

-----------------
I suggest to set such cases in config. Let's create a json-like oauth 
array config. Field auth_scheme shows what scheme we want to use. (see 
GUC description in pt1 of this email).
oauth = [
   {
          issuer: google
         add_err_codes: [428],
         auth_scheme: body
     },
     {
     issuer: someservice
         add_err_code: [403],
         auth_scheme: basic
     }

]

-- 
Best wishes,
Ivan Kush
Tantor Labs LLC

  358. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-21T16:57:33Z

    On Sun, Apr 20, 2025 at 10:12 AM Ivan Kush <ivan.kush@tantorlabs.com> wrote:
> I'm testing OAuth Device Flow implementation on Google. Met several
> problems.

Hi Ivan, thank you for testing and reporting! Unfortunately, yeah,
Google is a known problem [1]. They've taken several liberties with
the spec, as you point out.

We have some options for dealing with them, since their documentation
instructs clients to hardcode their API entry points instead of using
discovery. (That makes it easy for us to figure out when we're talking
to Google, and potentially switch to a quirks mode.)

But! Before we do that: How do you intend to authorize tokens issued
by Google? Last I checked, they still had no way to register an
application-specific scope, making it very dangerous IMO to use a
public flow [2]. Do you have an architecture where this usage is safe,
and/or have they added custom scopes? (I deprioritized handling the
nonstandard behavior when I couldn't prove to myself that it was
possible to use the Google version of Device Authorization safely, but
I'm happy to jump back into that if we have a good use case.)

> 1) In Device Authorization Request Google returns 428 code on pending
> https://developers.google.com/identity/protocols/oauth2/limited-input-device#authorization-pending

Right. I believe there were other nonstandard errors in other corner
cases, too. :(

> I suggest to add a GUC in postgresql.conf that contains additional
> non-standard error codes for a specific service.
> oauth_add_error_codes = [
>    {
>           issuer: google
>          add_err_codes: [428],
>      },
>      {
>          issuer: someservice
>          add_err_code: [403],
>      }
> ]
> So Google can contain 400,401,428

The server config doesn't help us much, since this is a client-side
feature. Any "global" configuration is probably going to be done
through environment variables or a service file [3].

> Additionally write parsing of such json-like config-values. Will be cool
> to create serializer, that matches struct to such json-like GUC.

I'm not too excited about a separate configuration DSL. I'm guessing
most end users, if they really want Google as their Device
Authorization provider, would rather have us switch over to "Google
mode" once we notice the magic Google endpoint is in use.

> 2) Google requires client_secret only in the Device Access Token Request
> (Section 3.3 RFC-8628).
> ...
> But Postgres sends client_secret in both request, also in Device
> Authorization Request.

Yes. See 3.1 (Device Authorization Request):

   The client authentication requirements of Section 3.2.1 of [RFC6749]
   apply to requests on this endpoint, which means that confidential
   clients (those that have established client credentials) authenticate
   in the same manner as when making requests to the token endpoint, and
   public clients provide the "client_id" parameter to identify
   themselves.

> I suggest to remove send secret on Device Authorization Request.

This breaks Okta, at minimum. We can't do it across the board. (As for
Azure, I haven't figured out how to configure it to *require* a
confidential client secret for the device flow -- which makes a
certain amount of sense since the flow is public -- but its v2
endpoint doesn't mind being *sent* a secret.)

> 3) Additionally if secret exists PG sends it only using Basic Auth. But
> RFC contain only MAY word about Basic Auth. Section 2.3.1 RFC 6749,

>From 2.3.1:

   The authorization server MUST support the HTTP Basic
   authentication scheme for authenticating clients that were issued a
   client password.

We rely on that MUST, at the moment. We can add an exception for a
provider, certainly, but it needs to be limited for safety reasons:
"Including the client credentials in the request-body using the two
parameters is NOT RECOMMENDED and SHOULD be limited to clients unable
to directly utilize the HTTP Basic authentication scheme..."

(Authentication is its own nasty minefield; OAuth introduced its own
encoding requirements on top of HTTP that a bunch of servers ignored,
but in practice we cross our fingers that servers will only issue
ASCII credentials if they're not willing to follow the encoding
rules...)

So to recap: I'm happy to add a Google compatibility mode, but I'd
like to gather some evidence that their device flow can actually
authorize tokens for third parties safely, before we commit to that.
Thoughts?

Thanks!
--Jacob

[1] https://postgr.es/m/CAOYmi%2BkTumP6FHwLnUKX0DVKrTv%3DN9xSOAu7YMH_XKSMP7ozfA%40mail.gmail.com
[2] https://postgr.es/m/CAOYmi%2B%3DMFyrjDps-YNtem3%3DGr3mUsgZ49m7bfMCgr1TDjHL58g%40mail.gmail.com
[3] https://www.postgresql.org/docs/current/libpq-pgservice.html

  359. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Devrim Gündüz <devrim@gunduz.org> — 2025-04-21T18:49:53Z

    Hi,

On Sat, 2025-04-19 at 14:03 +0200, Christoph Berg wrote:
>   No libpq OAuth flows are available. (Try installing the libpq-oauth
> package.)
> 
> People who have custom flows will likely know that they have to do
> anyway.
> 
> Devrim: Does that match the package name you'd use?

On PGDG RPM world it would be libpq5-oauth -- but I need to read the
whole thread first as I don't know yet why we need to split out oauth
into a separate package (at least in the RPM world)

Regards,
-- 
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org

  360. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-21T23:19:06Z

    On Sat, Apr 19, 2025 at 5:04 AM Christoph Berg <myon@debian.org> wrote:
> How about this:
>
>   No libpq OAuth flows are available. (Try installing the libpq-oauth package.)

Tweaked for capitalization/punctuation rules, and removing the first
"libpq" mention (which I don't think helps a user of, say, psql):

    no OAuth flows are available (try installing the libpq-oauth package)

v8 also makes the following changes:

- Per ABI comment upthread, we are back to major-minor versioning for
the shared library (e.g. libpq-oauth-18-0.so). 0001 adds the macros
and makefile variables to make this easy, and 0002 is the bulk of the
change now.
- Since libpq-oauth.a is going to be discovered at compile time, not
runtime, I've removed the versioning from that filename. Static
clients need to match them anyway, so we don't need that additional
packaging headache.
- conn->errorMessage is now decoupled from oauth-curl.c. Separate
object file builds are made using the same technique as libpq.

Thanks,
--Jacob

  361. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-22T10:02:42Z

    > On 22 Apr 2025, at 01:19, Jacob Champion <jacob.champion@enterprisedb.com> wrote:

> v8 also makes the following changes:

Thanks for this version, a few small comments:

+  if oauth_flow_supported
+    cdata.set('USE_LIBCURL', 1)
+  elif libcurlopt.enabled()
+    error('client OAuth is not supported on this platform')
+  endif
We already know that libcurlopt.enabled() is true here so maybe just doing
if-else-endif would make it more readable and save readers thinking it might
have changed?  Also, "client OAuth" reads a bit strange, how about "client-side
OAuth" or "OAuth flow module"?


-       appendPQExpBufferStr(&conn->errorMessage,
-                            libpq_gettext(actx->errctx));
-       appendPQExpBufferStr(&conn->errorMessage, ": ");
+       appendPQExpBufferStr(errbuf, libpq_gettext(actx->errctx));
+       appendPQExpBufferStr(errbuf, ": ");
I think we should take this opportunity to turn this into a appendPQExpBuffer()
with a format string instead of two calls.


+       len = errbuf->len;
+       if (len >= 2 && errbuf->data[len - 2] == '\n')
Now that the actual variable, errbuf->len, is short and very descriptive I
wonder if we shouldn't just use this as it makes the code even clearer IMO.

--
Daniel Gustafsson

  362. Re: [PoC] Federated Authn/z with OAUTHBEARER

    ivan.kush@tantorlabs.com — 2025-04-22T21:29:10Z

    Hi Jacob, thank you for detailed explanation and links!

Am I right that classic OAuth flow "create user account based on a 
token" is implemented using custom validators?

1) In pg_hba.conf set user to all and  "delegate_ident_mapping=1"

"local all all oauth issuer=$issuer scope=$scope delegate_ident_mapping=1"

2) Write a custom validator that will "execute" in C `CREATE USER 
token.name WITH token.listofOptions` after verification of a token.

On 25-04-21 19:57, Jacob Champion wrote:
> We have some options for dealing with them, since their documentation
> instructs clients to hardcode their API entry points instead of using
> discovery. (That makes it easy for us to figure out when we're talking
> to Google, and potentially switch to a quirks mode.)

What do you mean by "discovery"? OpenID link that returns endpoint?

Google has this link

https://accounts.google.com/.well-known/openid-configuration

OUTPUT:
     {
         "issuer": "https://accounts.google.com",
         "authorization_endpoint": 
"https://accounts.google.com/o/oauth2/v2/auth",
         "device_authorization_endpoint": 
"https://oauth2.googleapis.com/device/code",
         "token_endpoint": "https://oauth2.googleapis.com/token",
         "userinfo_endpoint": 
"https://openidconnect.googleapis.com/v1/userinfo",
         "revocation_endpoint": "https://oauth2.googleapis.com/revoke",
         "jwks_uri": "https://www.googleapis.com/oauth2/v3/certs",
............
     }

Here it's described

https://developers.google.com/identity/openid-connect/openid-connect

> But! Before we do that: How do you intend to authorize tokens issued
> by Google? Last I checked, they still had no way to register an
> application-specific scope, making it very dangerous IMO to use a
> public flow [2].

I've also thought as Antonin about 
https://www.googleapis.com/oauth2/v3/userinfo for verification

As I understand from [2], the current problem is security, Google 
doesn't want to add new scopes.

-- 
Best wishes,
Ivan Kush
Tantor Labs LLC

  363. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-22T23:41:50Z

    On Tue, Apr 22, 2025 at 3:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> +  if oauth_flow_supported
> +    cdata.set('USE_LIBCURL', 1)
> +  elif libcurlopt.enabled()
> +    error('client OAuth is not supported on this platform')
> +  endif
> We already know that libcurlopt.enabled() is true here so maybe just doing
> if-else-endif would make it more readable and save readers thinking it might
> have changed?

Features are tri-state, so libcurlopt.disabled() and
libcurlopt.enabled() can both be false. :( My intent is to fall
through nicely in the case where -Dlibcurl=auto.

(Our minimum version of Meson is too old to switch to syntax that
makes this more readable, like .allowed(), .require(), .disable_if(),
etc...)

> Also, "client OAuth" reads a bit strange, how about "client-side
> OAuth" or "OAuth flow module"?
> ...
> I think we should take this opportunity to turn this into a appendPQExpBuffer()
> with a format string instead of two calls.
> ...
> Now that the actual variable, errbuf->len, is short and very descriptive I
> wonder if we shouldn't just use this as it makes the code even clearer IMO.

All three done in v9, attached.

Thanks!
--Jacob

  364. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-23T15:39:08Z

    Re: Jacob Champion
> - Per ABI comment upthread, we are back to major-minor versioning for
> the shared library (e.g. libpq-oauth-18-0.so). 0001 adds the macros
> and makefile variables to make this easy, and 0002 is the bulk of the
> change now.

This will cause problems when programs are running while packages are
updated on disk. That program then tries to dlopen 18-0.so when there
is already 18-1.so installed. Relevant when the first oauth connection
is made way after startup.

This is trading one problem for another, but within-a-major ABI
changes should be much rarer than normal minor updates with
applications restarting only later.

Alternatively, there could be a dedicated SONAME for the plugin that
only changes when necessary, but perhaps the simple "18.so" solution
is good enough.

Christoph

  365. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-23T16:07:52Z

    On Wed, Apr 23, 2025 at 8:39 AM Christoph Berg <myon@debian.org> wrote:
> This will cause problems when programs are running while packages are
> updated on disk. That program then tries to dlopen 18-0.so when there
> is already 18-1.so installed. Relevant when the first oauth connection
> is made way after startup.

Ugh, good point. This hazard applies to the previous suggestion of
pkglibdir, too, but in that case it would have been silent...

> This is trading one problem for another, but within-a-major ABI
> changes should be much rarer than normal minor updates with
> applications restarting only later.

But the consequences are much worse for a silent ABI mismatch. Imagine
if libpq-oauth examines the wrong pointer inside PGconn for a
security-critical check.

> Alternatively, there could be a dedicated SONAME for the plugin that
> only changes when necessary, but perhaps the simple "18.so" solution
> is good enough.

I don't think SONAME helps us, does it? We're not using it in dlopen().

We could all agree to bump the second number in the filename whenever
there's an internal ABI change. That works from a technical
perspective, but it's hard to test and enforce and... just not forget.
Or, I may still be able to thread the needle with a fuller lookup
table, and remove the dependency on libpq-int.h; it's just not going
to be incredibly pretty. Thinking...

Thanks so much for your continued review!

--Jacob

  366. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-23T16:38:07Z

    Re: Jacob Champion
> But the consequences are much worse for a silent ABI mismatch. Imagine
> if libpq-oauth examines the wrong pointer inside PGconn for a
> security-critical check.

True.

> > Alternatively, there could be a dedicated SONAME for the plugin that
> > only changes when necessary, but perhaps the simple "18.so" solution
> > is good enough.
> 
> I don't think SONAME helps us, does it? We're not using it in dlopen().

That was paraphrasing, with SONAME I meant "library file name that
changes when the ABI changes".

> We could all agree to bump the second number in the filename whenever
> there's an internal ABI change. That works from a technical
> perspective, but it's hard to test and enforce and... just not forget.

It's hopefully not harder than checking ABI compatibility of any other
libpq change, just a different number. If that number is in the
meson.build in the same directory, people should be able to connect
the dots.

Btw, if we have that number, we might as well drop the MAJOR part as
well... apt.pg.o is always shipping the latest libpq5, so major libpq
upgrades while apps are running are going to happen. (But this is just
once a year and much less problematic than minor upgrades and I'm not
going to complain if MAJOR is kept.)

> Or, I may still be able to thread the needle with a fuller lookup
> table, and remove the dependency on libpq-int.h; it's just not going
> to be incredibly pretty. Thinking...

Don't overdesign it...

Christoph

  367. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-23T17:46:14Z

    On Wed, Apr 23, 2025 at 9:38 AM Christoph Berg <myon@debian.org> wrote:
> > We could all agree to bump the second number in the filename whenever
> > there's an internal ABI change. That works from a technical
> > perspective, but it's hard to test and enforce and... just not forget.
>
> It's hopefully not harder than checking ABI compatibility of any other
> libpq change, just a different number. If that number is in the
> meson.build in the same directory, people should be able to connect
> the dots.

I think it is harder, simply because no one has to do it today, and
that change would sign them up to do it, forever, adding to the
backport checklist. It's one thing if there's a bunch of committers
who pile into the thread right now saying "yes, that's okay", but I
don't really feel comfortable making that decision for them right this
instant.

If we had robust ABI compatibility checks as part of the farm [1], I
think we could do that. Doesn't feel like an 18 thing, though.

> Btw, if we have that number, we might as well drop the MAJOR part as
> well... apt.pg.o is always shipping the latest libpq5, so major libpq
> upgrades while apps are running are going to happen. (But this is just
> once a year and much less problematic than minor upgrades and I'm not
> going to complain if MAJOR is kept.)

I don't want to introduce another testing matrix dimension if I can
avoid it. ("I have this bug where libpq.so.5.18 is using
libpq-oauth.so from PG20 and I had no idea it was doing that and the
problem went away when I restarted and...")

And the intent is for this to be temporary until we have a user-facing
API. If this is the solution we go with, I think it'd wise to prepare
for a -19 version of libpq-oauth, but I'm going to try my best to get
custom modules in ASAP. People are going to be annoyed that v1 of the
feature doesn't let them swap the flow for our utilities. Ideally they
only have to deal with that for a single major release.

Also: since the libpq-oauth-18 and libpq-oauth-19 packages can be
installed side-by-side safely, isn't the upgrade hazard significantly
diminished? (If a user uninstalls the previous libpq-oauth version
while they're still running that version of libpq in memory, _and_
they've somehow never used OAuth until right that instant... it's easy
enough for them to undo their mistake while the application is still
running.)

> > Or, I may still be able to thread the needle with a fuller lookup
> > table, and remove the dependency on libpq-int.h; it's just not going
> > to be incredibly pretty. Thinking...
>
> Don't overdesign it...

Oh, I agree... but my "minimal" ABI designs have all had corner cases
so far. I may need to just bite the bullet.

Are there any readers who feel like an internal ABI version for
`struct pg_conn`, bumped during breaking backports, would be
acceptable? (More definitively: are there any readers who would veto
that?) You're still signing up for delayed errors in the long-lived
client case, so it's not a magic bullet, but the breakage is easy to
see and it's not a crash. The client application "just" has to restart
after a libpq upgrade.

Thanks,
--Jacob

[1] https://www.postgresql.org/message-id/B142EE8A-5D38-48B9-A4BB-82D69A854B55%40justatheory.com

  368. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-23T20:12:55Z

    Re: Jacob Champion
> Also: since the libpq-oauth-18 and libpq-oauth-19 packages can be
> installed side-by-side safely, isn't the upgrade hazard significantly
> diminished? (If a user uninstalls the previous libpq-oauth version
> while they're still running that version of libpq in memory, _and_
> they've somehow never used OAuth until right that instant... it's easy
> enough for them to undo their mistake while the application is still
> running.)

Uhm, so far the plan was to have one "libpq-oauth" package, not several.
Since shipping a single libpq5.deb package for all PG majors has worked well
for the past decades, I wouldn't want to complicate that now.

Christoph

  369. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-24T17:02:25Z

    On Wed, Apr 23, 2025 at 1:13 PM Christoph Berg <myon@debian.org> wrote:
> Uhm, so far the plan was to have one "libpq-oauth" package, not several.

I think the system is overconstrained at that point. If you want to
support clients that delay-load the ABI they're compiled against,
_and_ have them continue to work seamlessly after the system has
upgraded the ABI underneath them, without restarting the client... is
there any option other than side-by-side installation?

> Since shipping a single libpq5.deb package for all PG majors has worked well
> for the past decades, I wouldn't want to complicate that now.

I'm not sure if it's possible to ship a client-side module system
without something getting more complicated, though... I'm trying hard
not to overcomplicate it for you, but I also don't think the
complexity is going to remain the same.

--Jacob

  370. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-04-25T09:02:57Z

    Re: Jacob Champion
> I think the system is overconstrained at that point. If you want to
> support clients that delay-load the ABI they're compiled against,
> _and_ have them continue to work seamlessly after the system has
> upgraded the ABI underneath them, without restarting the client... is
> there any option other than side-by-side installation?

My point is that we should be trying to change the ABI-as-coded-in-the-
filename as rarely as possible. Then side-by-side should not be required.

Christoph

  371. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-25T20:31:16Z

    On Fri, Apr 25, 2025 at 2:03 AM Christoph Berg <myon@debian.org> wrote:
> My point is that we should be trying to change the ABI-as-coded-in-the-
> filename as rarely as possible.

I agree, but I'm also trying to say I can't unilaterally declare
pieces of our internal structs to be covered by an ABI guarantee.
Maybe the rest of the ABI will never change because it'll be perfect,
but I point to the immediately preceding thread as evidence against
the likelihood of perfection on the first try. I'm trying to build in
air bags so we don't have to regret a mistake.

> Then side-by-side should not be required.

It's still required _during_ an ABI bump, though, if you don't want
things to break. Right?

--Jacob

  372. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-29T00:10:06Z

    On Wed, Apr 23, 2025 at 10:46 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Are there any readers who feel like an internal ABI version for
> `struct pg_conn`, bumped during breaking backports, would be
> acceptable? (More definitively: are there any readers who would veto
> that?)

To keep things moving: I assume this is unacceptable. So v10 redirects
every access to a PGconn struct member through a shim, similarly to
how conn->errorMessage was translated in v9. This adds plenty of new
boilerplate, but not a whole lot of complexity. To try to keep us
honest, libpq-int.h has been removed from the libpq-oauth includes.

This will now handle in-place minor version upgrades that swap pg_conn
internals around, so I've gone back to -MAJOR versioning alone.
fe_oauth_state is still exported; it now has an ABI warning above it.
(I figure that's easier to draw a line around during backports,
compared to everything in PGconn. We can still break things there
during major version upgrades.)

Thanks,
--Jacob

  373. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-30T12:55:07Z

    > On 29 Apr 2025, at 02:10, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Wed, Apr 23, 2025 at 10:46 AM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
>> Are there any readers who feel like an internal ABI version for
>> `struct pg_conn`, bumped during breaking backports, would be
>> acceptable? (More definitively: are there any readers who would veto
>> that?)
> 
> To keep things moving: I assume this is unacceptable. So v10 redirects
> every access to a PGconn struct member through a shim, similarly to
> how conn->errorMessage was translated in v9. This adds plenty of new
> boilerplate, but not a whole lot of complexity. To try to keep us
> honest, libpq-int.h has been removed from the libpq-oauth includes.

That admittedly seems like a win regardless.

> This will now handle in-place minor version upgrades that swap pg_conn
> internals around, so I've gone back to -MAJOR versioning alone.
> fe_oauth_state is still exported; it now has an ABI warning above it.
> (I figure that's easier to draw a line around during backports,
> compared to everything in PGconn. We can still break things there
> during major version upgrades.)

While I'm far from the expert on this subject (luckily there are such in this
thread), I am unable to see any sharp edges from reading and testing this
version of the patch. A few small comments:


+libpq-oauth is an optional module implementing the Device Authorization flow for
+OAuth clients (RFC 8628). It was originally developed as part of libpq core and
+later split out as its own shared library in order to isolate its dependency on
+libcurl. (End users who don't want the Curl dependency can simply choose not to
+install this module.)

We should either clarify that it was never shipped as part of libpq core, or
remove this altogether.  I would vote for the latter since we typically don't
document changes that happen during the devcycle.  How about something like:

+libpq-oauth is an optional module implementing the Device Authorization flow for
+OAuth clients (RFC 8628). It is maintained as its own shared library in order to
+isolate its dependency on libcurl. (End users who don't want the Curl dependency
+can simply choose not to install this module.)


+- void libpq_oauth_init(pgthreadlock_t threadlock,
+             <snip>
+At the moment, pg_fe_run_oauth_flow() relies on libpq's pg_g_threadlock and
+libpq_gettext(), which must be injected by libpq using this initialization
+function before the flow is run.

I think this explanatory paragraph should come before the function prototype.
The following paragraph on the setters/getters make sense where it is though.


+#if defined(USE_DYNAMIC_OAUTH) && defined(LIBPQ_INT_H)
+#error do not rely on libpq-int.h in libpq-oauth.so
+#endif

Nitpick, but it won't be .so everywhere.  Would this be clearar if spelled out
with something like "do not rely on libpq-int.h when building libpq-oauth as
dynamic shared lib"?

--
Daniel Gustafsson

  374. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-04-30T17:59:43Z

    On Wed, Apr 30, 2025 at 5:55 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > To keep things moving: I assume this is unacceptable. So v10 redirects
> > every access to a PGconn struct member through a shim, similarly to
> > how conn->errorMessage was translated in v9. This adds plenty of new
> > boilerplate, but not a whole lot of complexity. To try to keep us
> > honest, libpq-int.h has been removed from the libpq-oauth includes.
>
> That admittedly seems like a win regardless.

Yeah, it moves us much closer to the long-term goal.

> We should either clarify that it was never shipped as part of libpq core, or
> remove this altogether.

Done in v11, with your suggested wording.

> I think this explanatory paragraph should come before the function prototype.

Done.

> Nitpick, but it won't be .so everywhere.  Would this be clearar if spelled out
> with something like "do not rely on libpq-int.h when building libpq-oauth as
> dynamic shared lib"?

I went with "do not rely on libpq-int.h in dynamic builds of
libpq-oauth", since devs are hopefully going to be the only people who
see it. I've also fixed up an errant #endif label right above it.

I'd ideally like to get a working split in for beta. Barring
objections, I plan to get this pushed tomorrow so that the buildfarm
has time to highlight any corner cases well before the Saturday
freeze. I still see the choice of naming (with its forced-ABI break
every major version) as needing more scrutiny, and probably worth a
Revisit entry.

The CI still looks happy, and I will spend today with VMs and more
testing on the Autoconf side. I'll try to peer at Alpine and musl
libc, too; dogfish and basilisk are the Curl-enabled animals that
caught my attention most.

Thanks!
--Jacob

  375. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-04-30T18:09:28Z

    > On 30 Apr 2025, at 19:59, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Wed, Apr 30, 2025 at 5:55 AM Daniel Gustafsson <daniel@yesql.se> wrote:

>> Nitpick, but it won't be .so everywhere.  Would this be clearar if spelled out
>> with something like "do not rely on libpq-int.h when building libpq-oauth as
>> dynamic shared lib"?
> 
> I went with "do not rely on libpq-int.h in dynamic builds of
> libpq-oauth", since devs are hopefully going to be the only people who
> see it. I've also fixed up an errant #endif label right above it.

That's indeed better than my suggestion.

> I'd ideally like to get a working split in for beta.

+Many

> Barring
> objections, I plan to get this pushed tomorrow so that the buildfarm
> has time to highlight any corner cases well before the Saturday
> freeze.

I'll try to kick the tyres a bit more as well.

--
Daniel Gustafsson

  376. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-01T17:38:15Z

    On Wed, Apr 30, 2025 at 11:09 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> I'll try to kick the tyres a bit more as well.

Thanks! Alpine seems to be happy with the dlopen() arrangement. And
I've thrown some more Autoconf testing at Rocky, Mac, and Ubuntu.

So, committed. Thanks everyone for all the excellent feedback!
(Further feedback is still very welcome.)

--Jacob

  377. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-01T19:24:19Z

    On Thu, May 1, 2025 at 10:38 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I've thrown some more Autoconf testing at Rocky, Mac, and Ubuntu.
>
> So, committed.

I forgot --enable-nls in my Mac testing, so indri complains about my
omission of -lintl... I'd incorrectly thought it was no longer needed
after all the gettext motion.

I'm running the attached fixup through CI now.

--Jacob

  378. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-01T20:31:05Z

    On Thu, May 1, 2025 at 12:24 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> I'm running the attached fixup through CI now.

(Pushed, and indri is happy again.)

--Jacob

  379. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-01T20:41:30Z

    On Mon, Apr 21, 2025 at 9:57 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> So to recap: I'm happy to add a Google compatibility mode, but I'd
> like to gather some evidence that their device flow can actually
> authorize tokens for third parties safely, before we commit to that.
> Thoughts?

Hi Ivan, I know the thread has been deep in discussion around the
module split, but I was wondering if you'd had any thoughts on the
Google safety problem?

--Jacob

  380. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Nathan Bossart <nathandbossart@gmail.com> — 2025-05-02T15:11:08Z

    After commit b0635bf, I'm seeing the following meson build failures on
macOS:

	In file included from ../postgresql/src/interfaces/libpq-oauth/oauth-curl.c:51:
	../postgresql/src/interfaces/libpq/libpq-int.h:70:10: fatal error: 'openssl/ssl.h' file not found
	   70 | #include <openssl/ssl.h>
	      |          ^~~~~~~~~~~~~~~
	1 error generated.

The following patch seems to resolve it.  I'm curious if commit 4ea1254
might apply to meson, too, but FWIW I haven't noticed any related failures
on my machine.

diff --git a/meson.build b/meson.build
index 29d46c8ad01..19ad03042d3 100644
--- a/meson.build
+++ b/meson.build
@@ -3295,6 +3295,7 @@ libpq_deps += [

 libpq_oauth_deps += [
   libcurl,
+  ssl,
 ]

 subdir('src/interfaces/libpq')

-- 
nathan

  381. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T15:46:21Z

    On Fri, May 2, 2025 at 8:11 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
>
> After commit b0635bf, I'm seeing the following meson build failures on
> macOS:

Thanks for the report, and sorry for the breakage.

>         In file included from ../postgresql/src/interfaces/libpq-oauth/oauth-curl.c:51:
>         ../postgresql/src/interfaces/libpq/libpq-int.h:70:10: fatal error: 'openssl/ssl.h' file not found
>            70 | #include <openssl/ssl.h>
>               |          ^~~~~~~~~~~~~~~
>         1 error generated.

Hm. My test setup here is Homebrew with -Dextra_include_dirs, which
may explain why it's not failing for me. Looks like Cirrus also has
-Dextra_include_dirs...

> The following patch seems to resolve it.  I'm curious if commit 4ea1254
> might apply to meson, too, but FWIW I haven't noticed any related failures
> on my machine.

Yeah, I wonder if libintl is being similarly "cheated" on the Meson side.

> diff --git a/meson.build b/meson.build
> index 29d46c8ad01..19ad03042d3 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -3295,6 +3295,7 @@ libpq_deps += [
>
>  libpq_oauth_deps += [
>    libcurl,
> +  ssl,
>  ]

Thanks! I think the include directory is the only thing needed for the
static library, not the full link dependency. But I'll try to build up
a new Meson setup, with minimal added settings, to see if I can
reproduce here. Can you share your Meson configuration?

--Jacob

  382. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T15:59:18Z

    On Fri, May 2, 2025 at 8:46 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Yeah, I wonder if libintl is being similarly "cheated" on the Meson side.

libintl is already coming in via frontend_stlib_code, so that's fine.
So now I'm wondering if any other static clients of libpq-int.h (if
there are any) need the ssl dependency too, for correctness, or if
it's just me.

> But I'll try to build up
> a new Meson setup, with minimal added settings, to see if I can
> reproduce here. Can you share your Meson configuration?

(Never mind -- this was pretty easy to hit in a from-scratch configuration.)

--Jacob

  383. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T17:05:26Z

    On Fri, May 2, 2025 at 8:59 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> libintl is already coming in via frontend_stlib_code, so that's fine.
> So now I'm wondering if any other static clients of libpq-int.h (if
> there are any) need the ssl dependency too, for correctness, or if
> it's just me.

Looks like it's just me. And using partial_dependency for the includes
seems like overkill, so I've kept the full ssl dependency object, but
moved it to the staticlib only, which is enough to solve the breakage
on my machine.

Nathan, if you get a chance, does the attached patch work for you?

Thanks!
--Jacob

  384. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Nathan Bossart <nathandbossart@gmail.com> — 2025-05-02T17:31:27Z

    On Fri, May 02, 2025 at 10:05:26AM -0700, Jacob Champion wrote:
> Nathan, if you get a chance, does the attached patch work for you?

Yup, thanks!

-- 
nathan

  385. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-05-02T17:35:30Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> Looks like it's just me. And using partial_dependency for the includes
> seems like overkill, so I've kept the full ssl dependency object, but
> moved it to the staticlib only, which is enough to solve the breakage
> on my machine.
> Nathan, if you get a chance, does the attached patch work for you?

FWIW, on my Mac a meson build from HEAD goes through fine, with or
without this patch.  I'm getting openssl and libcurl from MacPorts
not Homebrew, but I don't know why that would make any difference.

			regards, tom lane

  386. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T17:42:25Z

    On Fri, May 2, 2025 at 10:35 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> FWIW, on my Mac a meson build from HEAD goes through fine, with or
> without this patch.  I'm getting openssl and libcurl from MacPorts
> not Homebrew, but I don't know why that would make any difference.

Do your <libintl.h> and <openssl/*.h> live in the same place? Mine do,
so I had to disable NLS to get this to break.

--Jacob

  387. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T17:42:34Z

    On Fri, May 2, 2025 at 10:31 AM Nathan Bossart <nathandbossart@gmail.com> wrote:
> Yup, thanks!

Great, thanks. I'll push it soon.

--Jacob

  388. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Nathan Bossart <nathandbossart@gmail.com> — 2025-05-02T17:59:12Z

    On Fri, May 02, 2025 at 10:42:25AM -0700, Jacob Champion wrote:
> On Fri, May 2, 2025 at 10:35 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> FWIW, on my Mac a meson build from HEAD goes through fine, with or
>> without this patch.  I'm getting openssl and libcurl from MacPorts
>> not Homebrew, but I don't know why that would make any difference.
> 
> Do your <libintl.h> and <openssl/*.h> live in the same place? Mine do,
> so I had to disable NLS to get this to break.

I enabled NLS and the problem disappeared for me, but that seems to be a
side effect of setting -Dextra_{include,lib}_dirs to point to my Homebrew
directories, which I needed to do to get NLS to work.

-- 
nathan

  389. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-05-02T18:25:49Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Fri, May 2, 2025 at 10:35 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> FWIW, on my Mac a meson build from HEAD goes through fine, with or
>> without this patch.  I'm getting openssl and libcurl from MacPorts
>> not Homebrew, but I don't know why that would make any difference.

> Do your <libintl.h> and <openssl/*.h> live in the same place? Mine do,
> so I had to disable NLS to get this to break.

Yeah, they are both under /opt/local/include in a MacPorts setup.
But disabling NLS doesn't break it for me.  I tried

meson setup build --auto-features=disabled -Dlibcurl=enabled

to make sure that /opt/local/include wasn't getting pulled in
some other way, and it still builds.

Apropos of that: our fine manual claims that option is spelled
--auto_features, but that fails for me.  Is that a typo in our
manual, or do some meson versions accept the underscore?

			regards, tom lane

  390. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T18:46:15Z

    On Fri, May 2, 2025 at 11:26 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Yeah, they are both under /opt/local/include in a MacPorts setup.
> But disabling NLS doesn't break it for me.  I tried
>
> meson setup build --auto-features=disabled -Dlibcurl=enabled
>
> to make sure that /opt/local/include wasn't getting pulled in
> some other way, and it still builds.

Hm. If you clear out the build artifacts under
src/interfaces/libpq-oauth, and then build with

    $ ninja -v src/interfaces/libpq-oauth/libpq-oauth.a

does that help surface anything interesting?

> Apropos of that: our fine manual claims that option is spelled
> --auto_features, but that fails for me.  Is that a typo in our
> manual, or do some meson versions accept the underscore?

--auto_features doesn't work for me either. That looks like an
accidental mashup of --auto-features and -Dauto_features.

--Jacob

  391. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-05-02T18:52:36Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> Hm. If you clear out the build artifacts under
> src/interfaces/libpq-oauth, and then build with
>     $ ninja -v src/interfaces/libpq-oauth/libpq-oauth.a
> does that help surface anything interesting?

$ rm -rf src/interfaces/libpq-oauth
$ ninja -v src/interfaces/libpq-oauth/libpq-oauth.a
[1/2] ccache cc -Isrc/interfaces/libpq-oauth/libpq-oauth.a.p -Isrc/interfaces/libpq-oauth -I../src/interfaces/libpq-oauth -Isrc/interfaces/libpq -I../src/interfaces/libpq -Isrc/port -I../src/port -Isrc/include -I../src/include -I/opt/local/include -I/opt/local/libexec/openssl3/include -fdiagnostics-color=always -Wall -Winvalid-pch -O2 -g -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.4.sdk -fno-strict-aliasing -fwrapv -fexcess-precision=standard -Wmissing-prototypes -Wpointer-arith -Werror=vla -Werror=unguarded-availability-new -Wendif-labels -Wmissing-format-attribute -Wcast-function-type -Wformat-security -Wdeclaration-after-statement -Wmissing-variable-declarations -Wno-unused-command-line-argument -Wno-compound-token-split-by-macro -Wno-format-truncation -Wno-cast-function-type-strict -MD -MQ src/interfaces/libpq-oauth/libpq-oauth.a.p/oauth-curl.c.o -MF src/interfaces/libpq-oauth/libpq-oauth.a.p/oauth-curl.c.o.d -o src/interfaces/libpq-oauth/libpq-oauth.a.p/oauth-curl.c.o -c ../src/interfaces/libpq-oauth/oauth-curl.c
[2/2] rm -f src/interfaces/libpq-oauth/libpq-oauth.a && ar csr src/interfaces/libpq-oauth/libpq-oauth.a src/interfaces/libpq-oauth/libpq-oauth.a.p/oauth-curl.c.o && ranlib -c src/interfaces/libpq-oauth/libpq-oauth.a

So it's getting -I/opt/local/include and also
-I/opt/local/libexec/openssl3/include from somewhere,
which I guess must be libcurl's pkg-config data ... yup:

$ pkg-config --cflags libcurl
-I/opt/local/include -I/opt/local/libexec/openssl3/include -I/opt/local/include

I bet Homebrew's libcurl packaging doesn't do that.

			regards, tom lane

  392. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T18:56:46Z

    On Fri, May 2, 2025 at 11:52 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> $ pkg-config --cflags libcurl
> -I/opt/local/include -I/opt/local/libexec/openssl3/include -I/opt/local/include
>
> I bet Homebrew's libcurl packaging doesn't do that.

Nope, Homebrew breaks them out into smaller pieces:

    % PKG_CONFIG_PATH=/opt/homebrew/opt/curl/lib/pkgconfig pkg-config
--cflags libcurl
    -I/opt/homebrew/Cellar/curl/8.13.0/include
-I/opt/homebrew/Cellar/brotli/1.1.0/include
-I/opt/homebrew/opt/zstd/include
-I/opt/homebrew/Cellar/libssh2/1.11.1/include
-I/opt/homebrew/Cellar/rtmpdump/2.4-20151223_3/include
-I/opt/homebrew/Cellar/openssl@3/3.5.0/include
-I/opt/homebrew/Cellar/libnghttp2/1.65.0/include

--Jacob

  393. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-02T19:05:10Z

    On Fri, May 2, 2025 at 11:56 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> -I/opt/homebrew/Cellar/openssl@3/3.5.0/include

Except it _is_ right there.

Oh, ha -- I'm not using Homebrew's Curl in this minimal build. Looks
like it's coming from the sysroot.

    % ls -l /Library/Developer/CommandLineTools/SDKs/MacOSX15.2.sdk/usr/include/curl
    total 208
    -rw-r--r--  1 root  wheel  129052 Nov  9 22:54 curl.h
    -rw-r--r--  1 root  wheel    3044 Nov  9 22:54 curlver.h
    ...

Well, that was fun.

--Jacob

  394. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-05-02T19:05:26Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Fri, May 2, 2025 at 11:26 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Apropos of that: our fine manual claims that option is spelled
>> --auto_features, but that fails for me.  Is that a typo in our
>> manual, or do some meson versions accept the underscore?

> --auto_features doesn't work for me either. That looks like an
> accidental mashup of --auto-features and -Dauto_features.

Ah, I see somebody already complained of this [1], but apparently
we did nothing about it.  I shall go fix it now.

			regards, tom lane

[1] https://www.postgresql.org/message-id/flat/172465652540.862882.17808523044292761256%40wrigleys.postgresql.org

  395. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Christoph Berg <myon@debian.org> — 2025-05-03T14:54:59Z

    Re: Jacob Champion
> So, committed. Thanks everyone for all the excellent feedback!

The package split between libpq5 and libpq-oauth in Debian has already
been accepted into the experimental branch.

Thanks,
Christoph

  396. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Wolfgang Walther <walther@technowledgy.de> — 2025-05-04T12:58:48Z

    Jacob Champion:
>> libintl is already coming in via frontend_stlib_code, so that's fine.
>> So now I'm wondering if any other static clients of libpq-int.h (if
>> there are any) need the ssl dependency too, for correctness, or if
>> it's just me.
> 
> Looks like it's just me. And using partial_dependency for the includes
> seems like overkill, so I've kept the full ssl dependency object, but
> moved it to the staticlib only, which is enough to solve the breakage
> on my machine.
> 
> Nathan, if you get a chance, does the attached patch work for you?

I couldn't reproduce the problem, so did not test the latest patch. But 
I tested a lot of scenarios on nixpkgs with latest master (250a718a):

- aarch64 + x86_64 architectures, both Linux and MacOS

- Autoconf and Meson

- Various features enabled / disabled in different configurations (NLS, 
OpenSSL, GSSAPI)

- And additionally some cross-compiling from x86_64 Linux to aarch64 
Linux and x86_64 FreeBSD

Worked very well.


The only inconsistency I was able to find is the autoconf-generated 
libpq.pc file, which has this:

   Requires.private: libssl, libcrypto libcurl

Note the missing "," before libcurl.

It does *not* affect functionality, though:

   pkg-config --print-requires-private libpq
   libssl
   libcrypto
   libcurl


The meson-generated libpq.pc looks like this:

   Requires.private: openssl, krb5-gssapi, libcurl >=  7.61.0

I was only able to test the latter in an end-to-end fully static build 
of a downstream dependency - works great. The final executable has all 
the expected oauth strings in it.

Best,

Wolfgang

  397. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Devrim Gündüz <devrim@gunduz.org> — 2025-05-05T12:37:45Z

    Hi,

On Sat, 2025-05-03 at 16:54 +0200, Christoph Berg wrote:
> The package split between libpq5 and libpq-oauth in Debian has already
> been accepted into the experimental branch.

RPMs will ship postgresql18-libs and postgresql18-libs-oauth. The latter
depends on the former for sure.

Regards,

-- 
Devrim Gündüz
Open Source Solution Architect, PostgreSQL Major Contributor
BlueSky: @devrim.gunduz.org , @gunduz.org

  398. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-05-06T17:49:31Z

    On Sun, May 4, 2025 at 5:58 AM Wolfgang Walther <walther@technowledgy.de> wrote:
> The only inconsistency I was able to find is the autoconf-generated
> libpq.pc file, which has this:
>
>    Requires.private: libssl, libcrypto libcurl

Oh, I see what I did. Will fix, thanks.

> I was only able to test the latter in an end-to-end fully static build
> of a downstream dependency - works great. The final executable has all
> the expected oauth strings in it.

Thank you so much for all the detailed testing!

--Jacob

  399. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-06-12T19:58:44Z

    On Thu, Mar 6, 2025 at 12:57 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> 3) There is a related performance bug on other platforms. If a Curl
> timeout happens partway through a request (so libcurl won't clear it),
> the timer-expired event will stay set and CPU will be burned to spin
> pointlessly on drive_request(). This is much easier to notice after
> taking Happy Eyeballs out of the picture. It doesn't cause logical
> failures -- Curl basically discards the unnecessary calls -- but it's
> definitely unintended.
>
> ...
>
> I plan to defer working on Problem 3, which should just be a
> performance bug, until the tests are green again. And I would like to
> eventually add some stronger unit tests for the timer behavior, to
> catch other potential OS-specific problems in the future.

To follow up on this: I had intended to send a patch fixing the timer
bug this week, but after fixing it, the performance problem did not
disappear. Turns out: other file descriptors can get stuck open on
BSD, depending on how complicated Curl wants to make the order of
operations, and the existing tests aren't always enough to expose it.
(It also depends on the Curl version installed.)

I will split this off into its own thread soon, because this
megathread is just too big, but I wanted to make a note here and file
an open item. As part of that, I have a set of more rigorous unit
tests for the libcurl-libpq interaction that I'm working on, since the
external view of "the flow worked/didn't work" is not enough to
indicate internal health.

--Jacob

  400. Re: [PoC] Federated Authn/z with OAUTHBEARER

    ivan.kush@tantorlabs.com — 2025-06-20T10:08:35Z

    Hello!

This patch fixes CPPFLAGS, LDFLAGS, LIBS when checking AsyncDNS libcurl 
support in configure

Custom parameters and paths to libcurl were mistakenly excluded from 
CPPFLAGS, LDFLAGS, and LIBS, although AsyncDNS check was OK.

For example, the command `pkg-config --libs libcurl` gives 
`-L/usr/local/lib -lcurl`. LDFLAGS will not contain `-L/usr/local/lib`.

This patch fixes such behaviour.


Test case:

I've tested custom Postgres in an old Debian based Linux distro. This 
distro contains old libcurl (< 7.61, package libcurl3) that was compiled 
with symbols CURL_OPENSSL_3. So I've installed newer version of 
libcurlssl, package libcurl4-openssl-dev, that contains symbols 
CURL_OPENSSL_4 and compiled my libcurl version > 7.61.

After compilation during testing some Postgres shared libraries or 
binaries that was linked with libcurl showed an error "version 
CURL_OPENSSL_3 not found (required by …/libcurl.so.4)"

-- 
Best wishes,
Ivan Kush
Tantor Labs LLC

  401. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-06-23T15:32:26Z

    On Fri, Jun 20, 2025 at 3:08 AM Ivan Kush <ivan.kush@tantorlabs.com> wrote:
>
> Hello!
>
> This patch fixes CPPFLAGS, LDFLAGS, LIBS when checking AsyncDNS libcurl
> support in configure

Hi Ivan, thanks for the report! Your patch puts new logic directly
after an AC_MSG_ERROR() call, so any effect has to come from the fact
that we're no longer restoring the old compiler and linker flags.
That's not what we want -- Curl needs to be isolated from the rest of
the build.

Let's focus on the error you're seeing:

> After compilation during testing some Postgres shared libraries or
> binaries that was linked with libcurl showed an error "version
> CURL_OPENSSL_3 not found (required by …/libcurl.so.4)"

What's your configure line? You need to make sure that your custom
libcurl is used at configure-time, compile-time, and run-time.

And which binaries are complaining? The only thing that should ever be
linked against libcurl is libpq-oauth-18.so.

Thanks,
--Jacob

  402. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-06-30T16:58:18Z

    Hi,

On 2025-05-02 10:42:34 -0700, Jacob Champion wrote:
> Great, thanks. I'll push it soon.

I just noticed that I think the dependencies on the meson build aren't quite
sufficient:

andres@awork3:/srv/dev/build/postgres/m-dev-assert$ ninja install-quiet
[2205/2205 1 100%] Generating install-quiet with a custom command
FAILED: install-quiet
/usr/bin/python3 /home/andres/src/meson/meson.py install --quiet --no-rebuild

ERROR: File 'src/interfaces/libpq-oauth/libpq-oauth.a' could not be found
ninja: build stopped: subcommand failed.


Probably just needs to be added to the installed_targets list.

Greetings,

Andres Freund

  403. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-06-30T17:01:50Z

    > On 30 Jun 2025, at 18:58, Andres Freund <andres@anarazel.de> wrote:
> 
> Hi,
> 
> On 2025-05-02 10:42:34 -0700, Jacob Champion wrote:
>> Great, thanks. I'll push it soon.
> 
> I just noticed that I think the dependencies on the meson build aren't quite
> sufficient:
> 
> andres@awork3:/srv/dev/build/postgres/m-dev-assert$ ninja install-quiet
> [2205/2205 1 100%] Generating install-quiet with a custom command
> FAILED: install-quiet
> /usr/bin/python3 /home/andres/src/meson/meson.py install --quiet --no-rebuild
> 
> ERROR: File 'src/interfaces/libpq-oauth/libpq-oauth.a' could not be found
> ninja: build stopped: subcommand failed.
> 
> Probably just needs to be added to the installed_targets list.

Thanks for the report, I'll take a look today to get it fixed.

--
Daniel Gustafsson

  404. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-06-30T18:33:27Z

    On Mon, Jun 30, 2025 at 10:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > On 30 Jun 2025, at 18:58, Andres Freund <andres@anarazel.de> wrote:
> > Probably just needs to be added to the installed_targets list.
>
> Thanks for the report, I'll take a look today to get it fixed.

Thanks both!

Looking at the installed_targets stuff, though... why do we use `meson
install --no-rebuild` in combination with `depends:
installed_targets`? Can't we just use Meson's dependency tracking
during installation, and avoid this hazard?

--Jacob

  405. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-06-30T22:52:49Z

    > On 30 Jun 2025, at 20:33, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> 
> On Mon, Jun 30, 2025 at 10:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
>>> On 30 Jun 2025, at 18:58, Andres Freund <andres@anarazel.de> wrote:
>>> Probably just needs to be added to the installed_targets list.
>> 
>> Thanks for the report, I'll take a look today to get it fixed.
> 
> Thanks both!
> 
> Looking at the installed_targets stuff, though... why do we use `meson
> install --no-rebuild` in combination with `depends:
> installed_targets`? Can't we just use Meson's dependency tracking
> during installation, and avoid this hazard?

I suspect it is because without --no-rebuild the quiet target isn't entirely
quiet.  Still, I was unable to make something that work in all build
combinations while keeping --no-rebuild (which isn't indicative of it being
possible to do).  Is --no-rebuild just there to reduce output noise, or is
there another reason that I don't see?

--
Daniel Gustafsson

  406. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-06-30T23:42:51Z

    Hi,

On 2025-07-01 00:52:49 +0200, Daniel Gustafsson wrote:
> > On 30 Jun 2025, at 20:33, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > 
> > On Mon, Jun 30, 2025 at 10:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> >>> On 30 Jun 2025, at 18:58, Andres Freund <andres@anarazel.de> wrote:
> >>> Probably just needs to be added to the installed_targets list.
> >> 
> >> Thanks for the report, I'll take a look today to get it fixed.
> > 
> > Thanks both!
> > 
> > Looking at the installed_targets stuff, though... why do we use `meson
> > install --no-rebuild` in combination with `depends:
> > installed_targets`? Can't we just use Meson's dependency tracking
> > during installation, and avoid this hazard?

I don't think that's really possible - the dependency tracking is useful to
generate granular *rebuild* information, but doesn't help with the first
build.

If we had dependency generation for the install target it could be helpful to
discover missing dependencies though.


> I suspect it is because without --no-rebuild the quiet target isn't entirely
> quiet.

No - the issue is that you're not allowed to run ninja while ninja is running,
as that would corrupt it's tracking (and build things multiple times). meson
install --no-rebuild would run ninja to build things...


> Still, I was unable to make something that work in all build combinations
> while keeping --no-rebuild (which isn't indicative of it being possible to
> do).

Hm, what problem did you encounter? I don't think there should be any
difficulty?

Greetings,

Andres Freund

  407. Re: [PoC] Federated Authn/z with OAUTHBEARER

    ivan.kush@tantorlabs.com — 2025-07-02T12:45:30Z

    Thanks for the clarification! I thought linker flags should be installed 
globally for all compilation targets.


Another question:

Why don't we set LIBS in the configure in "checking for curl_multi_init" 
using LIBCURL_LIBS or LIBCURL_LDFLAGS?
https://github.com/postgres/postgres/blob/master/configure#L12734

Like this:
     LIBS="$(LIBCURL_LDFLAGS) $(LIBCURL_LDLIBS)"

And set LIBS with -lcurl.

As I understand we need to check the properties of libcurl we are 
compiling with?
It may be some local libcurl from /opt/my_libcurl. So LIBCURL_... may 
contain a flag like -L/opt/my_libcurl
Without these LIBCURL... variables we will check a system libcurl, not 
our local.

I mean why don't we set LIBS


current *configure*

$as_echo_n "checking for curl_multi_init in -lcurl... " >&6; }
....
else
   ac_check_lib_save_LIBS=$LIBS
LIBS="-lcurl  $LIBS"
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h.  */

https://github.com/postgres/postgres/blob/master/configure#L12734

For example, I've logged flags after this code sample and they don't 
contain -L/opt/my_libcurl

     IVK configure:13648: CFLAGS=-Wall -Wmissing-prototypes 
-Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels 
-Wmissing-format-attribute -Wformat-security -fno-strict-aliasing 
-fwrapv -fexcess-precision=standard -pipe -O2
     IVK configure:13649: LDFLAGS=-Wl,-z,relro -Wl,-z,now -flto=auto 
-ffat-lto-objects -L/usr/lib/llvm-10/lib -L/usr/local/lib/zstd
     IVK configure:13650: LIBS=-lcurl  -lz -lreadline -lpthread -lrt 
-ldl -lm
     IVK configure:13651: LDLIBS=

On 25-06-23 18:32, Jacob Champion wrote:
> On Fri, Jun 20, 2025 at 3:08 AM Ivan Kush <ivan.kush@tantorlabs.com> wrote:
>> Hello!
>>
>> This patch fixes CPPFLAGS, LDFLAGS, LIBS when checking AsyncDNS libcurl
>> support in configure
> Hi Ivan, thanks for the report! Your patch puts new logic directly
> after an AC_MSG_ERROR() call, so any effect has to come from the fact
> that we're no longer restoring the old compiler and linker flags.
> That's not what we want -- Curl needs to be isolated from the rest of
> the build.
>
> Let's focus on the error you're seeing:
>
>> After compilation during testing some Postgres shared libraries or
>> binaries that was linked with libcurl showed an error "version
>> CURL_OPENSSL_3 not found (required by …/libcurl.so.4)"
> What's your configure line? You need to make sure that your custom
> libcurl is used at configure-time, compile-time, and run-time.
>
> And which binaries are complaining? The only thing that should ever be
> linked against libcurl is libpq-oauth-18.so.
>
> Thanks,
> --Jacob

-- 
Best wishes,
Ivan Kush
Tantor Labs LLC

  408. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-02T14:46:40Z

    On Wed, Jul 2, 2025 at 5:45 AM Ivan Kush <ivan.kush@tantorlabs.com> wrote:
>
> Thanks for the clarification! I thought linker flags should be installed
> globally for all compilation targets.

Not for libcurl, since the libpq-oauth module split.

> Another question:
>
> Why don't we set LIBS in the configure in "checking for curl_multi_init"
> using LIBCURL_LIBS or LIBCURL_LDFLAGS?
> [...]
> Without these LIBCURL... variables we will check a system libcurl, not
> our local.

Ah, that's definitely a bug. I've tested alternate PKG_CONFIG_PATHs,
but I haven't regularly tested on systems that have no system libcurl
at all. So those header and lib checks need to be moved after the use
of LIBCURL_CPPFLAGS and LIBCURL_LDFLAGS to prevent a false failure.
Otherwise they're only useful for the LIBCURL_LDLIBS assignment.

I wonder if I should just get rid of those to better match the Meson
implementation... but the error messages from the checks will likely
be nicer than compilation failures during the later test programs. Hm.

(Thanks for the report!)

--Jacob

  409. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-09T17:36:26Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Wed, Jul 2, 2025 at 5:45 AM Ivan Kush <ivan.kush@tantorlabs.com> wrote:
>> Why don't we set LIBS in the configure in "checking for curl_multi_init"
>> using LIBCURL_LIBS or LIBCURL_LDFLAGS?
>> [...]
>> Without these LIBCURL... variables we will check a system libcurl, not
>> our local.

> Ah, that's definitely a bug.

I just ran into a vaguely-related failure: on RHEL8, building
with --with-libcurl leads to failures during check-world:

../../../../src/interfaces/libpq/libpq.so: undefined reference to `dlopen'
../../../../src/interfaces/libpq/libpq.so: undefined reference to `dlclose'
../../../../src/interfaces/libpq/libpq.so: undefined reference to `dlerror'
../../../../src/interfaces/libpq/libpq.so: undefined reference to `dlsym'
collect2: error: ld returned 1 exit status

Per "man dlopen", you have to link with libdl to use these functions
on this platform.  (Curiously, although RHEL9 still says that in the
documentation, it doesn't seem to actually need -ldl.)  I was able
to resolve this by adding -ldl in libpq's Makefile:

-SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
+SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -ldl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)

It doesn't look like the Meson support needs such explicit tracking of
required libraries, but perhaps I'm missing something?  I'm not able
to test that directly for lack of a usable ninja version on this
platform.

Apologies for not noticing this sooner.  I don't think I'd tried
--with-libcurl since the changes to split out libpq-oauth.

			regards, tom lane

  410. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-07-09T17:46:29Z

    Hi,

On 2025-07-09 13:36:26 -0400, Tom Lane wrote:
> It doesn't look like the Meson support needs such explicit tracking of
> required libraries, but perhaps I'm missing something?

It should be fine, -ldl is added to "os_deps" if needed, and os_deps is used
for all code in pg.

Greetings,

Andres Freund

  411. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-09T17:59:47Z

    On Wed, Jul 9, 2025 at 10:36 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Per "man dlopen", you have to link with libdl to use these functions
> on this platform.  (Curiously, although RHEL9 still says that in the
> documentation, it doesn't seem to actually need -ldl.)  I was able
> to resolve this by adding -ldl in libpq's Makefile:
>
> -SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)
> +SHLIB_LINK += $(filter -lcrypt -ldes -lcom_err -lcrypto -lk5crypto -lkrb5 -lgssapi_krb5 -lgss -lgssapi -lssl -lsocket -lnsl -lresolv -lintl -ldl -lm, $(LIBS)) $(LDAP_LIBS_FE) $(PTHREAD_LIBS)

Hmm, okay. That analysis and fix look good to me. (It looks like none
of the RHEL animals are testing with Curl yet, and locally I was using
Rocky 9...)

I'll work up a patch to send through the CI. I can't currently test
RHEL8 easily -- Rocky 8 is incompatible with my Macbook? -- which I
will need to rectify eventually, but I can't this week.

Thanks!
--Jacob

  412. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-09T18:13:08Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> I'll work up a patch to send through the CI. I can't currently test
> RHEL8 easily -- Rocky 8 is incompatible with my Macbook? -- which I
> will need to rectify eventually, but I can't this week.

No need, I already tested locally.  Will push shortly.

			regards, tom lane

  413. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-09T18:39:41Z

    On Wed, Jul 9, 2025 at 11:13 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Jacob Champion <jacob.champion@enterprisedb.com> writes:
> > I'll work up a patch to send through the CI. I can't currently test
> > RHEL8 easily -- Rocky 8 is incompatible with my Macbook? -- which I
> > will need to rectify eventually, but I can't this week.
>
> No need, I already tested locally.  Will push shortly.

Thank you very much!

Here is a draft patch for Ivan's reported issue; I still need to put
it through its paces with some more unusual setups, but I want to get
cfbot on it.

--Jacob

  414. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-09T19:07:07Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> Here is a draft patch for Ivan's reported issue; I still need to put
> it through its paces with some more unusual setups, but I want to get
> cfbot on it.

I'm confused about why this moves up the temporary changes of
CPPFLAGS and LDFLAGS, but not LIBS?  Maybe that's actually correct,
but it looks strange (and perhaps deserves a comment about why).

			regards, tom lane

  415. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-09T19:28:52Z

    On Wed, Jul 9, 2025 at 12:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Jacob Champion <jacob.champion@enterprisedb.com> writes:
> > Here is a draft patch for Ivan's reported issue; I still need to put
> > it through its paces with some more unusual setups, but I want to get
> > cfbot on it.
>
> I'm confused about why this moves up the temporary changes of
> CPPFLAGS and LDFLAGS, but not LIBS?  Maybe that's actually correct,
> but it looks strange (and perhaps deserves a comment about why).

Yeah, that's fair. It's because LIBCURL_LDLIBS isn't set until that
AC_CHECK_LIB test is run, and the test needs LIBCURL_LDFLAGS to be in
force.

(Upthread, I was idly wondering if those AC_CHECKs should just be
removed -- after all, PKG_CHECK_MODULES just told us where Curl was --
but I'm nervous that this might make more niche use cases like
cross-compilation harder to use in practice?)

Does the attached help clarify?

Thanks,
--Jacob

  416. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Tom Lane <tgl@sss.pgh.pa.us> — 2025-07-09T19:42:30Z

    Jacob Champion <jacob.champion@enterprisedb.com> writes:
> On Wed, Jul 9, 2025 at 12:07 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> I'm confused about why this moves up the temporary changes of
>> CPPFLAGS and LDFLAGS, but not LIBS?  Maybe that's actually correct,
>> but it looks strange (and perhaps deserves a comment about why).

> Does the attached help clarify?

Yes, thanks.

> (Upthread, I was idly wondering if those AC_CHECKs should just be
> removed -- after all, PKG_CHECK_MODULES just told us where Curl was --
> but I'm nervous that this might make more niche use cases like
> cross-compilation harder to use in practice?)

Nah, let's keep them.  We do document for at least some libraries
how to manually specify the include and link options without
depending on pkg-config.  If someone tries that with libcurl,
it'd be good to have sanity checks on the results.

			regards, tom lane

  417. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-09T23:54:02Z

    On Wed, Jul 9, 2025 at 12:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Nah, let's keep them.  We do document for at least some libraries
> how to manually specify the include and link options without
> depending on pkg-config.  If someone tries that with libcurl,
> it'd be good to have sanity checks on the results.

Sounds good, thanks for the review!

On Wed, Jul 9, 2025 at 11:39 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Here is a draft patch for Ivan's reported issue; I still need to put
> it through its paces with some more unusual setups, but I want to get
> cfbot on it.

On HEAD, Rocky 9 fails to build with a custom Curl PKG_CONFIG_PATH and
no libcurl-devel installed. With this patch, that build now succeeds,
and it still succeeds after libcurl-devel is reinstalled, with the
compiler tests continuing to use the custom libcurl and not the
system's.

So I'll give Ivan a little time in case he'd like to test/review
again, but otherwise I plan to push it this week.

Thanks,
--Jacob

  418. Re: [PoC] Federated Authn/z with OAUTHBEARER

    ivan.kush@tantorlabs.com — 2025-07-10T14:41:07Z

    I agree with the patch. Works in my OSes

On 7/10/25 2:54 AM, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> On Wed, Jul 9, 2025 at 12:42 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
> > Nah, let's keep them.  We do document for at least some libraries
> > how to manually specify the include and link options without
> > depending on pkg-config.  If someone tries that with libcurl,
> > it'd be good to have sanity checks on the results.
> 
> Sounds good, thanks for the review!
> 
> On Wed, Jul 9, 2025 at 11:39 AM Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
> > Here is a draft patch for Ivan's reported issue; I still need to put
> > it through its paces with some more unusual setups, but I want to get
> > cfbot on it.
> 
> On HEAD, Rocky 9 fails to build with a custom Curl PKG_CONFIG_PATH and
> no libcurl-devel installed. With this patch, that build now succeeds,
> and it still succeeds after libcurl-devel is reinstalled, with the
> compiler tests continuing to use the custom libcurl and not the
> system's.
> 
> So I'll give Ivan a little time in case he'd like to test/review
> again, but otherwise I plan to push it this week.
> 
> Thanks,
> --Jacob
>

  419. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-11T17:33:20Z

    On Thu, Jul 10, 2025 at 7:41 AM <ivan.kush@tantorlabs.com> wrote:
> I agree with the patch. Works in my OSes

Thanks Ivan! Committed.

--Jacob

  420. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Andres Freund <andres@anarazel.de> — 2025-07-18T17:26:17Z

    Hi,

On 2025-06-30 19:42:51 -0400, Andres Freund wrote:
> On 2025-07-01 00:52:49 +0200, Daniel Gustafsson wrote:
> > > On 30 Jun 2025, at 20:33, Jacob Champion <jacob.champion@enterprisedb.com> wrote:
> > > 
> > > On Mon, Jun 30, 2025 at 10:02 AM Daniel Gustafsson <daniel@yesql.se> wrote:
> > >>> On 30 Jun 2025, at 18:58, Andres Freund <andres@anarazel.de> wrote:
> > >>> Probably just needs to be added to the installed_targets list.
> > >> 
> > >> Thanks for the report, I'll take a look today to get it fixed.
> > > 
> > > Thanks both!
> > > 
> > > Looking at the installed_targets stuff, though... why do we use `meson
> > > install --no-rebuild` in combination with `depends:
> > > installed_targets`? Can't we just use Meson's dependency tracking
> > > during installation, and avoid this hazard?
> 
> I don't think that's really possible - the dependency tracking is useful to
> generate granular *rebuild* information, but doesn't help with the first
> build.
> 
> If we had dependency generation for the install target it could be helpful to
> discover missing dependencies though.
> 
> 
> > I suspect it is because without --no-rebuild the quiet target isn't entirely
> > quiet.
> 
> No - the issue is that you're not allowed to run ninja while ninja is running,
> as that would corrupt it's tracking (and build things multiple times). meson
> install --no-rebuild would run ninja to build things...
> 
> 
> > Still, I was unable to make something that work in all build combinations
> > while keeping --no-rebuild (which isn't indicative of it being possible to
> > do).
> 
> Hm, what problem did you encounter? I don't think there should be any
> difficulty?

Ping?

Greetings,

Andres Freund

  421. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Daniel Gustafsson <daniel@yesql.se> — 2025-07-18T22:29:08Z

    > On 18 Jul 2025, at 19:26, Andres Freund <andres@anarazel.de> wrote:

>> Hm, what problem did you encounter? I don't think there should be any
>> difficulty?
> 
> Ping?

Ugh, In preparing for going on vacation this fell off the radar.  I'll try to
get to looking at it tomorrow during downtime unless beaten to it.

--
Daniel Gustafsson

  422. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-07-18T23:31:21Z

    On Fri, Jul 18, 2025 at 3:29 PM Daniel Gustafsson <daniel@yesql.se> wrote:
> Ugh, In preparing for going on vacation this fell off the radar.  I'll try to
> get to looking at it tomorrow during downtime unless beaten to it.

Your earlier mail made me worried I'd missed something, but is the
attached diff what Andres was asking for? A `ninja clean; ninja
install-quiet` now works for me with this applied.

--Jacob

  423. libpq-oauth: a mid-beta naming check

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-08-04T23:20:29Z

    On Wed, Apr 30, 2025 at 10:59 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
>
> I still see the choice of naming (with its forced-ABI break
> every major version) as needing more scrutiny, and probably worth a
> Revisit entry.

It is now time to revisit.

= The Status Quo =

The libpq-oauth module is loaded on-demand, during the first use of
OAuth authentication, so users who don't want the behavior don't have
to install it. This module is named "libpq-oauth-18.so" for the PG18
release. So libpq v18 will always load the 18 OAuth behavior, libpq
v19 will load the 19 OAuth behavior, etc. Builds on HEAD have already
switched to -19, which is not yet any different from -18.

The internal API injects some libpq internals into the libpq-oauth
module. The ABI for this is assumed to break during each major version
release, so I don't have to watch the boundary like a hawk, and other
maintainers hopefully won't be saddled with breakage reports if I get
hit by a bus. (This is another advantage to using the -MAJOR naming
scheme.) And pg_conn in particular is given more protections: we can
still change its member offsets in minor versions without any ABI
breakage.

During major-version upgrades, if a packager doesn't provide a
side-by-side installation of the -18 and -19 modules, there is a
hazard: an already-loaded v18 libpq might find that the -18 module no
longer exists on disk, which would require a restart of the affected
application to pick up the v19 libpq. This is not really a consequence
of the -MAJOR naming scheme -- it's a consequence of delay-loaded
libraries that go through an ABI version bump -- but the naming scheme
makes the problem extremely visible.

The annoying part is that, if 19 doesn't change anything in the OAuth
flow compared to 18, I will basically have made busywork for our
packagers for no reason. But my goal for v19 is to replace the
internally coupled API with a public API, so that users can swap in
their own flows for use with our utilities. As far as I know, that
work necessarily includes designing a stable ABI and figuring out a
trusted place that users can put their plugins into. If we can do
both, I think we can get rid of the -MAJOR versioning scheme entirely,
because our use case will have been subsumed by the more general
framework.

So, as we approach Beta 3: can anyone think of a way that this plan will fail?

Thanks,
--Jacob

  424. Re: libpq-oauth: a mid-beta naming check

    Jelte Fennema-Nio <postgres@jeltef.nl> — 2025-08-05T09:39:10Z

    On Tue, 5 Aug 2025 at 01:20, Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> As far as I know, that
> work necessarily includes designing a stable ABI and figuring out a
> trusted place that users can put their plugins into. If we can do
> both, I think we can get rid of the -MAJOR versioning scheme entirely,
> because our use case will have been subsumed by the more general
> framework.
>
> So, as we approach Beta 3: can anyone think of a way that this plan will fail?

It's not entirely clear what plan exactly you talk about here. Are you
saying you want to remove the -MAJOR suffix now for PG18? Or you want
to postpone doing that until PG19, when you would have designed a
stable API?

Based on my current understanding from what you wrote, I think that
second option would make sense, and the first option seems sketchy.
Because we don't know yet what the PG19 API will look like.

Also, the breakage during libpq major upgrades that you describe,
while unfortunate, doesn't seem completely terrible. This only impacts
people updating system packages in place on machines, which (based on
my experience) has started to become a minority in production setups.
Also this will obviously only impact oauth users, which I expect not
to be that many right away. If your goal is to remove this
during-upgrade breakage after PG19, then I'd say that seems totally
fine for a new feature.

  425. Re: libpq-oauth: a mid-beta naming check

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-08-05T15:21:14Z

    On Tue, Aug 5, 2025 at 2:39 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> On Tue, 5 Aug 2025 at 01:20, Jacob Champion
> <jacob.champion@enterprisedb.com> wrote:
> > So, as we approach Beta 3: can anyone think of a way that this plan will fail?
>
> It's not entirely clear what plan exactly you talk about here. Are you
> saying you want to remove the -MAJOR suffix now for PG18? Or you want
> to postpone doing that until PG19, when you would have designed a
> stable API?

That is a PG19 plan. I don't want to make any changes for 18 unless
someone can see a fatal flaw; this is just my mid-beta check.

> If your goal is to remove this
> during-upgrade breakage after PG19, then I'd say that seems totally
> fine for a new feature.

That's the hope, yes.

Thanks!
--Jacob

  426. Re: libpq-oauth: a mid-beta naming check

    Christoph Berg <myon@debian.org> — 2025-08-05T15:44:15Z

    Re: Jacob Champion
> On Tue, Aug 5, 2025 at 2:39 AM Jelte Fennema-Nio <postgres@jeltef.nl> wrote:
> > On Tue, 5 Aug 2025 at 01:20, Jacob Champion
> > <jacob.champion@enterprisedb.com> wrote:
> > > So, as we approach Beta 3: can anyone think of a way that this plan will fail?
> >
> > It's not entirely clear what plan exactly you talk about here. Are you
> > saying you want to remove the -MAJOR suffix now for PG18? Or you want
> > to postpone doing that until PG19, when you would have designed a
> > stable API?
> 
> That is a PG19 plan. I don't want to make any changes for 18 unless
> someone can see a fatal flaw; this is just my mid-beta check.

FTR, fine with me.

> > If your goal is to remove this
> > during-upgrade breakage after PG19, then I'd say that seems totally
> > fine for a new feature.
> 
> That's the hope, yes.

Well, at the PG19 release time it will no longer be new. But the plan
sounds like a good one.

Christoph

  427. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-08-05T18:54:35Z

    On Fri, Jul 18, 2025 at 4:31 PM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Your earlier mail made me worried I'd missed something, but is the
> attached diff what Andres was asking for? A `ninja clean; ninja
> install-quiet` now works for me with this applied.

Ping. I'll plan to commit this by the beta3 cutoff but it'd be nice to
verify that I'm not missing something obvious. :D

--Jacob

  428. Re: libpq-oauth: a mid-beta naming check

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-08-07T20:52:53Z

    On Tue, Aug 5, 2025 at 8:44 AM Christoph Berg <myon@debian.org> wrote:
> Well, at the PG19 release time it will no longer be new. But the plan
> sounds like a good one.

Excellent. Thanks everybody!

--Jacob

  429. Re: [PoC] Federated Authn/z with OAUTHBEARER

    Jacob Champion <jacob.champion@enterprisedb.com> — 2025-08-08T16:23:30Z

    On Tue, Aug 5, 2025 at 11:54 AM Jacob Champion
<jacob.champion@enterprisedb.com> wrote:
> Ping. I'll plan to commit this by the beta3 cutoff but it'd be nice to
> verify that I'm not missing something obvious. :D

(Committed yesterday.)

I wonder if there's a Meson feature request in here somewhere, to be
able to compose targets from other builtin targets without having to
duplicate the internal bookkeeping.

--Jacob