Thread

1. pg_threads.h take II

   Thomas Munro <thomas.munro@gmail.com> — 2026-07-06T12:09:00Z

   Hi,
   
   Here's another go at port/pg_threads.h, a C11 <threads.h>-like
   interface, with some patches to use it (some just sketch-quality).
   Previous work:
   
   https://www.postgresql.org/message-id/flat/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
   
   We still can't require C11 <threads.h> yet, though the interface has a
   clear future upgrade/remap path to do that eventually.  Since I last
   wrote about this topic, we gained the ability to count on C11
   thread_local being available (it's a language/compiler feature and
   doesn't need a <threads.h> header or library), and partly because of
   that, but also because I don't think anyone really likes it, I ripped
   out that tss_t stuff.  I came up with a narrower API to get
   thread-exit cleanup callbacks if you need them for your thread_local
   resources, same pthread_key_t/FlsAlloc stuff underneath it but without
   having to get lost in the weeds of <threads.h> conformance.
   
   v1-0001-ecpg-Fix-auto_mem-cleanup-on-thread-exit.patch
   
   Already discussed in another thread, a couple of existing ECPG
   thread-exit cleanup bugs need to be fixed (bugs that cancel each other
   out in master, but leak, and with my patches the leak was fixed in
   passing but then a double-free would crash):
   
   https://www.postgresql.org/message-id/flat/CA%2BhUKGLv202ndx%3DkP1fZJapNcbfnpq-rezHNyt%2BcQ4pdxr1akg%40mail.gmail.com
   
   v1-0002-port-Provide-minimal-pg_threads.h-API.patch
   
   The main thing I'm wondering about is how to make the error reporting
   a bit nicer.  Neither pthreads nor C11 threads set errno, but it's
   nice if you can use %m.  Hmm.
   
   v1-0003-port-Use-pg_threads.h-API-for-pg_localconv_r.patch
   
   Trivial.
   
   v1-0004-ecpg-Improve-variable-name.patch
   v1-0005-ecpg-Use-pg_threads.h.patch
   
   These demonstrate the replacement of pthread_key_t with thread_local,
   and where needed, pg_thrd_atexit_t.  That's the narrower thing I came
   up with to replace pthread_t/tss_t with something that more directly
   models what we want, I think.
   
   I could do pg_tss_t, but it takes a lot of extra book-keeping code to
   make Windows FlsAlloc match the spec.
   
   v1-0006-pgbench-Use-pg_threads.h.patch
   
   Kills another local thread-porting abstraction.
   
   v1-0007-libpq-Use-pg_threads.h.patch
   
   Kills another local thread-porting abstraction.
   
   v1-0008-pg_basebackup-Use-pg_threads.h.patch
   
   Kills the fork() mode used for Unix, harmonises Unix and Windows.
   
   v1-0009-pg_dump-Remove-TerminateThread-call.patch
   
   This one already has a discussion thread (I figured it might be
   another back-patchable issue, IDK), and it looks like the solution I
   used here isn't what we want to go with for the future, but to do it
   the way Heikki and Jelte are suggesting will probably require deciding
   how to access atomics or at least compiler barrires in front-end code,
   which I'll look into this week.
   
   Consider this approach a stand-in for now, as I needed to get rid of a
   TerminateThread() call somehow.  We can't have that in pg_threads.h
   for various reasons (it's all kinds of undefined).
   
   https://www.postgresql.org/message-id/flat/CA%2BhUKGJgO%3Do-vLFahGdR2WesuX3h1-0j%3Da8z72fChc-MG1Hveg%40mail.gmail.com
   
   v1-0010-pg_dump-Block-signals-during-signal-handler.patch
   
   Not very interesting, I was just triggered by steampunk Unix
   anachronisms in passing...  an experimental patch further down
   replaces all of this anyway.
   
   v1-0011-pg_dump-Use-pg_threads.h-and-remove-fork-path.patch
   
   Kills the fork() mode used for Unix, harmonises Unix and Windows.
   After this patch, the cancellation stuff admittedly looks pretty
   shaky, but I wanted to separate the fork()-ectomy from any larger
   refactoring, so bear with me...
   
   Next I was planning to rip out the pipes that fork()-mode needed and
   use an in-memory work queue, and probably make a reusable pg_thrd_pool
   instead of open coding it here, but it turned out that Bryan was
   already way ahead of me there:
   
   https://www.postgresql.org/message-id/flat/8c712d76-ecf7-4749-a6d8-dddc01f298ec@gmail.com
   
   Luckily it looks like he started working at that end of the problem
   and I started with this fork()-ectomy/harmonisation piece, so I
   think/hope that we can fit those efforts together quite easily... I
   will look at his patches this week.
   
   v1-0012-pg_signal_processor-Infrastructure-for-cleanup.patch
   v1-0013-fe_utils-Provide-cancel_set-for-fast-quit-paths.patch
   v1-0014-pg_dump-Refactor-connection-cancel-management.patch
   
   I considered lots of different ways to make pg_dump cleanup better,
   and this is the approach that *seemed* to be roughly along the lines
   that Jelte suggested he wanted for various reasons, over in the
   TerminateThread() thread.  Namely, make it so that Unix and Windows
   both run your cleanup code on a thread, but also force Windows to
   serialise the handlers, so that the semantics match and we can escape
   from the constraints of async-signal-safe signal handlers.  On Unix it
   has a thread sitting there waiting for you to hit ^C, just looping
   over sigwait() and calling handlers serially, that are then free to
   use arbitrarily complex code before _exit() turns out the lights.
   
   Warning/disclaimer: 0012-0014 are only experimental sketches trying
   stuff out, and might not work that well in various ways, I just wanted
   to get your take on this general direction.  Could skin this in lots
   of other ways too and since you guys are actively hacking on this area
   you might already have another idea in mind or even patches?
   
   After these patches, there are a few places left that use raw Windows
   threading APIs, but they're places that are creating threads to solve
   signaling and I/O problems that have better solutions, topics for
   another email (but briefly I mean: the interrupts/latches refactoring
   patch can kill "our" signal-emulation threads, and support for
   overlapped I/O can allow multiplexing pipe I/O with latches without
   intermediate data-pumping threads (cf Run::IPC which also has that
   kind of Windows-hates-select() problem), and support for process-exit
   WaitEvents can kill another helper thread, ... yada yada).
   

2. Re: pg_threads.h take II

   Peter Eisentraut <peter@eisentraut.org> — 2026-07-06T12:31:57Z

   On 06.07.26 14:09, Thomas Munro wrote:
   > We still can't require C11 <threads.h> yet, though the interface has a
   > clear future upgrade/remap path to do that eventually.  Since I last
   > wrote about this topic, we gained the ability to count on C11
   > thread_local being available
   
   We still support platforms/compilers where it's not available, but
   
   https://www.postgresql.org/message-id/2a965ac6-fa42-4054-bee0-b1618e7729d6%40eisentraut.org
   
   which is ready to be committed, will de-support those, so by the time 
   this patch set is ready, we'll be fine.
   
   
   
   
   

3. Re: pg_threads.h take II

   Thomas Munro <thomas.munro@gmail.com> — 2026-07-06T12:32:12Z

   On Tue, Jul 7, 2026 at 12:09 AM Thomas Munro <thomas.munro@gmail.com> wrote:
   > used here isn't what we want to go with for the future, but to do it
   > the way Heikki and Jelte are suggesting will probably require deciding
   > how to access atomics or at least compiler barrires in front-end code,
   > which I'll look into this week.
   
   Correction: at least memory barriers
   
   
   
   

4. Re: pg_threads.h take II

   Heikki Linnakangas <hlinnaka@iki.fi> — 2026-07-06T20:48:16Z

   On 06/07/2026 15:09, Thomas Munro wrote:
   > Hi,
   > 
   > Here's another go at port/pg_threads.h, a C11 <threads.h>-like
   > interface, with some patches to use it (some just sketch-quality).
   > Previous work:
   > 
   > https://www.postgresql.org/message-id/flat/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
   > 
   > We still can't require C11 <threads.h> yet, though the interface has a
   > clear future upgrade/remap path to do that eventually.  Since I last
   > wrote about this topic, we gained the ability to count on C11
   > thread_local being available (it's a language/compiler feature and
   > doesn't need a <threads.h> header or library), and partly because of
   > that, but also because I don't think anyone really likes it, I ripped
   > out that tss_t stuff.  I came up with a narrower API to get
   > thread-exit cleanup callbacks if you need them for your thread_local
   > resources, same pthread_key_t/FlsAlloc stuff underneath it but without
   > having to get lost in the weeds of <threads.h> conformance.
   
   Nice!
   
   I've now reviewed patches 0001-0007. I think they're ready to be 
   committed, with some small comments below:
   
   > v1-0002-port-Provide-minimal-pg_threads.h-API.patch
   > 
   > The main thing I'm wondering about is how to make the error reporting
   > a bit nicer.  Neither pthreads nor C11 threads set errno, but it's
   > nice if you can use %m.  Hmm.
   
   Perhaps provide a function like:
   
   void
   set_errno_from_pg_thrd_error(int error)
   {
        switch((pg_thrd_error_t) error)
        {
            case pg_thrd_nomem:
                errno = ENOMEM;
                break;
            case pg_thrd_busy:
                errno = EBUSY;
                break;
           ...
        }
   }
   
   > +/* Convert native error to pg_thrd_error_t. */
   > +static inline int
   > +pg_thrd_maperror(int error)
   > +{
   > +#ifdef WIN32
   > +	return error ? pg_thrd_success : pg_thrd_error;
   > +#else
   > +	return error == 0 ? pg_thrd_success : pg_thrd_error;
   > +#endif
   > +}
   
   The WIN32 version of this is confusing. There is only one caller of this 
   with WIN32:
   
   > +#elif defined(WIN32)
   > +	return pg_thrd_maperror(InitializeSynchronizationBarrier(barrier, count, 0));
   > +#else
   
   So that's not wrong. But I'd suggest only defining pg_thrd_maperror() 
   with pthreads, and handling that one WIN32 caller directly without the 
   helper function. Maybe rename pg_thrd_maperror() to something like 
   map_pthread_retval_to_pg_thrd_error() or something.
   
   Do the pthread functions return an errno on error? Currently, all errors 
   get squashed to pg_thrd_error, which is a little unfortunate.
   
   > +/* Like C11 mtx_type_t. */
   > +typedef enum pg_mtx_type_t
   > +{
   > +	pg_mtx_plain = 0
   > +} pg_mtx_type_t;
   > +
   > +/* Like C11 mtx_init(). */
   > +static inline int
   > +pg_mtx_init(pg_mtx_t *mutex, int type)
   > +{
   > +#ifdef WIN32
   > +	return pg_rwlock_init(mutex);
   > +#else
   > +	return pg_thrd_maperror(pthread_mutex_init(mutex, NULL));
   > +#endif
   > +}
   
   Since we only support plain mutexes, how about "Assert(type == 
   pg_mtx_plan)" here?
   
   
   > +/*-------------------------------------------------------------------------
   > + *
   > + * Barriers.  Not in C11.
   > + *
   > + *-------------------------------------------------------------------------
   > + */
   
   I wonder if we really need barriers. They're not that useful IMHO. I'd 
   tend to just open-code this directly with a mutex and the condition 
   variable in most cases. (Not a strong objection, there's little harm in 
   having it either)
   
   > +/*
   > + * Wait for all expected threads to arrive at the barrier, and elect one
   > + * arbitrary thread to perform a phase of computation serially.  Sets
   > + * *elected_thread to true in the elected thread, and false in all others.
   > + */
   > +static inline int
   > +pg_barrier_wait_and_elect(pg_barrier_t *barrier, bool *elected_thread)
   > +{
   
   No callers use the *elected_thread return value. What was the idea here?
   
   > +#ifdef WIN32
   > +
   > +	/*
   > +	 * Retrieve handle passed here by pg_thrd_create() before allowing this
   > +	 * thread to run.  (pg_thrd_current() can't use CurrentThread(), because
   > +	 * that returns a pseudo-handle with the same value in all threads.)
   > +	 */
   > +	Assert(start_info->self);
   > +	my_thrd_handle = start_info->self;
   > +#endif
   
   Does that refer to the GetCurrentThreadId() function? We use that in a 
   few places currently.
   
   How can it return the same value in all threads, isn't that completely 
   useless? And does that mean all our current uses of it are broken?
   
   - Heikki
   
   
   
   
   

5. Re: pg_threads.h take II

   Bryan Green <dbryan.green@gmail.com> — 2026-07-06T21:01:10Z

   On 7/6/26 3:48 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
   > On 06/07/2026 15:09, Thomas Munro wrote:
   > > Hi,
   > >
   > > Here's another go at port/pg_threads.h, a C11 <threads.h>-like
   > > interface, with some patches to use it (some just sketch-quality).
   > > Previous work:
   > >
   > > https://www.postgresql.org/message-id/flat/
   > > CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
   > >
   > > We still can't require C11 <threads.h> yet, though the interface has a
   > > clear future upgrade/remap path to do that eventually.  Since I last
   > > wrote about this topic, we gained the ability to count on C11
   > > thread_local being available (it's a language/compiler feature and
   > > doesn't need a <threads.h> header or library), and partly because of
   > > that, but also because I don't think anyone really likes it, I ripped
   > > out that tss_t stuff.  I came up with a narrower API to get
   > > thread-exit cleanup callbacks if you need them for your thread_local
   > > resources, same pthread_key_t/FlsAlloc stuff underneath it but without
   > > having to get lost in the weeds of <threads.h> conformance.
   > 
   > Nice!
   > 
   > I've now reviewed patches 0001-0007. I think they're ready to be
   > committed, with some small comments below:
   > 
   > > v1-0002-port-Provide-minimal-pg_threads.h-API.patch
   > >
   > > The main thing I'm wondering about is how to make the error reporting
   > > a bit nicer.  Neither pthreads nor C11 threads set errno, but it's
   > > nice if you can use %m.  Hmm.
   > 
   > Perhaps provide a function like:
   > 
   > void
   > set_errno_from_pg_thrd_error(int error)
   > {
   >     switch((pg_thrd_error_t) error)
   >     {
   >         case pg_thrd_nomem:
   >             errno = ENOMEM;
   >             break;
   >         case pg_thrd_busy:
   >             errno = EBUSY;
   >             break;
   >        ...
   >     }
   > }
   > 
   > > +/* Convert native error to pg_thrd_error_t. */
   > > +static inline int
   > > +pg_thrd_maperror(int error)
   > > +{
   > > +#ifdef WIN32
   > > +    return error ? pg_thrd_success : pg_thrd_error;
   > > +#else
   > > +    return error == 0 ? pg_thrd_success : pg_thrd_error;
   > > +#endif
   > > +}
   > 
   > The WIN32 version of this is confusing. There is only one caller of this
   > with WIN32:
   > 
   > > +#elif defined(WIN32)
   > > +    return pg_thrd_maperror(InitializeSynchronizationBarrier(barrier,
   > > count, 0));
   > > +#else
   > 
   > So that's not wrong. But I'd suggest only defining pg_thrd_maperror()
   > with pthreads, and handling that one WIN32 caller directly without the
   > helper function. Maybe rename pg_thrd_maperror() to something like
   > map_pthread_retval_to_pg_thrd_error() or something.
   > 
   > Do the pthread functions return an errno on error? Currently, all errors
   > get squashed to pg_thrd_error, which is a little unfortunate.
   > 
   > > +/* Like C11 mtx_type_t. */
   > > +typedef enum pg_mtx_type_t
   > > +{
   > > +    pg_mtx_plain = 0
   > > +} pg_mtx_type_t;
   > > +
   > > +/* Like C11 mtx_init(). */
   > > +static inline int
   > > +pg_mtx_init(pg_mtx_t *mutex, int type)
   > > +{
   > > +#ifdef WIN32
   > > +    return pg_rwlock_init(mutex);
   > > +#else
   > > +    return pg_thrd_maperror(pthread_mutex_init(mutex, NULL));
   > > +#endif
   > > +}
   > 
   > Since we only support plain mutexes, how about "Assert(type ==
   > pg_mtx_plan)" here?
   > 
   > 
   > > +/
   > > *-------------------------------------------------------------------------
   > > + *
   > > + * Barriers.  Not in C11.
   > > + *
   > > +
   > > *-------------------------------------------------------------------------
   > > + */
   > 
   > I wonder if we really need barriers. They're not that useful IMHO. I'd
   > tend to just open-code this directly with a mutex and the condition
   > variable in most cases. (Not a strong objection, there's little harm in
   > having it either)
   > 
   > > +/*
   > > + * Wait for all expected threads to arrive at the barrier, and elect one
   > > + * arbitrary thread to perform a phase of computation serially.  Sets
   > > + * *elected_thread to true in the elected thread, and false in all
   > > others.
   > > + */
   > > +static inline int
   > > +pg_barrier_wait_and_elect(pg_barrier_t *barrier, bool *elected_thread)
   > > +{
   > 
   > No callers use the *elected_thread return value. What was the idea here?
   > 
   > > +#ifdef WIN32
   > > +
   > > +    /*
   > > +     * Retrieve handle passed here by pg_thrd_create() before
   > > allowing this
   > > +     * thread to run.  (pg_thrd_current() can't use CurrentThread(),
   > > because
   > > +     * that returns a pseudo-handle with the same value in all threads.)
   > > +     */
   > > +    Assert(start_info->self);
   > > +    my_thrd_handle = start_info->self;
   > > +#endif
   > 
   > Does that refer to the GetCurrentThreadId() function? We use that in a
   > few places currently.
   > 
   > How can it return the same value in all threads, isn't that completely
   > useless? And does that mean all our current uses of it are broken?
   > 
   > - Heikki
   > 
   > 
   GetCurrentThread() not GetCurrentThreadId()...  GetCurrentThread() 
   returns a special sentinel handle (-2) that when passed to another 
   Win32 function causes the kernel to resolve to whatever thread made
   the call....
   
   So, no-- not broken.
   
   
   
   

6. Re: pg_threads.h take II

   Heikki Linnakangas <hlinnaka@iki.fi> — 2026-07-06T21:13:37Z

   On 07/07/2026 00:01, dbryan.green@gmail.com wrote:
   > On 7/6/26 3:48 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
   >> On 06/07/2026 15:09, Thomas Munro wrote:
   >>> +#ifdef WIN32
   >>> +
   >>> +    /*
   >>> +     * Retrieve handle passed here by pg_thrd_create() before
   >>> allowing this
   >>> +     * thread to run.  (pg_thrd_current() can't use CurrentThread(),
   >>> because
   >>> +     * that returns a pseudo-handle with the same value in all threads.)
   >>> +     */
   >>> +    Assert(start_info->self);
   >>> +    my_thrd_handle = start_info->self;
   >>> +#endif
   >>
   >> Does that refer to the GetCurrentThreadId() function? We use that in a
   >> few places currently.
   >>
   >> How can it return the same value in all threads, isn't that completely
   >> useless? And does that mean all our current uses of it are broken?
   >
   > GetCurrentThread() not GetCurrentThreadId()...  GetCurrentThread()
   > returns a special sentinel handle (-2) that when passed to another
   > Win32 function causes the kernel to resolve to whatever thread made
   > the call....
   > 
   > So, no-- not broken.
   
   Ah gotcha. Could we use GetCurrentThreadId() here then? (I have no 
   problem with the way it's currently done in the patch either though, 
   just curious)
   
   - Heikki
   
   
   
   
   

7. Re: pg_threads.h take II

   Bryan Green <dbryan.green@gmail.com> — 2026-07-06T21:28:14Z

   On 7/6/2026 4:13 PM, Heikki Linnakangas wrote:
   > On 07/07/2026 00:01, dbryan.green@gmail.com wrote:
   >> On 7/6/26 3:48 PM, Heikki Linnakangas <hlinnaka@iki.fi> wrote:
   >>> On 06/07/2026 15:09, Thomas Munro wrote:
   >>>> +#ifdef WIN32
   >>>> +
   >>>> +    /*
   >>>> +     * Retrieve handle passed here by pg_thrd_create() before
   >>>> allowing this
   >>>> +     * thread to run.  (pg_thrd_current() can't use CurrentThread(),
   >>>> because
   >>>> +     * that returns a pseudo-handle with the same value in all
   >>>> threads.)
   >>>> +     */
   >>>> +    Assert(start_info->self);
   >>>> +    my_thrd_handle = start_info->self;
   >>>> +#endif
   >>>
   >>> Does that refer to the GetCurrentThreadId() function? We use that in a
   >>> few places currently.
   >>>
   >>> How can it return the same value in all threads, isn't that completely
   >>> useless? And does that mean all our current uses of it are broken?
   >>
   >> GetCurrentThread() not GetCurrentThreadId()...  GetCurrentThread()
   >> returns a special sentinel handle (-2) that when passed to another
   >> Win32 function causes the kernel to resolve to whatever thread made
   >> the call....
   >>
   >> So, no-- not broken.
   > 
   > Ah gotcha. Could we use GetCurrentThreadId() here then? (I have no
   > problem with the way it's currently done in the patch either though,
   > just curious)
   > 
   > - Heikki
   > 
   I haven't looked at all of the code, but I would assume there is a
   WaitFor...Object() somewhere and those only work with actual
   handles...not id's.  GetCurrentThreadId() returns a dword for id.  I'll
   look over the patches as well.
   
   -- 
   Bryan Green
   EDB: https://www.enterprisedb.com