Horton

Refactoring backend fork+exec code

Heikki Linnakangas <hlinnaka@iki.fi>

View thread
From: Heikki Linnakangas <hlinnaka@iki.fi>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Date: 2023-06-18T11:22:33Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Avoid 0-length memcpy to NULL with EXEC_BACKEND

  2. Tighten check for --forkchild argument when spawning child process

  3. Add missing PGDLLIMPORT markings

  4. Remove assertions that some compiler say are tautological

  5. Move code for backend startup to separate file

  6. Refactor postmaster child process launching

  7. Move some functions from postmaster.c to a new source file

  8. Split registration of Win32 deadchild callback to separate function

  9. Improve log messages referring to background worker processes

  10. Move initialization of the Port struct to the child process

  11. Pass CAC as an argument to the backend process

  12. Set socket options in child process after forking

  13. Fix references to renamed function in comments

  14. Use MyBackendType in more places to check what process this is

  15. Remove MyAuxProcType, use MyBackendType instead

  16. Replace BackendIds with 0-based ProcNumbers

  17. Redefine backend ID to be an index into the proc array

  18. Remove superfluous 'pgprocno' field from PGPROC

  19. Fix incorrect comment on how BackendStatusArray is indexed

  20. Refactor how InitProcess is called

  21. Pass BackgroundWorker entry in the parameter file in EXEC_BACKEND mode

  22. Refactor CreateSharedMemoryAndSemaphores

  23. Silence Valgrind complaint with EXEC_BACKEND

  24. Adjust obsolete comment explaining set_stack_base().

  25. Print lwlock stats also for aux processes, when built with LWLOCK_STATS

  26. Move canAcceptConnections check from ProcessStartupPacket to caller.

  27. Fix crash on syslogger startup

  28. Refactor ListenSocket array.

  29. Use FD_CLOEXEC on ListenSockets

  30. Don't leak descriptors into subprograms.

  31. Use WaitEventSet API for postmaster's event loop.

  32. Rearrange postmaster's startup sequence for better syslogger results.

  33. Fix and enhance the assertion of no palloc's in a critical section.

Attachments

I started to look at the code in postmaster.c related to launching child 
processes. I tried to reduce the difference between EXEC_BACKEND and 
!EXEC_BACKEND code paths, and put the code that needs to differ behind a 
better abstraction. I started doing this to help with implementing 
multi-threading, but it doesn't introduce anything thread-related yet 
and I think this improves readability anyway.

This is still work-inprogress, especially the last, big, patch in the 
patch set. Mainly, I need to clean up the comments in the new 
launch_backend.c file. But the other patches are in pretty good shape, 
and if you ignore launch_backend.c, you can see the effect on the other 
source files.

With these patches, there is a new function for launching a postmaster 
child process:

pid_t postmaster_child_launch(PostmasterChildType child_type, char 
*startup_data, size_t startup_data_len, ClientSocket *client_sock);

This function hides the differences between EXEC_BACKEND and 
!EXEC_BACKEND cases.

In 'startup_data', the caller can pass a blob of data to the child 
process, with different meaning for different kinds of child processes. 
For a backend process, for example, it's used to pass the CAC_state, 
which indicates whether the backend accepts the connection or just sends 
"too many clients" error. And for background workers, it's used to pass 
the BackgroundWorker struct. The startup data is passed to the child 
process in the

ClientSocket is a new struct holds a socket FD, and the local and remote 
address info. Before this patch set, postmaster initializes the Port 
structs but only fills in those fields in it. With this patch set, we 
have a new ClientSocket struct just for those fields, which makes it 
more clear which fields are initialized where.

I haven't done much testing yet, and no testing at all on Windows, so 
that's probably still broken.

-- 
Heikki Linnakangas
Neon (https://neon.tech)