Re: Add jsonlog log_destination for JSON server logs
Sehrope Sarkuni <sehrope@jackdb.com>
From: Sehrope Sarkuni <sehrope@jackdb.com>
To: Michael Paquier <michael@paquier.xyz>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>, david@fetter.org
Date: 2021-09-16T21:27:20Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Introduce log_destination=jsonlog
- dc686681e079 15.0 landed
-
Refactor set of routines specific to elog.c
- ac7c80758a7a 15.0 landed
-
Fix compilation warning in syslogger.c
- 05c4248ad1bf 15.0 landed
-
Refactor per-destination file rotation in logging collector
- 5c6e33f07153 15.0 landed
-
Refactor output file handling when forking syslogger under EXEC_BACKEND
- 5b0b699f748e 15.0 landed
-
Add regression tests for csvlog with the logging collector
- 72b76f76161c 15.0 landed
-
Refactor the syslogger pipe protocol to use a bitmask for its options
- 2d77d835403a 15.0 landed
Attachments
- v4-0001-Add-constant-for-list-of-log-destinations-that-use-f.patch (text/x-patch) patch v4-0001
- v4-0002-Split-out-syslogger-EXEC_BACKEND-fd-serialization-an.patch (text/x-patch) patch v4-0002
- v4-0003-Refactor-syslogger-to-consolidate-common-tasks-for-f.patch (text/x-patch) patch v4-0003
Attached three patches refactor the syslogger handling of file based destinations a bit ... and then a lot. First patch adds a new constant to represent both file based destinations. This should make it easier to ensure additional destinations are handled in "For all file destinations..." situations (e.g. when we add the jsonlog destination). Second patch refactors the file descriptor serialization and re-opening in EXEC_BACKEND forking. Previously the code was duplicated for both stderr and csvlog. Again, this should simplify adding new destinations as they'd just invoke the same helper. There's an existing comment about not handling failed opens in syslogger_parseArgs(...) and this patch doesn't fix that, but it does provide a single location to do so in the future. The third patch adds a new internal (to syslogger.c) structure, FileLogDestination, for file based log destinations and modifies the existing handling for syslogFile and csvlogFile to defer to a bunch of helper functions. It also renames "syslogFile" to "stderr_file_info". Working through this patch, it was initially confusing that the stderr log file was named "syslogFile", the C file is named syslogger.c, and there's an entirely separate syslog (the message logging API) destination. I think this clears that up a bit. The patches pass check-world on Linux. I haven't tested it on Windows but it does pass check-world with EXEC_BACKEND defined to try out the forking code paths. Definitely needs some review to ensure it's functionally correct for the different log files. I haven't tried splitting the csvlog code out or adding the new jsonlog atop this yet. There's enough changes here that I'd like to get this settled first. Regards, -- Sehrope Sarkuni Founder & CEO | JackDB, Inc. | https://www.jackdb.com/