Add jsonlog log_destination for JSON server logs
Sehrope Sarkuni <sehrope@jackdb.com>
From: Sehrope Sarkuni <sehrope@jackdb.com>
To: Pg Hackers <pgsql-hackers@postgresql.org>
Cc: Michael Paquier <michael@paquier.xyz>, david@fetter.org
Date: 2021-08-31T15:34:56Z
Lists: pgsql-hackers
Commits
Same data as JSON:
GET /api/v1/messages/:b64id/commits
the thread's linked commits as JSON, with link sources.
API reference →
-
Introduce log_destination=jsonlog
- dc686681e079 15.0 landed
-
Refactor set of routines specific to elog.c
- ac7c80758a7a 15.0 landed
-
Fix compilation warning in syslogger.c
- 05c4248ad1bf 15.0 landed
-
Refactor per-destination file rotation in logging collector
- 5c6e33f07153 15.0 landed
-
Refactor output file handling when forking syslogger under EXEC_BACKEND
- 5b0b699f748e 15.0 landed
-
Add regression tests for csvlog with the logging collector
- 72b76f76161c 15.0 landed
-
Refactor the syslogger pipe protocol to use a bitmask for its options
- 2d77d835403a 15.0 landed
Attachments
- v001-0001-Adds-separate-dest-field-to-log-protocol-PipeProtoHe.patch (text/x-patch) patch v1-0001
- v001-0002-Add-TAP-test-for-csvlog.patch (text/x-patch) patch v1-0002
- v001-0003-Add-jsonlog-log_destination-for-JSON-server-logs.patch (text/x-patch) patch v1-0003
Hi,
This patch adds a new log_destination, "jsonlog", that writes log entries
as lines of JSON. It was originally started by David Fetter using
the jsonlog module by Michael Paquier (
https://github.com/michaelpq/pg_plugins/blob/master/jsonlog/jsonlog.c) as a
basis for how to serialize the log messages. Thanks to both of them because
this wouldn't be possible without that starting point.
The first commit splits out the destination in log pipe messages into its
own field. Previously it would piggyback on the "is_last" field. This adds
an int to the message size but makes the rest of the code easier to follow.
The second commit adds a TAP test for log_destination "csvlog". This was
done to both confirm that the previous change didn't break anything and as
a skeleton for the test in the next commit.
The third commit adds the new log_destination "jsonlog". The output format
is one line per entry with the top level output being a JSON object keyed
with the log fields. Newlines in the output fields are escaped as \n so the
output file has exactly one line per log entry. It also includes a new test
for verifying the JSON output with some basic regex checks (similar to the
csvlog test).
Here's a sample of what the log entries look like:
{"timestamp":"2021-08-31 10:15:25.129
EDT","user":"sehrope","dbname":"postgres","pid":12012,"remote_host":"[local]","session_id":"612e397d.2eec","line_num":1,"ps":"idle","session_start":"2021-08-31
10:15:25
EDT","vxid":"3/2","txid":"0","error_severity":"LOG","application_name":"
006_jsonlog.pl","message":"statement: SELECT 1/0"}
It builds and passes "make check-world" on Linux. It also includes code to
handle Windows as well but I have not actually tried building it there.
Regards,
-- Sehrope Sarkuni
Founder & CEO | JackDB, Inc. | https://www.jackdb.com/