Horton

Re: Add jsonlog log_destination for JSON server logs

Michael Paquier <michael@paquier.xyz>

View thread
From: Michael Paquier <michael@paquier.xyz>
To: Sehrope Sarkuni <sehrope@jackdb.com>
Cc: Pg Hackers <pgsql-hackers@postgresql.org>, david@fetter.org
Date: 2021-09-08T06:58:57Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Introduce log_destination=jsonlog

  2. Refactor set of routines specific to elog.c

  3. Fix compilation warning in syslogger.c

  4. Refactor per-destination file rotation in logging collector

  5. Refactor output file handling when forking syslogger under EXEC_BACKEND

  6. Add regression tests for csvlog with the logging collector

  7. Refactor the syslogger pipe protocol to use a bitmask for its options

On Wed, Sep 01, 2021 at 04:39:43PM -0400, Sehrope Sarkuni wrote:
> That makes the elog.c changes in the JSON logging patch minimal as all it's
> really doing is invoking the new write_jsonlog(...) function.

Looking at 0001, to do things in order.

> @@ -46,8 +46,8 @@ typedef struct
>  	char		nuls[2];		/* always \0\0 */
>  	uint16		len;			/* size of this chunk (counts data only) */
>  	int32		pid;			/* writer's pid */
> -	char		is_last;		/* last chunk of message? 't' or 'f' ('T' or
> -								 * 'F' for CSV case) */
> +	int32		dest;			/* log destination */
> +	char		is_last;        /* last chunk of message? 't' or 'f'*/
>  	char		data[FLEXIBLE_ARRAY_MEMBER];	/* data payload starts here */
>  } PipeProtoHeader;

Making PipeProtoHeader larger is not free, and that could penalize
workloads with a lot of short messages and many backends as the
syslogger relies on pipes with sync calls.  Why not switching is_last
to bits8 flags instead?  That should be enough for the addition of
JSON.  3 bits are enough at the end: one to know if it is the last
chunk of message, one for CSV and one for JSON.
--
Michael