Horton

Re: making the backend's json parser work in frontend code

David Steele <david@pgmasters.net>

View thread View on postgr.es
From: David Steele <david@pgmasters.net>
To: Andres Freund <andres@anarazel.de>, Robert Haas <robertmhaas@gmail.com>
Cc: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-16T18:58:55Z
Lists: pgsql-hackers 
On 1/15/20 4:40 PM, Andres Freund wrote:
 >
 > I'm not sure where I come down between using json and a simple ad-hoc
 > format, when the dependency for the former is making the existing json
 > parser work in the frontend. But if the alternative is to add a second
 > json parser, it very clearly shifts towards using an ad-hoc
 > format. Having to maintain a simple ad-hoc parser is a lot less
 > technical debt than having a second full blown json parser.

Maybe at first, but it will grow and become more complex as new features 
are added.  This has been our experience with pgBackRest, at least.

 > Imo even
 > when an external project or three also has to have that simple parser.

I don't agree here.  Especially if we outgrow the format and they need 
two parsers, depending on the version of PostgreSQL.

To do page-level incrementals (which this feature is intended to enable) 
the user will need to be able to associate full and incremental backups 
and the only way I see to do that (currently) is to read the manifests, 
since the prior backup should be stored there.  I think this means that 
parsing the manifest is not really optional -- it will be required to do 
any kind of automation with incrementals.

It's easy enough for a tool like pgBackRest to do something like that, 
much harder for a user hacking together a tool in bash based on 
pg_basebackup.

 > If the alternative were to use that newly proposed json parser to
 > *replace* the backend one too, the story would again be different.

That was certainly not my intention.

Regards,
-- 
-David
david@pgmasters.net

Commits

  1. Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.

  2. Move jsonapi.c and jsonapi.h to src/common.

  3. Move some code from jsonapi.c to jsonfuncs.c.

  4. Adjust pg_parse_json() so that it does not directly ereport().

  5. Remove jsonapi.c's lex_accept().

  6. Split JSON lexer/parser from 'json' data type support.

  7. Rationalize code placement between wchar.c, encnames.c, and mbutils.c.

  8. Move wchar.c and encnames.c to src/common/.

  9. Update header comments for wchar.c and encnames.c.

  10. Make StringInfo available to frontend code.

  11. Use SASLprep to normalize passwords for SCRAM authentication.