making the backend's json parser work in frontend code
Robert Haas <robertmhaas@gmail.com>
From: Robert Haas <robertmhaas@gmail.com>
To: "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-15T21:02:49Z
Lists: pgsql-hackers
Attachments
- 0001-Move-wchar.c-to-src-common.patch (application/octet-stream) patch 0001
- 0002-Adjust-src-include-utils-jsonapi.h-so-it-s-not-backe.patch (application/octet-stream) patch 0002
- 0003-Split-JSON-lexer-parser-from-json-data-type-support.patch (application/octet-stream) patch 0003
- 0004-Introduce-json_error-macro.patch (application/octet-stream) patch 0004
- 0005-Frontendify-jsonapi.c.patch (application/octet-stream) patch 0005
- 0006-Move-jsonapi.c-to-src-common.patch (application/octet-stream) patch 0006
The discussion on the backup manifest thread has gotten bogged down on the issue of the format that should be used to store the backup manifest file. I want something simple and ad-hoc; David Steele and Stephen Frost prefer JSON. That is problematic because our JSON parser does not work in frontend code, and I want to be able to validate a backup against its manifest, which involves being able to parse the manifest from frontend code. The latest development over there is that David Steele has posted the JSON parser that he wrote for pgbackrest with an offer to try to adapt it for use in front-end PostgreSQL code, an offer which I genuinely appreciate. I'll write more about that over on that thread. However, I decided to spend today doing some further investigation of an alternative approach, namely making the backend's existing JSON parser work in frontend code as well. I did not solve all the problems there, but I did come up with some patches which I think would be worth committing on independent grounds, and I think the whole series is worth posting. So here goes. 0001 moves wchar.c from src/backend/utils/mb to src/common. Unless I'm missing something, this seems like an overdue cleanup. It's long been the case that wchar.c is actually compiled and linked into both frontend and backend code. Commit 60f11b87a2349985230c08616fa8a34ffde934c8 added code into src/common that depends on wchar.c being available, but didn't actually make wchar.c part of src/common, which seems like an odd decision: the functions in the library are dependent on code that is not part of any library but whose source files get copied around where needed. Eh? 0002 does some basic header cleanup to make it possible to include the existing header file jsonapi.h in frontend code. The state of the JSON headers today looks generally poor. There seems not to have been much attempt to get the prototypes for a given source file, say foo.c, into a header file with the same name, say foo.h. Also, dependencies between various header files seem to be have added somewhat freely. This patch does not come close to fixing all that, but I consider it a modest down payment on a cleanup that probably ought to be taken further. 0003 splits json.c into two files, json.c and jsonapi.c. All the lexing and parsing stuff (whose prototypes are in jsonapi.h) goes into jsonapi.c, while the stuff that pertains to the 'json' data type remains in json.c. This also seems like a good cleanup, because to me, at least, it's not a great idea to mix together code that is used by both the json and jsonb data types as well as other things in the system that want to generate or parse json together with things that are specific to the 'json' data type. As far as I know all three of the above patches are committable as-is; review and contrary opinions welcome. On the other hand, 0004, 0005, and 0006 are charitably described as experimental or WIP. 0004 and 0005 hack up jsonapi.c so that it can still be compiled even if #include "postgres.h" is changed to #include "postgres-fe.h" and 0006 moves it into src/common. Note that I say that they make it compile, not work. It's not just untested; it's definitely broken. But it gives a feeling for what the remaining obstacles to making this code available in a frontend environment are. Since I wrote my very first email complaining about the difficulty of making the backend's JSON parser work in a frontend environment, one obstacle has been knocked down: StringInfo is now available in front-end code (commit 26aaf97b683d6258c098859e6b1268e1f5da242f). The remaining problems (that I know about) have to do with error reporting and multibyte character support; a read of the patches is suggested for those wanting further details. Suggestions welcome. Thanks, -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company
Commits
-
Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.
- 006b9dcad419 13.0 landed
-
Move jsonapi.c and jsonapi.h to src/common.
- beb4699091e9 13.0 landed
-
Move some code from jsonapi.c to jsonfuncs.c.
- 73ce2a03f30b 13.0 landed
-
Adjust pg_parse_json() so that it does not directly ereport().
- 1f3a021730be 13.0 landed
-
Remove jsonapi.c's lex_accept().
- 530609aa4263 13.0 landed
-
Split JSON lexer/parser from 'json' data type support.
- 11b5e3e35d39 13.0 landed
-
Rationalize code placement between wchar.c, encnames.c, and mbutils.c.
- 5afaa2e42655 13.0 landed
-
Move wchar.c and encnames.c to src/common/.
- e6afa8918c46 13.0 landed
-
Update header comments for wchar.c and encnames.c.
- 3d4cb5d6c180 13.0 landed
-
Make StringInfo available to frontend code.
- 26aaf97b683d 13.0 cited
-
Use SASLprep to normalize passwords for SCRAM authentication.
- 60f11b87a234 10.0 cited