Horton

Re: making the backend's json parser work in frontend code

David Steele <david@pgmasters.net>

View thread View on postgr.es
From: David Steele <david@pgmasters.net>
To: Robert Haas <robertmhaas@gmail.com>, Bruce Momjian <bruce@momjian.us>
Cc: Alvaro Herrera <alvherre@2ndquadrant.com>, Mark Dilger <mark.dilger@enterprisedb.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-24T16:29:48Z
Lists: pgsql-hackers 
On 1/23/20 11:05 AM, Robert Haas wrote:
 > On Thu, Jan 23, 2020 at 12:49 PM Bruce Momjian <bruce@momjian.us> wrote:
 >> Another idea is to use base64 for all non-ASCII file names, so we don't
 >> need to check if the file name is valid UTF8 before outputting --- we
 >> just need to check for non-ASCII, which is much easier.
 >
 > I think that we have the infrastructure available to check in a
 > convenient way whether it's valid as UTF-8, so this might not be
 > necessary, but I will look into it further unless there is a consensus
 > to go another direction entirely.
 >
 >> Another
 >> problem, though, is how do you _flag_ file names as being
 >> base64-encoded?  Use another JSON field to specify that?
 >
 > Alvaro's proposed solution in the message to which you replied was to
 > call the field either 'path' or 'path_base64' depending on whether
 > base-64 escaping was used. That seems better to me than having a field
 > called 'path' and a separate field called 'is_path_base64' or
 > whatever.

+1. I'm not excited about this solution but don't have a better idea.

It might be nice to have a strict mode where non-ASCII/UTF8 characters 
will error instead, but that can be added on later.

Regards,
-- 
-David
david@pgmasters.net

Commits

  1. Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.

  2. Move jsonapi.c and jsonapi.h to src/common.

  3. Move some code from jsonapi.c to jsonfuncs.c.

  4. Adjust pg_parse_json() so that it does not directly ereport().

  5. Remove jsonapi.c's lex_accept().

  6. Split JSON lexer/parser from 'json' data type support.

  7. Rationalize code placement between wchar.c, encnames.c, and mbutils.c.

  8. Move wchar.c and encnames.c to src/common/.

  9. Update header comments for wchar.c and encnames.c.

  10. Make StringInfo available to frontend code.

  11. Use SASLprep to normalize passwords for SCRAM authentication.