Horton

Re: making the backend's json parser work in frontend code

Tom Lane <tgl@sss.pgh.pa.us>

View thread View on postgr.es
From: Tom Lane <tgl@sss.pgh.pa.us>
To: Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Mark Dilger <mark.dilger@enterprisedb.com>, Alvaro Herrera <alvherre@2ndquadrant.com>, David Steele <david@pgmasters.net>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-24T16:27:44Z
Lists: pgsql-hackers 
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> writes:
> On 2020-01-23 18:04, Robert Haas wrote:
>> Now, you might say "well, why don't we just do an encoding
>> conversion?", but we can't. When the filesystem tells us what the file
>> names are, it does not tell us what encoding the person who created
>> those files had in mind. We don't know that they had*any*  encoding in
>> mind. IIUC, a file in the data directory can have a name that consists
>> of any sequence of bytes whatsoever, so long as it doesn't contain
>> prohibited characters like a path separator or \0 byte. But only some
>> of those possible octet sequences can be stored in a manifest that has
>> to be valid UTF-8.

> I think it wouldn't be unreasonable to require that file names in the 
> database directory be consistently encoded (as defined by pg_control, 
> probably).  After all, this information is sometimes also shown in 
> system views, so it's already difficult to process total junk.  In 
> practice, this shouldn't be an onerous requirement.

I don't entirely follow why we're discussing this at all, if the
requirement is backing up a PG data directory.  There are not, and
are never likely to be, any legitimate files with non-ASCII names
in that context.  Why can't we just skip any such files?

			regards, tom lane

Commits

  1. Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.

  2. Move jsonapi.c and jsonapi.h to src/common.

  3. Move some code from jsonapi.c to jsonfuncs.c.

  4. Adjust pg_parse_json() so that it does not directly ereport().

  5. Remove jsonapi.c's lex_accept().

  6. Split JSON lexer/parser from 'json' data type support.

  7. Rationalize code placement between wchar.c, encnames.c, and mbutils.c.

  8. Move wchar.c and encnames.c to src/common/.

  9. Update header comments for wchar.c and encnames.c.

  10. Make StringInfo available to frontend code.

  11. Use SASLprep to normalize passwords for SCRAM authentication.