Horton

Re: making the backend's json parser work in frontend code

Alvaro Herrera <alvherre@2ndquadrant.com>

View thread View on postgr.es
From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Tom Lane <tgl@sss.pgh.pa.us>
Cc: Mark Dilger <mark.dilger@enterprisedb.com>, David Steele <david@pgmasters.net>, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>, Robert Haas <robertmhaas@gmail.com>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>
Date: 2020-01-24T18:03:12Z
Lists: pgsql-hackers 
On 2020-Jan-24, Tom Lane wrote:

> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > On 2020-Jan-24, Mark Dilger wrote:
> >> I would expect, therefore, that we only back up files which match our
> >> expected file name pattern and ignore (perhaps with a warning)
> >> everything else.
> 
> > That risks missing files placed in the datadir by extensions;
> 
> I agree that assuming we know everything that will appear in the
> data directory is a pretty unsafe assumption.  But no rational
> extension is going to use a non-ASCII file name, either, if only
> because it can't predict what the filesystem encoding will be.

I see two different arguments. One is about the file encoding. Those
files are rare and would be placed by the user manually.  We can fix
that by encoding the name.  We can have a debug mode that encodes all
names that way, just to ensure the tools are prepared for it.

The other is Mark's point about "expected file pattern", which seems a
slippery slope to me.  If the pattern is /^[a-zA-Z0-9_.]*$/ then I'm
okay with it (maybe add a few other punctuation chars); as you say no
sane extension would use names much weirder than that.  But we should
not be stricter, such as counting the number of periods/underscores
allowed or where are alpha chars expected (except maybe disallow period
at start of filename), or anything too specific like that.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Commits

  1. Add jsonapi.c to Mkvcbuild.pm's @pgcommonallfiles.

  2. Move jsonapi.c and jsonapi.h to src/common.

  3. Move some code from jsonapi.c to jsonfuncs.c.

  4. Adjust pg_parse_json() so that it does not directly ereport().

  5. Remove jsonapi.c's lex_accept().

  6. Split JSON lexer/parser from 'json' data type support.

  7. Rationalize code placement between wchar.c, encnames.c, and mbutils.c.

  8. Move wchar.c and encnames.c to src/common/.

  9. Update header comments for wchar.c and encnames.c.

  10. Make StringInfo available to frontend code.

  11. Use SASLprep to normalize passwords for SCRAM authentication.