Re: BUG #18735: Specific multibyte character in psql file path command parameter for Windows

Hello;

Very short response.

Lexical analysis of backshash commands in psql is handled
by psqlscanslash.l and this module scans iput byte-by-byte, not
character-by-character.   I'm afraid that the cause of the bug is in
this part..   Is there any way to make this flex syntax local-dependent?

We need to analyze the behavior of this flex module to get practical idea
for fix.

Regards;
---
Koichi Suzuki
https://www.linkedin.com/in/koichidbms



2024年12月6日(金) 3:50 Tom Lane <tgl@sss.pgh.pa.us>:

> PG Bug reporting form <noreply@postgresql.org> writes:
> > Analysis:
> > * Latter byte valueof the character in question is same as '\'
> (backslash).
> >   It looks that this byte value is handled as escape characters.   This
> > happns SHIFT JIS client encoding.
> > * The issue happens in \i, \ir and \copy but does not happen in \cd, \o
> and
> > \! command.
>
> I imagine what is happening here is that canonicalize_path() interprets
> the backslash bytes as directory separators.
>
> The only thing I can think of to improve that is to make
> canonicalize_path() encoding-aware and have it skip over multibyte
> characters.  Unfortunately, I fear that would introduce as many
> misbehaviors as it would remove, because we don't always know the
> relevant encoding.  We might be able to limit the hazard by
> confining the encoding-awareness to the initial Windows-only
> conversion of '\' to '/', but it'd still be pretty squishy.
>
> > * The similar issue may happen if the latter byte value of a multibyte
> > character is same as '/' (directory delimiter).
>
> I don't believe Shift-JIS uses '/' as part of multibyte characters,
> so it should be sufficient to consider '\'.
>
> BTW, according to wikipedia[1], backslash is not even part of the
> Shift-JIS character set:
>
>     The single-byte characters 0x00 to 0x7F match the ASCII encoding,
>     except for a yen sign (U+00A5) at 0x5C and an overline (U+203E) at
>     0x7E in place of the ASCII character set's backslash and tilde
>     respectively (these deviations from ASCII align with JIS X
>     0201). The single-byte characters from 0xA1 to 0xDF map to the
>     half-width katakana characters found in JIS X 0201.
>
>     For double-byte characters, the first byte is always in the range
>     0x81 to 0x9F or the range 0xE0 to 0xEF (these ranges are
>     unassigned in JIS X 0201). If the first byte is odd, the second
>     byte must be in the range 0x40 to 0x9E (but cannot be 0x7F); if
>     the first byte is even, the second byte must in the range 0x9F to
>     0xFC.
>
> This might mean that it'd be okay to just skip the backslash-to-slash
> conversion loops altogether if we think the encoding is Shift-JIS.
>
> There's still the question of how we determine the relevant encoding.
> I don't think client_encoding is what to use (and we won't have that
> at hand anyway, in programs other than psql).  What we want to know
> is what fopen and related system calls will do with the path: they
> must have different behavior for Shift-JIS than other encodings,
> else none of your examples could work at all.  I assume there's
> a way to find out what they think the relevant encoding is.
>
> make_native_path() adds even more fun: when should we convert '/'
> back to '\'?  From the comments, this function is concerned with
> producing something that will be accepted as a command-line
> argument by other programs, so I wonder if we can even know what
> to do with any certainty.
>
> (In case it's not clear, I'm not volunteering to write or test
> any of this.)
>
>                         regards, tom lane
>
> [1] https://en.wikipedia.org/wiki/Shift_JIS
>