Horton

Thread

  1. Re: UTF8 or Unicode

    Dave Page <dpage@vale-housing.co.uk> — 2005-02-18T12:49:36Z

    

-----Original Message-----
From: pgsql-hackers-owner@postgresql.org on behalf of Oliver Jowett
Sent: Fri 2/18/2005 11:27 AM
To: Karel Zak
Cc: List pgsql-hackers
Subject: Re: [HACKERS] UTF8 or Unicode
 
Karel Zak wrote:

>> Yes, I think we should fix it and remove UNICODE and WIN encoding names
>> from PG code.
>
> The JDBC driver asks for a UNICODE client encoding before it knows the 
> server version it is talking to. How do you avoid breaking this?

So does pgAdmin.

Regards, Dave

  2. Re: UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-19T03:35:07Z

    Dave Page wrote:
> Karel Zak wrote:
> 
> >> Yes, I think we should fix it and remove UNICODE and WIN encoding names
> >> from PG code.
> >
> > The JDBC driver asks for a UNICODE client encoding before it knows the 
> > server version it is talking to. How do you avoid breaking this?
> 
> So does pgAdmin.

I think we just need to _favor_ UTF8.  The question is where are we
favoring Unicode rather than UTF8?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  3. Re: UTF8 or Unicode

    Tom Lane <tgl@sss.pgh.pa.us> — 2005-02-19T03:58:34Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> I think we just need to _favor_ UTF8.

I agree.

> The question is where are we
> favoring Unicode rather than UTF8?

It's the canonical name of the encoding, both in the code and the docs.

regression=# create database e encoding 'utf-8';
CREATE DATABASE
regression=# \l
         List of databases
    Name    |  Owner   | Encoding  
------------+----------+-----------
 e          | postgres | UNICODE
 regression | postgres | SQL_ASCII
 template0  | postgres | SQL_ASCII
 template1  | postgres | SQL_ASCII
(5 rows)

As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
;-) we can fix it.

			regards, tom lane

  4. Re: UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-22T03:08:25Z

    Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > I think we just need to _favor_ UTF8.
> 
> I agree.
> 
> > The question is where are we
> > favoring Unicode rather than UTF8?
> 
> It's the canonical name of the encoding, both in the code and the docs.
> 
> regression=# create database e encoding 'utf-8';
> CREATE DATABASE
> regression=# \l
>          List of databases
>     Name    |  Owner   | Encoding  
> ------------+----------+-----------
>  e          | postgres | UNICODE
>  regression | postgres | SQL_ASCII
>  template0  | postgres | SQL_ASCII
>  template1  | postgres | SQL_ASCII
> (5 rows)
> 
> As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
> ;-) we can fix it.

I checked and it looks like "UTF-8" is the correct usage:

	http://www.unicode.org/glossary/

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  5. Re: UTF8 or Unicode

    Tatsuo Ishii <t-ishii@sra.co.jp> — 2005-02-23T02:44:29Z

    I do not object the changing UNICODE->UTF-8, but all these discussions
sound a little bit funny to me.

If you want to blame UNICODE, you should blame LATIN1 etc. as
well. LATIN1(ISO-8859-1) is actually a character set name, not an
encoding name. ISO-8859-1 can be encoded in 8-bit single byte
stream. But it can be encoded in 7-bit too. So when we refer to
LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.
--
Tatsuo Ishii

From: Bruce Momjian <pgman@candle.pha.pa.us>
Subject: Re: [HACKERS] UTF8 or Unicode
Date: Mon, 21 Feb 2005 22:08:25 -0500 (EST)
Message-ID: <200502220308.j1M38PV03238@candle.pha.pa.us>

> Tom Lane wrote:
> > Bruce Momjian <pgman@candle.pha.pa.us> writes:
> > > I think we just need to _favor_ UTF8.
> > 
> > I agree.
> > 
> > > The question is where are we
> > > favoring Unicode rather than UTF8?
> > 
> > It's the canonical name of the encoding, both in the code and the docs.
> > 
> > regression=# create database e encoding 'utf-8';
> > CREATE DATABASE
> > regression=# \l
> >          List of databases
> >     Name    |  Owner   | Encoding  
> > ------------+----------+-----------
> >  e          | postgres | UNICODE
> >  regression | postgres | SQL_ASCII
> >  template0  | postgres | SQL_ASCII
> >  template1  | postgres | SQL_ASCII
> > (5 rows)
> > 
> > As soon as we decide whether the canonical name is "UTF8" or "UTF-8"
> > ;-) we can fix it.
> 
> I checked and it looks like "UTF-8" is the correct usage:
> 
> 	http://www.unicode.org/glossary/
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman@candle.pha.pa.us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 6: Have you searched our list archives?
> 
>                http://archives.postgresql.org
>

  6. Re: UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-25T04:51:16Z

    Tatsuo Ishii wrote:
> I do not object the changing UNICODE->UTF-8, but all these discussions
> sound a little bit funny to me.
> 
> If you want to blame UNICODE, you should blame LATIN1 etc. as
> well. LATIN1(ISO-8859-1) is actually a character set name, not an
> encoding name. ISO-8859-1 can be encoded in 8-bit single byte
> stream. But it can be encoded in 7-bit too. So when we refer to
> LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.

Wow, Tatsuo has a point here.  Looking at encnames.c, I see:

        "UNICODE", PG_UTF8

but also:

        "WIN", PG_WIN1251
        "LATIN1", PG_LATIN1

and I see conversions for those:

        "iso88591", PG_LATIN1
        "win", PG_WIN1251

so I see what he is saying.  We are not consistent in favoring the
official names vs. the common names.

I will work on a patch that people can review and test.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  7. Re: UTF8 or Unicode

    Peter Eisentraut <peter_e@gmx.net> — 2005-02-25T07:02:45Z

    Bruce Momjian wrote:
> We are not consistent in favoring the
> official names vs. the common names.

The problem is rather that there are too many standards and conventions 
to choose from.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

  8. Re: UTF8 or Unicode

    Karel Zak <zakkr@zf.jcu.cz> — 2005-02-25T08:59:25Z

    On Thu, 2005-02-24 at 23:51 -0500, Bruce Momjian wrote:
> Tatsuo Ishii wrote:
> > I do not object the changing UNICODE->UTF-8, but all these discussions
> > sound a little bit funny to me.
> > 
> > If you want to blame UNICODE, you should blame LATIN1 etc. as
> > well. LATIN1(ISO-8859-1) is actually a character set name, not an
> > encoding name. ISO-8859-1 can be encoded in 8-bit single byte
> > stream. But it can be encoded in 7-bit too. So when we refer to
> > LATIN1(ISO-8859-1), it's not clear if it's encoded in 7/8-bit.
> 
> Wow, Tatsuo has a point here.  Looking at encnames.c, I see:
> 
>         "UNICODE", PG_UTF8
> 
> but also:
> 
>         "WIN", PG_WIN1251
>         "LATIN1", PG_LATIN1

> so I see what he is saying.  We are not consistent in favoring the
> official names vs. the common names.

Yes. I said already. For example "WIN" is extremely bad alias. It all is
heritage from old versions.

> I will work on a patch that people can review and test.

Thanks.

	Karel

-- 
Karel Zak <zakkr@zf.jcu.cz>

  9. Re: UTF8 or Unicode

    Peter Eisentraut <peter_e@gmx.net> — 2005-02-25T10:01:44Z

    Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> so I see what he is saying.  We are not consistent in favoring the
> official names vs. the common names.
>
> I will work on a patch that people can review and test.

I think this is what we should do:

UNICODE => UTF8
ALT => WIN866
WIN => WIN1251
TCVN => WIN1258

That should clear it up.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

  10. Re: UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-25T15:26:05Z

    Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > so I see what he is saying.  We are not consistent in favoring the
> > official names vs. the common names.
> >
> > I will work on a patch that people can review and test.
> 
> I think this is what we should do:
> 
> UNICODE => UTF8
> ALT => WIN866
> WIN => WIN1251
> TCVN => WIN1258

OK, but what about latin1?

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  11. Re: UTF8 or Unicode

    Tom Lane <tgl@sss.pgh.pa.us> — 2005-02-25T15:33:13Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
> Peter Eisentraut wrote:
>> I think this is what we should do:
>> 
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258

> OK, but what about latin1?

I think LATIN1 is fine as-is.  It's a reasonably popular name for the
character set, and despite Tatsuo's complaint, it's not going to confuse
anyone in practice --- the 7-bit version of that standard has no traction.
The reason UNICODE is a bad name for UTF8 is exactly that there are
multiple physical encodings of Unicode that are in common use.

			regards, tom lane

  12. Re: UTF8 or Unicode

    Peter Eisentraut <peter_e@gmx.net> — 2005-02-25T16:32:21Z

    Am Freitag, 25. Februar 2005 16:26 schrieb Bruce Momjian:
> OK, but what about latin1?

The following character set names are specified in the SQL standard and 
therefore somewhat non-negotiable:

SQL_CHARACTER
GRAPHIC_IRV
LATIN1
ISO8BIT
UTF16
UTF8
UCS2
SQL_TEXT
SQL_IDENTIFIER

So we have to use LATIN1, even though it creates an inconsistency.  We 
discussed this a while ago during the last great renaming, I think.

Btw., I think ISO8BIT is the correct name for what we call SQL_ASCII, but I 
haven't analyzed that in detail, yet.

-- 
Peter Eisentraut
http://developer.postgresql.org/~petere/

  13. Re: UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-25T18:36:57Z

    Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 16:26 schrieb Bruce Momjian:
> > OK, but what about latin1?
> 
> The following character set names are specified in the SQL standard and 
> therefore somewhat non-negotiable:
> 
> SQL_CHARACTER
> GRAPHIC_IRV
> LATIN1
> ISO8BIT
> UTF16
> UTF8
> UCS2
> SQL_TEXT
> SQL_IDENTIFIER
> 
> So we have to use LATIN1, even though it creates an inconsistency.  We 
> discussed this a while ago during the last great renaming, I think.
> 

Oh, UTF8 and not UTF-8?  I thought UTF-8 was the standard name, but if
ANSI uses UTF8 we will have to use that.

> Btw., I think ISO8BIT is the correct name for what we call SQL_ASCII, but I 
> haven't analyzed that in detail, yet.

OK, please let us know.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  14. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-26T20:50:23Z

    Peter Eisentraut wrote:
> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > so I see what he is saying.  We are not consistent in favoring the
> > official names vs. the common names.
> >
> > I will work on a patch that people can review and test.
> 
> I think this is what we should do:
> 
> UNICODE => UTF8
> ALT => WIN866
> WIN => WIN1251
> TCVN => WIN1258
> 
> That should clear it up.

OK, here is a patch that makes those changes.

The only uncertainty I have is with the the use of the TCVN conversion
routine names, e.g.:

	SELECT CONVERT('foo' USING tcvn_to_utf_8);

I assume this is the same as:

	SELECT CONVERT('foo', 'WIN1258', 'UTF8');
and
	SELECT CONVERT('foo', 'TCVN', 'UTF8');   -- alias usage

So, why would people use the routine name?  Both forms are documented. 
The first one with USING does not accept aliases, while the others do.

I think this should be renamed to win1258_to_utf_8.  However, this would
be an incompatibility.  We should mention it in the release notes.

Other than that the other conversion files were already named fine, e.g.
ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
unsure how to handle these.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  15. Re: [HACKERS] UTF8 or Unicode

    lsunley@mb.sympatico.ca — 2005-02-26T23:08:52Z

    The routines that do the conversion could have alternate names specified
in the conversion_create.sql. There is not reason that I can see why you
cannot have two function names pointing to the same routine.

like

CREATE OR REPLACE FUNCTION ascii_to_mic (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

and 

CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

I just tried with those two and it works OK

That way you do not break compatibility with existing SQL scripts and/or 
programs using the existing function names.

Lorne


In <200502262050.j1QKoNi10358@candle.pha.pa.us>, on 02/26/05 
   at 03:50 PM, Bruce Momjian <pgman@candle.pha.pa.us> said:


>Peter Eisentraut wrote:
>> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
>> > so I see what he is saying.  We are not consistent in favoring the
>> > official names vs. the common names.
>> >
>> > I will work on a patch that people can review and test.
>> 
>> I think this is what we should do:
>> 
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258
>> 
>> That should clear it up.

>OK, here is a patch that makes those changes.

>The only uncertainty I have is with the the use of the TCVN conversion
>routine names, e.g.:

>	SELECT CONVERT('foo' USING tcvn_to_utf_8);

>I assume this is the same as:

>	SELECT CONVERT('foo', 'WIN1258', 'UTF8');
>and
>	SELECT CONVERT('foo', 'TCVN', 'UTF8');   -- alias usage

>So, why would people use the routine name?  Both forms are documented. 
>The first one with USING does not accept aliases, while the others do.

>I think this should be renamed to win1258_to_utf_8.  However, this would
>be an incompatibility.  We should mention it in the release notes.

>Other than that the other conversion files were already named fine, e.g.
>ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
>unsure how to handle these.



-- 
-----------------------------------------------------------
lsunley@mb.sympatico.ca
-----------------------------------------------------------

  16. Re: [HACKERS] UTF8 or Unicode

    lsunley@mb.sympatico.ca — 2005-02-26T23:49:36Z

    Further to my earlier e-mail, there would have to be two lines added to
conversion_create.sql for each alternate function name

Like:

CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
LANGUAGE 'c' STRICT;

CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
'MULE_INTERNAL' FROM ascii_to_whatever;

Lorne

In <200502262050.j1QKoNi10358@candle.pha.pa.us>, on 02/26/05 
   at 03:50 PM, Bruce Momjian <pgman@candle.pha.pa.us> said:


>Peter Eisentraut wrote:
>> Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
>> > so I see what he is saying.  We are not consistent in favoring the
>> > official names vs. the common names.
>> >
>> > I will work on a patch that people can review and test.
>> 
>> I think this is what we should do:
>> 
>> UNICODE => UTF8
>> ALT => WIN866
>> WIN => WIN1251
>> TCVN => WIN1258
>> 
>> That should clear it up.

>OK, here is a patch that makes those changes.

>The only uncertainty I have is with the the use of the TCVN conversion
>routine names, e.g.:

>	SELECT CONVERT('foo' USING tcvn_to_utf_8);

>I assume this is the same as:

>	SELECT CONVERT('foo', 'WIN1258', 'UTF8');
>and
>	SELECT CONVERT('foo', 'TCVN', 'UTF8');   -- alias usage

>So, why would people use the routine name?  Both forms are documented. 
>The first one with USING does not accept aliases, while the others do.

>I think this should be renamed to win1258_to_utf_8.  However, this would
>be an incompatibility.  We should mention it in the release notes.

>Other than that the other conversion files were already named fine, e.g.
>ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
>unsure how to handle these.



-- 
-----------------------------------------------------------
lsunley@mb.sympatico.ca
-----------------------------------------------------------

  17. Re: [HACKERS] UTF8 or Unicode

    lsunley@mb.sympatico.ca — 2005-02-26T23:56:59Z

    In <200502270055.j1R0te122973@candle.pha.pa.us>, on 02/26/05 
   at 07:55 PM, Bruce Momjian <pgman@candle.pha.pa.us> said:

>lsunley@mb.sympatico.ca wrote:
>> Further to my earlier e-mail, there would have to be two lines added to
>> conversion_create.sql for each alternate function name
>> 
>> Like:
>> 
>> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
>> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
>> LANGUAGE 'c' STRICT;
>> 
>> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
>> 'MULE_INTERNAL' FROM ascii_to_whatever;

>I would like to avoid bloating the system with aliases.  I think it is OK
>to have it happen with specific names but doing it for all the
>combinations is too much, and these duplicates are in the system tables,
>not in the C code.  It just doesn't seem worth it to me.

Good point, perhaps having a script generated that can be executed by the
DBA to insert any deprecated conversion routine names would be the best
route to take.

That way they would only be there if needed by the particular
installation.

-- 
-----------------------------------------------------------
lsunley@mb.sympatico.ca
-----------------------------------------------------------

  18. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-27T00:55:40Z

    lsunley@mb.sympatico.ca wrote:
> Further to my earlier e-mail, there would have to be two lines added to
> conversion_create.sql for each alternate function name
> 
> Like:
> 
> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
> LANGUAGE 'c' STRICT;
> 
> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
> 'MULE_INTERNAL' FROM ascii_to_whatever;

I would like to avoid bloating the system with aliases.  I think it is
OK to have it happen with specific names but doing it for all the
combinations is too much, and these duplicates are in the system tables,
not in the C code.  It just doesn't seem worth it to me.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  19. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-27T01:07:22Z

    lsunley@mb.sympatico.ca wrote:
> In <200502270055.j1R0te122973@candle.pha.pa.us>, on 02/26/05 
>    at 07:55 PM, Bruce Momjian <pgman@candle.pha.pa.us> said:
> 
> >lsunley@mb.sympatico.ca wrote:
> >> Further to my earlier e-mail, there would have to be two lines added to
> >> conversion_create.sql for each alternate function name
> >> 
> >> Like:
> >> 
> >> CREATE OR REPLACE FUNCTION ascii_to_whatever (INTEGER, INTEGER, CSTRING,
> >> CSTRING, INTEGER) RETURNS VOID AS '$libdir/ascmic', 'ascii_to_mic'
> >> LANGUAGE 'c' STRICT;
> >> 
> >> CREATE CONVERSION pg_catalog.ascii_to_whatever FOR 'SQL_ASCII' TO
> >> 'MULE_INTERNAL' FROM ascii_to_whatever;
> 
> >I would like to avoid bloating the system with aliases.  I think it is OK
> >to have it happen with specific names but doing it for all the
> >combinations is too much, and these duplicates are in the system tables,
> >not in the C code.  It just doesn't seem worth it to me.
> 
> Good point, perhaps having a script generated that can be executed by the
> DBA to insert any deprecated conversion routine names would be the best
> route to take.

I am unsure why anyone would use the awkward names when you can more
easily supply the actual encoding names, and all the aliases work there.

Also, consider this is going to be a major release change so they are
going to dump/reload anyway.  I see I need to update
conversion_create.sql too.  I was previously only looking in C files. 
Let me do a bigger search.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  20. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-02-27T04:09:43Z

    Here is an updated version that handles all cases.  It does rename the
routine names so the primary encoding name is used for the routine
names.  This will be documented in the release notes if anyone actually
uses those names in their code.

This patch requires renaming of the utf8_and_tcvn directory so it will
not apply cleanly.

I left the routines named utf_8 alone because the code splits encoding
names at breaks, like this iso_8859_7_to_utf_8.  I assume that is OK.

---------------------------------------------------------------------------

Bruce Momjian wrote:
> Peter Eisentraut wrote:
> > Am Freitag, 25. Februar 2005 05:51 schrieb Bruce Momjian:
> > > so I see what he is saying.  We are not consistent in favoring the
> > > official names vs. the common names.
> > >
> > > I will work on a patch that people can review and test.
> > 
> > I think this is what we should do:
> > 
> > UNICODE => UTF8
> > ALT => WIN866
> > WIN => WIN1251
> > TCVN => WIN1258
> > 
> > That should clear it up.
> 
> OK, here is a patch that makes those changes.
> 
> The only uncertainty I have is with the the use of the TCVN conversion
> routine names, e.g.:
> 
> 	SELECT CONVERT('foo' USING tcvn_to_utf_8);
> 
> I assume this is the same as:
> 
> 	SELECT CONVERT('foo', 'WIN1258', 'UTF8');
> and
> 	SELECT CONVERT('foo', 'TCVN', 'UTF8');   -- alias usage
> 
> So, why would people use the routine name?  Both forms are documented. 
> The first one with USING does not accept aliases, while the others do.
> 
> I think this should be renamed to win1258_to_utf_8.  However, this would
> be an incompatibility.  We should mention it in the release notes.
> 
> Other than that the other conversion files were already named fine, e.g.
> ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
> unsure how to handle these.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  21. Re: [HACKERS] UTF8 or Unicode

    lsunley@mb.sympatico.ca — 2005-02-27T04:50:12Z

    In <200502270409.j1R49hc08394@candle.pha.pa.us>, on 02/26/05 
   at 11:09 PM, Bruce Momjian <pgman@candle.pha.pa.us> said:



>Here is an updated version that handles all cases.  It does rename the
>routine names so the primary encoding name is used for the routine names. 
>This will be documented in the release notes if anyone actually uses
>those names in their code.

>This patch requires renaming of the utf8_and_tcvn directory so it will
>not apply cleanly.

>I left the routines named utf_8 alone because the code splits encoding
>names at breaks, like this iso_8859_7_to_utf_8.  I assume that is OK.


You cannot patch conversion_create.sql

You have to patch the makefile that generates it.

The CONVERSIONS variable has the names etc and the

@set $(CONVERSIONS) etc code generates the conversion_create.sql file

-- 
-----------------------------------------------------------
lsunley@mb.sympatico.ca
-----------------------------------------------------------

  22. Re: [HACKERS] UTF8 or Unicode

    lsunley@mb.sympatico.ca — 2005-02-27T04:53:48Z

    Ignore previous e-mail...

Missed the patch to the makefile

Sorry about that

-- 
-----------------------------------------------------------
lsunley@mb.sympatico.ca
-----------------------------------------------------------

  23. Re: [HACKERS] UTF8 or Unicode

    Markus Bertheau <twanger@bluetwanger.de> — 2005-03-02T09:48:43Z

    В Сбт, 26/02/2005 в 15:50 -0500, Bruce Momjian пишет:

> Other than that the other conversion files were already named fine, e.g.
> ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
> unsure how to handle these.

General remark about the spelling of this encoding:

The correct encoding name is "UTF-8".

"The official name and spelling of this encoding is UTF-8, where UTF
stands for UCS Transformation Format. Please do not write UTF-8 in any
documentation text in other ways (such as utf8 or UTF_8), unless of
course you refer to a variable name and not the encoding itself."

from

http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8

Thanks

-- 
Markus Bertheau ☭ <twanger@bluetwanger.de>

  24. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-03-02T13:31:11Z

    Markus Bertheau ? wrote:
> ? ???, 26/02/2005 ? 15:50 -0500, Bruce Momjian ?????:
> 
> > Other than that the other conversion files were already named fine, e.g.
> > ascii_to_utf_8 (no UNICODE), however it is utf_8 and not utf8.  I am
> > unsure how to handle these.
> 
> General remark about the spelling of this encoding:
> 
> The correct encoding name is "UTF-8".
> 
> "The official name and spelling of this encoding is UTF-8, where UTF
> stands for UCS Transformation Format. Please do not write UTF-8 in any
> documentation text in other ways (such as utf8 or UTF_8), unless of
> course you refer to a variable name and not the encoding itself."
> 
> from
> 
> http://www.cl.cam.ac.uk/~mgk25/unicode.html#utf-8

True, but Peter says the ANSI standard calls it UTF8 so that's what I
used.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

  25. Re: [HACKERS] UTF8 or Unicode

    Tom Lane <tgl@sss.pgh.pa.us> — 2005-03-02T17:54:20Z

    Bruce Momjian <pgman@candle.pha.pa.us> writes:
>> The correct encoding name is "UTF-8".

> True, but Peter says the ANSI standard calls it UTF8 so that's what I
> used.

What SQL99 actually says is

         -  UTF8 specifies the name of a character repertoire that consists
            of every character represented by The Unicode Standard Version
            2.0 and by ISO/IEC 10646 UTF-8, where each character is encoded
            using the UTF-8 encoding, occupying from 1 (one) through 6
            octets.

That is, "UTF8" is an identifier chosen to refer to an encoding which
they know perfectly well is really called UTF-8.  We should probably
follow the same convention of using UTF8 in code identifiers and UTF-8
in documentation.  In particular, UTF_8 with an underscore is sanctioned
by nobody and should be avoided.

			regards, tom lane

  26. Re: [HACKERS] UTF8 or Unicode

    Bruce Momjian <pgman@candle.pha.pa.us> — 2005-03-03T05:20:35Z

    Tom Lane wrote:
> Bruce Momjian <pgman@candle.pha.pa.us> writes:
> >> The correct encoding name is "UTF-8".
> 
> > True, but Peter says the ANSI standard calls it UTF8 so that's what I
> > used.
> 
> What SQL99 actually says is
> 
>          -  UTF8 specifies the name of a character repertoire that consists
>             of every character represented by The Unicode Standard Version
>             2.0 and by ISO/IEC 10646 UTF-8, where each character is encoded
>             using the UTF-8 encoding, occupying from 1 (one) through 6
>             octets.
> 
> That is, "UTF8" is an identifier chosen to refer to an encoding which
> they know perfectly well is really called UTF-8.  We should probably
> follow the same convention of using UTF8 in code identifiers and UTF-8
> in documentation.  In particular, UTF_8 with an underscore is sanctioned
> by nobody and should be avoided.

OK, new version with utf_8 changed to utf8.  The docs mention UTF-8 when
talking about the encoding, but UTF8 when listing the code to use.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman@candle.pha.pa.us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073