Horton

Thread

Commits

  1. Remove ruleutils.c's special case for BIT [VARYING] literals.

  2. Make pg_dump output more portable and more pleasing to look at.

  1. pg_ugprade test failure on data set with column with default value with type bit/varbit

    Paul Guo <paulguo@gmail.com> — 2018-03-30T09:36:10Z

    Hello,

While testing pg_upgrade we seemed to find an issue related to default
value of a column with type bit/varbit.
Below are the steps to reproduce. In this case we added two 'create table'
DDLs in the regression database.
Obviously we saw diff after pg_upgrade testing. The pg binaries are based
on the code pulled a couple of days ago.

[pguo@host67:/data2/postgres/src/bin/pg_upgrade]$ git diff
diff --git a/src/bin/pg_upgrade/test.sh b/src/bin/pg_upgrade/test.sh
old mode 100644
new mode 100755
index 39983abea1..a41105859e
--- a/src/bin/pg_upgrade/test.sh
+++ b/src/bin/pg_upgrade/test.sh
@@ -188,6 +188,9 @@ if "$MAKE" -C "$oldsrc" installcheck; then
        psql -X -d regression -c "$fix_sql;" || psql_fix_sql_status=$?
    fi

+   psql -X -d regression -c "CREATE TABLE t111 ( a40 bit varying(5)
DEFAULT '1');"
+   psql -X -d regression -c "CREATE TABLE t222 ( a40 bit varying(5)
DEFAULT B'1');"
+
    pg_dumpall --no-sync -f "$temp_root"/dump1.sql || pg_dumpall1_status=$?

    if [ "$newsrc" != "$oldsrc" ]; then

[pguo@host67:/data2/postgres/src/bin/pg_upgrade]$ make check


[pguo@host67:/data2/postgres/src/bin/pg_upgrade]$ diff -du
tmp_check/dump1.sql tmp_check/dump2.sql
--- tmp_check/dump1.sql 2018-03-30 16:31:44.329021348 +0800
+++ tmp_check/dump2.sql 2018-03-30 16:31:54.100478202 +0800
@@ -10956,7 +10956,7 @@
 --

 CREATE TABLE public.t111 (
-    a40 bit varying(5) DEFAULT B'1'::bit varying
+    a40 bit varying(5) DEFAULT (B'1'::"bit")::bit varying
 );

There is no diff in functionality of the dump SQLs, but it is annoying. The
simple patch below could fix this. Thanks.

[pguo@host67:/data2/postgres/src/bin/pg_upgrade]$ git diff
diff --git a/src/backend/utils/adt/ruleutils.c
b/src/backend/utils/adt/ruleutils.c
index 2cd54ec33f..ef2ab2d681 100644
--- a/src/backend/utils/adt/ruleutils.c
+++ b/src/backend/utils/adt/ruleutils.c
@@ -9389,7 +9389,7 @@ get_const_expr(Const *constval, deparse_context
*context, int showtype)

        case BITOID:
        case VARBITOID:
-           appendStringInfo(buf, "B'%s'", extval);
+           appendStringInfo(buf, "'%s'", extval);
            break;

        case BOOLOID:

  2. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Robert Haas <robertmhaas@gmail.com> — 2018-05-10T19:08:31Z

    On Fri, Mar 30, 2018 at 5:36 AM, Paul Guo <paulguo@gmail.com> wrote:
> There is no diff in functionality of the dump SQLs, but it is annoying. The
> simple patch below could fix this. Thanks.
>
> --- a/src/backend/utils/adt/ruleutils.c
> +++ b/src/backend/utils/adt/ruleutils.c
> @@ -9389,7 +9389,7 @@ get_const_expr(Const *constval, deparse_context
> *context, int showtype)
>
>         case BITOID:
>         case VARBITOID:
> -           appendStringInfo(buf, "B'%s'", extval);
> +           appendStringInfo(buf, "'%s'", extval);
>             break;
>
>         case BOOLOID:

My first reaction was to guess that this would be unsafe, but looking
it over I don't see a problem.  For the other types handled in that
switch statement, we rely on the custom representation to avoid
needing a typecast, but it seems that for BITOID and VARBITOID we
insert a typecast no matter what.  So maybe the presence of absence of
the "B" makes no difference.

This logic seems to have been added by commit
c828ec88205a232a9789f157d8cf9c3d82f85152, Peter Eisentraut, vintage
2002.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  3. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Paul Guo <paulguo@gmail.com> — 2018-05-17T08:20:50Z

    Thanks. I tentatively submitted a patch (See the attachment).

By the way, current pg_upgrade test script depends on the left data on test
database, but it seems that
a lot of tables are dropped in those test SQL files so this affects the
pg_upgrade test coverage much.
Maybe this needs to be addressed in the future. (Maybe when anyone checks
in test cases pg_upgrade
needs to be considered also?)


2018-05-11 3:08 GMT+08:00 Robert Haas <robertmhaas@gmail.com>:

> On Fri, Mar 30, 2018 at 5:36 AM, Paul Guo <paulguo@gmail.com> wrote:
> > There is no diff in functionality of the dump SQLs, but it is annoying.
> The
> > simple patch below could fix this. Thanks.
> >
> > --- a/src/backend/utils/adt/ruleutils.c
> > +++ b/src/backend/utils/adt/ruleutils.c
> > @@ -9389,7 +9389,7 @@ get_const_expr(Const *constval, deparse_context
> > *context, int showtype)
> >
> >         case BITOID:
> >         case VARBITOID:
> > -           appendStringInfo(buf, "B'%s'", extval);
> > +           appendStringInfo(buf, "'%s'", extval);
> >             break;
> >
> >         case BOOLOID:
>
> My first reaction was to guess that this would be unsafe, but looking
> it over I don't see a problem.  For the other types handled in that
> switch statement, we rely on the custom representation to avoid
> needing a typecast, but it seems that for BITOID and VARBITOID we
> insert a typecast no matter what.  So maybe the presence of absence of
> the "B" makes no difference.
>
> This logic seems to have been added by commit
> c828ec88205a232a9789f157d8cf9c3d82f85152, Peter Eisentraut, vintage
> 2002.
>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise PostgreSQL Company
>

  4. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Robert Haas <robertmhaas@gmail.com> — 2018-05-17T13:21:00Z

    On Thu, May 17, 2018 at 4:20 AM, Paul Guo <paulguo@gmail.com> wrote:
> Thanks. I tentatively submitted a patch (See the attachment).

You probably want to add this to the next Commitfest.

> By the way, current pg_upgrade test script depends on the left data on test
> database, but it seems that
> a lot of tables are dropped in those test SQL files so this affects the
> pg_upgrade test coverage much.
> Maybe this needs to be addressed in the future. (Maybe when anyone checks in
> test cases pg_upgrade
> needs to be considered also?)

There's been a deliberate attempt to leave a representative selection
of tables not dropped for this exact reason, but it's definitely
possible that interesting cases have been missed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

  5. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Davy Machado <machado.davy@gmail.com> — 2018-07-22T02:21:28Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            not tested

Hi Paul,

this is a review of the patch:
CABQrizc90sfkZgi4=+0Bbp1Zu3yEx9sM4rjBE1YNCvzf3qKHkA@mail.gmail.com

There hasn't been any problem, at least that I've been able to find.

This one applies cleanly. 

Compile, pg_upgrade and pg_dumpall passed without error too.

Follow below a comparison of the results of the pg_dumpall:

############# Without patch #############

...

CREATE TABLE public.t111 (
    a40 bit varying(5) DEFAULT (B'1'::"bit")::bit varying
);

...

CREATE TABLE public.t222 (
    a40 bit varying(5) DEFAULT B'1'::"bit"
);

############# With patch #############

...

CREATE TABLE public.t111 (
    a40 bit varying(5) DEFAULT ('1'::"bit")::bit varying
);

...

CREATE TABLE public.t222 (
    a40 bit varying(5) DEFAULT '1'::"bit"
);


The "B", used to indicated a bit-string constant, removed as expected.

+1 for committer review

--
Davy Machado

  6. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Thomas Munro <thomas.munro@enterprisedb.com> — 2018-07-30T01:38:18Z

    On Thu, May 17, 2018 at 8:20 PM, Paul Guo <paulguo@gmail.com> wrote:
> Thanks. I tentatively submitted a patch (See the attachment).

Hi Paul,

It looks like you missed a couple of changes in the contrib/btree_gist
bit and varbit tests, so make check-world fails:

- Index Cond: ((a >= B'1000000'::"bit") AND (a <= B'1000001'::"bit"))
+ Index Cond: ((a >= '1000000'::"bit") AND (a <= '1000001'::"bit"))

-- 
Thomas Munro
http://www.enterprisedb.com

  7. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Paul Guo <paulguo@gmail.com> — 2018-08-01T03:56:30Z

    Thanks. I updated the patch as attached.

Double-checked those tests passed.

2018-07-30 9:38 GMT+08:00 Thomas Munro <thomas.munro@enterprisedb.com>:

> On Thu, May 17, 2018 at 8:20 PM, Paul Guo <paulguo@gmail.com> wrote:
> > Thanks. I tentatively submitted a patch (See the attachment).
>
> Hi Paul,
>
> It looks like you missed a couple of changes in the contrib/btree_gist
> bit and varbit tests, so make check-world fails:
>
> - Index Cond: ((a >= B'1000000'::"bit") AND (a <= B'1000001'::"bit"))
> + Index Cond: ((a >= '1000000'::"bit") AND (a <= '1000001'::"bit"))
>
> --
> Thomas Munro
> http://www.enterprisedb.com
>

  8. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    John Naylor <jcnaylor@gmail.com> — 2018-09-09T07:47:50Z

    On 8/1/18, Paul Guo <paulguo@gmail.com> wrote:
> Thanks. I updated the patch as attached.
>
> Double-checked those tests passed.

I've verified make check-world passes. I've marked it Ready for Committer.

-John Naylor

  9. Re: pg_ugprade test failure on data set with column with default value with type bit/varbit

    Tom Lane <tgl@sss.pgh.pa.us> — 2018-09-11T20:37:10Z

    Paul Guo <paulguo@gmail.com> writes:
> [ 0001-Fix-pg_upgrade-test-failure-caused-by-the-DDL-below.v2.patch ]

Actually, there's an even easier way to fix this, which is to discard
the special case for BITOID/VARBITOID altogether, and let the "default"
case handle it.  Fixing things by removing code is always great when
possible.

Also, it's fairly customary to add a test case that actually exhibits
the behavior you want to fix, so I added a regression test table
that has some bit/varbit columns with defaults.  I confirmed that that
makes the pg_upgrade test fail without this ruleutils change.

Pushed with those changes.

			regards, tom lane