Horton

Thread

Commits

  1. Reword partitioning error message

  2. Doc: Improve clarity on partitioned table limitations

  1. ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Nagaraj Raj <nagaraj.sf@yahoo.com> — 2020-09-29T21:36:02Z

    
I recently upgraded the database from PostgreSQL v9.6 to v11.7. we have some partitioned table with `inherence` and planning to migrate them to the `declaration`.
Table DDL:

CREATE TABLE c_account_p(    billing_account_guid character varying(40)  NOT NULL,    ingestion_process_id bigint NOT NULL DEFAULT '-1'::integer,    load_dttm timestamp(6) without time zone NOT NULL,    ban integer NOT NULL,    CONSTRAINT billing_account_pkey PRIMARY KEY (billing_account_guid, ban)) PARTITION by RANGE(load_dttm);


When I try the create table, it's throwing below error:
ERROR:  insufficient columns in the PRIMARY KEY constraint definitionDETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.SQL state: 0A000

Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.
If the partition column should be supposed to be a PK, it's challenging to create a partition by range with the date column, cause the load_dttm column chances to have duplicate if data loaded `COPY`.

Could some please help me to understand this scenario?   https://dbfiddle.uk/?rdbms=postgres_11&fiddle=5000c4602c16350d77974868e62dc7c9  

Thanks.

  2. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Nagaraj Raj <nagaraj.sf@yahoo.com> — 2020-09-29T21:41:48Z

     Added three scenarios, Postgres 11 | db<>fiddle

| 
| 
|  | 
Postgres 11 | db<>fiddle

Free online SQL environment for experimenting and sharing.
 |

 |

 |




    On Tuesday, September 29, 2020, 02:36:17 PM PDT, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:  
 
 

I recently upgraded the database from PostgreSQL v9.6 to v11.7. we have some partitioned table with `inherence` and planning to migrate them to the `declaration`.
Table DDL:

CREATE TABLE c_account_p(    billing_account_guid character varying(40)  NOT NULL,    ingestion_process_id bigint NOT NULL DEFAULT '-1'::integer,    load_dttm timestamp(6) without time zone NOT NULL,    ban integer NOT NULL,    CONSTRAINT billing_account_pkey PRIMARY KEY (billing_account_guid, ban)) PARTITION by RANGE(load_dttm);


When I try the create table, it's throwing below error:
ERROR:  insufficient columns in the PRIMARY KEY constraint definitionDETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.SQL state: 0A000

Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.
If the partition column should be supposed to be a PK, it's challenging to create a partition by range with the date column, cause the load_dttm column chances to have duplicate if data loaded `COPY`.

Could some please help me to understand this scenario?   https://dbfiddle.uk/?rdbms=postgres_11&fiddle=5000c4602c16350d77974868e62dc7c9  

Thanks.

  3. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-29T22:05:42Z

    On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.

Yes, this is required.  There is no concept of an index over all
partitions in PostgreSQL.  The requirement of having the partition key
a subset of the primary key allows us to implement primary keys by
just having individual unique indexes on each partition.  The fact
that it does not work is not a bug.

There's mention in [1] section 5.10.2.3. "Unique constraints on
partitioned tables must include all the partition key columns. This
limitation exists because PostgreSQL can only enforce uniqueness in
each partition individually.". That text likely should also mention
PRIMARY KEY constraints. That probably should be changed

David

[1] https://www.postgresql.org/docs/11/ddl-partitioning.html

  4. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-29T22:22:29Z

    David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
>> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.

> Yes, this is required.

Indeed.  However, this complaint shows that the error message is not clear
enough.  I propose changing it to

ERROR: unique constraint on partitioned table must be a superset of the partitioning columns

or perhaps

ERROR: unique constraint on partitioned table must include all partitioning columns

The DETAIL seems fine as-is:

DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.

> There's mention in [1] section 5.10.2.3. "Unique constraints on
> partitioned tables must include all the partition key columns. This
> limitation exists because PostgreSQL can only enforce uniqueness in
> each partition individually.". That text likely should also mention
> PRIMARY KEY constraints. That probably should be changed

Meh.  If you've read that bit you probably already understand that
pkeys are unique constraints.  I think the problem is with the error
text not the docs.

			regards, tom lane

  5. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-29T22:29:37Z

    On Wed, 30 Sep 2020 at 11:22, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> David Rowley <dgrowleyml@gmail.com> writes:
> > On Wed, 30 Sep 2020 at 10:36, Nagaraj Raj <nagaraj.sf@yahoo.com> wrote:
> >> Is it mandatory/necessary that the `partition column` should be a primary key? cause if I  include `load_dttm` as `PK` then its working fine.
>
> > Yes, this is required.
>
> Indeed.  However, this complaint shows that the error message is not clear
> enough.  I propose changing it to
>
> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
>
> or perhaps
>
> ERROR: unique constraint on partitioned table must include all partitioning columns

I prefer the former. Although I'd rather see the constraint type
mentioned explicitly rather than using the word "unique" regardless of
what the constraint type is.

> The DETAIL seems fine as-is:
>
> DETAIL:  PRIMARY KEY constraint on table "l_billing_account_p" lacks column "load_dttm" which is part of the partition key.
>
> > There's mention in [1] section 5.10.2.3. "Unique constraints on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in
> > each partition individually.". That text likely should also mention
> > PRIMARY KEY constraints. That probably should be changed
>
> Meh.  If you've read that bit you probably already understand that
> pkeys are unique constraints.  I think the problem is with the error
> text not the docs.

I think you're assuming too much.  If you don't think too hard about
it, it might seem reasonable that we can implement something for a
primary key constraint, because there can only be at most 1 per table,
but not a unique constraint there can be any number.

David

  6. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-29T22:37:35Z

    David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 11:22, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>> Indeed.  However, this complaint shows that the error message is not clear
>> enough.  I propose changing it to
>> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
>> or perhaps
>> ERROR: unique constraint on partitioned table must include all partitioning columns

> I prefer the former. Although I'd rather see the constraint type
> mentioned explicitly rather than using the word "unique" regardless of
> what the constraint type is.

If the DETAIL has that, which it does, I'd rather have the primary text
lay out the general rule not a specific case.  Anybody who is not clear
that a pkey is a kind of unique constraint will get that from reading the
DETAIL.  Also, avoiding that insertion should allow more robust
translation of the primary message, which is something we should worry
about if we are concerned that users won't understand the message.

FWIW, after a few minutes' more thought I think I like the second wording;
it's just as precise and does not rely on two-dollar words.  I'm not dead
set on it though.

			regards, tom lane

  7. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Alvaro Herrera <alvherre@2ndquadrant.com> — 2020-09-29T22:39:07Z

    On 2020-Sep-29, Tom Lane wrote:

> Indeed.  However, this complaint shows that the error message is not clear
> enough.  I propose changing it to
> 
> ERROR: unique constraint on partitioned table must be a superset of the partitioning columns
> 
> or perhaps
> 
> ERROR: unique constraint on partitioned table must include all partitioning columns

Either of those work for me.  How common is the word "superset" in
English?  We don't seem seem to use it either in error messages, though
it does occur in docs.

> > There's mention in [1] section 5.10.2.3. "Unique constraints on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in
> > each partition individually.". That text likely should also mention
> > PRIMARY KEY constraints. That probably should be changed
> 
> Meh.  If you've read that bit you probably already understand that
> pkeys are unique constraints.  I think the problem is with the error
> text not the docs.

Maybe mention PKs in parens:

"Unique constraints (and[, by extension,] primary key constraints) on
partitioned tables must include all the partition key columns. This
limitation exists because PostgreSQL can only enforce uniqueness in each
partition individually.". That text likely should also mention PRIMARY
KEY constraints. That probably should be changed

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  8. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-29T22:43:38Z

    Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> On 2020-Sep-29, Tom Lane wrote:
>> Meh.  If you've read that bit you probably already understand that
>> pkeys are unique constraints.  I think the problem is with the error
>> text not the docs.

> Maybe mention PKs in parens:

> "Unique constraints (and[, by extension,] primary key constraints) on
> partitioned tables must include all the partition key columns. This
> limitation exists because PostgreSQL can only enforce uniqueness in each
> partition individually.".

I don't object to clarifying that (and that wording seems fine), but I
think fixing the error message is more important.  We'd not be having this
discussion if the OP had found that documentation.

			regards, tom lane

  9. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-29T23:08:43Z

    On Wed, 30 Sep 2020 at 11:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > On 2020-Sep-29, Tom Lane wrote:
> >> Meh.  If you've read that bit you probably already understand that
> >> pkeys are unique constraints.  I think the problem is with the error
> >> text not the docs.
>
> > Maybe mention PKs in parens:
>
> > "Unique constraints (and[, by extension,] primary key constraints) on
> > partitioned tables must include all the partition key columns. This
> > limitation exists because PostgreSQL can only enforce uniqueness in each
> > partition individually.".
>
> I don't object to clarifying that (and that wording seems fine), but I
> think fixing the error message is more important.  We'd not be having this
> discussion if the OP had found that documentation.

I do agree the error message is the first thing we should be changing.

I'll write a doc patch if you handle the error message.

David

  10. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-29T23:13:56Z

    On Wed, 30 Sep 2020 at 12:08, David Rowley <dgrowleyml@gmail.com> wrote:
>
> On Wed, 30 Sep 2020 at 11:43, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > > "Unique constraints (and[, by extension,] primary key constraints) on
> > > partitioned tables must include all the partition key columns. This
> > > limitation exists because PostgreSQL can only enforce uniqueness in each
> > > partition individually.".
> >
> > I don't object to clarifying that (and that wording seems fine), but I
> > think fixing the error message is more important.  We'd not be having this
> > discussion if the OP had found that documentation.
>
> I do agree the error message is the first thing we should be changing.
>
> I'll write a doc patch if you handle the error message.

I didn't go with the same wording.  The reason was that I didn't feel
the word "constraint" had to be mentioned twice.

I won't object if you or Alvaro want to keep Alvaro's suggestion though.

David

  11. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-29T23:21:30Z

    David Rowley <dgrowleyml@gmail.com> writes:
> On Wed, 30 Sep 2020 at 12:08, David Rowley <dgrowleyml@gmail.com> wrote:
>> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
>>> "Unique constraints (and[, by extension,] primary key constraints) on
>>> partitioned tables must include all the partition key columns.

> I didn't go with the same wording.  The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

I kind of like Alvaro's wording because it helps to reinforce the point
that pkeys are a type of unique constraint.  If you dislike repeating
"constraints", perhaps we could go with something like

Unique constraints (and hence primary keys) on partitioned tables ...

I'm not hugely against your wording though.

			regards, tom lane

  12. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Alvaro Herrera <alvherre@2ndquadrant.com> — 2020-09-29T23:22:25Z

    On 2020-Sep-30, David Rowley wrote:

> I didn't go with the same wording.  The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
> 
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

*Shrug* this seems good enough.  A purist could complain that it is
redundant, but in practice it's not important.

Here's the proposed error message fix, using the wording that saves
$1.99.  I agree that trying to cram the constraint type in the primary
message is uglier.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  13. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-29T23:53:49Z

    On Wed, 30 Sep 2020 at 12:21, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> I kind of like Alvaro's wording because it helps to reinforce the point
> that pkeys are a type of unique constraint.  If you dislike repeating
> "constraints", perhaps we could go with something like
>
> Unique constraints (and hence primary keys) on partitioned tables ...

ok, cool. I'll go with that.  Thanks

David

  14. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-30T00:06:45Z

    Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> Here's the proposed error message fix, using the wording that saves
> $1.99.  I agree that trying to cram the constraint type in the primary
> message is uglier.

WFM.

			regards, tom lane

  15. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    David Rowley <dgrowleyml@gmail.com> — 2020-09-30T00:12:18Z

    On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > Here's the proposed error message fix, using the wording that saves
> > $1.99.  I agree that trying to cram the constraint type in the primary
> > message is uglier.
>
> WFM.

Looks good to me too.

David

  16. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Nagaraj Raj <nagaraj.sf@yahoo.com> — 2020-09-30T00:45:45Z

     Noted, thank you all. 
    On Tuesday, September 29, 2020, 05:12:40 PM PDT, David Rowley <dgrowleyml@gmail.com> wrote:  
 
 On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
>
> Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > Here's the proposed error message fix, using the wording that saves
> > $1.99.  I agree that trying to cram the constraint type in the primary
> > message is uglier.
>
> WFM.

Looks good to me too.

David

  17. RE: [EXTERNAL] Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Godfrin, Philippe E <philippe.godfrin@nov.com> — 2020-09-30T13:31:35Z

    I am curious why this is considered Class 0A, versus 42 (syntax error?)

From: Alvaro Herrera <alvherre@2ndquadrant.com>
Sent: Tuesday, September 29, 2020 6:22 PM
To: David Rowley <dgrowleyml@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>; Nagaraj Raj <nagaraj.sf@yahoo.com>; Pg Bugs <pgsql-bugs@postgresql.org>
Subject: [EXTERNAL] Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

On 2020-Sep-30, David Rowley wrote:

> I didn't go with the same wording. The reason was that I didn't feel
> the word "constraint" had to be mentioned twice.
>
> I won't object if you or Alvaro want to keep Alvaro's suggestion though.

*Shrug* this seems good enough. A purist could complain that it is
redundant, but in practice it's not important.

Here's the proposed error message fix, using the wording that saves
$1.99. I agree that trying to cram the constraint type in the primary
message is uglier.

--
Álvaro Herrera https://www.2ndQuadrant.com/<https://www.2ndQuadrant.com>
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

  18. Re: [EXTERNAL] Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Tom Lane <tgl@sss.pgh.pa.us> — 2020-09-30T14:16:33Z

    "Godfrin, Philippe E" <Philippe.Godfrin@nov.com> writes:
> I am curious why this is considered Class 0A, versus 42 (syntax error?)

FEATURE_NOT_SUPPORTED seems perfectly appropriate from here.
What you tried to do is not semantically nonsensical, it's just something
we can't do given the current architecture for partitioned tables.

			regards, tom lane

  19. Re: ERROR: insufficient columns in the PRIMARY KEY constraint definition

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2020-09-30T21:27:09Z

    On 2020-Sep-30, David Rowley wrote:

> On Wed, 30 Sep 2020 at 13:06, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> >
> > Alvaro Herrera <alvherre@2ndquadrant.com> writes:
> > > Here's the proposed error message fix, using the wording that saves
> > > $1.99.  I agree that trying to cram the constraint type in the primary
> > > message is uglier.
> >
> > WFM.
> 
> Looks good to me too.

Great, thanks, pushed now.