Horton

Thread

Commits

  1. Fix check_exclusion_or_unique_constraint for UNIQUE NULLS NOT DISTINCT.

  2. Add CHECK_FOR_INTERRUPTS in ExecInsert's speculative insertion loop.

  1. BUG #17558: 15beta2: Endless loop with UNIQUE NULLS NOT DISTINCT and INSERT ... ON CONFLICT

    The Post Office <noreply@postgresql.org> — 2022-07-25T01:04:06Z

    The following bug has been logged on the website:

Bug reference:      17558
Logged by:          Andrew Kesper
Email address:      postgresql@middaysomewhere.com
PostgreSQL version: Unsupported/Unknown
Operating system:   Debian 11 (buster)
Description:        

In PostgreSQL 15 beta, when I try to insert a NULL value which violates a
"UNIQUE NULLS NOT DISTINCT" constraint, PostgreSQL will get stuck in an
endless loop if the command includes an "ON CONFLICT" clause. 

CREATE TABLE test (val TEXT UNIQUE NULLS NOT DISTINCT);
INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- inserts 'a'
INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- does
nothing
INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; -- inserts
NULL
INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; --
unresponsive

PostgreSQL will max out the CPU, complain that the WAL is growing too fast,
consume all available disk space, then crash.

  2. Re: BUG #17558: 15beta2: Endless loop with UNIQUE NULLS NOT DISTINCT and INSERT ... ON CONFLICT

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-07-25T15:39:40Z

    PG Bug reporting form <noreply@postgresql.org> writes:
> In PostgreSQL 15 beta, when I try to insert a NULL value which violates a
> "UNIQUE NULLS NOT DISTINCT" constraint, PostgreSQL will get stuck in an
> endless loop if the command includes an "ON CONFLICT" clause. 

> CREATE TABLE test (val TEXT UNIQUE NULLS NOT DISTINCT);
> INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- inserts 'a'
> INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- does
> nothing
> INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; -- inserts
> NULL
> INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; --
> unresponsive

Yup, still a problem in HEAD.  It correctly reports an error if
you just "INSERT INTO test (val) VALUES (NULL)", but something
in the ON CONFLICT code path seems not to be on board with this.
Peter?

			regards, tom lane

  3. Re: BUG #17558: 15beta2: Endless loop with UNIQUE NULLS NOT DISTINCT and INSERT ... ON CONFLICT

    Richard Guo <guofenglinux@gmail.com> — 2022-07-26T03:17:45Z

    On Mon, Jul 25, 2022 at 11:39 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:

> PG Bug reporting form <noreply@postgresql.org> writes:
> > In PostgreSQL 15 beta, when I try to insert a NULL value which violates a
> > "UNIQUE NULLS NOT DISTINCT" constraint, PostgreSQL will get stuck in an
> > endless loop if the command includes an "ON CONFLICT" clause.
>
> > CREATE TABLE test (val TEXT UNIQUE NULLS NOT DISTINCT);
> > INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- inserts
> 'a'
> > INSERT INTO test (val) VALUES ('a') ON CONFLICT DO NOTHING; -- does
> > nothing
> > INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; -- inserts
> > NULL
> > INSERT INTO test (val) VALUES (NULL) ON CONFLICT DO NOTHING; --
> > unresponsive
>
> Yup, still a problem in HEAD.  It correctly reports an error if
> you just "INSERT INTO test (val) VALUES (NULL)", but something
> in the ON CONFLICT code path seems not to be on board with this.
> Peter?


Yeah, I can see the same problem. _bt_check_unique() can catch violation
of unique index constraint successfully, so insert without 'ON CONFLICT'
has no problem. But ExecCheckIndexConstraints() fails to tell the NULL
tuple violates unique constraint, and the logic below is quite
suspicious to me.

/*
 * If any of the input values are NULL, the constraint check is assumed to
 * pass (i.e., we assume the operators are strict).
 */
for (i = 0; i < indnkeyatts; i++)
{
    if (isnull[i])
        return true;
}

Thanks
Richard

  4. Re: BUG #17558: 15beta2: Endless loop with UNIQUE NULLS NOT DISTINCT and INSERT ... ON CONFLICT

    Michael Paquier <michael@paquier.xyz> — 2022-08-02T10:19:09Z

    On Tue, Jul 26, 2022 at 11:17:45AM +0800, Richard Guo wrote:
> Yeah, I can see the same problem. _bt_check_unique() can catch violation
> of unique index constraint successfully, so insert without 'ON CONFLICT'
> has no problem. But ExecCheckIndexConstraints() fails to tell the NULL
> tuple violates unique constraint, and the logic below is quite
> suspicious to me.

This is still an open item assigned to Peter, and beta3 is planned for
the 11th of August.  Would a resolution by the beginning of next week
be doable?
--
Michael

  5. Re: BUG #17558: 15beta2: Endless loop with UNIQUE NULLS NOT DISTINCT and INSERT ... ON CONFLICT

    Tom Lane <tgl@sss.pgh.pa.us> — 2022-08-04T17:17:48Z

    Michael Paquier <michael@paquier.xyz> writes:
> On Tue, Jul 26, 2022 at 11:17:45AM +0800, Richard Guo wrote:
>> Yeah, I can see the same problem. _bt_check_unique() can catch violation
>> of unique index constraint successfully, so insert without 'ON CONFLICT'
>> has no problem. But ExecCheckIndexConstraints() fails to tell the NULL
>> tuple violates unique constraint, and the logic below is quite
>> suspicious to me.

> This is still an open item assigned to Peter, and beta3 is planned for
> the 11th of August.  Would a resolution by the beginning of next week
> be doable?

Since nothing seems to be happening here, I looked into it.  I think
Richard has correctly fingered the buggy code: we need to teach
check_exclusion_or_unique_constraint() that a NULL isn't necessarily
an instant pass.  As attached.

Since the presented test case caused an uninterruptible loop, I also added
a CHECK_FOR_INTERRUPTS() that could stop it.  (I was initially pretty
surprised that we could get through an index scan without hitting any CFI,
but on reflection it's less surprising, because no index scan actually
happens: check_exclusion_or_unique_constraint() short-circuits things
before very much code has been reached.)  I'm kind of inclined to
back-patch the nodeModifyTable.c change all the way, just for safety.

			regards, tom lane