Horton

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

Alexander Korotkov <aekorotkov@gmail.com>

View thread
From: Alexander Korotkov <aekorotkov@gmail.com>
To: Dmitry Koval <d.koval@postgrespro.ru>
Cc: Alexander Lakhin <exclusion@gmail.com>, Robert Haas <robertmhaas@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2024-05-08T19:19:08Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Adjust errcode in checkPartition()

  2. Fix usage of palloc() in MERGE/SPLIT PARTITION(s) code

  3. Implement ALTER TABLE ... SPLIT PARTITION ... command

  4. Implement ALTER TABLE ... MERGE PARTITIONS ... command

  5. Calculate agglevelsup correctly when Aggref contains a CTE.

  6. Use PqMsg_* macros in applyparallelworker.c.

  7. Restrict psql meta-commands in plain-text dumps.

  8. Ensure we have a snapshot when updating various system catalogs.

  9. Use specific collation where needed in new test

  10. Expand virtual generated columns in the planner

  11. Virtual generated columns

  12. Define PG_LOGICAL_DIR for path pg_logical/ in data folder

  13. Revert support for ALTER TABLE ... MERGE/SPLIT PARTITION(S) commands

  14. Avoid repeated table name lookups in createPartitionTable()

  15. Provide deterministic order for catalog queries in partition_split.sql

  16. Don't copy extended statistics during MERGE/SPLIT partition operations

  17. Fix the name collision detection in MERGE/SPLIT partition operations

  18. Fix regression tests conflict in 3ca43dbbb6

  19. Add permission check for MERGE/SPLIT partition operations

  20. Fix one more portability shortcoming in new test_pg_dump test.

  21. Inherit parent's AM for partition MERGE/SPLIT operations

  22. Add tab completion for partition MERGE/SPLIT operations

  23. Rename tables in tests of partition MERGE/SPLIT operations

  24. Make new partitions with parent's persistence during MERGE/SPLIT

  25. Document the way partition MERGE/SPLIT operations create new partitions

  26. Change the way ATExecMergePartitions() handles the name collision

  27. Grammar fixes for split/merge partitions code

  28. Checks for ALTER TABLE ... SPLIT/MERGE PARTITIONS ... commands

  29. Fix some grammer errors from error messages and codes comments

  30. Support TZ and OF format codes in to_timestamp().

  31. Support identity columns in partitioned tables

  32. Fix indentation in twophase.c

  33. Fix corner-case planner failure for MERGE.

  34. Doc: fix documentation example for bytea hex output format.

  35. Avoid repeated name lookups during table and index DDL.

On Wed, May 1, 2024 at 12:14 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 30.04.2024 6:00, Alexander Lakhin пишет:
> > Maybe I'm doing something wrong, but the following script:
> > CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
> > CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
> > CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
> >
> > CREATE TABLE t2 (LIKE t INCLUDING ALL);
> > CREATE TABLE tp2 (LIKE tp_0 INCLUDING ALL);
> > creates tables t2, tp2 without not-null constraints.
>
> To create partitions is used the "CREATE TABLE ... LIKE ..." command
> with the "EXCLUDING INDEXES" modifier (to speed up the insertion of
values).
>
> CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE(i);
> CREATE TABLE t2 (LIKE t INCLUDING ALL EXCLUDING INDEXES EXCLUDING
IDENTITY);
> \d+ t2;
> ...
> Not-null constraints:
>      "t2_i_not_null" NOT NULL "i"
> Access method: heap

I've explored this a little bit more.

If the parent table has explicit not null constraint than results of
MERGE/SPLIT look the same as result of CREATE TABLE ... PARTITION OF.  In
every case there is explicit not null constraint in all the cases.

# CREATE TABLE t (i int not null, PRIMARY KEY(i)) PARTITION BY RANGE(i);
# \d+ t
                                      Partitioned table "public.t"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition key: RANGE (i)
Indexes:
    "t_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i"
Number of partitions: 0
# CREATE TABLE tp_0_2 PARTITION OF t FOR VALUES FROM (0) TO (2);
# \d+ tp_0_2
                                         Table "public.tp_0_2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
Indexes:
    "tp_0_2_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i" (inherited)
Access method: heap
# ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
#    (PARTITION tp_0_1 FOR VALUES FROM (0) TO (1),
#     PARTITION tp_1_2 FOR VALUES FROM (1) TO (2))
# \d+ tp_0_1
                                         Table "public.tp_0_1"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (1)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 1))
Indexes:
    "tp_0_1_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i" (inherited)
Access method: heap

However, if not null constraint is implicit and derived from primary key,
the situation is different.  The partition created by CREATE TABLE ...
PARTITION OF doesn't have explicit not null constraint just like the
parent.  But the partition created by MERGE/SPLIT has explicit not null
contraint.

# CREATE TABLE t (i int not null, PRIMARY KEY(i)) PARTITION BY RANGE(i);
# \d+ t
                                      Partitioned table "public.t"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition key: RANGE (i)
Indexes:
    "t_pkey" PRIMARY KEY, btree (i)
Number of partitions: 0
# CREATE TABLE tp_0_2 PARTITION OF t FOR VALUES FROM (0) TO (2);
# \d+ tp_0_2
                                         Table "public.tp_0_2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
Indexes:
    "tp_0_2_pkey" PRIMARY KEY, btree (i)
Access method: heap
# ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
#    (PARTITION tp_0_1 FOR VALUES FROM (0) TO (1),
#     PARTITION tp_1_2 FOR VALUES FROM (1) TO (2))
# \d+ tp_0_1
                                         Table "public.tp_0_1"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (1)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 1))
Indexes:
    "tp_0_1_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "tp_0_1_i_not_null" NOT NULL "i"
Access method: heap

I think this is related to the fact that we create indexes later.  The same
applies to CREATE TABLE ... LIKE.  If we create indexes immediately, not
explicit not null contraints are created.  Not if we do without indexes, we
have an explicit not null constraint.

# CREATE TABLE t2 (LIKE t INCLUDING ALL);
# \d+ t2
                                           Table "public.t2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Not-null constraints:
    "t2_i_not_null" NOT NULL "i"
Access method: heap
# CREATE TABLE t3 (LIKE t INCLUDING ALL EXCLUDING IDENTITY);
# \d+ t3
                                           Table "public.t3"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Indexes:
    "t3_pkey" PRIMARY KEY, btree (i)
Access method: heap

I think this is feasible to avoid.  However, it's minor and we exactly
documented how we create new partitions.  So, I think it works "as
documented" and we don't have to fix this for v17.

------
Regards,
Alexander Korotkov
Supabase