Horton

Thread

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Adjust errcode in checkPartition()

  2. Fix usage of palloc() in MERGE/SPLIT PARTITION(s) code

  3. Implement ALTER TABLE ... SPLIT PARTITION ... command

  4. Implement ALTER TABLE ... MERGE PARTITIONS ... command

  5. Calculate agglevelsup correctly when Aggref contains a CTE.

  6. Use PqMsg_* macros in applyparallelworker.c.

  7. Restrict psql meta-commands in plain-text dumps.

  8. Ensure we have a snapshot when updating various system catalogs.

  9. Use specific collation where needed in new test

  10. Expand virtual generated columns in the planner

  11. Virtual generated columns

  12. Define PG_LOGICAL_DIR for path pg_logical/ in data folder

  13. Revert support for ALTER TABLE ... MERGE/SPLIT PARTITION(S) commands

  14. Avoid repeated table name lookups in createPartitionTable()

  15. Provide deterministic order for catalog queries in partition_split.sql

  16. Don't copy extended statistics during MERGE/SPLIT partition operations

  17. Fix the name collision detection in MERGE/SPLIT partition operations

  18. Fix regression tests conflict in 3ca43dbbb6

  19. Add permission check for MERGE/SPLIT partition operations

  20. Fix one more portability shortcoming in new test_pg_dump test.

  21. Inherit parent's AM for partition MERGE/SPLIT operations

  22. Add tab completion for partition MERGE/SPLIT operations

  23. Rename tables in tests of partition MERGE/SPLIT operations

  24. Make new partitions with parent's persistence during MERGE/SPLIT

  25. Document the way partition MERGE/SPLIT operations create new partitions

  26. Change the way ATExecMergePartitions() handles the name collision

  27. Grammar fixes for split/merge partitions code

  28. Checks for ALTER TABLE ... SPLIT/MERGE PARTITIONS ... commands

  29. Fix some grammer errors from error messages and codes comments

  30. Support TZ and OF format codes in to_timestamp().

  31. Support identity columns in partitioned tables

  32. Fix indentation in twophase.c

  33. Fix corner-case planner failure for MERGE.

  34. Doc: fix documentation example for bytea hex output format.

  35. Avoid repeated name lookups during table and index DDL.

  1. Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-05-31T09:32:43Z

    Hi, hackers!

There are not many commands in PostgreSQL for working with partitioned 
tables. This is an obstacle to their widespread use.
Adding SPLIT PARTITION/MERGE PARTITIONS operations can make easier to 
use partitioned tables in PostgreSQL.
(This is especially important when migrating projects from ORACLE DBMS.)

SPLIT PARTITION/MERGE PARTITIONS commands are supported for range 
partitioning (BY RANGE) and for list partitioning (BY LIST).
For hash partitioning (BY HASH) these operations are not supported.

=================
1 SPLIT PARTITION
=================
Command for split a single partition.

1.1 Syntax
----------

ALTER TABLE <name> SPLIT PARTITION <partition_name> INTO
(PARTITION <partition_name1> { FOR VALUES <partition_bound_spec> | 
DEFAULT },
   [ ... ]
   PARTITION <partition_nameN> { FOR VALUES <partition_bound_spec> | 
DEFAULT })

<partition_bound_spec>:
	IN ( <partition_bound_expr> [, ...] ) |
	FROM ( { <partition_bound_expr> | MINVALUE | MAXVALUE } [, ...] )
	TO ( { <partition_bound_expr> | MINVALUE | MAXVALUE } [, ...] )

1.2 Rules
---------

1.2.1 The <partition_name> partition should be split into two (or more) 
partitions.

1.2.2 New partitions should have different names (with existing 
partitions too).

1.2.3 Bounds of new partitions should not overlap with new and existing 
partitions.

1.2.4 In case split partition is DEFAULT partition, one of new 
partitions should be DEFAULT.

1.2.5 In case new partitions or existing partitions contains DEFAULT 
partition, new partitions <partition_name1>...<partition_nameN> can have 
any bounds inside split partition bound (can be spaces between 
partitions bounds).

1.2.6 In case partitioned table does not have DEFAULT partition, DEFAULT 
partition can be defined as one of new partition.

1.2.7 In case new partitions not contains DEFAULT partition and 
partitioned table does not have DEFAULT partition the following should 
be true: sum bounds of new partitions 
<partition_name1>...<partition_nameN> should be equal to bound of split 
partition <partition_name>.

1.2.8 One of the new partitions <partition_name1>-<partition_nameN> can 
have the same name as split partition <partition_name> (this is suitable 
in case splitting a DEFAULT partition: we split it, but after splitting 
we have a partition with the same name).

1.2.9 Only simple (non-partitioned) partitions can be split.

1.3 Examples
------------

1.3.1 Example for range partitioning (BY RANGE):

CREATE TABLE sales_range (salesman_id INT, salesman_name VARCHAR(30), 
sales_amount INT, sales_date DATE) PARTITION BY RANGE (sales_date);
CREATE TABLE sales_jan2022 PARTITION OF sales_range FOR VALUES FROM 
('2022-01-01') TO ('2022-02-01');
CREATE TABLE sales_feb_mar_apr2022 PARTITION OF sales_range FOR VALUES 
FROM ('2022-02-01') TO ('2022-05-01');
CREATE TABLE sales_others PARTITION OF sales_range DEFAULT;

ALTER TABLE sales_range SPLIT PARTITION sales_feb_mar_apr2022 INTO
    (PARTITION sales_feb2022 FOR VALUES FROM ('2022-02-01') TO 
('2022-03-01'),
     PARTITION sales_mar2022 FOR VALUES FROM ('2022-03-01') TO 
('2022-04-01'),
     PARTITION sales_apr2022 FOR VALUES FROM ('2022-04-01') TO 
('2022-05-01'));

1.3.2 Example for list partitioning (BY LIST):

CREATE TABLE sales_list
    (salesman_id INT GENERATED ALWAYS AS IDENTITY,
     salesman_name VARCHAR(30),
     sales_state VARCHAR(20),
     sales_amount INT,
     sales_date DATE)
PARTITION BY LIST (sales_state);

CREATE TABLE sales_nord PARTITION OF sales_list FOR VALUES IN 
('Murmansk', 'St. Petersburg', 'Ukhta');
CREATE TABLE sales_all PARTITION OF sales_list FOR VALUES IN ('Moscow', 
'Voronezh', 'Smolensk', 'Bryansk', 'Magadan', 'Kazan', 'Khabarovsk', 
'Volgograd', 'Vladivostok');
CREATE TABLE sales_others PARTITION OF sales_list DEFAULT;

ALTER TABLE sales_list SPLIT PARTITION sales_all INTO
    (PARTITION sales_west FOR VALUES IN ('Voronezh', 'Smolensk', 'Bryansk'),
     PARTITION sales_east FOR VALUES IN ('Magadan', 'Khabarovsk', 
'Vladivostok'),
     PARTITION sales_central FOR VALUES IN ('Moscow', 'Kazan', 
'Volgograd'));

1.4 ToDo:
---------

1.4.1 Possibility to specify tablespace for each of the new partitions 
(currently new partitions are created in the same tablespace as split 
partition).
1.4.2 Possibility to use CONCURRENTLY mode that allows (during the SPLIT 
operation) not blocking partitions that are not splitting.

==================
2 MERGE PARTITIONS
==================
Command for merge several partitions into one partition.

2.1 Syntax
----------

ALTER TABLE <name> MERGE PARTITIONS (<partition_name1>, 
<partition_name2>[, ...]) INTO <new_partition_name>;

2.2 Rules
---------

2.2.1 The number of partitions that are merged into the new partition 
<new_partition_name> should be at least two.

2.2.2
If DEFAULT partition is not in the list of partitions <partition_name1>, 
<partition_name2>[, ...]:
   * for range partitioning (BY RANGE) is necessary that the ranges of 
the partitions <partition_name1>, <partition_name2>[, ...] can be merged 
into one range without spaces and overlaps (otherwise an error will be 
generated).
     The combined range will be the range for the partition 
<new_partition_name>.
   * for list partitioning (BY LIST) the values lists of all partitions 
<partition_name1>, <partition_name2>[, ...] are combined and form a list 
of values of partition <new_partition_name>.

If DEFAULT partition is in the list of partitions <partition_name1>, 
<partition_name2>[, ...]:
   * the partition <new_partition_name> will be the DEFAULT partition;
   * for both partitioning types (BY RANGE, BY LIST) the ranges and 
lists of values of the merged partitions can be any.

2.2.3 The new partition <new_partition_name> can have the same name as 
one of the merged partitions.

2.2.4 Only simple (non-partitioned) partitions can be merged.

2.3 Examples
------------

2.3.1 Example for range partitioning (BY RANGE):

CREATE TABLE sales_range (salesman_id INT, salesman_name VARCHAR(30), 
sales_amount INT, sales_date DATE) PARTITION BY RANGE (sales_date);
CREATE TABLE sales_jan2022 PARTITION OF sales_range FOR VALUES FROM 
('2022-01-01') TO ('2022-02-01');
CREATE TABLE sales_feb2022 PARTITION OF sales_range FOR VALUES FROM 
('2022-02-01') TO ('2022-03-01');
CREATE TABLE sales_mar2022 PARTITION OF sales_range FOR VALUES FROM 
('2022-03-01') TO ('2022-04-01');
CREATE TABLE sales_apr2022 PARTITION OF sales_range FOR VALUES FROM 
('2022-04-01') TO ('2022-05-01');
CREATE TABLE sales_others PARTITION OF sales_range DEFAULT;

ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022, sales_mar2022, 
sales_apr2022) INTO sales_feb_mar_apr2022;

2.3.2 Example for list partitioning (BY LIST):

CREATE TABLE sales_list
(salesman_id INT GENERATED ALWAYS AS IDENTITY,
   salesman_name VARCHAR(30),
   sales_state VARCHAR(20),
   sales_amount INT,
   sales_date DATE)
PARTITION BY LIST (sales_state);

CREATE TABLE sales_nord PARTITION OF sales_list FOR VALUES IN 
('Murmansk', 'St. Petersburg', 'Ukhta');
CREATE TABLE sales_west PARTITION OF sales_list FOR VALUES IN 
('Voronezh', 'Smolensk', 'Bryansk');
CREATE TABLE sales_east PARTITION OF sales_list FOR VALUES IN 
('Magadan', 'Khabarovsk', 'Vladivostok');
CREATE TABLE sales_central PARTITION OF sales_list FOR VALUES IN 
('Moscow', 'Kazan', 'Volgograd');
CREATE TABLE sales_others PARTITION OF sales_list DEFAULT;

ALTER TABLE sales_list MERGE PARTITIONS (sales_west, sales_east, 
sales_central) INTO sales_all;

2.4 ToDo:
---------

2.4.1 Possibility to specify tablespace for the new partition (currently 
new partition is created in the same tablespace as partitioned table).
2.4.2 Possibility to use CONCURRENTLY mode that allows (during the MERGE 
operation) not blocking partitions that are not merging.
2.4.3 New syntax for ALTER TABLE ... MERGE PARTITIONS command for range 
partitioning (BY RANGE):

ALTER TABLE <name> MERGE PARTITIONS <partition_name1> TO 
<partition_name2> INTO <new_partition_name>;

This command can merge all partitions between <partition_name1> and 
<partition_name2> into new partition <new_partition_name>.
This can be useful for this example cases: need to merge all one-month 
partitions into a year partition or need to merge all one-day partitions 
into a month partition.

Your opinions are very much welcome!

-- 
With best regards,
Dmitry Koval.

  2. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Matthias van de Meent <boekewurm+postgres@gmail.com> — 2022-05-31T10:30:22Z

    On Tue, 31 May 2022 at 11:33, Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi, hackers!
>
> There are not many commands in PostgreSQL for working with partitioned
> tables. This is an obstacle to their widespread use.
> Adding SPLIT PARTITION/MERGE PARTITIONS operations can make easier to
> use partitioned tables in PostgreSQL.

That is quite a nice and useful feature to have.

> (This is especially important when migrating projects from ORACLE DBMS.)
>
> SPLIT PARTITION/MERGE PARTITIONS commands are supported for range
> partitioning (BY RANGE) and for list partitioning (BY LIST).
> For hash partitioning (BY HASH) these operations are not supported.

Just out of curiosity, why is SPLIT / MERGE support not included for
HASH partitions? Because sibling partitions can have a different
modulus, you should be able to e.g. split a partition with (modulus,
remainder) of (3, 1) into two partitions with (mod, rem) of (6, 1) and
(6, 4) respectively, with the reverse being true for merge operations,
right?


Kind regards,

Matthias van de Meent

  3. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Laurenz Albe <laurenz.albe@cybertec.at> — 2022-05-31T11:02:27Z

    On Tue, 2022-05-31 at 12:32 +0300, Dmitry Koval wrote:
> There are not many commands in PostgreSQL for working with partitioned 
> tables. This is an obstacle to their widespread use.
> Adding SPLIT PARTITION/MERGE PARTITIONS operations can make easier to 
> use partitioned tables in PostgreSQL.
> (This is especially important when migrating projects from ORACLE DBMS.)
> 
> SPLIT PARTITION/MERGE PARTITIONS commands are supported for range 
> partitioning (BY RANGE) and for list partitioning (BY LIST).
> For hash partitioning (BY HASH) these operations are not supported.

+1 on the general idea.

At least, it will makes these operations simpler, but probably also less
invasive (no need to detach the affected partitions).


I didn't read the patch, but what lock level does that place on the
partitioned table?  Anything more than ACCESS SHARE?


Yours,
Laurenz Albe

  4. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-05-31T19:43:16Z

     >Just out of curiosity, why is SPLIT / MERGE support not included for
 >HASH partitions? Because sibling partitions can have a different
 >modulus, you should be able to e.g. split a partition with (modulus,
 >remainder) of (3, 1) into two partitions with (mod, rem) of (6, 1) and
 >(6, 4) respectively, with the reverse being true for merge operations,
 >right?

You are right, SPLIT/MERGE operations can be added for HASH-partitioning 
in the future. But HASH-partitioning is rarer than RANGE- and 
LIST-partitioning and I decided to skip it in the first step.
Maybe community will say that SPLIT/MERGE commands are not needed... (At 
first step I would like to make sure that it is no true)

P.S. I attached patch with 1-line warning fix (for cfbot).
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  5. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-05-31T20:22:32Z

    > I didn't read the patch, but what lock level does that place on the
> partitioned table?  Anything more than ACCESS SHARE?

Current patch locks a partitioned table with ACCESS EXCLUSIVE lock. 
Unfortunately only this lock guarantees that other session can not work 
with partitions that are splitting or merging.

I want add CONCURRENTLY mode in future. With this mode partitioned table 
during SPLIT/MERGE operation will be locked with SHARE UPDATE EXCLUSIVE 
(as ATTACH/DETACH PARTITION commands in CONCURRENTLY mode).
But in this case queries from other sessions that want to work with 
partitions that are splitting/merging at this time should receive an 
error (like "Partition data is moving. Repeat the operation later") 
because old partitions will be deleted at the end of SPLIT/MERGE operation.
I hope exists a better solution, but I don't know it now...

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  6. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-05-31T20:43:26Z

    On Tue, May 31, 2022 at 12:43 PM Dmitry Koval <d.koval@postgrespro.ru>
wrote:

>  >Just out of curiosity, why is SPLIT / MERGE support not included for
>  >HASH partitions? Because sibling partitions can have a different
>  >modulus, you should be able to e.g. split a partition with (modulus,
>  >remainder) of (3, 1) into two partitions with (mod, rem) of (6, 1) and
>  >(6, 4) respectively, with the reverse being true for merge operations,
>  >right?
>
> You are right, SPLIT/MERGE operations can be added for HASH-partitioning
> in the future. But HASH-partitioning is rarer than RANGE- and
> LIST-partitioning and I decided to skip it in the first step.
> Maybe community will say that SPLIT/MERGE commands are not needed... (At
> first step I would like to make sure that it is no true)
>
> P.S. I attached patch with 1-line warning fix (for cfbot).
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com


Hi,
For attachPartTable, the parameter wqueue is missing from comment.
The parameters of CloneRowTriggersToPartition are called parent
and partition. I think it is better to name the parameters to
attachPartTable in a similar manner.

For struct SplitPartContext, SplitPartitionContext would be better name.

+       /* Store partition contect into list. */
contect -> context

Cheers

  7. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-05-31T22:14:25Z

    On Tue, May 31, 2022 at 1:43 PM Zhihong Yu <zyu@yugabyte.com> wrote:

>
>
> On Tue, May 31, 2022 at 12:43 PM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
>
>>  >Just out of curiosity, why is SPLIT / MERGE support not included for
>>  >HASH partitions? Because sibling partitions can have a different
>>  >modulus, you should be able to e.g. split a partition with (modulus,
>>  >remainder) of (3, 1) into two partitions with (mod, rem) of (6, 1) and
>>  >(6, 4) respectively, with the reverse being true for merge operations,
>>  >right?
>>
>> You are right, SPLIT/MERGE operations can be added for HASH-partitioning
>> in the future. But HASH-partitioning is rarer than RANGE- and
>> LIST-partitioning and I decided to skip it in the first step.
>> Maybe community will say that SPLIT/MERGE commands are not needed... (At
>> first step I would like to make sure that it is no true)
>>
>> P.S. I attached patch with 1-line warning fix (for cfbot).
>> --
>> With best regards,
>> Dmitry Koval
>>
>> Postgres Professional: http://postgrespro.com
>
>
> Hi,
> For attachPartTable, the parameter wqueue is missing from comment.
> The parameters of CloneRowTriggersToPartition are called parent
> and partition. I think it is better to name the parameters to
> attachPartTable in a similar manner.
>
> For struct SplitPartContext, SplitPartitionContext would be better name.
>
> +       /* Store partition contect into list. */
> contect -> context
>
> Cheers
>
Hi,
For transformPartitionCmdForMerge(), nested loop is used to detect
duplicate names.
If the number of partitions in partcmd->partlist, we should utilize map to
speed up the check.

For check_parent_values_in_new_partitions():

+           if (!find_value_in_new_partitions(&key->partsupfunc[0],
+                                             key->partcollation, parts,
nparts, datum, false))
+               found = false;

It seems we can break out of the loop when found is false.

Cheers

  8. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-06-01T18:58:42Z

    Hi,

1)
> For attachPartTable, the parameter wqueue is missing from comment.
> The parameters of CloneRowTriggersToPartition are called parent and partition.
> I think it is better to name the parameters to attachPartTable in a similar manner.
> 
> For struct SplitPartContext, SplitPartitionContext would be better name.
> 
> +       /* Store partition contect into list. */
> contect -> context

Thanks, changed.

2)
> For transformPartitionCmdForMerge(), nested loop is used to detect duplicate names.
> If the number of partitions in partcmd->partlist, we should utilize map to speed up the check.

I'm not sure what we should utilize map in this case because chance that 
number of merging partitions exceed dozens is low.
Is there a function example that uses a map for such a small number of 
elements?

3)
> For check_parent_values_in_new_partitions():
> 
> +           if (!find_value_in_new_partitions(&key->partsupfunc[0],
> +                                             key->partcollation, parts, nparts, datum, false))
> +               found = false;
> 
> It seems we can break out of the loop when found is false.

We have implicit "break" in "for" construction:

+	for (i = 0; i < boundinfo->ndatums && found; i++)

I'll change it to explicit "break;" to avoid confusion.


Attached patch with the changes described above.
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  9. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-06-01T19:10:22Z

    On Wed, Jun 1, 2022 at 11:58 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:

> Hi,
>
> 1)
> > For attachPartTable, the parameter wqueue is missing from comment.
> > The parameters of CloneRowTriggersToPartition are called parent and
> partition.
> > I think it is better to name the parameters to attachPartTable in a
> similar manner.
> >
> > For struct SplitPartContext, SplitPartitionContext would be better name.
> >
> > +       /* Store partition contect into list. */
> > contect -> context
>
> Thanks, changed.
>
> 2)
> > For transformPartitionCmdForMerge(), nested loop is used to detect
> duplicate names.
> > If the number of partitions in partcmd->partlist, we should utilize map
> to speed up the check.
>
> I'm not sure what we should utilize map in this case because chance that
> number of merging partitions exceed dozens is low.
> Is there a function example that uses a map for such a small number of
> elements?
>
> 3)
> > For check_parent_values_in_new_partitions():
> >
> > +           if (!find_value_in_new_partitions(&key->partsupfunc[0],
> > +                                             key->partcollation, parts,
> nparts, datum, false))
> > +               found = false;
> >
> > It seems we can break out of the loop when found is false.
>
> We have implicit "break" in "for" construction:
>
> +       for (i = 0; i < boundinfo->ndatums && found; i++)
>
> I'll change it to explicit "break;" to avoid confusion.
>
>
> Attached patch with the changes described above.
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com

Hi,
Thanks for your response.

w.r.t. #2, I think using nested loop is fine for now.
If, when this feature is merged, some user comes up with long merge list,
we can revisit this topic.

Cheers

  10. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-07-13T18:27:44Z

    Hi!

Patch stop applying due to changes in upstream.
Here is a rebased version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  11. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-07-13T19:03:46Z

    On Wed, Jul 13, 2022 at 11:28 AM Dmitry Koval <d.koval@postgrespro.ru>
wrote:

> Hi!
>
> Patch stop applying due to changes in upstream.
> Here is a rebased version.
>
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com

Hi,

+attachPartTable(List **wqueue, Relation rel, Relation partition,
PartitionBoundSpec *bound)

I checked naming of existing methods, such as AttachPartitionEnsureIndexes.
I think it would be better if the above method is
named attachPartitionTable.

+   if (!defaultPartCtx && OidIsValid(defaultPartOid))
+   {
+       pc = createSplitPartitionContext(table_open(defaultPartOid,
AccessExclusiveLock));

Since the value of pc would be passed to defaultPartCtx, there is no need
to assign to pc above. You can assign directly to defaultPartCtx.

+   /* Drop splitted partition. */

splitted -> split

+   /* Rename new partition if it is need. */

need -> needed.

Cheers

  12. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-07-13T20:05:44Z

    Thanks you!
I've fixed all things mentioned.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  13. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-07-13T20:17:30Z

    On Wed, Jul 13, 2022 at 1:05 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:

> Thanks you!
> I've fixed all things mentioned.
>
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com

Hi,
Toward the end of ATExecSplitPartition():

+       /* Unlock new partition. */
+       table_close(newPartRel, NoLock);

 Why is NoLock passed (instead of AccessExclusiveLock) ?

Cheers

  14. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-07-13T20:33:45Z

    > +       /* Unlock new partition. */
> +       table_close(newPartRel, NoLock);
> 
>   Why is NoLock passed (instead of AccessExclusiveLock) ?

Thanks!

You're right, I replaced the comment with "Keep the lock until commit.".

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  15. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2022-07-14T08:12:14Z

    This is not a review, but I think the isolation tests should be
expanded.  At least, include the case of serializable transactions being
involved.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"Pensar que el espectro que vemos es ilusorio no lo despoja de espanto,
sólo le suma el nuevo terror de la locura" (Perelandra, C.S. Lewis)

  16. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-07-15T11:00:48Z

    > This is not a review, but I think the isolation tests should be
> expanded.  At least, include the case of serializable transactions being
> involved.

Thanks!
I will expand the tests for the next commitfest.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  17. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-08-11T06:56:37Z

    Hi!

Patch stop applying due to changes in upstream.
Here is a rebased version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  18. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-08-29T16:56:47Z

    > I will expand the tests for the next commitfest.

Hi!

Combinations of isolation modes (READ COMMITTED/REPEATABLE 
READ/SERIALIZABLE) were added to test

src/test/isolation/specs/partition-split-merge.spec

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  19. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-09-07T17:03:09Z

    Hi!

Patch stop applying due to changes in upstream.
Here is a rebased version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  20. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2022-09-07T18:43:34Z

    On Wed, Sep 07, 2022 at 08:03:09PM +0300, Dmitry Koval wrote:
> Hi!
> 
> Patch stop applying due to changes in upstream.
> Here is a rebased version.

This crashes on freebsd with -DRELCACHE_FORCE_RELEASE
https://cirrus-ci.com/task/6565371623768064
https://cirrus-ci.com/task/6145355992530944

Note that that's a modified cirrus script from my CI improvements branch
which also does some extra/different things.

-- 
Justin

  21. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-09-08T11:35:24Z

    Thanks a lot Justin!

After compilation PostgreSQL+patch with macros
RELCACHE_FORCE_RELEASE,
COPY_PARSE_PLAN_TREES,
WRITE_READ_PARSE_PLAN_TREES,
RAW_EXPRESSION_COVERAGE_TEST,
RANDOMIZE_ALLOCATED_MEMORY,
I saw a problem on Windows 10, MSVC2019.

(I hope this problem was the same as on Cirrus CI).

Attached patch with fix.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  22. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2022-09-08T12:26:04Z

    On Thu, Sep 08, 2022 at 02:35:24PM +0300, Dmitry Koval wrote:
> Thanks a lot Justin!
> 
> After compilation PostgreSQL+patch with macros
> RELCACHE_FORCE_RELEASE,
> RANDOMIZE_ALLOCATED_MEMORY,
> I saw a problem on Windows 10, MSVC2019.

Yes, it passes tests on my CI improvements branch.
https://github.com/justinpryzby/postgres/runs/8248668269
Thanks to Alexander Pyhalov for reminding me about
RELCACHE_FORCE_RELEASE last year ;)

On Tue, May 31, 2022 at 12:32:43PM +0300, Dmitry Koval wrote:
> This can be useful for this example cases: 
> need to merge all one-day partitions
> into a month partition.

+1, we would use this (at least the MERGE half).

I wonder if it's possible to reduce the size of this patch (I'm starting
to try to absorb it).  Is there a way to refactor/reuse existing code to
reduce its footprint ?

partbounds.c is adding 500+ LOC about checking if proposed partitions
meet the requirements (don't overlap, etc).  But a lot of those checks
must already happen, no?  Can you re-use/refactor the existing checks ?

An UPDATE on a partitioned table will move tuples from one partition to
another.  Is there a way to re-use that ?  Also, postgres already
supports concurrent DDL (CREATE+ATTACH and DETACH CONCURRENTLY).  Is it 
possible to leverage that ?  (Mostly to reduce the patch size, but also
because maybe some cases could be concurrent?).

If the patch were split into separate parts for MERGE and SPLIT, would
the first patch be significantly smaller than the existing patch
(hopefully half as big) ?  That would help to review it, even if both
halves were ultimately squished together.  (An easy way to do this is to
open up all the files in separate editor instances, trim out the parts
that aren't needed for the first patch, save the files but don't quit
the editors, test compilation and regression tests, then git commit
--amend -a.  Then in each editor, "undo" all the trimmed changes, save,
and git commit -a).

Would it save much code if "default" partitions weren't handled in the
first patch ?

-- 
Justin

  23. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2022-09-08T14:10:54Z

    On 2022-Sep-08, Justin Pryzby wrote:

> If the patch were split into separate parts for MERGE and SPLIT, would
> the first patch be significantly smaller than the existing patch
> (hopefully half as big) ?  That would help to review it, even if both
> halves were ultimately squished together.  (An easy way to do this is to
> open up all the files in separate editor instances, trim out the parts
> that aren't needed for the first patch, save the files but don't quit
> the editors, test compilation and regression tests, then git commit
> --amend -a.  Then in each editor, "undo" all the trimmed changes, save,
> and git commit -a).

An easier (IMO) way to do that is to use "git gui" or even "git add -p",
which allow you to selectively add changed lines/hunks to the index.
You add a few, commit, then add the rest, commit again.  With "git add
-p" you can even edit individual hunks in an editor in case you have a
mix of both wanted and unwanted in a single hunk (after "s"plitting, of
course), which turns out to be easier than it sounds.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/
"El sudor es la mejor cura para un pensamiento enfermo" (Bardia)

  24. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-09-08T14:26:51Z

    Thanks for your advice, Justin and Alvaro!

I'll try to reduce the size of this patch and split it into separate 
parts (for MERGE and SPLIT).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  25. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-09-19T19:26:28Z

    Hi!

Two separate parts for MERGE and SPLIT partitions (without refactoring; 
it will be later)

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  26. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2022-09-19T19:56:42Z

    On Tue, May 31, 2022 at 5:33 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> There are not many commands in PostgreSQL for working with partitioned
> tables. This is an obstacle to their widespread use.
> Adding SPLIT PARTITION/MERGE PARTITIONS operations can make easier to
> use partitioned tables in PostgreSQL.
> (This is especially important when migrating projects from ORACLE DBMS.)
>
> SPLIT PARTITION/MERGE PARTITIONS commands are supported for range
> partitioning (BY RANGE) and for list partitioning (BY LIST).
> For hash partitioning (BY HASH) these operations are not supported.

This may be a good idea, but I would like to point out one
disadvantage of this approach.

If you know that a certain partition is not changing, and you would
like to split it, you can create two or more new standalone tables and
populate them from the original partition using INSERT .. SELECT. Then
you can BEGIN a transaction, DETACH the existing partitions, and
ATTACH the replacement ones. By doing this, you take an ACCESS
EXCLUSIVE lock on the partitioned table only for a brief period. The
same kind of idea can be used to merge partitions.

It seems hard to do something comparable with built-in DDL for SPLIT
PARTITION and MERGE PARTITION. You could start by taking e.g. SHARE
lock on the existing partition(s) and then wait until the end to take
ACCESS EXCLUSIVE lock on the partitions, but we typically avoid such
coding patterns, because the lock upgrade might deadlock and then a
lot of work would be wasted. So most likely with the approach you
propose here you will end up acquiring ACCESS EXCLUSIVE lock at the
beginning of the operation and then shuffle a lot of data around while
still holding it, which is pretty painful.

Because of this problem, I find it hard to believe that these commands
would get much use, except perhaps on small tables or in
non-production environments, unless people just didn't know about the
alternatives. That's not to say that something like this has no value.
As a convenience feature, it's fine. It's just hard for me to see it
as any more than that.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  27. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-09-19T20:42:38Z

    Thanks for comments and advice!
I thought about this problem and discussed about it with colleagues.
Unfortunately, I don't know of a good general solution.

19.09.2022 22:56, Robert Haas пишет:
> If you know that a certain partition is not changing, and you would
> like to split it, you can create two or more new standalone tables and
> populate them from the original partition using INSERT .. SELECT. Then
> you can BEGIN a transaction, DETACH the existing partitions, and
> ATTACH the replacement ones. By doing this, you take an ACCESS
> EXCLUSIVE lock on the partitioned table only for a brief period. The
> same kind of idea can be used to merge partitions.

But for specific situation like this (certain partition is not changing) 
we can add CONCURRENTLY modifier.
Our DDL query can be like

ALTER TABLE...SPLIT PARTITION [CONCURRENTLY];

With CONCURRENTLY modifier we can lock partitioned table in 
ShareUpdateExclusiveLock mode and split partition - in 
AccessExclusiveLock mode. So we don't lock partitioned table in 
AccessExclusiveLock mode and can modify other partitions during SPLIT 
operation (except split partition).
If smb try to modify split partition, he will receive error "relation 
does not exist" at end of operation (because split partition will be drop).


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  28. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2022-09-20T12:20:52Z

    On Mon, Sep 19, 2022 at 4:42 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Thanks for comments and advice!
> I thought about this problem and discussed about it with colleagues.
> Unfortunately, I don't know of a good general solution.

Yeah, me neither.

> But for specific situation like this (certain partition is not changing)
> we can add CONCURRENTLY modifier.
> Our DDL query can be like
>
> ALTER TABLE...SPLIT PARTITION [CONCURRENTLY];
>
> With CONCURRENTLY modifier we can lock partitioned table in
> ShareUpdateExclusiveLock mode and split partition - in
> AccessExclusiveLock mode. So we don't lock partitioned table in
> AccessExclusiveLock mode and can modify other partitions during SPLIT
> operation (except split partition).
> If smb try to modify split partition, he will receive error "relation
> does not exist" at end of operation (because split partition will be drop).

I think that a built-in DDL command can't really assume that the user
won't modify anything. You'd have to take a ShareLock.

But you might be able to have a CONCURRENTLY variant of the command
that does the same kind of multi-transaction thing as, e.g., CREATE
INDEX CONCURRENTLY. You would probably have to be quite careful about
race conditions (e.g. you commit the first transaction and before you
start the second one, someone drops or detaches the partition you were
planning to merge or split). Might take some thought, but feels
possibly doable. I've never been excited enough about this kind of
thing to want to put a lot of energy into engineering it, because
doing it "manually" feels so much nicer to me, and doubly so given
that we now have ATTACH CONCURRENTLY and DETACH CONCURRENTLY, but it
does seem like a thing some people would probably use and value.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  29. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-10-11T16:21:54Z

    Hi!

Fixed couple warnings (for cfbot).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  30. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-10-11T16:58:01Z

    On Tue, Oct 11, 2022 at 9:22 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:

> Hi!
>
> Fixed couple warnings (for cfbot).
>
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com

Hi,
For v12-0001-PGPRO-ALTER-TABLE-MERGE-PARTITIONS-command.patch:

+       if (equal(name, cmd->name))
+           /* One new partition can have the same name as merged
partition. */
+           isSameName = true;

I think there should be a check before assigning true to isSameName - if
isSameName is true, that means there are two partitions with this same name.

Cheers

  31. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Zhihong Yu <zyu@yugabyte.com> — 2022-10-11T17:15:05Z

    On Tue, Oct 11, 2022 at 9:58 AM Zhihong Yu <zyu@yugabyte.com> wrote:

>
>
> On Tue, Oct 11, 2022 at 9:22 AM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
>
>> Hi!
>>
>> Fixed couple warnings (for cfbot).
>>
>> --
>> With best regards,
>> Dmitry Koval
>>
>> Postgres Professional: http://postgrespro.com
>
> Hi,
> For v12-0001-PGPRO-ALTER-TABLE-MERGE-PARTITIONS-command.patch:
>
> +       if (equal(name, cmd->name))
> +           /* One new partition can have the same name as merged
> partition. */
> +           isSameName = true;
>
> I think there should be a check before assigning true to isSameName - if
> isSameName is true, that means there are two partitions with this same name.
>
> Cheers
>

Pardon - I see that transformPartitionCmdForMerge() compares the partition
names.
Maybe you can add a comment in ATExecMergePartitions referring to
transformPartitionCmdForMerge() so that people can more easily understand
the logic.

  32. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-10-13T08:57:33Z

    Hi!

 >Maybe you can add a comment in ATExecMergePartitions referring to
 >transformPartitionCmdForMerge() so that people can more easily
 >understand the logic.

Thanks, comment added.

Patch stop applying due to changes in upstream.
Here is a fixed version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  33. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2022-11-28T22:30:14Z

    I'm sorry, I couldn't answer earlier...

1.
 > partbounds.c is adding 500+ LOC about checking if proposed partitions
 > meet the requirements (don't overlap, etc).  But a lot of those
 > checks must already happen, no?  Can you re-use/refactor the existing
 > checks ?

I a bit reduced the number of lines in partbounds.c and added comments.
Unfortunately, it is very difficult to re-use existing checks for other 
partitioned tables operations, because mostly part of PostgreSQL 
commands works with a single partition.
So for SPLIT/MERGE commands were created new checks for several partitions.

2.
 > Also, postgres already supports concurrent DDL (CREATE+ATTACH and
 > DETACH CONCURRENTLY).  Is it possible to leverage that ?
 > (Mostly to reduce the patch size, but also because maybe some cases
 > could be concurrent?).

Probably "ATTACH CONCURRENTLY" is not supported?
A few words about "DETACH CONCURRENTLY".
"DETACH CONCURRENTLY" can works because this command not move rows 
during detach partition (and so no reason to block detached partition).
"DETACH CONCURRENTLY" do not changes data, but changes partition 
description (partition is marked as "inhdetachpending = true" etc.).

For SPLIT and MERGE the situation is completely different - these 
commands transfer rows between sections.
Therefore partitions must be LOCKED EXCLUSIVELY during rows transfer.
Probably we can use concurrently partitions not participating in SPLIT 
and MERGE.
But now PostgreSQL has no possibilities to forbid using a part of 
partitions of a partitioned table (until the end of data transfer by 
SPLIT/MERGE commands).
Simple locking is not quite suitable here.
I see only one variant of SPLIT/MERGE CONCURRENTLY implementation that 
can be realized now:

* ShareUpdateExclusiveLock on partitioned table;
* AccessExclusiveLock on partition(s) which will be deleted and will be 
created during SPLIT/MEGRE command;
* transferring data between locked sections; operations with non-blocked 
partitions are allowed;
* sessions which want to use partition(s) which will be deleted, waits 
on locks;
* finally we release AccessExclusiveLock on partition(s) which will be 
deleted and delete them;
* waiting sessions will get errors "relation ... does not exist" (we can 
transform it to "relation structure was changed ... please try again"?).

It doesn't look pretty.
Therefore for the SPLIT/MERGE command the partitioned table is locked 
with AccessExclusiveLock.

3.
 > An UPDATE on a partitioned table will move tuples from one partition
 > to another.  Is there a way to re-use that?

This could be realized using methods that are called from 
ExecCrossPartitionUpdate().
But using these methods is more expensive than the current 
implementation of the SPLIT/MERGE commands.
SPLIT/MERGE commands uses "bulk insert" and there is low overhead for 
finding a partition to insert data: for MERGE is not need to search 
partition; for SPLIT need to use simple search from several partitions 
(listed in the SPLIT command).
Below is a test example.

a. Transferring data from the table "test2" to partitions "partition1" 
and "partition2" using the current implementation of tuple routing in 
PostgreSQL:

CREATE TABLE test (a int, b char(10)) PARTITION BY RANGE (a);
CREATE TABLE partition1 PARTITION OF test FOR VALUES FROM (10) TO (20);
CREATE TABLE partition2 PARTITION OF test FOR VALUES FROM (20) TO (30);
CREATE TABLE test2 (a int, b char(10));
INSERT INTO test2 (a, b) SELECT 11, 'a' FROM generate_series(1, 1000000);
INSERT INTO test2 (a, b) SELECT 22, 'b' FROM generate_series(1, 1000000);
INSERT INTO test(a, b) SELECT a, b FROM test2;
DROP TABLE test2;
DROP TABLE test;

Three attempts (the results are little different), the best result:

INSERT 0 2000000
Time: 4467,814 ms (00:04,468)

b. Transferring data from the partition "partition0" to partitions 
"partition 1" and "partition2" using SPLIT command:

CREATE TABLE test (a int, b char(10)) PARTITION BY RANGE (a);
CREATE TABLE partition0 PARTITION OF test FOR VALUES FROM (0) TO (30);
INSERT INTO test (a, b) SELECT 11, 'a' FROM generate_series(1, 1000000);
INSERT INTO test (a, b) SELECT 22, 'b' FROM generate_series(1, 1000000);
ALTER TABLE test SPLIT PARTITION partition0 INTO
   (PARTITION partition0 FOR VALUES FROM (0) TO (10),
    PARTITION partition1 FOR VALUES FROM (10) TO (20),
    PARTITION partition2 FOR VALUES FROM (20) TO (30));
DROP TABLE test;

Three attempts (the results are little different), the best result:

ALTER TABLE
Time: 3840,127 ms (00:03,840)

So the current implementation of tuple routing is ~16% slower than the 
SPLIT command.
That's quite a lot.


With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  34. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2023-03-19T20:45:13Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            tested, failed

Feature is clearly missing with partition handling in PostgreSQL, so, this patch is very welcome (as are futur steps)
Code presents good, comments are explicit
Patch v14 apply nicely on 4f46f870fa56fa73d6678273f1bd059fdd93d5e6
Compilation ok with meson compile
LCOV after meson test shows good new code coverage.
Documentation is missing in v14.

  35. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-03-28T08:28:05Z

    Hi!

> Documentation:            tested, failed

Added documentation (as separate commit).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  36. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2023-03-28T19:34:33Z

    Hi,

Patch v15-0001-ALTER-TABLE-MERGE-PARTITIONS-command.patch
Apply nicely.
One warning on meson compile (configure -Dssl=openssl -Dldap=enabled -Dauto_features=enabled -DPG_TEST_EXTRA='ssl,ldap,kerberos' -Dbsd_auth=disabled -Dbonjour=disabled -Dpam=disabled -Dpltcl=disabled -Dsystemd=disabled -Dzstd=disabled  -Db_coverage=true)

../../src/pgmergesplit/src/test/modules/test_ddl_deparse/test_ddl_deparse.c: In function ‘get_altertable_subcmdinfo’:
../../src/pgmergesplit/src/test/modules/test_ddl_deparse/test_ddl_deparse.c:112:17: warning: enumeration value ‘AT_MergePartitions’ not handled in switch [-Wswitch]
  112 |                 switch (subcmd->subtype)
      |                 ^~~~~~
Should be the same with 0002...

meson test perfect, patch coverage is very good.

Patch v15-0002-ALTER-TABLE-SPLIT-PARTITION-command.patch
Doesn't apply on 326a33a289c7ba2dbf45f17e610b7be98dc11f67

Patch v15-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch
Apply with one warning  1 line add space error (translate from french "warning: 1 ligne a ajouté des erreurs d'espace").
v15-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch:54: trailing whitespace.
      One of the new partitions <replaceable class="parameter">partition_name1</replaceable>, 
Comment are ok for me. A non native english speaker.
Perhaps you could add some remarks in ddl.html and alter-ddl.html

Stéphane

The new status of this patch is: Waiting on Author

  37. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-03-28T20:43:45Z

    Thank you!

Corrected version in attachment.
Strange that cfbot didn't show this warning ...

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  38. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2023-03-29T10:13:37Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            tested, failed

Hi,
Just a minor warning with documentation patch 
git apply ../v16-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch
../v16-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch:54: trailing whitespace.
      One of the new partitions <replaceable class="parameter">partition_name1</replaceable>, 
warning: 1 ligne a ajouté des erreurs d'espace.
(perhaps due to my Ubuntu 22.04.2 french install)
Everything else is ok.

Thanks a lot for your work
Stéphane

  39. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-03-29T13:32:36Z

    Thanks!

I missed the trailing whitespace.
Corrected.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  40. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Daniel Gustafsson <daniel@yesql.se> — 2023-07-06T16:10:28Z

    This patch no longer applies to master, please submit a rebased version to the
thread. I've marked the CF entry as waiting for author in the meantime.

--
Daniel Gustafsson

  41. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-07-06T18:43:23Z

    Thanks, Daniel!

 > This patch no longer applies to master, please submit a rebased
 > version to the thread.

Here is a rebased version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  42. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2023-07-18T12:51:41Z

    The following review has been posted through the commitfest application:
make installcheck-world:  not tested
Implements feature:       not tested
Spec compliant:           not tested
Documentation:            not tested

Only documentation patch applied on 4e465aac36ce9a9533c68dbdc83e67579880e628
Checked with v18

The new status of this patch is: Waiting on Author

  43. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-07-19T13:43:47Z

    Thank you, Stephane!

Rebased version attached to email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  44. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2023-07-20T11:56:33Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            tested, passed

It is just a rebase
I check with make and meson
run manual split and merge on list and range partition
Doc fits

The new status of this patch is: Ready for Committer

  45. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-11-11T10:26:03Z

    Rebased version attached to email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  46. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2023-12-04T07:52:06Z

    Hello!

Added commit v21-0004-SPLIT-PARTITION-optimization.patch.

Three already existing commits did not change 
(v21-0001-ALTER-TABLE-MERGE-PARTITIONS-command.patch, 
v21-0002-ALTER-TABLE-SPLIT-PARTITION-command.patch, 
v21-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch).

The new commit is an optimization for the SPLIT PARTITION command.

Description of optimization:
1) optimization is used for the SPLIT PARTITION command for tables with 
BY RANGE partitioning in case the partitioning key has a b-tree index;
2) the point of optimization is that, if after dividing of the old 
partition, all its records according to the range conditions must be 
inserted into ONE new partition, then instead of transferring data (from 
the old partition to new partition), the old partition will be renamed.

Example.
Suppose we have a BY RANGE-partitioned table "test" (indexed by 
partitioning key) with a single partition "test_default", which we want 
to split into two partitions ("test_1" and "test_default"), and all 
records should be moved to the "test_1" partition.
When executing the script below, the "test_default" partition will be 
renamed to "test_1".

----
CREATE TABLE test(d date, v text) PARTITION BY RANGE (d);
CREATE TABLE test_default PARTITION OF test DEFAULT;

CREATE INDEX idx_test_d ON test USING btree (d);

INSERT INTO test (d, v)
  SELECT d, 'value_' || md5(random()::text) FROM
   generate_series('2024-01-01', '2024-01-25', interval '10 seconds')
    AS d;

-- Oid of table 'test_default':
SELECT 'test_default'::regclass::oid AS previous_partition_oid;

ALTER TABLE test SPLIT PARTITION test_default INTO
   (PARTITION test_1 FOR VALUES FROM ('2024-01-01') TO ('2024-02-01'),
    PARTITION test_default DEFAULT);

-- Oid of table 'test_1' (should be the same as "previous_partition_oid"):
SELECT 'test_1'::regclass::oid AS current_partition_oid;

DROP TABLE test CASCADE;

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  47. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    vignesh C <vignesh21@gmail.com> — 2024-01-26T12:50:07Z

    On Mon, 4 Dec 2023 at 13:22, Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hello!
>
> Added commit v21-0004-SPLIT-PARTITION-optimization.patch.

CFBot shows that the patch does not apply anymore as in [1]:
=== Applying patches on top of PostgreSQL commit ID
8ba6fdf905d0f5aef70ced4504c6ad297bfe08ea ===
=== applying patch ./v21-0001-ALTER-TABLE-MERGE-PARTITIONS-command.patch
patching file src/backend/commands/tablecmds.c
...
Hunk #7 FAILED at 18735.
Hunk #8 succeeded at 20608 (offset 315 lines).
1 out of 8 hunks FAILED -- saving rejects to file
src/backend/commands/tablecmds.c.rej
patching file src/backend/parser/gram.y

Please post an updated version for the same.

[1] - http://cfbot.cputube.org/patch_46_3659.log

Regards,
Vignesh

  48. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-01-26T14:01:52Z

    On 2024-Jan-26, vignesh C wrote:

> Please post an updated version for the same.

Here's a rebase.  I only fixed the conflicts, didn't review.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/

  49. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-01-26T16:36:33Z

    On 2024-Jan-26, Alvaro Herrera wrote:

> On 2024-Jan-26, vignesh C wrote:
> 
> > Please post an updated version for the same.
> 
> Here's a rebase.  I only fixed the conflicts, didn't review.

Hmm, but I got the attached regression.diffs with it.  I didn't
investigate further, but it looks like the recent changes to replication
identity for partitioned tables has broken the regression tests.

-- 
Álvaro Herrera               48°01'N 7°57'E  —  https://www.EnterpriseDB.com/
"This is what I like so much about PostgreSQL.  Most of the surprises
are of the "oh wow!  That's cool" Not the "oh shit!" kind.  :)"
Scott Marlowe, http://archives.postgresql.org/pgsql-admin/2008-10/msg00152.php

  50. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-01-26T17:08:08Z

    git format-patch -4 HEAD -v 23

=============================

Thanks!

I excluded regression test "Test: split partition witch identity column" 
from script src/test/regress/sql/partition_split.sql because after 
commit [1] partitions cannot contain identity columns and queries

CREATE TABLE salesmans2_5(salesman_id INT GENERATED ALWAYS AS IDENTITY 
PRIMARY KEY, salesman_name VARCHAR(30));
ALTER TABLE salesmans ATTACH PARTITION salesmans2_5 FOR VALUES FROM (2) 
TO (5);

returns

ERROR:  table "salesmans2_5" being attached contains an identity column 
"salesman_id"
DETAIL:  The new partition may not contain an identity column.

[1] 
https://github.com/postgres/postgres/commit/699586315704a8268808e3bdba4cb5924a038c49
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  51. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-01-26T18:36:59Z

    I thought it's wrong to exclude the IDENTITY-column test, so I fixed the 
test and return it back.
Changes in attachment (commit 
v24-0002-ALTER-TABLE-SPLIT-PARTITION-command.patch).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  52. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Andrey Borodin <x4mmm@yandex-team.ru> — 2024-03-08T10:26:17Z

    
> On 26 Jan 2024, at 23:36, Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 
> <v24-0001-ALTER-TABLE-MERGE-PARTITIONS-command.patch><v24-0002-ALTER-TABLE-SPLIT-PARTITION-command.patch><v24-0003-Documentation-for-ALTER-TABLE-SPLIT-PARTITION-ME.patch><v24-0004-SPLIT-PARTITION-optimization.patch>

The CF entry was in Ready for Committer state no so long ago.
Stephane, you might want to review recent version after it was rebased on current HEAD. CFbot's test passed successfully.

Thanks!


Best regards, Andrey Borodin.

  53. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-03-12T16:45:28Z

    Hi!

Rebased version attached to email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  54. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2024-03-19T13:06:41Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, failed
Implements feature:       not tested
Spec compliant:           not tested
Documentation:            not tested

Hi,
I have failing tap test after patches apply:

ok 201       + partition_merge                          2635 ms
not ok 202   + partition_split                          5719 ms

@@ -805,6 +805,7 @@
   (PARTITION salesmans2_3 FOR VALUES FROM (2) TO (3),
    PARTITION salesmans3_4 FOR VALUES FROM (3) TO (4),
    PARTITION salesmans4_5 FOR VALUES FROM (4) TO (5));
+ERROR:  no owned sequence found
 INSERT INTO salesmans (salesman_name) VALUES ('May');
 INSERT INTO salesmans (salesman_name) VALUES ('Ford');
 SELECT * FROM salesmans1_2;
@@ -814,23 +815,17 @@
 (1 row)

 SELECT * FROM salesmans2_3;
- salesman_id | salesman_name 
--------------+---------------
-           2 | Ivanov
-(1 row)
-
+ERROR:  relation "salesmans2_3" does not exist
+LINE 1: SELECT * FROM salesmans2_3;

The new status of this patch is: Waiting on Author

  55. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2024-03-19T14:29:47Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       not tested
Spec compliant:           not tested
Documentation:            not tested

Sorry, tests passed when applying all patches.
I planned to check without optimisation first.

The new status of this patch is: Needs review

  56. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-03-19T14:43:33Z

    Hi!

> The following review has been posted through the commitfest application:
> make installcheck-world:  tested, passed

Thanks for info!
I was unable to reproduce the problem and I wanted to ask for 
clarification. But your message was ahead of my question.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  57. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-03-25T10:28:36Z

    On Tue, Sep 20, 2022 at 3:21 PM Robert Haas <robertmhaas@gmail.com> wrote:
> On Mon, Sep 19, 2022 at 4:42 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > Thanks for comments and advice!
> > I thought about this problem and discussed about it with colleagues.
> > Unfortunately, I don't know of a good general solution.
>
> Yeah, me neither.
>
> > But for specific situation like this (certain partition is not changing)
> > we can add CONCURRENTLY modifier.
> > Our DDL query can be like
> >
> > ALTER TABLE...SPLIT PARTITION [CONCURRENTLY];
> >
> > With CONCURRENTLY modifier we can lock partitioned table in
> > ShareUpdateExclusiveLock mode and split partition - in
> > AccessExclusiveLock mode. So we don't lock partitioned table in
> > AccessExclusiveLock mode and can modify other partitions during SPLIT
> > operation (except split partition).
> > If smb try to modify split partition, he will receive error "relation
> > does not exist" at end of operation (because split partition will be drop).
>
> I think that a built-in DDL command can't really assume that the user
> won't modify anything. You'd have to take a ShareLock.
>
> But you might be able to have a CONCURRENTLY variant of the command
> that does the same kind of multi-transaction thing as, e.g., CREATE
> INDEX CONCURRENTLY. You would probably have to be quite careful about
> race conditions (e.g. you commit the first transaction and before you
> start the second one, someone drops or detaches the partition you were
> planning to merge or split). Might take some thought, but feels
> possibly doable. I've never been excited enough about this kind of
> thing to want to put a lot of energy into engineering it, because
> doing it "manually" feels so much nicer to me, and doubly so given
> that we now have ATTACH CONCURRENTLY and DETACH CONCURRENTLY, but it
> does seem like a thing some people would probably use and value.

+1
Currently people are using external tools to implement this kind of
task.  However, having this functionality in core would be great.
Implementing concurrent merge/split seems quite a difficult task,
which needs careful design.  It might be too hard to carry around the
syntax altogether.  So, I think having basic syntax in-core is a good
step forward.  But I think we need a clear notice in the documentation
about the concurrency to avoid wrong user expectations.

------
Regards,
Alexander Korotkov

  58. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-03-26T23:39:04Z

    On Tue, Mar 19, 2024 at 4:43 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Thanks for info!
> I was unable to reproduce the problem and I wanted to ask for
> clarification. But your message was ahead of my question.

I've revised the patchset.  I mostly did some refactoring, code
improvements and wrote new comments.

If I apply just the first two patches, I get the same error as [1].
This error happens when createPartitionTable() tries to copy the
identity of another partition.  I've fixed that by skipping a copy of
the identity of another partition (remove CREATE_TABLE_LIKE_IDENTITY
from TableLikeClause.options).   BTW, the same error happened to me
when I manually ran CREATE TABLE ... (LIKE ... INCLUDING IDENTITY) for
a partition of the table with identity.  So, this probably deserves a
separate fix, but I think not directly related to this patch.

I have one question.  When merging partitions you're creating a merged
partition like the parent table.   But when splitting a partition
you're creating new partitions like the partition being split.  What
motivates this difference?

Links.
1. https://www.postgresql.org/message-id/171085360143.2046436.7217841141682511557.pgcf%40coridan.postgresql.org

------
Regards,
Alexander Korotkov

  59. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-03-27T20:18:00Z

    Hi!

 > I've fixed that by skipping a copy of the identity of another
 > partition (remove CREATE_TABLE_LIKE_IDENTITY from
 > TableLikeClause.options).

Thanks for correction!
Probably I should have looked at the code more closely after commit [1]. 
I'm also very glad that situation [2] was reproduced.

 > When merging partitions you're creating a merged partition like the
 > parent table.   But when splitting a partition you're creating new
 > partitions like the partition being split.  What motivates this
 > difference?

When splitting a partition, I planned to set parameters for each of the 
new partitions (for example, tablespace parameter).
It would make sense if we want to transfer part of the data of splitting 
partition to a slower (archive) storage device.
Right now I haven't seen any interest in this functionality, so it 
hasn't been implemented yet. But I think this will be needed in the future.

Special thanks for the hint that new structures should be added to the 
list src\tools\pgindent\typedefs.list.

Links.
[1] 
https://github.com/postgres/postgres/commit/699586315704a8268808e3bdba4cb5924a038c49

[2] 
https://www.postgresql.org/message-id/171085360143.2046436.7217841141682511557.pgcf%40coridan.postgresql.org

--
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  60. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-03-30T12:40:43Z

    Hi, Dmitry!

Thank you for your feedback!

On Wed, Mar 27, 2024 at 10:18 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>  > I've fixed that by skipping a copy of the identity of another
>  > partition (remove CREATE_TABLE_LIKE_IDENTITY from
>  > TableLikeClause.options).
>
> Thanks for correction!
> Probably I should have looked at the code more closely after commit [1].
> I'm also very glad that situation [2] was reproduced.
>
>  > When merging partitions you're creating a merged partition like the
>  > parent table.   But when splitting a partition you're creating new
>  > partitions like the partition being split.  What motivates this
>  > difference?
>
> When splitting a partition, I planned to set parameters for each of the
> new partitions (for example, tablespace parameter).
> It would make sense if we want to transfer part of the data of splitting
> partition to a slower (archive) storage device.
> Right now I haven't seen any interest in this functionality, so it
> hasn't been implemented yet. But I think this will be needed in the future.

OK, I've changed the code to use the parent table as a template for
new partitions in split case.  So, now it's the same in both split and
merge cases.

I also added a special note into docs about ACCESS EXCLUSIVE lock,
because I believe that's a significant limitation for usage of this
functionality.

I think 0001, 0002 and 0003 could be considered for pg17.  I will
continue reviewing them.

0004 might require more work.  I didn't rebase it for now.  I suggest
we can rebase it later and consider for pg18.

------
Regards,
Alexander Korotkov

  61. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-03-31T00:56:50Z

    Hi, Alexander!

Thank you very much for your work on refactoring the commits!
Yesterday I received an email from adjkldd@126.com <winterloo@126.com> 
with a proposal for optimization (MERGE PARTITION command) for cases 
where the target partition has a name identical to one of the merging 
partition names.
I think this optimization is worth considering.
A simplified version of the optimization is attached to this letter 
(difference is 10-15 lines).
All changes made in one commit 
(v28-0001-ALTER-TABLE-MERGE-PARTITIONS-command.patch) and in one 
function (ATExecMergePartitions).

In your opinion, should we added this optimization now or should it be 
left for later?

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  62. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-03-31T02:12:19Z

    Hi!

Patch stop applying due to changes in upstream.
Here is a rebased version.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  63. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-04T19:17:45Z

    Hi!

On Sun, Mar 31, 2024 at 5:12 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Patch stop applying due to changes in upstream.
> Here is a rebased version.

I've revised the patchset.  Now there are two self-contained patches
coming with the documentation.  Also, now each command has a paragraph
in the "Data definition" chapter.  Also documentation and tests
contain geographical partitioning with all Russian cities. I think
that might create a country-centric feeling for the reader.   I've
edited that to make cities spread around the world to reflect the
international spirit.  Hope you're OK with this.  Now, both merge and
split commands make new partitions using the parent table as the
template.  And some other edits to comments, commit messages,
documentation etc.

I think this patch is well-reviewed and also has quite straightforward
implementation.  The major limitation of holding ACCESS EXCLUSIVE LOCK
on the parent table is well-documented.  I'm going to push this if no
objections.

------
Regards,
Alexander Korotkov

  64. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-05T13:00:44Z

    Hi!

> I've revised the patchset.

Thanks for the corrections (especially ddl.sgml).
Could you also look at a small optimization for the MERGE PARTITIONS 
command (in a separate file 
v31-0003-Additional-patch-for-ALTER-TABLE-.-MERGE-PARTITI.patch, I wrote 
about it in an email 2024-03-31 00:56:50)?

Files v31-0001-*.patch, v31-0002-*.patch are the same as 
v30-0001-*.patch, v30-0002-*.patch (after rebasing because patch stopped 
applying due to changes in upstream).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  65. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2024-04-05T19:06:03Z

    The following review has been posted through the commitfest application:
make installcheck-world:  tested, passed
Implements feature:       tested, passed
Spec compliant:           tested, passed
Documentation:            tested, passed

All three patches applied nivcely.
Code fits standart, comments are relevant.

The new status of this patch is: Ready for Committer

  66. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-06T22:22:51Z

    Hi, Dmitry!

On Fri, Apr 5, 2024 at 4:00 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > I've revised the patchset.
>
> Thanks for the corrections (especially ddl.sgml).
> Could you also look at a small optimization for the MERGE PARTITIONS
> command (in a separate file
> v31-0003-Additional-patch-for-ALTER-TABLE-.-MERGE-PARTITI.patch, I wrote
> about it in an email 2024-03-31 00:56:50)?
>
> Files v31-0001-*.patch, v31-0002-*.patch are the same as
> v30-0001-*.patch, v30-0002-*.patch (after rebasing because patch stopped
> applying due to changes in upstream).

I've pushed 0001 and 0002.  I didn't push 0003 for the following reasons.
1) This doesn't keep functionality equivalent to 0001.  With 0003, the
merged partition will inherit indexes, constraints, and so on from the
one of merging partitions.
2) This is not necessarily an optimization. Without 0003 indexes on
the merged partition are created after moving the rows in
attachPartitionTable(). With 0003 we merge data into the existing
partition which saves its indexes.  That might cause a significant
performance loss because mass inserts into indexes may be much slower
than building indexes from scratch.
I think both aspects need to be carefully considered.  Even if we
accept them, this needs to be documented.  I think now it's too late
for both of these.  So, this should wait for v18.

------
Regards,
Alexander Korotkov

  67. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-06T22:38:56Z

    Hi, Alexander!

> I didn't push 0003 for the following reasons. ....

Thanks for clarifying. You are right, these are serious reasons.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  68. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-07T19:00:00Z

    Hi Alexander and Dmitry,

07.04.2024 01:22, Alexander Korotkov wrote:
> I've pushed 0001 and 0002.  I didn't push 0003 for the following reasons.

Please try the following (erroneous) query:
CREATE TABLE t1(i int, t text) PARTITION BY LIST (t);
CREATE TABLE t1pa PARTITION OF t1 FOR VALUES IN ('A');

CREATE TABLE t2 (i int, t text) PARTITION BY RANGE (t);
ALTER TABLE t2 SPLIT PARTITION t1pa INTO
   (PARTITION t2a FOR VALUES FROM ('A') TO ('B'),
    PARTITION t2b FOR VALUES FROM ('B') TO ('C'));

that triggers an assertion failure:
TRAP: failed Assert("datums != NIL"), File: "partbounds.c", Line: 3434, PID: 1841459

or a segfault (in a non-assert build):
Program terminated with signal SIGSEGV, Segmentation fault.

#0  pg_detoast_datum_packed (datum=0x0) at fmgr.c:1866
1866            if (VARATT_IS_COMPRESSED(datum) || VARATT_IS_EXTERNAL(datum))
(gdb) bt
#0  pg_detoast_datum_packed (datum=0x0) at fmgr.c:1866
#1  0x000055f38c5d5e3f in bttextcmp (...) at varlena.c:1834
#2  0x000055f38c6030dd in FunctionCall2Coll (...) at fmgr.c:1161
#3  0x000055f38c417c83 in partition_rbound_cmp (...) at partbounds.c:3525
#4  check_partition_bounds_for_split_range (...) at partbounds.c:5221
#5  check_partitions_for_split (...) at partbounds.c:5688
#6  0x000055f38c256c49 in transformPartitionCmdForSplit (...) at parse_utilcmd.c:3451
#7  transformAlterTableStmt (...) at parse_utilcmd.c:3810
#8  0x000055f38c2bdf9c in ATParseTransformCmd (...) at tablecmds.c:5650
...

Best regards,
Alexander

  69. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-07T22:15:06Z

    Hi, Alexander!

On Sun, Apr 7, 2024 at 10:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
> 07.04.2024 01:22, Alexander Korotkov wrote:
> > I've pushed 0001 and 0002.  I didn't push 0003 for the following reasons.
>
> Please try the following (erroneous) query:
> CREATE TABLE t1(i int, t text) PARTITION BY LIST (t);
> CREATE TABLE t1pa PARTITION OF t1 FOR VALUES IN ('A');
>
> CREATE TABLE t2 (i int, t text) PARTITION BY RANGE (t);
> ALTER TABLE t2 SPLIT PARTITION t1pa INTO
>    (PARTITION t2a FOR VALUES FROM ('A') TO ('B'),
>     PARTITION t2b FOR VALUES FROM ('B') TO ('C'));
>
> that triggers an assertion failure:
> TRAP: failed Assert("datums != NIL"), File: "partbounds.c", Line: 3434, PID: 1841459
>
> or a segfault (in a non-assert build):
> Program terminated with signal SIGSEGV, Segmentation fault.
>
> #0  pg_detoast_datum_packed (datum=0x0) at fmgr.c:1866
> 1866            if (VARATT_IS_COMPRESSED(datum) || VARATT_IS_EXTERNAL(datum))
> (gdb) bt
> #0  pg_detoast_datum_packed (datum=0x0) at fmgr.c:1866
> #1  0x000055f38c5d5e3f in bttextcmp (...) at varlena.c:1834
> #2  0x000055f38c6030dd in FunctionCall2Coll (...) at fmgr.c:1161
> #3  0x000055f38c417c83 in partition_rbound_cmp (...) at partbounds.c:3525
> #4  check_partition_bounds_for_split_range (...) at partbounds.c:5221
> #5  check_partitions_for_split (...) at partbounds.c:5688
> #6  0x000055f38c256c49 in transformPartitionCmdForSplit (...) at parse_utilcmd.c:3451
> #7  transformAlterTableStmt (...) at parse_utilcmd.c:3810
> #8  0x000055f38c2bdf9c in ATParseTransformCmd (...) at tablecmds.c:5650

Thank you for spotting this.  This seems like a missing check.  I'm
going to get a closer look at this tomorrow.

------
Regards,
Alexander Korotkov

  70. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-08T04:00:00Z

    08.04.2024 01:15, Alexander Korotkov wrote:
> Thank you for spotting this.  This seems like a missing check.  I'm
> going to get a closer look at this tomorrow.
>

Thanks!

There is also an anomaly with the MERGE command:
CREATE TABLE t1 (i int, a int, b int, c int) PARTITION BY RANGE (a, b);
CREATE TABLE t1p1 PARTITION OF t1 FOR VALUES FROM (1, 1) TO (1, 2);

CREATE TABLE t2 (i int, t text) PARTITION BY RANGE (t);
CREATE TABLE t2pa PARTITION OF t2 FOR VALUES FROM ('A') TO ('C');

CREATE TABLE t3 (i int, t text);

ALTER TABLE t2 MERGE PARTITIONS (t1p1, t2pa, t3) INTO t2p;

leads to:
ERROR:  partition bound for relation "t3" is null
WARNING:  problem in alloc set PortalContext: detected write past chunk end in block 0x55f1ef42f820, chunk 0x55f1ef42ff40
WARNING:  problem in alloc set PortalContext: detected write past chunk end in block 0x55f1ef42f820, chunk 0x55f1ef42ff40

(I'm also not sure that the error message is clear enough (can't we say
"relation X is not a partition of relation Y" in this context, as in
MarkInheritDetached(), for example?).)

Whilst with
ALTER TABLE t2 MERGE PARTITIONS (t1p1, t2pa) INTO t2p;

I get:
Program terminated with signal SIGSEGV, Segmentation fault.

#0  pg_detoast_datum_packed (datum=0x1) at fmgr.c:1866
1866            if (VARATT_IS_COMPRESSED(datum) || VARATT_IS_EXTERNAL(datum))
(gdb) bt
#0  pg_detoast_datum_packed (datum=0x1) at fmgr.c:1866
#1  0x000055d77d00fde2 in bttextcmp (...) at ../../../../src/include/postgres.h:314
#2  0x000055d77d03fa27 in FunctionCall2Coll (...) at fmgr.c:1161
#3  0x000055d77ce1572f in partition_rbound_cmp (...) at partbounds.c:3525
#4  0x000055d77ce157b9 in qsort_partition_rbound_cmp (...) at partbounds.c:3816
#5  0x000055d77d0982ef in qsort_arg (...) at ../../src/include/lib/sort_template.h:316
#6  0x000055d77ce1d109 in calculate_partition_bound_for_merge (...) at partbounds.c:5786
#7  0x000055d77cc24b2b in transformPartitionCmdForMerge (...) at parse_utilcmd.c:3524
#8  0x000055d77cc2b555 in transformAlterTableStmt (...) at parse_utilcmd.c:3812
#9  0x000055d77ccab17c in ATParseTransformCmd (...) at tablecmds.c:5650
#10 0x000055d77ccafd09 in ATExecCmd (...) at tablecmds.c:5589
...

Best regards,
Alexander

  71. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-08T06:16:54Z

    Alexander Lakhin, thanks for the problems you found!

Unfortunately I can't watch them immediately (event [1]).
I will try to start solving them in 12-14 hours.

[1] https://pgconf.ru/2024
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  72. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Tender Wang <tndrwang@gmail.com> — 2024-04-08T10:43:51Z

    Hi all,
  I went through the MERGE/SPLIT partition codes today, thanks for the
works.  I found some grammar errors:
 i. in error messages(Users can see this grammar errors, not friendly).
ii. in codes comments



Alexander Korotkov <aekorotkov@gmail.com> 于2024年4月7日周日 06:23写道：

> Hi, Dmitry!
>
> On Fri, Apr 5, 2024 at 4:00 PM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
> > > I've revised the patchset.
> >
> > Thanks for the corrections (especially ddl.sgml).
> > Could you also look at a small optimization for the MERGE PARTITIONS
> > command (in a separate file
> > v31-0003-Additional-patch-for-ALTER-TABLE-.-MERGE-PARTITI.patch, I wrote
> > about it in an email 2024-03-31 00:56:50)?
> >
> > Files v31-0001-*.patch, v31-0002-*.patch are the same as
> > v30-0001-*.patch, v30-0002-*.patch (after rebasing because patch stopped
> > applying due to changes in upstream).
>
> I've pushed 0001 and 0002.  I didn't push 0003 for the following reasons.
> 1) This doesn't keep functionality equivalent to 0001.  With 0003, the
> merged partition will inherit indexes, constraints, and so on from the
> one of merging partitions.
> 2) This is not necessarily an optimization. Without 0003 indexes on
> the merged partition are created after moving the rows in
> attachPartitionTable(). With 0003 we merge data into the existing
> partition which saves its indexes.  That might cause a significant
> performance loss because mass inserts into indexes may be much slower
> than building indexes from scratch.
> I think both aspects need to be carefully considered.  Even if we
> accept them, this needs to be documented.  I think now it's too late
> for both of these.  So, this should wait for v18.
>
> ------
> Regards,
> Alexander Korotkov
>
>
>

-- 
Tender Wang
OpenPie:  https://en.openpie.com/

  73. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-08T12:00:00Z

    Hi Tender Wang,

08.04.2024 13:43, Tender Wang wrote:
> Hi all,
>   I went through the MERGE/SPLIT partition codes today, thanks for the works.  I found some grammar errors:
>  i. in error messages(Users can see this grammar errors, not friendly).
> ii. in codes comments
>

On a quick glance, I saw also:
NULL-value
partitionde
splited
temparary

And a trailing whitespace at:
      the quarter partition back to monthly partitions:
warning: 1 line adds whitespace errors.

I'm also confused by "administrators" here:
https://www.postgresql.org/docs/devel/ddl-partitioning.html

(We can find on the same page, for instance:
... whereas table inheritance allows data to be divided in a manner of
the user's choosing.
It seems to me, that "users" should work for merging partitions as well.)

Though the documentation addition requires more than just a quick glance,
of course.

Best regards,
Alexander

  74. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-08T20:43:21Z

    Hi!

Attached fix for the problems found by Alexander Lakhin.

About grammar errors.
Unfortunately, I don't know English well.
Therefore, I plan (in the coming days) to show the text to specialists 
who perform technical translation of documentation.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  75. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-09T23:03:40Z

    On Mon, Apr 8, 2024 at 11:43 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Attached fix for the problems found by Alexander Lakhin.
>
> About grammar errors.
> Unfortunately, I don't know English well.
> Therefore, I plan (in the coming days) to show the text to specialists
> who perform technical translation of documentation.

Thank you.  I've pushed this fix with minor corrections from me.

------
Regards,
Alexander Korotkov

  76. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-10T09:00:00Z

    Hello Alexander and Dmitry,

10.04.2024 02:03, Alexander Korotkov wrote:
> On Mon, Apr 8, 2024 at 11:43 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>> Attached fix for the problems found by Alexander Lakhin.
>>
>> About grammar errors.
>> Unfortunately, I don't know English well.
>> Therefore, I plan (in the coming days) to show the text to specialists
>> who perform technical translation of documentation.
> Thank you.  I've pushed this fix with minor corrections from me.

Thank you for fixing that defect!

Please look at an error message emitted for foreign tables:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE FOREIGN TABLE ftp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1)
   SERVER loopback OPTIONS (table_name 'lt_0_1');
CREATE FOREIGN TABLE ftp_1_2 PARTITION OF t
   FOR VALUES FROM (1) TO (2)
   SERVER loopback OPTIONS (table_name 'lt_1_2');
ALTER TABLE t MERGE PARTITIONS (ftp_0_1, ftp_1_2) INTO ftp_0_2;
ERROR:  "ftp_0_1" is not a table

Shouldn't it be more correct/precise?

Best regards,
Alexander

  77. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-10T12:00:00Z

    10.04.2024 12:00, Alexander Lakhin wrote:
> Hello Alexander and Dmitry,
>
> 10.04.2024 02:03, Alexander Korotkov wrote:
>> Thank you.  I've pushed this fix with minor corrections from me.
>

Please look at another anomaly with MERGE.

CREATE TEMP TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_2 PARTITION OF t
   FOR VALUES FROM (0) TO (2);
fails with
ERROR:  cannot create a permanent relation as partition of temporary relation "t"

But
CREATE TEMP TABLE t (i int) PARTITION BY RANGE (i);
CREATE TEMP TABLE tp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1);
CREATE TEMP TABLE tp_1_2 PARTITION OF t
   FOR VALUES FROM (1) TO (2);
ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO tp_0_2;
succeeds and we get:
regression=# \d+ t*
                                     Partitioned table "pg_temp_1.t"
  Column |  Type   | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
  i      | integer |           |          |         | plain |             |              |
Partition key: RANGE (i)
Partitions: tp_0_2 FOR VALUES FROM (0) TO (2)

                                          Table "public.tp_0_2"
  Column |  Type   | Collation | Nullable | Default | Storage | Compression | Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
  i      | integer |           |          |         | plain |             |              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))

Best regards,
Alexander

  78. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-10T17:22:35Z

    Hi!

Alexander Korotkov, thanks for the commit of previous fix.
Alexander Lakhin, thanks for the problem you found.

There are two corrections attached to the letter:

1) v1-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch - fix 
for the problem [1].

2) v1-0002-Fixes-for-english-text.patch - fixes for English text 
(comments, error messages etc.).

Links:
[1] 
https://www.postgresql.org/message-id/dbc8b96c-3cf0-d1ee-860d-0e491da20485%40gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  79. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Richard Guo <guofenglinux@gmail.com> — 2024-04-11T07:57:12Z

    On Thu, Apr 11, 2024 at 1:22 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:

> 2) v1-0002-Fixes-for-english-text.patch - fixes for English text
> (comments, error messages etc.).


FWIW, I also proposed a patch earlier that fixes error messages and
comments in the split partition code at
https://www.postgresql.org/message-id/flat/CAMbWs49DDsknxyoycBqiE72VxzL_sYHF6zqL8dSeNehKPJhkKg%40mail.gmail.com

Thanks
Richard

  80. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-11T08:59:10Z

    Hi!

> FWIW, I also proposed a patch earlier that fixes error messages and
> comments in the split partition code

Sorry, I thought all the fixes you suggested were already included in 
v1-0002-Fixes-for-english-text.patch (but they are not).
Added missing lines to v2-0002-Fixes-for-english-text.patch.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  81. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-11T12:00:00Z

    Hi Dmitry,

11.04.2024 11:59, Dmitry Koval wrote:
>
>> FWIW, I also proposed a patch earlier that fixes error messages and
>> comments in the split partition code
>
> Sorry, I thought all the fixes you suggested were already included in v1-0002-Fixes-for-english-text.patch (but they 
> are not).
> Added missing lines to v2-0002-Fixes-for-english-text.patch.
>

It seems to me that v2-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch
is not complete either.
Take a look, please:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
SET search_path = pg_temp, public;
CREATE TABLE tp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1);
-- fails with:
ERROR:  cannot create a temporary relation as partition of permanent relation "t"

But:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1_2 PARTITION OF t
   FOR VALUES FROM (1) TO (2);
INSERT INTO t VALUES(0), (1);
SELECT * FROM t;
-- the expected result is:
  i
---
  0
  1
(2 rows)

SET search_path = pg_temp, public;
ALTER TABLE t
MERGE PARTITIONS (tp_0_1, tp_1_2) INTO tp_0_2;
-- succeeds, and
\c -
SELECT * FROM t;
-- gives:
  i
---
(0 rows)

Please also ask your tech writers to check contents of src/test/sql/*, if
possible (perhaps, they'll fix "salesmans" and improve grammar).

Best regards,
Alexander

  82. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-11T13:27:40Z

    Hi!

1.
Alexander Lakhin sent a question about index name after MERGE (partition 
name is the same as one of the merged partitions):

----start of quote----
I'm also confused by an index name after MERGE:
CREATE TABLE t (i int) PARTITION BY RANGE (i);

CREATE TABLE tp_0_1 PARTITION OF t FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1_2 PARTITION OF t FOR VALUES FROM (1) TO (2);

CREATE INDEX tidx ON t(i);
ALTER TABLE t MERGE PARTITIONS (tp_1_2, tp_0_1) INTO tp_1_2;
\d+ t*

                                          Table "public.tp_1_2"
  Column |  Type   | Collation | Nullable | Default | Storage | 
Compression | Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
  i      | integer |           |          |         | plain   | 
    |              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
Indexes:
     "merge-16385-3A14B2-tmp_i_idx" btree (i)

Is the name "merge-16385-3A14B2-tmp_i_idx" valid or it's something 
temporary?
----end of quote----

Fix for this case added to file 
v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.

----

2.
 >It seems to me that v2-0001-Fix-for-SPLIT-MERGE-partitions-of-
 >temporary-table.patch is not complete either.

Added correction (and test), see 
v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  83. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-11T14:21:17Z

    Hi, Dmitry!

On Thu, Apr 11, 2024 at 4:27 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 1.
> Alexander Lakhin sent a question about index name after MERGE (partition
> name is the same as one of the merged partitions):
>
> ----start of quote----
> I'm also confused by an index name after MERGE:
> CREATE TABLE t (i int) PARTITION BY RANGE (i);
>
> CREATE TABLE tp_0_1 PARTITION OF t FOR VALUES FROM (0) TO (1);
> CREATE TABLE tp_1_2 PARTITION OF t FOR VALUES FROM (1) TO (2);
>
> CREATE INDEX tidx ON t(i);
> ALTER TABLE t MERGE PARTITIONS (tp_1_2, tp_0_1) INTO tp_1_2;
> \d+ t*
>
>                                           Table "public.tp_1_2"
>   Column |  Type   | Collation | Nullable | Default | Storage |
> Compression | Stats target | Description
> --------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
>   i      | integer |           |          |         | plain   |
>     |              |
> Partition of: t FOR VALUES FROM (0) TO (2)
> Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
> Indexes:
>      "merge-16385-3A14B2-tmp_i_idx" btree (i)
>
> Is the name "merge-16385-3A14B2-tmp_i_idx" valid or it's something
> temporary?
> ----end of quote----
>
> Fix for this case added to file
> v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.
>
> ----
>
> 2.
>  >It seems to me that v2-0001-Fix-for-SPLIT-MERGE-partitions-of-
>  >temporary-table.patch is not complete either.
>
> Added correction (and test), see
> v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.

Thank you, I'll review this later today.

------
Regards,
Alexander Korotkov

  84. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-11T17:00:00Z

    11.04.2024 16:27, Dmitry Koval wrote:
>
> Added correction (and test), see v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.
>

Thank you for the correction, but may be an attempt to merge into implicit
pg_temp should fail just like CREATE TABLE ... PARTITION OF ... does?

Please look also at another anomaly with schemas:
CREATE SCHEMA s1;
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_2 PARTITION OF t
   FOR VALUES FROM (0) TO (2);
ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
   (PARTITION s1.tp0 FOR VALUES FROM (0) TO (1), PARTITION s1.tp1 FOR VALUES FROM (1) TO (2));
results in:
\d+ s1.*
Did not find any relation named "s1.*"
\d+ tp*
                                           Table "public.tp0"
...
                                            Table "public.tp1"
...

Best regards,
Alexander

  85. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-12T01:53:43Z

    On Thu, Apr 11, 2024 at 8:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
> 11.04.2024 16:27, Dmitry Koval wrote:
> >
> > Added correction (and test), see v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.
> >
>
> Thank you for the correction, but may be an attempt to merge into implicit
> pg_temp should fail just like CREATE TABLE ... PARTITION OF ... does?
>
> Please look also at another anomaly with schemas:
> CREATE SCHEMA s1;
> CREATE TABLE t (i int) PARTITION BY RANGE (i);
> CREATE TABLE tp_0_2 PARTITION OF t
>    FOR VALUES FROM (0) TO (2);
> ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
>    (PARTITION s1.tp0 FOR VALUES FROM (0) TO (1), PARTITION s1.tp1 FOR VALUES FROM (1) TO (2));
> results in:
> \d+ s1.*
> Did not find any relation named "s1.*"
> \d+ tp*
>                                            Table "public.tp0"
> ...
>                                             Table "public.tp1"

+1
I think we shouldn't unconditionally copy schema name and
relpersistence from the parent table.  Instead we should throw the
error on a mismatch like CREATE TABLE ... PARTITION OF ... does.  I'm
working on revising this fix.

------
Regards,
Alexander Korotkov

  86. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-04-12T02:20:53Z

    On Thu, Apr 11, 2024 at 9:54 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> I think we shouldn't unconditionally copy schema name and
> relpersistence from the parent table.  Instead we should throw the
> error on a mismatch like CREATE TABLE ... PARTITION OF ... does.  I'm
> working on revising this fix.

We definitely shouldn't copy the schema name from the parent table. It
should be possible to schema-qualify the new partition names, and if
you don't, then the search_path should determine where they get
placed.

But I am inclined to think that relpersistence should be copied. It's
weird that you split an unlogged partition and you get logged
partitions.

One of the things I dislike about this type of feature -- not this
implementation specifically, but just this kind of idea in general --
is that the syntax mentions a whole bunch of tables but in a way where
you can't set their properties. Persistence, reloptions, whatever.
There's just no place to mention any of that stuff - and if you wanted
to create a place, you'd have to invent special syntax for each
separate thing. That's why I think it's good that the normal way of
creating a partition is CREATE TABLE .. PARTITION OF. Because that
way, we know that the full power of the CREATE TABLE statement is
always available, and you can set anything that you could set for a
table that is not a partition.

Of course, that is not to say that some people won't like to have a
feature of this sort. I expect they will. The approach does have some
drawbacks, though.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  87. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-12T13:04:23Z

    Hi!

Attached is a patch with corrections based on comments in previous 
letters (I think these corrections are not final).
I'll be very grateful for feedbacks and bug reports.

11.04.2024 20:00, Alexander Lakhin wrote:
 > may be an attempt to merge into implicit
 > pg_temp should fail just like CREATE TABLE ... PARTITION OF ... does?

Corrected. Result is:

\d+ s1.*
Table "s1.tp0"
...
Table "s1.tp1"
...
\d+ tp*
Did not find any relation named "tp*".


12.04.2024 4:53, Alexander Korotkov wrote:
 > I think we shouldn't unconditionally copy schema name and
 > relpersistence from the parent table. Instead we should throw the
 > error on a mismatch like CREATE TABLE ... PARTITION OF ... does.
12.04.2024 5:20, Robert Haas wrote:
 > We definitely shouldn't copy the schema name from the parent table.

Fixed.

12.04.2024 5:20, Robert Haas wrote:
 > One of the things I dislike about this type of feature -- not this
 > implementation specifically, but just this kind of idea in general --
 > is that the syntax mentions a whole bunch of tables but in a way where
 > you can't set their properties. Persistence, reloptions, whatever.

In next releases I want to allow specifying options (probably, first of 
all, specifying tablespace of the partitions).
But before that, I would like to get a users reaction - what options 
they really need?

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  88. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-12T17:00:00Z

    Hi Dmitry,

12.04.2024 16:04, Dmitry Koval wrote:
> Hi!
>
> Attached is a patch with corrections based on comments in previous letters (I think these corrections are not final).
> I'll be very grateful for feedbacks and bug reports.
>
> 11.04.2024 20:00, Alexander Lakhin wrote:
> > may be an attempt to merge into implicit
> > pg_temp should fail just like CREATE TABLE ... PARTITION OF ... does?
>
> Corrected. Result is:

Thank you!
Still now we're able to create a partition in the pg_temp schema
explicitly. Please try:
ALTER TABLE t
MERGE PARTITIONS (tp_0_1, tp_1_2) INTO pg_temp.tp_0_2;

in the scenario [1] and you'll get the same empty table.

[1] https://www.postgresql.org/message-id/fdaa003e-919c-cbc9-4f0c-e4546e96bd65%40gmail.com

Best regards,
Alexander

  89. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-12T19:59:57Z

    Thanks, Alexander!

> Still now we're able to create a partition in the pg_temp schema
> explicitly.

Attached patches with fix.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  90. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-13T10:04:58Z

    Hi, Dmitry!

On Fri, Apr 12, 2024 at 10:59 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Thanks, Alexander!
>
> > Still now we're able to create a partition in the pg_temp schema
> > explicitly.
>
> Attached patches with fix.

Please, find a my version of this fix attached.  I think we need to
check relpersistence in a similar way ATTACH PARTITION or CREATE TABLE
... PARTITION OF do.  I'm going to polish this a little bit more.

------
Regards,
Alexander Korotkov

  91. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-04-15T14:30:57Z

    On Sat, Apr 13, 2024 at 6:05 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> Please, find a my version of this fix attached.  I think we need to
> check relpersistence in a similar way ATTACH PARTITION or CREATE TABLE
> ... PARTITION OF do.  I'm going to polish this a little bit more.

+ errmsg("\"%s\" is not an ordinary table",

This is not a phrasing that we use in any other error message. We
always just say "is not a table".

+ * Open the new partition and acquire exclusive lock on it.  This will

A minor nitpick is that this should probably say access exclusive
rather than exclusive. But the bigger thing that confuses me here is
that if we just created the partition, surely we must *already* hold
AccessExclusiveLoc on it. No?

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  92. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-15T15:00:00Z

    Hello Robert,

15.04.2024 17:30, Robert Haas wrote:
> On Sat, Apr 13, 2024 at 6:05 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
>> Please, find a my version of this fix attached.  I think we need to
>> check relpersistence in a similar way ATTACH PARTITION or CREATE TABLE
>> ... PARTITION OF do.  I'm going to polish this a little bit more.
> + errmsg("\"%s\" is not an ordinary table",
>
> This is not a phrasing that we use in any other error message. We
> always just say "is not a table".

Initially I was confused by that message, because of:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE FOREIGN TABLE ftp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1)
   SERVER loopback OPTIONS (table_name 'lt_0_1');
CREATE FOREIGN TABLE ftp_1_2 PARTITION OF t
   FOR VALUES FROM (1) TO (2)
   SERVER loopback OPTIONS (table_name 'lt_1_2');
ALTER TABLE t MERGE PARTITIONS (ftp_0_1, ftp_1_2) INTO ftp_0_2;
ERROR:  "ftp_0_1" is not a table
(Isn't a foreign table a table?)

And also:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1);
CREATE TABLE t2 (i int) PARTITION BY RANGE (i);
ALTER TABLE t MERGE PARTITIONS (tp_0_1, t2) INTO tpn;
ERROR:  "t2" is not a table
(Isn't a partitioned table a table?)

And in fact, an ordinary table is not suitable for MERGE anyway:
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_1 PARTITION OF t
   FOR VALUES FROM (0) TO (1);
CREATE TABLE t2 (i int);
ALTER TABLE t MERGE PARTITIONS (tp_0_1, t2) INTO tpn;
ERROR:  "t2" is not a partition

So I don't think that "an ordinary table" is a good (unambiguous) term
either.

Best regards,
Alexander

  93. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-15T15:26:56Z

    Hi!

> Please, find a my version of this fix attached.

Is it possible to make a small addition to the file v6-0001 ... .patch 
(see attachment)?

Most important:
1) Line 19:

+ mergePartName = makeRangeVar(cmd->name->schemaname, tmpRelName, -1);

(temporary table should use the same schema as the partition);

2) Lines 116-123:

+RESET search_path;
+
+-- Can't merge persistent partitions into a temporary partition
+ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO pg_temp.tp_0_2;
+
+SET search_path = pg_temp, public;

(Alexandr Lakhin's test for using of pg_temp schema explicitly).


The rest of the changes in v6_afterfix.diff are not very important and 
can be ignored.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  94. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-04-15T15:38:04Z

    On Mon, Apr 15, 2024 at 11:00 AM Alexander Lakhin <exclusion@gmail.com> wrote:
> Initially I was confused by that message, because of:
> CREATE TABLE t (i int) PARTITION BY RANGE (i);
> CREATE FOREIGN TABLE ftp_0_1 PARTITION OF t
>    FOR VALUES FROM (0) TO (1)
>    SERVER loopback OPTIONS (table_name 'lt_0_1');
> CREATE FOREIGN TABLE ftp_1_2 PARTITION OF t
>    FOR VALUES FROM (1) TO (2)
>    SERVER loopback OPTIONS (table_name 'lt_1_2');
> ALTER TABLE t MERGE PARTITIONS (ftp_0_1, ftp_1_2) INTO ftp_0_2;
> ERROR:  "ftp_0_1" is not a table
> (Isn't a foreign table a table?)

I agree that this can be confusing, but a patch that is about adding
SPLIT and MERGE PARTITION operations cannot decide to also invent a
new error message phraseology and use it only in one place. We need to
maintain consistency across the whole code base.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  95. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-18T10:35:41Z

    Hi, Dmitry!

On Mon, Apr 15, 2024 at 6:26 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
>
> > Please, find a my version of this fix attached.
>
> Is it possible to make a small addition to the file v6-0001 ... .patch
> (see attachment)?
>
> Most important:
> 1) Line 19:
>
> + mergePartName = makeRangeVar(cmd->name->schemaname, tmpRelName, -1);
>
> (temporary table should use the same schema as the partition);
>
> 2) Lines 116-123:
>
> +RESET search_path;
> +
> +-- Can't merge persistent partitions into a temporary partition
> +ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO pg_temp.tp_0_2;
> +
> +SET search_path = pg_temp, public;
>
> (Alexandr Lakhin's test for using of pg_temp schema explicitly).
>
>
> The rest of the changes in v6_afterfix.diff are not very important and
> can be ignored.

Thank you.  I've integrated your changes.

The revised patchset is attached.
1) I've split the fix for the CommandCounterIncrement() issue and the
fix for relation persistence issue into a separate patch.
2) I've validated that the lock on the new partition is held in
createPartitionTable() after ProcessUtility() as pointed out by
Robert.  So, no need to place the lock again.
3) Added fix for problematic error message as a separate patch [1].
4) Added rename "salemans" => "salesmen" for tests as a separate patch.

I think these fixes are reaching committable shape, but I'd like
someone to check it before I push.

Links.
1. https://postgr.es/m/20240408.152402.1485994009160660141.horikyota.ntt%40gmail.com

------
Regards,
Alexander Korotkov

  96. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-18T16:00:00Z

    Hi Alexander,

18.04.2024 13:35, Alexander Korotkov wrote:
>
> The revised patchset is attached.
> 1) I've split the fix for the CommandCounterIncrement() issue and the
> fix for relation persistence issue into a separate patch.
> 2) I've validated that the lock on the new partition is held in
> createPartitionTable() after ProcessUtility() as pointed out by
> Robert.  So, no need to place the lock again.
> 3) Added fix for problematic error message as a separate patch [1].
> 4) Added rename "salemans" => "salesmen" for tests as a separate patch.
>
> I think these fixes are reaching committable shape, but I'd like
> someone to check it before I push.

I think the feature implementation should also provide tab completion for
SPLIT/MERGE.
(ALTER TABLE t S<Tab>
fills in only SET now.)

Also, the following MERGE operation:
CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
ALTER TABLE t MERGE PARTITIONS (tp_0, tp_1) INTO tp_0;

leaves a strange constraint:
\d+ t*
                                           Table "public.tp_0"
...
Not-null constraints:
     "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"

Best regards,
Alexander

  97. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dagfinn Ilmari Mannsåker <ilmari@ilmari.org> — 2024-04-18T17:03:21Z

    Alexander Lakhin <exclusion@gmail.com> writes:

> Hi Alexander,
>
> 18.04.2024 13:35, Alexander Korotkov wrote:
>>
>> The revised patchset is attached.
>> 1) I've split the fix for the CommandCounterIncrement() issue and the
>> fix for relation persistence issue into a separate patch.
>> 2) I've validated that the lock on the new partition is held in
>> createPartitionTable() after ProcessUtility() as pointed out by
>> Robert.  So, no need to place the lock again.
>> 3) Added fix for problematic error message as a separate patch [1].
>> 4) Added rename "salemans" => "salesmen" for tests as a separate patch.
>>
>> I think these fixes are reaching committable shape, but I'd like
>> someone to check it before I push.
>
> I think the feature implementation should also provide tab completion for
> SPLIT/MERGE.
> (ALTER TABLE t S<Tab>
> fills in only SET now.)

Here's a patch for that.  One thing I noticed while testing it was that
the tab completeion for partitions (Query_for_partition_of_table) shows
all the schemas in the DB, even ones that don't contain any partitions
of the table being altered.

- ilmari

  98. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2024-04-18T17:49:22Z

    On 2024-Apr-18, Alexander Lakhin wrote:

> I think the feature implementation should also provide tab completion
> for SPLIT/MERGE.

I don't think that we should be imposing on feature authors or
committers the task of filling in tab-completion for whatever features
they contribute.  I mean, if they want to add that, cool; but if not,
somebody else can do that, too.  It's not a critical piece.

Now, if we're talking about whether a patch to add tab-completion to a
feature post feature-freeze is acceptable, I think it absolutely is
(even though you could claim that it's a new psql feature).  But for
sure we shouldn't mandate that a feature be reverted just because it
lacks tab-completion -- such lack is not an open-item against the
feature in that sense.

-- 
Álvaro Herrera        Breisgau, Deutschland  —  https://www.EnterpriseDB.com/
"That sort of implies that there are Emacs keystrokes which aren't obscure.
I've been using it daily for 2 years now and have yet to discover any key
sequence which makes any sense."                        (Paul Thomas)

  99. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-04-18T19:59:01Z

    On Thu, Apr 18, 2024 at 6:35 AM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> The revised patchset is attached.
> 1) I've split the fix for the CommandCounterIncrement() issue and the
> fix for relation persistence issue into a separate patch.
> 2) I've validated that the lock on the new partition is held in
> createPartitionTable() after ProcessUtility() as pointed out by
> Robert.  So, no need to place the lock again.
> 3) Added fix for problematic error message as a separate patch [1].
> 4) Added rename "salemans" => "salesmen" for tests as a separate patch.
>
> I think these fixes are reaching committable shape, but I'd like
> someone to check it before I push.

Reviewing 0001:

- Seems mostly fine. I think the comment /* Unlock and drop merged
partitions. */ is wrong. I think it should say something like /* Drop
the current partitions before adding the new one. */ because (a) it
doesn't unlock anything, and there's another comment saying that and
(b) we now know that the drop vs. add order matters.

Reviewing 0002:

- Commit message typos: behavious, corresponsing

- Given the change to the header comment of createPartitionTable, it's
rather surprising to me that this patch doesn't touch the
documentation. Isn't that a big change in semantics?

- My previous review comment was really about the code comment, I
believe, rather than the use of AccessExclusiveLock. NoLock is
probably fine, but if it were me I'd be tempted to write
AccessExclusiveLock and just make the comment say something like /* We
should already have the lock, but do it this way just to be certain
*/. But what you have is probably fine, too. Mostly, I want to clarify
the intent of my previous comment.

- Do we, or can we, have a test that if you split a partition that's
not in the search path, the resulting partitions end up in your
creation namespace? And similarly for merge? And maybe also that
schema-qualification works properly?

I haven't exhaustively verified the patch, but these are some things I
noticed when scrolling through it.

Reviewing 0003:

- Are you sure this can't dereference datum when datum is NULL, in
either the upper or lower half? It sure looks strange to have code
that looks like it can make datum a null pointer, and then an
unconditional deference just after.

- In general I think the wording changes are improvements. I'm
slightly suspicious that there might be an even better way to word it,
but I can't think of it right at this very moment.

- I'm kind of unhappy (but not totally unhappy) with the semantics.
Suppose I have a partition that allows values from 0 to 1000, but
actually only contains values that are either between 0 and 99 or
between 901 and 1000. If I try to to split the partition into one that
allows 0..100 and a second that allows 900..1000, it will fail. Maybe
that's good, because that means that if a failure is going to happen,
it will happen right at the beginning, rather than maybe after doing a
lot of work. But on the other hand, it also kind of stinks, because it
feels like I'm being told I can't do something that I know is
perfectly fine.

Reviewing 0004:

- Obviously this is quite trivial and there's no real problem with it,
but if we're changing it anyway, how about a gender-neutral term
(salesperson/salespeople)?

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  100. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-18T20:51:31Z

    Here are some additional fixes to docs.

  101. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-18T23:26:07Z

    Hi!

18.04.2024 19:00, Alexander Lakhin wrote:
> leaves a strange constraint:
> \d+ t*
>                                            Table "public.tp_0"
> ...
> Not-null constraints:
>      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"

Thanks!
Attached fix (with test) for this case.
The patch should be applied after patches
v6-0001- ... .patch ... v6-0004- ... .patch

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  102. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-19T09:00:00Z

    18.04.2024 20:49, Alvaro Herrera wrote:
> On 2024-Apr-18, Alexander Lakhin wrote:
>
>> I think the feature implementation should also provide tab completion
>> for SPLIT/MERGE.
> I don't think that we should be imposing on feature authors or
> committers the task of filling in tab-completion for whatever features
> they contribute.  I mean, if they want to add that, cool; but if not,
> somebody else can do that, too.  It's not a critical piece.

I agree, I just wanted to note the lack of the current implementation.
But now, thanks to Dagfinn, we have the tab completion too.

I have also a question regarding "ALTER TABLE ... SET ACCESS METHOD". The
current documentation says:
When applied to a partitioned table, there is no data to rewrite, but
partitions created afterwards will default to the given access method
unless overridden by a USING clause.

But MERGE/SPLIT behave differently (if one can assume that MERGE/SPLIT
create new partitions under the hood):
CREATE ACCESS METHOD heap2 TYPE TABLE HANDLER heap_tableam_handler;

CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
ALTER TABLE t SET ACCESS METHOD heap2;
CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
\d t+
                                       Partitioned table "public.t"
...
Access method: heap2

                                           Table "public.tp_0"
...
Access method: heap2

                                           Table "public.tp_1"
...
Access method: heap2

ALTER TABLE t MERGE PARTITIONS (tp_0, tp_1) INTO tp_0;
                                       Partitioned table "public.t"
...
Access method: heap2

                                           Table "public.tp_0"
...
Access method: heap

Shouldn't it be changed, what do you think?

Best regards,
Alexander

  103. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-19T11:34:46Z

    On Thu, Apr 11, 2024 at 10:20:53PM -0400, Robert Haas wrote:
> On Thu, Apr 11, 2024 at 9:54 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > I think we shouldn't unconditionally copy schema name and
> > relpersistence from the parent table.  Instead we should throw the
> > error on a mismatch like CREATE TABLE ... PARTITION OF ... does.  I'm
> > working on revising this fix.
> 
> We definitely shouldn't copy the schema name from the parent table. It
> should be possible to schema-qualify the new partition names, and if
> you don't, then the search_path should determine where they get
> placed.

+1.  Alexander Lakhin reported an issue with schemas and SPLIT, and I
noticed an issue with schemas with MERGE.  The issue I hit is occurs
when MERGE'ing into a partition with the same name, and it's fixed like
so:

--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21526,8 +21526,7 @@ ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
 	{
 		/* Create partition table with generated temporary name. */
 		sprintf(tmpRelName, "merge-%u-%X-tmp", RelationGetRelid(rel), MyProcPid);
-		mergePartName = makeRangeVar(get_namespace_name(RelationGetNamespace(rel)),
-									 tmpRelName, -1);
+		mergePartName = makeRangeVar(mergePartName->schemaname, tmpRelName, -1);
 	}
 	createPartitionTable(mergePartName,
 						 makeRangeVar(get_namespace_name(RelationGetNamespace(rel)),

> One of the things I dislike about this type of feature -- not this
> implementation specifically, but just this kind of idea in general --
> is that the syntax mentions a whole bunch of tables but in a way where
> you can't set their properties. Persistence, reloptions, whatever.
> There's just no place to mention any of that stuff - and if you wanted
> to create a place, you'd have to invent special syntax for each
> separate thing. That's why I think it's good that the normal way of
> creating a partition is CREATE TABLE .. PARTITION OF. Because that
> way, we know that the full power of the CREATE TABLE statement is
> always available, and you can set anything that you could set for a
> table that is not a partition.

Right.  The current feature is useful and will probably work for 90% of
people's partitioned tables.

Currently, CREATE TABLE .. PARTITION OF does not create stats objects on
the child table, but MERGE PARTITIONS does, which seems strange.
Maybe stats should not be included on the new child ?

Note that stats on parent table are not analagous to indexes -
partitioned indexes do nothing other than cause indexes to be created on
any new/attached partitions.  But stats objects on the parent 1) cause
extended stats to be collected and computed across the whole partition
heirarchy, and 2) do not cause stats to be computed for the individual
partitions.

Partitions can have different column definitions, for example null
constraints, FKs, defaults.  And currently, if you MERGE partitions,
those will all be lost (or rather, replaced by whatever LIKE parent
gives).  I think that's totally fine - anyone using different defaults
on child tables could either not use MERGE PARTITIONS, or fix up the
defaults afterwards.  There's not much confusion that the details of the
differences between individual partitions will be lost when the
individual partitions are merged and no longer exist.
But I think it'd be useful to document how the new partitions will be
constructed.

-- 
Justin

  104. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-19T13:29:44Z

    On Fri, Apr 19, 2024 at 2:26 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
>
> 18.04.2024 19:00, Alexander Lakhin wrote:
> > leaves a strange constraint:
> > \d+ t*
> >                                            Table "public.tp_0"
> > ...
> > Not-null constraints:
> >      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"
>
> Thanks!
> Attached fix (with test) for this case.
> The patch should be applied after patches
> v6-0001- ... .patch ... v6-0004- ... .patch

I've incorporated this fix with 0001 patch.

Also added to the patchset
005 – tab completion by Dagfinn [1]
006 – draft fix for table AM issue spotted by Alexander Lakhin [2]
007 – doc review by Justin [3]

I'm continuing work on this.

Links
1. https://www.postgresql.org/message-id/87plumiox2.fsf%40wibble.ilmari.org
2. https://www.postgresql.org/message-id/84ada05b-be5c-473e-6d1c-ebe5dd21b190%40gmail.com
3. https://www.postgresql.org/message-id/ZiGH0xc1lxJ71ZfB%40pryzbyj2023

------
Regards,
Alexander Korotkov

  105. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-22T10:31:48Z

    Hi!

On Fri, Apr 19, 2024 at 4:29 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> On Fri, Apr 19, 2024 at 2:26 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > 18.04.2024 19:00, Alexander Lakhin wrote:
> > > leaves a strange constraint:
> > > \d+ t*
> > >                                            Table "public.tp_0"
> > > ...
> > > Not-null constraints:
> > >      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"
> >
> > Thanks!
> > Attached fix (with test) for this case.
> > The patch should be applied after patches
> > v6-0001- ... .patch ... v6-0004- ... .patch
>
> I've incorporated this fix with 0001 patch.
>
> Also added to the patchset
> 005 – tab completion by Dagfinn [1]
> 006 – draft fix for table AM issue spotted by Alexander Lakhin [2]
> 007 – doc review by Justin [3]
>
> I'm continuing work on this.
>
> Links
> 1. https://www.postgresql.org/message-id/87plumiox2.fsf%40wibble.ilmari.org
> 2. https://www.postgresql.org/message-id/84ada05b-be5c-473e-6d1c-ebe5dd21b190%40gmail.com
> 3. https://www.postgresql.org/message-id/ZiGH0xc1lxJ71ZfB%40pryzbyj2023

0001
The way we handle name collisions during MERGE PARTITIONS operation is
reworked by integration of patch [3].  This makes note about commit in
[2] not relevant.

0002
The persistence of the new partition is copied as suggested in [1].
But the checks are in-place, because search_path could influence new
table persistence.  Per review [2], commit message typos are fixed,
documentation is revised, revised tests to cover schema-qualification,
usage of search_path.

0003
Making code more clear that we're not going to dereference the NULL
datum per note in [2].

0004
Gender-neutral terms are used per suggestions in [2].

0005
Commit message revised

0006
Revise documentation mentioning we're going to copy the parent's table
AM.  Regression tests are added.  Commit message revised.

0007
Commit message revised

Links
1. https://www.postgresql.org/message-id/CA%2BTgmoYcjL%2Bw2BQzku5iNXKR5fyxJMSP3avQta8xngioTX7D7A%40mail.gmail.com
2. https://www.postgresql.org/message-id/CA%2BTgmoY_4r6BeeSCTim04nAiCmmXg-1pG1toxQovZOP2qaFJ0A%40mail.gmail.com
3. https://www.postgresql.org/message-id/f8b5cbf5-965e-4e5b-b506-33bbf41b0d50%40postgrespro.ru

------
Regards,
Alexander Korotkov

  106. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-24T20:26:47Z

    On Mon, Apr 22, 2024 at 01:31:48PM +0300, Alexander Korotkov wrote:
> Hi!
> 
> On Fri, Apr 19, 2024 at 4:29 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > On Fri, Apr 19, 2024 at 2:26 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > > 18.04.2024 19:00, Alexander Lakhin wrote:
> > > > leaves a strange constraint:
> > > > \d+ t*
> > > >                                            Table "public.tp_0"
> > > > ...
> > > > Not-null constraints:
> > > >      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"
> > >
> > > Thanks!
> > > Attached fix (with test) for this case.
> > > The patch should be applied after patches
> > > v6-0001- ... .patch ... v6-0004- ... .patch
> >
> > I've incorporated this fix with 0001 patch.
> >
> > Also added to the patchset
> > 005 – tab completion by Dagfinn [1]
> > 006 – draft fix for table AM issue spotted by Alexander Lakhin [2]
> > 007 – doc review by Justin [3]
> >
> > I'm continuing work on this.
> >
> > Links
> > 1. https://www.postgresql.org/message-id/87plumiox2.fsf%40wibble.ilmari.org
> > 2. https://www.postgresql.org/message-id/84ada05b-be5c-473e-6d1c-ebe5dd21b190%40gmail.com
> > 3. https://www.postgresql.org/message-id/ZiGH0xc1lxJ71ZfB%40pryzbyj2023
> 
> 0001
> The way we handle name collisions during MERGE PARTITIONS operation is
> reworked by integration of patch [3].  This makes note about commit in
> [2] not relevant.

This patch also/already fixes the schema issue I reported.  Thanks.

If you wanted to include a test case for that:

begin;
CREATE SCHEMA s;
CREATE SCHEMA t;
CREATE TABLE p(i int) PARTITION BY RANGE(i);
CREATE TABLE s.c1 PARTITION OF p FOR VALUES FROM (1)TO(2);
CREATE TABLE s.c2 PARTITION OF p FOR VALUES FROM (2)TO(3);
ALTER TABLE p MERGE PARTITIONS (s.c1, s.c2) INTO s.c1; -- misbehaves if merging into the same name as an existing partition
\d+ p
...
Partitions: c1 FOR VALUES FROM (1) TO (3)

> 0002
> The persistence of the new partition is copied as suggested in [1].
> But the checks are in-place, because search_path could influence new
> table persistence.  Per review [2], commit message typos are fixed,
> documentation is revised, revised tests to cover schema-qualification,
> usage of search_path.

Subject: [PATCH v8 2/7] Make new partitions with parent's persistence during MERGE/SPLIT operations

This patch adds documentation saying:
+      Any indexes, constraints and user-defined row-level triggers that exist
+      in the parent table are cloned on new partitions [...]

Which is good to say, and addresses part of my message [0]
[0] ZiJW1g2nbQs9ekwK@pryzbyj2023

But it doesn't have anything to do with "creating new partitions with
parent's persistence".  Maybe there was a merge conflict and the docs
ended up in the wrong patch ?

Also, defaults, storage options, compression are also copied.  As will
be anything else from LIKE.  And since anything added in the future will
also be copied, maybe it's better to just say that the tables will be
created the same way as "LIKE .. INCLUDING ALL EXCLUDING ..", or
similar.  Otherwise, the next person who adds a new option for LIKE
would have to remember to update this paragraph...

Also, extended stats objects are currently cloned to new child tables.
But I suggested in [0] that they probably shouldn't be.

> 007 – doc review by Justin [3]

I suggest to drop this patch for now.  I'll send some more minor fixes to
docs and code comments once the other patches are settled.

-- 
Justin

  107. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Pavel Borisov <pashkin.elfe@gmail.com> — 2024-04-26T13:33:33Z

    Hi, Hackers!

On Thu, 25 Apr 2024 at 00:26, Justin Pryzby <pryzby@telsasoft.com> wrote:

> On Mon, Apr 22, 2024 at 01:31:48PM +0300, Alexander Korotkov wrote:
> > Hi!
> >
> > On Fri, Apr 19, 2024 at 4:29 PM Alexander Korotkov <aekorotkov@gmail.com>
> wrote:
> > > On Fri, Apr 19, 2024 at 2:26 AM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
> > > > 18.04.2024 19:00, Alexander Lakhin wrote:
> > > > > leaves a strange constraint:
> > > > > \d+ t*
> > > > >                                            Table "public.tp_0"
> > > > > ...
> > > > > Not-null constraints:
> > > > >      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"
> > > >
> > > > Thanks!
> > > > Attached fix (with test) for this case.
> > > > The patch should be applied after patches
> > > > v6-0001- ... .patch ... v6-0004- ... .patch
> > >
> > > I've incorporated this fix with 0001 patch.
> > >
> > > Also added to the patchset
> > > 005 – tab completion by Dagfinn [1]
> > > 006 – draft fix for table AM issue spotted by Alexander Lakhin [2]
> > > 007 – doc review by Justin [3]
> > >
> > > I'm continuing work on this.
> > >
> > > Links
> > > 1.
> https://www.postgresql.org/message-id/87plumiox2.fsf%40wibble.ilmari.org
> > > 2.
> https://www.postgresql.org/message-id/84ada05b-be5c-473e-6d1c-ebe5dd21b190%40gmail.com
> > > 3.
> https://www.postgresql.org/message-id/ZiGH0xc1lxJ71ZfB%40pryzbyj2023
> >
> > 0001
> > The way we handle name collisions during MERGE PARTITIONS operation is
> > reworked by integration of patch [3].  This makes note about commit in
> > [2] not relevant.
>
> This patch also/already fixes the schema issue I reported.  Thanks.
>
> If you wanted to include a test case for that:
>
> begin;
> CREATE SCHEMA s;
> CREATE SCHEMA t;
> CREATE TABLE p(i int) PARTITION BY RANGE(i);
> CREATE TABLE s.c1 PARTITION OF p FOR VALUES FROM (1)TO(2);
> CREATE TABLE s.c2 PARTITION OF p FOR VALUES FROM (2)TO(3);
> ALTER TABLE p MERGE PARTITIONS (s.c1, s.c2) INTO s.c1; -- misbehaves if
> merging into the same name as an existing partition
> \d+ p
> ...
> Partitions: c1 FOR VALUES FROM (1) TO (3)
>
> > 0002
> > The persistence of the new partition is copied as suggested in [1].
> > But the checks are in-place, because search_path could influence new
> > table persistence.  Per review [2], commit message typos are fixed,
> > documentation is revised, revised tests to cover schema-qualification,
> > usage of search_path.
>
> Subject: [PATCH v8 2/7] Make new partitions with parent's persistence
> during MERGE/SPLIT operations
>
> This patch adds documentation saying:
> +      Any indexes, constraints and user-defined row-level triggers that
> exist
> +      in the parent table are cloned on new partitions [...]
>
> Which is good to say, and addresses part of my message [0]
> [0] ZiJW1g2nbQs9ekwK@pryzbyj2023
>
> But it doesn't have anything to do with "creating new partitions with
> parent's persistence".  Maybe there was a merge conflict and the docs
> ended up in the wrong patch ?
>
> Also, defaults, storage options, compression are also copied.  As will
> be anything else from LIKE.  And since anything added in the future will
> also be copied, maybe it's better to just say that the tables will be
> created the same way as "LIKE .. INCLUDING ALL EXCLUDING ..", or
> similar.  Otherwise, the next person who adds a new option for LIKE
> would have to remember to update this paragraph...
>
> Also, extended stats objects are currently cloned to new child tables.
> But I suggested in [0] that they probably shouldn't be.
>
> > 007 – doc review by Justin [3]
>
> I suggest to drop this patch for now.  I'll send some more minor fixes to
> docs and code comments once the other patches are settled.
>
I've looked at the patchset:

0001 Look good.
0002 Also right with docs modification proposed by Justin.
0003:
Looks like unused code
5268             datum = cmpval ? list_nth(spec->lowerdatums, abs(cmpval) -
1) : NULL;
overridden by
5278                     datum = list_nth(spec->upperdatums, abs(cmpval) -
1);
and
5290                     datum = list_nth(spec->upperdatums, abs(cmpval) -
1);

Otherwise - good.

0004:
I suggest also getting rid of thee-noun compound words like:
salesperson_name. Maybe salesperson -> clerk? Or maybe use the same terms
like in pgbench: branches, tellers, accounts, balance.

0005: Good
0006: Patch is right
In comments:
+      New partitions will have the same table access method,
+      same column names and types as the partitioned table to which they
belong.
(I'd suggest to remove second "same")

Tests are passed. I suppose that it's better to add similar tests for
SPLIT/MERGE PARTITION(S)  to those covering ATTACH/DETACH PARTITION (e.g.:
subscription/t/013_partition.pl and regression tests)

Overall, great work! Thanks!

Regards,
Pavel Borisov,
Supabase.

  108. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-28T00:59:37Z

    Hi, Pavel.

Thank you for the review.

On Fri, Apr 26, 2024 at 4:33 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> I've looked at the patchset:
>
> 0001 Look good.
> 0002 Also right with docs modification proposed by Justin.

Modified as proposed by Justin.  The documentation for the way new
partitions are created is now in separate patch.

> 0003:
> Looks like unused code
> 5268             datum = cmpval ? list_nth(spec->lowerdatums, abs(cmpval) - 1) : NULL;
> overridden by
> 5278                     datum = list_nth(spec->upperdatums, abs(cmpval) - 1);
> and
> 5290                     datum = list_nth(spec->upperdatums, abs(cmpval) - 1);
>
> Otherwise - good.

Fixed, thanks.

> 0004:
> I suggest also getting rid of thee-noun compound words like: salesperson_name. Maybe salesperson -> clerk? Or maybe use the same terms like in pgbench: branches, tellers, accounts, balance.

Thank you, but I'd like to prefer keeping these modifications simple.
It's just regression tests, we don't need to have perfect naming here.
My intention is to fix just obvious errors.

> 0005: Good
> 0006: Patch is right
> In comments:
> +      New partitions will have the same table access method,
> +      same column names and types as the partitioned table to which they belong.
> (I'd suggest to remove second "same")

Documentation is modified per proposal by Justin.  Thus double "same"
is already gone.

> Tests are passed. I suppose that it's better to add similar tests for SPLIT/MERGE PARTITION(S)  to those covering ATTACH/DETACH PARTITION (e.g.: subscription/t/013_partition.pl and regression tests)

The revised patchset is attached.  I'm going to push it if there are
no objections.

Thank you for your suggestions about adding tests similar to
subscription/t/013_partition.pl.  I will work on this after pushing
this patchset.

------
Regards,
Alexander Korotkov
Supabase

  109. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-28T01:04:54Z

    Hi Justin,

Thank you for your review.  Please check v9 of the patchset [1].

On Wed, Apr 24, 2024 at 11:26 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> This patch also/already fixes the schema issue I reported.  Thanks.
>
> If you wanted to include a test case for that:
>
> begin;
> CREATE SCHEMA s;
> CREATE SCHEMA t;
> CREATE TABLE p(i int) PARTITION BY RANGE(i);
> CREATE TABLE s.c1 PARTITION OF p FOR VALUES FROM (1)TO(2);
> CREATE TABLE s.c2 PARTITION OF p FOR VALUES FROM (2)TO(3);
> ALTER TABLE p MERGE PARTITIONS (s.c1, s.c2) INTO s.c1; -- misbehaves if merging into the same name as an existing partition
> \d+ p
> ...
> Partitions: c1 FOR VALUES FROM (1) TO (3)

There is already a test which checks merging into the same name as an
existing partition.  And there are tests with schema-qualified names.
I'm not yet convinced we need a test with both these properties
together.

> > 0002
> > The persistence of the new partition is copied as suggested in [1].
> > But the checks are in-place, because search_path could influence new
> > table persistence.  Per review [2], commit message typos are fixed,
> > documentation is revised, revised tests to cover schema-qualification,
> > usage of search_path.
>
> Subject: [PATCH v8 2/7] Make new partitions with parent's persistence during MERGE/SPLIT operations
>
> This patch adds documentation saying:
> +      Any indexes, constraints and user-defined row-level triggers that exist
> +      in the parent table are cloned on new partitions [...]
>
> Which is good to say, and addresses part of my message [0]
> [0] ZiJW1g2nbQs9ekwK@pryzbyj2023
>
> But it doesn't have anything to do with "creating new partitions with
> parent's persistence".  Maybe there was a merge conflict and the docs
> ended up in the wrong patch ?

Makes sense.  Extracted this into a separate patch in v10.

> Also, defaults, storage options, compression are also copied.  As will
> be anything else from LIKE.  And since anything added in the future will
> also be copied, maybe it's better to just say that the tables will be
> created the same way as "LIKE .. INCLUDING ALL EXCLUDING ..", or
> similar.  Otherwise, the next person who adds a new option for LIKE
> would have to remember to update this paragraph...

Reworded that way.  Thank you.

> Also, extended stats objects are currently cloned to new child tables.
> But I suggested in [0] that they probably shouldn't be.

I will explore this.  Do we copy extended stats when we do CREATE
TABLE ... PARTITION OF?  I think we need to do the same here.

> > 007 – doc review by Justin [3]
>
> I suggest to drop this patch for now.  I'll send some more minor fixes to
> docs and code comments once the other patches are settled.

Your edits are welcome.  Dropped this for now.  And waiting for the
next revision from you.

Links.
1. https://www.postgresql.org/message-id/CAPpHfduYuYECrqpHMgcOsNr%2B4j3uJK%2BJPUJ_zDBn-tqjjh3p1Q%40mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase

  110. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-28T11:00:00Z

    Hello,

28.04.2024 03:59, Alexander Korotkov wrote:
> The revised patchset is attached.  I'm going to push it if there are
> no objections.

I have one additional question regarding security, if you don't mind:
What permissions should a user have to perform split/merge?

When we deal with mixed ownership, say, bob is an owner of a
partitioned table, but not an owner of a partition, should we
allow him to perform merge with that partition?
Consider the following script:
CREATE ROLE alice;
GRANT CREATE ON SCHEMA public TO alice;

SET SESSION AUTHORIZATION alice;
CREATE TABLE t (i int PRIMARY KEY, t text, u text) PARTITION BY RANGE (i);
CREATE TABLE tp_00 PARTITION OF t FOR VALUES FROM (0) TO (10);
CREATE TABLE tp_10 PARTITION OF t FOR VALUES FROM (10) TO (20);

CREATE POLICY p1 ON tp_00 USING (u = current_user);
ALTER TABLE tp_00 ENABLE ROW LEVEL SECURITY;

INSERT INTO t(i, t, u)  VALUES (0, 'info for bob', 'bob');
INSERT INTO t(i, t, u)  VALUES (1, 'info for alice', 'alice');
RESET SESSION AUTHORIZATION;

CREATE ROLE bob;
GRANT CREATE ON SCHEMA public TO bob;
ALTER TABLE t OWNER TO bob;
GRANT SELECT ON TABLE tp_00 TO bob;

SET SESSION AUTHORIZATION bob;
SELECT * FROM tp_00;
--- here bob can see his info only
\d
  Schema | Name  |       Type        | Owner
--------+-------+-------------------+-------
  public | t     | partitioned table | bob
  public | tp_00 | table             | alice
  public | tp_10 | table             | alice

-- but then bob can do:
ALTER TABLE t MERGE PARTITIONS (tp_00, tp_10) INTO tp_00;
-- (yes, he also can detach the partition tp_00, but then he couldn't
-- re-attach nor read it)

\d
  Schema | Name  |       Type        | Owner
--------+-------+-------------------+-------
  public | t     | partitioned table | bob
  public | tp_00 | table             | bob

Thus bob effectively have captured the partition with the data.

What do you think, does this create a new security risk?

Best regards,
Alexander

  111. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-04-28T11:36:51Z

    On Sun, Apr 28, 2024 at 2:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
> 28.04.2024 03:59, Alexander Korotkov wrote:
> > The revised patchset is attached.  I'm going to push it if there are
> > no objections.
>
> I have one additional question regarding security, if you don't mind:
> What permissions should a user have to perform split/merge?
>
> When we deal with mixed ownership, say, bob is an owner of a
> partitioned table, but not an owner of a partition, should we
> allow him to perform merge with that partition?
> Consider the following script:
> CREATE ROLE alice;
> GRANT CREATE ON SCHEMA public TO alice;
>
> SET SESSION AUTHORIZATION alice;
> CREATE TABLE t (i int PRIMARY KEY, t text, u text) PARTITION BY RANGE (i);
> CREATE TABLE tp_00 PARTITION OF t FOR VALUES FROM (0) TO (10);
> CREATE TABLE tp_10 PARTITION OF t FOR VALUES FROM (10) TO (20);
>
> CREATE POLICY p1 ON tp_00 USING (u = current_user);
> ALTER TABLE tp_00 ENABLE ROW LEVEL SECURITY;
>
> INSERT INTO t(i, t, u)  VALUES (0, 'info for bob', 'bob');
> INSERT INTO t(i, t, u)  VALUES (1, 'info for alice', 'alice');
> RESET SESSION AUTHORIZATION;
>
> CREATE ROLE bob;
> GRANT CREATE ON SCHEMA public TO bob;
> ALTER TABLE t OWNER TO bob;
> GRANT SELECT ON TABLE tp_00 TO bob;
>
> SET SESSION AUTHORIZATION bob;
> SELECT * FROM tp_00;
> --- here bob can see his info only
> \d
>   Schema | Name  |       Type        | Owner
> --------+-------+-------------------+-------
>   public | t     | partitioned table | bob
>   public | tp_00 | table             | alice
>   public | tp_10 | table             | alice
>
> -- but then bob can do:
> ALTER TABLE t MERGE PARTITIONS (tp_00, tp_10) INTO tp_00;
> -- (yes, he also can detach the partition tp_00, but then he couldn't
> -- re-attach nor read it)
>
> \d
>   Schema | Name  |       Type        | Owner
> --------+-------+-------------------+-------
>   public | t     | partitioned table | bob
>   public | tp_00 | table             | bob
>
> Thus bob effectively have captured the partition with the data.
>
> What do you think, does this create a new security risk?

Alexander, thank you for discovering this.  I believe that the one who
merges partitions should have permissions for all the partitions
merged.  I'll recheck this and provide the patch.

------
Regards,
Alexander Korotkov

  112. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-28T13:18:42Z

    On Sun, Apr 28, 2024 at 04:04:54AM +0300, Alexander Korotkov wrote:
> Hi Justin,
> 
> Thank you for your review.  Please check v9 of the patchset [1].
> 
> On Wed, Apr 24, 2024 at 11:26 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > This patch also/already fixes the schema issue I reported.  Thanks.
> >
> > If you wanted to include a test case for that:
> >
> > begin;
> > CREATE SCHEMA s;
> > CREATE SCHEMA t;
> > CREATE TABLE p(i int) PARTITION BY RANGE(i);
> > CREATE TABLE s.c1 PARTITION OF p FOR VALUES FROM (1)TO(2);
> > CREATE TABLE s.c2 PARTITION OF p FOR VALUES FROM (2)TO(3);
> > ALTER TABLE p MERGE PARTITIONS (s.c1, s.c2) INTO s.c1; -- misbehaves if merging into the same name as an existing partition
> > \d+ p
> > ...
> > Partitions: c1 FOR VALUES FROM (1) TO (3)
> 
> There is already a test which checks merging into the same name as an
> existing partition.  And there are tests with schema-qualified names.
> I'm not yet convinced we need a test with both these properties
> together.

I mentioned that the combination of schemas and merge-into-same-name is
what currently doesn't work right.

> > Also, extended stats objects are currently cloned to new child tables.
> > But I suggested in [0] that they probably shouldn't be.
> 
> I will explore this.  Do we copy extended stats when we do CREATE
> TABLE ... PARTITION OF?  I think we need to do the same here.

Right, they're not copied because an extended stats objs on the parent
does something different than putting stats objects on each child.
I've convinced myself that it's wrong to copy the parent's stats obj.
If someone wants stats objects on each child, they'll have to handle
them specially after MERGE/SPLIT, just as they would for per-child
defaults/constraints/etc.

On Sun, Apr 28, 2024 at 04:04:54AM +0300, Alexander Korotkov wrote:
> On Wed, Apr 24, 2024 at 11:26 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > This patch adds documentation saying:
> > +      Any indexes, constraints and user-defined row-level triggers that exist
> > +      in the parent table are cloned on new partitions [...]
> >
> > Which is good to say, and addresses part of my message [0]
> > [0] ZiJW1g2nbQs9ekwK@pryzbyj2023
> 
> Makes sense.  Extracted this into a separate patch in v10.

I adjusted the language some and fixed a typo in the commit message.

s/parition/partition/

-- 
Justin

  113. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    David G. Johnston <david.g.johnston@gmail.com> — 2024-04-28T13:42:59Z

    On Sunday, April 28, 2024, Alexander Lakhin <exclusion@gmail.com> wrote:

>
> When we deal with mixed ownership, say, bob is an owner of a
> partitioned table, but not an owner of a partition, should we
> allow him to perform merge with that partition?
>
>
IIUC Merge causes the source tables to be dropped, their data having been
effectively moved into the new partition.  bob must not be allowed to drop
Alice’s tables.  Only an owner may do that.  So if we do allow bob to build
a new partition using his select access, the tables he selected from would
have to remain behind if he is not an owner of them.

David J.

  114. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    David G. Johnston <david.g.johnston@gmail.com> — 2024-04-28T14:09:09Z

    On Sunday, April 28, 2024, Alexander Lakhin <exclusion@gmail.com> wrote:

>
> When we deal with mixed ownership, say, bob is an owner of a
> partitioned table, but not an owner of a partition, should we
> allow him to perform merge with that partition?
>
>
Attaching via alter table requires the user to own both the partitioned
table and the table being acted upon.  Merge needs to behave similarly.

The fact that we let the superuser break the requirement of common
ownership is unfortunate but I guess understandable.  But given the
existing behavior of attach merge should likewise fail if it find the user
doesn’t own the partitions being merged.  The fact that the user can select
from those tables can be acted upon manually if desired; these
administrative commands should all ensure common ownership and fail if that
precondition is not met.

David J.

  115. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-28T14:54:16Z

    On Sun, Apr 28, 2024 at 08:18:42AM -0500, Justin Pryzby wrote:
> > I will explore this.  Do we copy extended stats when we do CREATE
> > TABLE ... PARTITION OF?  I think we need to do the same here.
> 
> Right, they're not copied because an extended stats objs on the parent
> does something different than putting stats objects on each child.
> I've convinced myself that it's wrong to copy the parent's stats obj.
> If someone wants stats objects on each child, they'll have to handle
> them specially after MERGE/SPLIT, just as they would for per-child
> defaults/constraints/etc.

I dug up this thread, in which the idea of copying extended stats from
parent to child was considered some 6 years ago, but never implemented;
for consistency, MERGE/SPLIT shouldn't copy extended stats, either.

https://www.postgresql.org/message-id/20180305195750.aecbpihhcvuskzba%40alvherre.pgsql

-- 
Justin

  116. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-29T18:00:01Z

    Hi Dmitry,

19.04.2024 02:26, Dmitry Koval wrote:
>
> 18.04.2024 19:00, Alexander Lakhin wrote:
>> leaves a strange constraint:
>> \d+ t*
>>                                            Table "public.tp_0"
>> ...
>> Not-null constraints:
>>      "merge-16385-26BCB0-tmp_i_not_null" NOT NULL "i"
>
> Thanks!
> Attached fix (with test) for this case.
> The patch should be applied after patches
> v6-0001- ... .patch ... v6-0004- ... .patch

I still wonder, why that constraint (now with a less questionable name) is
created during MERGE?

That is, before MERGE, two partitions have only PRIMARY KEY indexes,
with no not-null constraint, and you can manually remove the constraint
after MERGE, so maybe it's not necessary...

Best regards,
Alexander

  117. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-30T00:10:47Z

    Hi!

1.
29.04.2024 21:00, Alexander Lakhin wrote:
> I still wonder, why that constraint (now with a less questionable name) is
> created during MERGE?

The SPLIT/MERGE PARTITION(S) commands for creating partitions reuse the 
existing code of CREATE TABLE .. LIKE ... command. A new partition was 
created with the name "merge-16385-26BCB0-tmp" (since there was an old 
partition with the same name). The constraint 
"merge-16385-26BCB0-tmp_i_not_null" was created too together with the 
partition. Subsequently, the table was renamed, but the constraint was not.
Now a new partition is immediately created with the correct name (the 
old partition is renamed).

2.
Just in case, I am attaching a small fix v9_fix.diff for situation [1].

[1] 
https://www.postgresql.org/message-id/0520c72e-8d97-245e-53f9-173beca2ab2e%40gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  118. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-04-30T03:00:00Z

    30.04.2024 03:10, Dmitry Koval wrote:
> Hi!
>
> 1.
> 29.04.2024 21:00, Alexander Lakhin wrote:
>> I still wonder, why that constraint (now with a less questionable name) is
>> created during MERGE?
>
> The SPLIT/MERGE PARTITION(S) commands for creating partitions reuse the existing code of CREATE TABLE .. LIKE ... 
> command. A new partition was created with the name "merge-16385-26BCB0-tmp" (since there was an old partition with the 
> same name). The constraint "merge-16385-26BCB0-tmp_i_not_null" was created too together with the partition. 
> Subsequently, the table was renamed, but the constraint was not.
> Now a new partition is immediately created with the correct name (the old partition is renamed).

Maybe I'm doing something wrong, but the following script:
CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);

CREATE TABLE t2 (LIKE t INCLUDING ALL);
CREATE TABLE tp2 (LIKE tp_0 INCLUDING ALL);
creates tables t2, tp2 without not-null constraints.

But after
ALTER TABLE t MERGE PARTITIONS (tp_0, tp_1) INTO tp_0;
I see:
\d+ tp_0
...
Indexes:
     "tp_0_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
     "tp_0_i_not_null" NOT NULL "i"

Best regards,
Alexander

  119. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-04-30T20:15:05Z

    On Thu, Apr 11, 2024 at 08:00:00PM +0300, Alexander Lakhin wrote:
> 11.04.2024 16:27, Dmitry Koval wrote:
> > 
> > Added correction (and test), see v3-0001-Fix-for-SPLIT-MERGE-partitions-of-temporary-table.patch.
> 
> Thank you for the correction, but may be an attempt to merge into implicit
> pg_temp should fail just like CREATE TABLE ... PARTITION OF ... does?
> 
> Please look also at another anomaly with schemas:
> CREATE SCHEMA s1;
> CREATE TABLE t (i int) PARTITION BY RANGE (i);
> CREATE TABLE tp_0_2 PARTITION OF t
>   FOR VALUES FROM (0) TO (2);
> ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
>   (PARTITION s1.tp0 FOR VALUES FROM (0) TO (1), PARTITION s1.tp1 FOR VALUES FROM (1) TO (2));
> results in:
> \d+ s1.*
> Did not find any relation named "s1.*"
> \d+ tp*
>                                           Table "public.tp0"

Hi,

Is this issue already fixed ?

I wasn't able to reproduce it.  Maybe it only happened with earlier
patch versions applied ?

Thanks,
-- 
Justin

  120. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-04-30T21:14:07Z

    Hi!

30.04.2024 6:00, Alexander Lakhin пишет:
> Maybe I'm doing something wrong, but the following script:
> CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
> CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
> CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
> 
> CREATE TABLE t2 (LIKE t INCLUDING ALL);
> CREATE TABLE tp2 (LIKE tp_0 INCLUDING ALL);
> creates tables t2, tp2 without not-null constraints.

To create partitions is used the "CREATE TABLE ... LIKE ..." command 
with the "EXCLUDING INDEXES" modifier (to speed up the insertion of values).

CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE(i);
CREATE TABLE t2 (LIKE t INCLUDING ALL EXCLUDING INDEXES EXCLUDING IDENTITY);
\d+ t2;
...
Not-null constraints:
     "t2_i_not_null" NOT NULL "i"
Access method: heap


[1] 
https://github.com/postgres/postgres/blob/d12b4ba1bd3eedd862064cf1dad5ff107c5cba90/src/backend/commands/tablecmds.c#L21215
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  121. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-05-01T19:51:24Z

    Hi!

30.04.2024 23:15, Justin Pryzby пишет:
> Is this issue already fixed ?
> I wasn't able to reproduce it.  Maybe it only happened with earlier
> patch versions applied ?

I think this was fixed in commit [1].

[1] 
https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  122. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-05-03T13:23:14Z

    On Wed, May 01, 2024 at 10:51:24PM +0300, Dmitry Koval wrote:
> Hi!
> 
> 30.04.2024 23:15, Justin Pryzby пишет:
> > Is this issue already fixed ?
> > I wasn't able to reproduce it.  Maybe it only happened with earlier
> > patch versions applied ?
> 
> I think this was fixed in commit [1].
> 
> [1] https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91

I tried to reproduce it at fcf80c5d5f~, but couldn't.  
I don't see how that patch would fix it anyway.
I'm hoping Alexander can confirm what happened.

The other remaining issues I'm aware of are for EXCLUDING STATISTICS and
refusing to ALTER if the owners don't match.

Note that the error that led to "EXCLUDING IDENTITY" is being discused
over here:
https://www.postgresql.org/message-id/3b8a9dc1-bbc7-0ef5-6863-c432afac7d59@gmail.com

It's possible that once that's addressed, the exclusion should be
removed here, too.

-- 
Justin

  123. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-03T13:32:25Z

    On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Wed, May 01, 2024 at 10:51:24PM +0300, Dmitry Koval wrote:
> > 30.04.2024 23:15, Justin Pryzby пишет:
> > > Is this issue already fixed ?
> > > I wasn't able to reproduce it.  Maybe it only happened with earlier
> > > patch versions applied ?
> >
> > I think this was fixed in commit [1].
> >
> > [1] https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91
>
> I tried to reproduce it at fcf80c5d5f~, but couldn't.
> I don't see how that patch would fix it anyway.
> I'm hoping Alexander can confirm what happened.

This problem is only relevant for an old version of fix [1], which
overrides schemas for new partitions.  That version was never
committed.

> The other remaining issues I'm aware of are for EXCLUDING STATISTICS and
> refusing to ALTER if the owners don't match.

These two are in my list.  I'm planning to work on them in the next few days.

> Note that the error that led to "EXCLUDING IDENTITY" is being discused
> over here:
> https://www.postgresql.org/message-id/3b8a9dc1-bbc7-0ef5-6863-c432afac7d59@gmail.com
>
> It's possible that once that's addressed, the exclusion should be
> removed here, too.

+1

Links.
1. https://www.postgresql.org/message-id/edfbd846-dcc1-42d1-ac26-715691b687d3%40postgrespro.ru

------
Regards,
Alexander Korotkov
Supabase

  124. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-08T18:00:10Z

    On Fri, May 3, 2024 at 4:32 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > On Wed, May 01, 2024 at 10:51:24PM +0300, Dmitry Koval wrote:
> > > 30.04.2024 23:15, Justin Pryzby пишет:
> > > > Is this issue already fixed ?
> > > > I wasn't able to reproduce it.  Maybe it only happened with earlier
> > > > patch versions applied ?
> > >
> > > I think this was fixed in commit [1].
> > >
> > > [1] https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91
> >
> > I tried to reproduce it at fcf80c5d5f~, but couldn't.
> > I don't see how that patch would fix it anyway.
> > I'm hoping Alexander can confirm what happened.
>
> This problem is only relevant for an old version of fix [1], which
> overrides schemas for new partitions.  That version was never
> committed.

Here are the patches.
0001 Adds permission checks on the partitions before doing MERGE/SPLIT
0002 Skips copying extended statistics while creating new partitions
in MERGE/SPLIT

0001 looks quite simple and trivial for me.  I'm going to push it if
no objections.
For 0002 I'd like to hear some feedback on wordings used in docs and comments.

------
Regards,
Alexander Korotkov
Supabase

  125. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-08T19:19:08Z

    On Wed, May 1, 2024 at 12:14 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 30.04.2024 6:00, Alexander Lakhin пишет:
> > Maybe I'm doing something wrong, but the following script:
> > CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE (i);
> > CREATE TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1);
> > CREATE TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
> >
> > CREATE TABLE t2 (LIKE t INCLUDING ALL);
> > CREATE TABLE tp2 (LIKE tp_0 INCLUDING ALL);
> > creates tables t2, tp2 without not-null constraints.
>
> To create partitions is used the "CREATE TABLE ... LIKE ..." command
> with the "EXCLUDING INDEXES" modifier (to speed up the insertion of
values).
>
> CREATE TABLE t (i int, PRIMARY KEY(i)) PARTITION BY RANGE(i);
> CREATE TABLE t2 (LIKE t INCLUDING ALL EXCLUDING INDEXES EXCLUDING
IDENTITY);
> \d+ t2;
> ...
> Not-null constraints:
>      "t2_i_not_null" NOT NULL "i"
> Access method: heap

I've explored this a little bit more.

If the parent table has explicit not null constraint than results of
MERGE/SPLIT look the same as result of CREATE TABLE ... PARTITION OF.  In
every case there is explicit not null constraint in all the cases.

# CREATE TABLE t (i int not null, PRIMARY KEY(i)) PARTITION BY RANGE(i);
# \d+ t
                                      Partitioned table "public.t"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition key: RANGE (i)
Indexes:
    "t_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i"
Number of partitions: 0
# CREATE TABLE tp_0_2 PARTITION OF t FOR VALUES FROM (0) TO (2);
# \d+ tp_0_2
                                         Table "public.tp_0_2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
Indexes:
    "tp_0_2_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i" (inherited)
Access method: heap
# ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
#    (PARTITION tp_0_1 FOR VALUES FROM (0) TO (1),
#     PARTITION tp_1_2 FOR VALUES FROM (1) TO (2))
# \d+ tp_0_1
                                         Table "public.tp_0_1"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (1)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 1))
Indexes:
    "tp_0_1_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "t_i_not_null" NOT NULL "i" (inherited)
Access method: heap

However, if not null constraint is implicit and derived from primary key,
the situation is different.  The partition created by CREATE TABLE ...
PARTITION OF doesn't have explicit not null constraint just like the
parent.  But the partition created by MERGE/SPLIT has explicit not null
contraint.

# CREATE TABLE t (i int not null, PRIMARY KEY(i)) PARTITION BY RANGE(i);
# \d+ t
                                      Partitioned table "public.t"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition key: RANGE (i)
Indexes:
    "t_pkey" PRIMARY KEY, btree (i)
Number of partitions: 0
# CREATE TABLE tp_0_2 PARTITION OF t FOR VALUES FROM (0) TO (2);
# \d+ tp_0_2
                                         Table "public.tp_0_2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (2)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
Indexes:
    "tp_0_2_pkey" PRIMARY KEY, btree (i)
Access method: heap
# ALTER TABLE t SPLIT PARTITION tp_0_2 INTO
#    (PARTITION tp_0_1 FOR VALUES FROM (0) TO (1),
#     PARTITION tp_1_2 FOR VALUES FROM (1) TO (2))
# \d+ tp_0_1
                                         Table "public.tp_0_1"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Partition of: t FOR VALUES FROM (0) TO (1)
Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 1))
Indexes:
    "tp_0_1_pkey" PRIMARY KEY, btree (i)
Not-null constraints:
    "tp_0_1_i_not_null" NOT NULL "i"
Access method: heap

I think this is related to the fact that we create indexes later.  The same
applies to CREATE TABLE ... LIKE.  If we create indexes immediately, not
explicit not null contraints are created.  Not if we do without indexes, we
have an explicit not null constraint.

# CREATE TABLE t2 (LIKE t INCLUDING ALL);
# \d+ t2
                                           Table "public.t2"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Not-null constraints:
    "t2_i_not_null" NOT NULL "i"
Access method: heap
# CREATE TABLE t3 (LIKE t INCLUDING ALL EXCLUDING IDENTITY);
# \d+ t3
                                           Table "public.t3"
 Column |  Type   | Collation | Nullable | Default | Storage | Compression
| Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 i      | integer |           | not null |         | plain   |
|              |
Indexes:
    "t3_pkey" PRIMARY KEY, btree (i)
Access method: heap

I think this is feasible to avoid.  However, it's minor and we exactly
documented how we create new partitions.  So, I think it works "as
documented" and we don't have to fix this for v17.

------
Regards,
Alexander Korotkov
Supabase

  126. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-05-08T21:37:46Z

    On Wed, May 08, 2024 at 09:00:10PM +0300, Alexander Korotkov wrote:
> On Fri, May 3, 2024 at 4:32 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > On Wed, May 01, 2024 at 10:51:24PM +0300, Dmitry Koval wrote:
> > > > 30.04.2024 23:15, Justin Pryzby пишет:
> > > > > Is this issue already fixed ?
> > > > > I wasn't able to reproduce it.  Maybe it only happened with earlier
> > > > > patch versions applied ?
> > > >
> > > > I think this was fixed in commit [1].
> > > >
> > > > [1] https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91
> > >
> > > I tried to reproduce it at fcf80c5d5f~, but couldn't.
> > > I don't see how that patch would fix it anyway.
> > > I'm hoping Alexander can confirm what happened.
> >
> > This problem is only relevant for an old version of fix [1], which
> > overrides schemas for new partitions.  That version was never
> > committed.
> 
> Here are the patches.
> 0002 Skips copying extended statistics while creating new partitions in MERGE/SPLIT
> 
> For 0002 I'd like to hear some feedback on wordings used in docs and comments.

commit message:

Currenlty => Currently
partiions => partitios
copying => by copying

> However, parent's table extended statistics already covers all its
> children.

=> That's the wrong explanation.  It's not that "stats on the parent
table cover its children".  It's that there are two types of stats:
stats for the "table hierarchy" and stats for the individual table.
That's true for single-column stats as well as for extended stats.
In both cases, that's indicated by the inh flag in the code and in the
catalog.

The right explanation is that extended stats on partitioned tables are
not similar to indexes.  Indexes on parent table are nothing other than
a mechanism to create indexes on the child tables.  That's not true for
stats.

See also my prior messages
ZiJW1g2nbQs9ekwK@pryzbyj2023
Zi5Msg74C61DjJKW@pryzbyj2023

I think EXCLUDE IDENTITY can/should now also be removed - see 509199587.
I'm not able to reproduce that problem anyway, even before that...

-- 
Justin

  127. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-08T21:51:32Z

    On Thu, May 9, 2024 at 12:37 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
>
> On Wed, May 08, 2024 at 09:00:10PM +0300, Alexander Korotkov wrote:
> > On Fri, May 3, 2024 at 4:32 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > > On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > > On Wed, May 01, 2024 at 10:51:24PM +0300, Dmitry Koval wrote:
> > > > > 30.04.2024 23:15, Justin Pryzby пишет:
> > > > > > Is this issue already fixed ?
> > > > > > I wasn't able to reproduce it.  Maybe it only happened with earlier
> > > > > > patch versions applied ?
> > > > >
> > > > > I think this was fixed in commit [1].
> > > > >
> > > > > [1] https://github.com/postgres/postgres/commit/fcf80c5d5f0f3787e70fca8fd029d2e08a923f91
> > > >
> > > > I tried to reproduce it at fcf80c5d5f~, but couldn't.
> > > > I don't see how that patch would fix it anyway.
> > > > I'm hoping Alexander can confirm what happened.
> > >
> > > This problem is only relevant for an old version of fix [1], which
> > > overrides schemas for new partitions.  That version was never
> > > committed.
> >
> > Here are the patches.
> > 0002 Skips copying extended statistics while creating new partitions in MERGE/SPLIT
> >
> > For 0002 I'd like to hear some feedback on wordings used in docs and comments.
>
> commit message:
>
> Currenlty => Currently
> partiions => partitios
> copying => by copying


Thank you!

>
> > However, parent's table extended statistics already covers all its
> > children.
>
> => That's the wrong explanation.  It's not that "stats on the parent
> table cover its children".  It's that there are two types of stats:
> stats for the "table hierarchy" and stats for the individual table.
> That's true for single-column stats as well as for extended stats.
> In both cases, that's indicated by the inh flag in the code and in the
> catalog.
>
> The right explanation is that extended stats on partitioned tables are
> not similar to indexes.  Indexes on parent table are nothing other than
> a mechanism to create indexes on the child tables.  That's not true for
> stats.
>
> See also my prior messages
> ZiJW1g2nbQs9ekwK@pryzbyj2023
> Zi5Msg74C61DjJKW@pryzbyj2023

Yes, I understand that parents pg_statistic entry with stainherit ==
true includes statistics for the children.  I tried to express this by
word "covers".  But you're right, this is the wrong explanation.

Can I, please, ask you to revise the patch?

> I think EXCLUDE IDENTITY can/should now also be removed - see 509199587.
> I'm not able to reproduce that problem anyway, even before that...

I will check this.

------
Regards,
Alexander Korotkov
Supabase

  128. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-05-11T09:00:00Z

    Hello Dmitry and Alexander,

Please look at one more anomaly with temporary tables:
CREATE TEMP TABLE t (a int) PARTITION BY RANGE (a);
CREATE TEMP TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (1) ;
CREATE TEMP TABLE tp_1 PARTITION OF t FOR VALUES FROM (1) TO (2);
ALTER TABLE t MERGE PARTITIONS (tp_0, tp_1) INTO tp_0;
-- succeeds, but:
ALTER TABLE t SPLIT PARTITION tp_0 INTO
   (PARTITION tp_0 FOR VALUES FROM (0) TO (1),  PARTITION tp_1 FOR VALUES FROM (1) TO (2));
-- fails with:
ERROR:  relation "tp_0" already exists

Though the same SPLIT succeeds with non-temporary tables...

Best regards,
Alexander

  129. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-05-11T13:19:38Z

    Hi!

11.05.2024 12:00, Alexander Lakhin wrote:
> Please look at one more anomaly with temporary tables:

Thank you, Alexander!

The problem affects the SPLIT PARTITION command.

CREATE TEMP TABLE t (a int) PARTITION BY RANGE (a);
CREATE TEMP TABLE tp_0 PARTITION OF t FOR VALUES FROM (0) TO (2) ;
-- ERROR:  relation "tp_0" already exists
ALTER TABLE t SPLIT PARTITION tp_0 INTO
    (PARTITION tp_0 FOR VALUES FROM (0) TO (1),  PARTITION tp_1 FOR 
VALUES FROM (1) TO (2));

I'll try to fix it soon.
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  130. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-05-12T14:43:40Z

    Hi!

Attached draft version of fix for [1].

[1] 
https://www.postgresql.org/message-id/86b4f1e3-0b5d-315c-9225-19860d64d685%40gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  131. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Daniel Gustafsson <daniel@yesql.se> — 2024-05-13T08:45:57Z

    Commit 3ca43dbbb67f which adds the permission checks seems to cause conflicts
in the pg_upgrade tests:

https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=piculet&dt=2024-05-13%2008%3A36%3A37

There is an issue with dropping and creating roles which seems to stem from
this commit:

 CREATE ROLE regress_partition_merge_alice;
+ERROR: role "regress_partition_merge_alice" already exists

--
Daniel Gustafsson

  132. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-05-13T09:45:49Z

    Hi!

13.05.2024 11:45, Daniel Gustafsson пишет:
> Commit 3ca43dbbb67f which adds the permission checks seems to cause conflicts
> in the pg_upgrade tests

Thanks!

It will probably be enough to rename the roles:

regress_partition_merge_alice -> regress_partition_split_alice
regress_partition_merge_bob -> regress_partition_split_bob

(changes in attachment)
-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  133. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-13T10:37:31Z

    On Mon, May 13, 2024 at 12:45 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 13.05.2024 11:45, Daniel Gustafsson пишет:
> > Commit 3ca43dbbb67f which adds the permission checks seems to cause conflicts
> > in the pg_upgrade tests
>
> Thanks!
>
> It will probably be enough to rename the roles:
>
> regress_partition_merge_alice -> regress_partition_split_alice
> regress_partition_merge_bob -> regress_partition_split_bob

Thanks to Danial for spotting this.
Thanks to Dmitry for the proposed fix.

The actual problem appears to be a bit more complex.  Additionally to
the role names, the lack of permissions on schemas lead to creation of
tables in public schema and potential conflict there.  Fixed in
2a679ae94e.

------
Regards,
Alexander Korotkov

  134. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-05-14T14:49:53Z

    On Thu, May 09, 2024 at 12:51:32AM +0300, Alexander Korotkov wrote:
> > > However, parent's table extended statistics already covers all its
> > > children.
> >
> > => That's the wrong explanation.  It's not that "stats on the parent
> > table cover its children".  It's that there are two types of stats:
> > stats for the "table hierarchy" and stats for the individual table.
> > That's true for single-column stats as well as for extended stats.
> > In both cases, that's indicated by the inh flag in the code and in the
> > catalog.
> >
> > The right explanation is that extended stats on partitioned tables are
> > not similar to indexes.  Indexes on parent table are nothing other than
> > a mechanism to create indexes on the child tables.  That's not true for
> > stats.
> >
> > See also my prior messages
> > ZiJW1g2nbQs9ekwK@pryzbyj2023
> > Zi5Msg74C61DjJKW@pryzbyj2023
> 
> Yes, I understand that parents pg_statistic entry with stainherit ==
> true includes statistics for the children.  I tried to express this by
> word "covers".  But you're right, this is the wrong explanation.
> 
> Can I, please, ask you to revise the patch?

I tried to make this clear but it'd be nice if someone (Tomas/Alvaro?)
would check that this says what's wanted.

-- 
Justin

  135. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-17T10:05:01Z

    On Tue, May 14, 2024 at 5:49 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Thu, May 09, 2024 at 12:51:32AM +0300, Alexander Korotkov wrote:
> > > > However, parent's table extended statistics already covers all its
> > > > children.
> > >
> > > => That's the wrong explanation.  It's not that "stats on the parent
> > > table cover its children".  It's that there are two types of stats:
> > > stats for the "table hierarchy" and stats for the individual table.
> > > That's true for single-column stats as well as for extended stats.
> > > In both cases, that's indicated by the inh flag in the code and in the
> > > catalog.
> > >
> > > The right explanation is that extended stats on partitioned tables are
> > > not similar to indexes.  Indexes on parent table are nothing other than
> > > a mechanism to create indexes on the child tables.  That's not true for
> > > stats.
> > >
> > > See also my prior messages
> > > ZiJW1g2nbQs9ekwK@pryzbyj2023
> > > Zi5Msg74C61DjJKW@pryzbyj2023
> >
> > Yes, I understand that parents pg_statistic entry with stainherit ==
> > true includes statistics for the children.  I tried to express this by
> > word "covers".  But you're right, this is the wrong explanation.
> >
> > Can I, please, ask you to revise the patch?
>
> I tried to make this clear but it'd be nice if someone (Tomas/Alvaro?)
> would check that this says what's wanted.

Thank you!

I've assembled the patches with the pending fixes.
0001 – The patch by Dmitry Koval for fixing detection of name
collision in SPLIT partition operation.  Also, I found that name
collision detection doesn't work well for MERGE partitions.  I've
added fix for that to this patch as well.
0002 -– Patch for skipping copy of extended statistics.  I would
appreciate more feedback about wording, but I'd like to get a correct
behavior into the source tree sooner.  If the docs and/or comments
need further improvements, we can fix that later.

I'm going to push both if no objections.

Links.
1. https://www.postgresql.org/message-id/147426d9-b793-4571-a5e5-7438affeeb5a%40postgrespro.ru

------
Regards,
Alexander Korotkov
Supabase

  136. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Pavel Borisov <pashkin.elfe@gmail.com> — 2024-05-17T11:02:40Z

    Hi, Alexander:

On Fri, 17 May 2024 at 14:05, Alexander Korotkov <aekorotkov@gmail.com>
wrote:

> On Tue, May 14, 2024 at 5:49 PM Justin Pryzby <pryzby@telsasoft.com>
> wrote:
> > On Thu, May 09, 2024 at 12:51:32AM +0300, Alexander Korotkov wrote:
> > > > > However, parent's table extended statistics already covers all its
> > > > > children.
> > > >
> > > > => That's the wrong explanation.  It's not that "stats on the parent
> > > > table cover its children".  It's that there are two types of stats:
> > > > stats for the "table hierarchy" and stats for the individual table.
> > > > That's true for single-column stats as well as for extended stats.
> > > > In both cases, that's indicated by the inh flag in the code and in
> the
> > > > catalog.
> > > >
> > > > The right explanation is that extended stats on partitioned tables
> are
> > > > not similar to indexes.  Indexes on parent table are nothing other
> than
> > > > a mechanism to create indexes on the child tables.  That's not true
> for
> > > > stats.
> > > >
> > > > See also my prior messages
> > > > ZiJW1g2nbQs9ekwK@pryzbyj2023
> > > > Zi5Msg74C61DjJKW@pryzbyj2023
> > >
> > > Yes, I understand that parents pg_statistic entry with stainherit ==
> > > true includes statistics for the children.  I tried to express this by
> > > word "covers".  But you're right, this is the wrong explanation.
> > >
> > > Can I, please, ask you to revise the patch?
> >
> > I tried to make this clear but it'd be nice if someone (Tomas/Alvaro?)
> > would check that this says what's wanted.
>
> Thank you!
>
> I've assembled the patches with the pending fixes.
> 0001 – The patch by Dmitry Koval for fixing detection of name
> collision in SPLIT partition operation.  Also, I found that name
> collision detection doesn't work well for MERGE partitions.  I've
> added fix for that to this patch as well.
> 0002 -– Patch for skipping copy of extended statistics.  I would
> appreciate more feedback about wording, but I'd like to get a correct
> behavior into the source tree sooner.  If the docs and/or comments
> need further improvements, we can fix that later.
>
> I'm going to push both if no objections.
>
Thank you for working on this patch set!

Some minor things:
0001:
partition_split.sql
157 +-- Check that detection, that the new partition has the same name as
one of
158 +-- the merged partitions, works correctly for temporary partitions
Test for split with comment for merge. Maybe better something like:
"Split partition of a temporary table when one of the partitions after
split has the same name as the partition being split"

0002:
analgous -> analogous (maybe better using "like" instead of "analogous to")
heirarchy -> hierarchy

alter_table.sgml:
Maybe in documentation it's better not to provide reasoning, just state how
it works:
for consistency with <command>CREATE TABLE PARTITION OF</command> ->
similar to <command>CREATE TABLE PARTITION OF</command>

Regards,
Pavel Borisov

  137. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-17T11:33:40Z

    Hi, Pavel!

On Fri, May 17, 2024 at 2:02 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> On Fri, 17 May 2024 at 14:05, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>>
>> On Tue, May 14, 2024 at 5:49 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
>> > On Thu, May 09, 2024 at 12:51:32AM +0300, Alexander Korotkov wrote:
>> > > > > However, parent's table extended statistics already covers all its
>> > > > > children.
>> > > >
>> > > > => That's the wrong explanation.  It's not that "stats on the parent
>> > > > table cover its children".  It's that there are two types of stats:
>> > > > stats for the "table hierarchy" and stats for the individual table.
>> > > > That's true for single-column stats as well as for extended stats.
>> > > > In both cases, that's indicated by the inh flag in the code and in the
>> > > > catalog.
>> > > >
>> > > > The right explanation is that extended stats on partitioned tables are
>> > > > not similar to indexes.  Indexes on parent table are nothing other than
>> > > > a mechanism to create indexes on the child tables.  That's not true for
>> > > > stats.
>> > > >
>> > > > See also my prior messages
>> > > > ZiJW1g2nbQs9ekwK@pryzbyj2023
>> > > > Zi5Msg74C61DjJKW@pryzbyj2023
>> > >
>> > > Yes, I understand that parents pg_statistic entry with stainherit ==
>> > > true includes statistics for the children.  I tried to express this by
>> > > word "covers".  But you're right, this is the wrong explanation.
>> > >
>> > > Can I, please, ask you to revise the patch?
>> >
>> > I tried to make this clear but it'd be nice if someone (Tomas/Alvaro?)
>> > would check that this says what's wanted.
>>
>> Thank you!
>>
>> I've assembled the patches with the pending fixes.
>> 0001 – The patch by Dmitry Koval for fixing detection of name
>> collision in SPLIT partition operation.  Also, I found that name
>> collision detection doesn't work well for MERGE partitions.  I've
>> added fix for that to this patch as well.
>> 0002 -– Patch for skipping copy of extended statistics.  I would
>> appreciate more feedback about wording, but I'd like to get a correct
>> behavior into the source tree sooner.  If the docs and/or comments
>> need further improvements, we can fix that later.
>>
>> I'm going to push both if no objections.
>
> Thank you for working on this patch set!
>
> Some minor things:
> 0001:
> partition_split.sql
> 157 +-- Check that detection, that the new partition has the same name as one of
> 158 +-- the merged partitions, works correctly for temporary partitions
> Test for split with comment for merge. Maybe better something like:
> "Split partition of a temporary table when one of the partitions after split has the same name as the partition being split"

Thank you, fixed as proposed.

> 0002:
> analgous -> analogous (maybe better using "like" instead of "analogous to")
> heirarchy -> hierarchy

Changed "are not analgous to" to "don't behave like".

> alter_table.sgml:
> Maybe in documentation it's better not to provide reasoning, just state how it works:
> for consistency with <command>CREATE TABLE PARTITION OF</command> -> similar to <command>CREATE TABLE PARTITION OF</command>

I'd like to keep this.  This is the question, which should naturally
arise when you read: "Why this is not just INCLUDING ALL?"

------
Regards,
Alexander Korotkov
Supabase

  138. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Tom Lane <tgl@sss.pgh.pa.us> — 2024-05-24T19:29:39Z

    The partition_split test has unstable results, as shown at [1].
I suggest adding "ORDER BY conname" to the two queries shown
to fail there.  Better look at other queries in the test for
possible similar problems, too.

			regards, tom lane

[1] https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=jackdaw&dt=2024-05-24%2015%3A58%3A17

  139. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Lakhin <exclusion@gmail.com> — 2024-05-24T20:00:00Z

    Hello,

24.05.2024 22:29, Tom Lane wrote:
> The partition_split test has unstable results, as shown at [1].
> I suggest adding "ORDER BY conname" to the two queries shown
> to fail there.  Better look at other queries in the test for
> possible similar problems, too.

Yes, I've just reproduced it on an aarch64 device as follows:
echo "autovacuum_naptime = 1
autovacuum_vacuum_threshold = 1
autovacuum_analyze_threshold = 1
" > ~/temp.config
TEMP_CONFIG=~/temp.config TESTS="$(printf 'partition_split %.0s' `seq 100`)" make -s check-tests
...
ok 80        - partition_split                           749 ms
not ok 81    - partition_split                           728 ms
ok 82        - partition_split                           732 ms

$ cat src/test/regress/regression.diffs
diff -U3 .../src/test/regress/expected/partition_split.out .../src/test/regress/results/partition_split.out
--- .../src/test/regress/expected/partition_split.out   2024-05-15 17:15:57.171999830 +0000
+++ .../src/test/regress/results/partition_split.out    2024-05-24 19:28:37.329999749 +0000
@@ -625,8 +625,8 @@
  SELECT pg_get_constraintdef(oid), conname, conkey FROM pg_constraint WHERE conrelid = 
'sales_feb_mar_apr2022'::regclass::oid;
pg_get_constraintdef                         | conname             | conkey
  ---------------------------------------------------------------------+---------------------------------+--------
- CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
   FOREIGN KEY (salesperson_id) REFERENCES salespeople(salesperson_id) | sales_range_salesperson_id_fkey | {1}
+ CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
  (2 rows)

  ALTER TABLE sales_range SPLIT PARTITION sales_feb_mar_apr2022 INTO

Best regards,
Alexander

  140. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-25T12:53:11Z

    On Fri, May 24, 2024 at 11:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
>
> 24.05.2024 22:29, Tom Lane wrote:
> > The partition_split test has unstable results, as shown at [1].
> > I suggest adding "ORDER BY conname" to the two queries shown
> > to fail there.  Better look at other queries in the test for
> > possible similar problems, too.
>
> Yes, I've just reproduced it on an aarch64 device as follows:
> echo "autovacuum_naptime = 1
> autovacuum_vacuum_threshold = 1
> autovacuum_analyze_threshold = 1
> " > ~/temp.config
> TEMP_CONFIG=~/temp.config TESTS="$(printf 'partition_split %.0s' `seq 100`)" make -s check-tests
> ...
> ok 80        - partition_split                           749 ms
> not ok 81    - partition_split                           728 ms
> ok 82        - partition_split                           732 ms
>
> $ cat src/test/regress/regression.diffs
> diff -U3 .../src/test/regress/expected/partition_split.out .../src/test/regress/results/partition_split.out
> --- .../src/test/regress/expected/partition_split.out   2024-05-15 17:15:57.171999830 +0000
> +++ .../src/test/regress/results/partition_split.out    2024-05-24 19:28:37.329999749 +0000
> @@ -625,8 +625,8 @@
>   SELECT pg_get_constraintdef(oid), conname, conkey FROM pg_constraint WHERE conrelid =
> 'sales_feb_mar_apr2022'::regclass::oid;
> pg_get_constraintdef                         | conname             | conkey
>   ---------------------------------------------------------------------+---------------------------------+--------
> - CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
>    FOREIGN KEY (salesperson_id) REFERENCES salespeople(salesperson_id) | sales_range_salesperson_id_fkey | {1}
> + CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
>   (2 rows)
>
>   ALTER TABLE sales_range SPLIT PARTITION sales_feb_mar_apr2022 INTO

Tom, Alexander, thank you for spotting this.
I'm going to care about it later today.

------
Regards,
Alexander Korotkov
Supabase

  141. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Justin Pryzby <pryzby@telsasoft.com> — 2024-05-25T17:53:17Z

    On Fri, May 03, 2024 at 04:32:25PM +0300, Alexander Korotkov wrote:
> On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > Note that the error that led to "EXCLUDING IDENTITY" is being discused
> > over here:
> > https://www.postgresql.org/message-id/3b8a9dc1-bbc7-0ef5-6863-c432afac7d59@gmail.com
> >
> > It's possible that once that's addressed, the exclusion should be
> > removed here, too.
> 
> +1

Can EXCLUDING IDENTITY be removed now ?

I wasn't able to find why it was needed - at one point, I think there
was a test case that threw an error, but now when I remove the EXCLUDE,
nothing goes wrong.

-- 
Justin

  142. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-26T03:56:33Z

    On Sat, May 25, 2024 at 8:53 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> On Fri, May 03, 2024 at 04:32:25PM +0300, Alexander Korotkov wrote:
> > On Fri, May 3, 2024 at 4:23 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
> > > Note that the error that led to "EXCLUDING IDENTITY" is being discused
> > > over here:
> > > https://www.postgresql.org/message-id/3b8a9dc1-bbc7-0ef5-6863-c432afac7d59@gmail.com
> > >
> > > It's possible that once that's addressed, the exclusion should be
> > > removed here, too.
> >
> > +1
>
> Can EXCLUDING IDENTITY be removed now ?
>
> I wasn't able to find why it was needed - at one point, I think there
> was a test case that threw an error, but now when I remove the EXCLUDE,
> nothing goes wrong.

Yes, it was broken before [1][2], but now it seems to work.  At the
same time, I'm not sure if we need to remove the EXCLUDE now.
IDENTITY is anyway successfully created when the new partition gets
attached.

Links.
1. https://www.postgresql.org/message-id/171085360143.2046436.7217841141682511557.pgcf@coridan.postgresql.org
2. https://www.postgresql.org/message-id/flat/ZiGH0xc1lxJ71ZfB%40pryzbyj2023#297b6aef85cb089abb38e9b1a9a7ffff

------
Regards,
Alexander Korotkov
Supabase

  143. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-05-26T03:58:11Z

    On Sat, May 25, 2024 at 3:53 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> On Fri, May 24, 2024 at 11:00 PM Alexander Lakhin <exclusion@gmail.com> wrote:
> >
> > 24.05.2024 22:29, Tom Lane wrote:
> > > The partition_split test has unstable results, as shown at [1].
> > > I suggest adding "ORDER BY conname" to the two queries shown
> > > to fail there.  Better look at other queries in the test for
> > > possible similar problems, too.
> >
> > Yes, I've just reproduced it on an aarch64 device as follows:
> > echo "autovacuum_naptime = 1
> > autovacuum_vacuum_threshold = 1
> > autovacuum_analyze_threshold = 1
> > " > ~/temp.config
> > TEMP_CONFIG=~/temp.config TESTS="$(printf 'partition_split %.0s' `seq 100`)" make -s check-tests
> > ...
> > ok 80        - partition_split                           749 ms
> > not ok 81    - partition_split                           728 ms
> > ok 82        - partition_split                           732 ms
> >
> > $ cat src/test/regress/regression.diffs
> > diff -U3 .../src/test/regress/expected/partition_split.out .../src/test/regress/results/partition_split.out
> > --- .../src/test/regress/expected/partition_split.out   2024-05-15 17:15:57.171999830 +0000
> > +++ .../src/test/regress/results/partition_split.out    2024-05-24 19:28:37.329999749 +0000
> > @@ -625,8 +625,8 @@
> >   SELECT pg_get_constraintdef(oid), conname, conkey FROM pg_constraint WHERE conrelid =
> > 'sales_feb_mar_apr2022'::regclass::oid;
> > pg_get_constraintdef                         | conname             | conkey
> >   ---------------------------------------------------------------------+---------------------------------+--------
> > - CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
> >    FOREIGN KEY (salesperson_id) REFERENCES salespeople(salesperson_id) | sales_range_salesperson_id_fkey | {1}
> > + CHECK ((sales_amount > 1))                                          | sales_range_sales_amount_check  | {2}
> >   (2 rows)
> >
> >   ALTER TABLE sales_range SPLIT PARTITION sales_feb_mar_apr2022 INTO
>
> Tom, Alexander, thank you for spotting this.
> I'm going to care about it later today.

ORDER BY is added in d53a4286d7 in these queries altogether with other
catalog queries with potentially unstable result.

------
Regards,
Alexander Korotkov
Supabase

  144. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Noah Misch <noah@leadboat.com> — 2024-08-08T17:13:51Z

    On Sun, Apr 07, 2024 at 01:22:51AM +0300, Alexander Korotkov wrote:
> I've pushed 0001 and 0002

The partition MERGE (1adf16b8f) and SPLIT (87c21bb94) v17 patches introduced
createPartitionTable() with this code:

	createStmt->relation = newPartName;
...
	wrapper->utilityStmt = (Node *) createStmt;
...
	ProcessUtility(wrapper,
...
	newRel = table_openrv(newPartName, NoLock);

This breaks from the CVE-2014-0062 (commit 5f17304) principle of not repeating
name lookups.  The attached demo uses this defect to make one partition have
two parents.

  145. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-08T22:43:11Z

    On Thu, Aug 8, 2024 at 8:14 PM Noah Misch <noah@leadboat.com> wrote:
> On Sun, Apr 07, 2024 at 01:22:51AM +0300, Alexander Korotkov wrote:
> > I've pushed 0001 and 0002
>
> The partition MERGE (1adf16b8f) and SPLIT (87c21bb94) v17 patches introduced
> createPartitionTable() with this code:
>
>         createStmt->relation = newPartName;
> ...
>         wrapper->utilityStmt = (Node *) createStmt;
> ...
>         ProcessUtility(wrapper,
> ...
>         newRel = table_openrv(newPartName, NoLock);
>
> This breaks from the CVE-2014-0062 (commit 5f17304) principle of not repeating
> name lookups.  The attached demo uses this defect to make one partition have
> two parents.

Thank you for a valuable report.  I will dig into and fix that.

------
Regards,
Alexander Korotkov
Supabase

  146. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-08-09T07:18:29Z

    > This breaks from the CVE-2014-0062 (commit 5f17304) principle of not repeating
> name lookups.  The attached demo uses this defect to make one partition have
> two parents.

Thank you very much for information (especially for the demo)!

I'm not sure that we can get the identifier of the newly created 
partition from the ProcessUtility() function...
Maybe it would be enough to check that the new partition is located in 
the namespace in which we created it (see attachment)?

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  147. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-10T15:43:59Z

    On Fri, Aug 9, 2024 at 10:18 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > This breaks from the CVE-2014-0062 (commit 5f17304) principle of not repeating
> > name lookups.  The attached demo uses this defect to make one partition have
> > two parents.
>
> Thank you very much for information (especially for the demo)!
>
> I'm not sure that we can get the identifier of the newly created
> partition from the ProcessUtility() function...
> Maybe it would be enough to check that the new partition is located in
> the namespace in which we created it (see attachment)?

The new partition doesn't necessarily get created in the same
namespace as parent partition.  I think it would be better to somehow
open partition by its oid.

It would be quite unfortunate to replicate significant part of
ProcessUtilitySlow().  So, the question is how to get the oid of newly
created relation from ProcessUtility().  I don't like to change the
signature of ProcessUtility() especially as a part of backpatch.  So,
I tried to fit this into existing parameters.  Probably
QueryCompletion struct fits this purpose best from the existing
parameters.  Attached draft patch implements returning oid of newly
created relation as part of QueryCompletion.  Thoughts?

------
Regards,
Alexander Korotkov
Supabase

  148. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-08-10T15:57:48Z

    > Probably
> QueryCompletion struct fits this purpose best from the existing
> parameters.  Attached draft patch implements returning oid of newly
> created relation as part of QueryCompletion.  Thoughts?

I agree, returning the oid of the newly created relation is the best way 
to solve the problem.
(Excuse me, I won't have access to a laptop for the next week - and 
won't be able to look at the source code).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  149. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-18T22:24:02Z

    On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > Probably
> > QueryCompletion struct fits this purpose best from the existing
> > parameters.  Attached draft patch implements returning oid of newly
> > created relation as part of QueryCompletion.  Thoughts?
>
> I agree, returning the oid of the newly created relation is the best way
> to solve the problem.
> (Excuse me, I won't have access to a laptop for the next week - and
> won't be able to look at the source code).

Thank you for your feedback.  Although, I decided QueryCompletion is
not a good place for this new field.  It looks more appropriate to
place it to TableLikeClause, which already contains one relation oid
inside.  The revised patch is attached.

------
Regards,
Alexander Korotkov
Supabase

  150. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Pavel Borisov <pashkin.elfe@gmail.com> — 2024-08-21T10:48:45Z

    Hi, Alexander!

On Mon, 19 Aug 2024 at 02:24, Alexander Korotkov <aekorotkov@gmail.com>
wrote:

> On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
> > > Probably
> > > QueryCompletion struct fits this purpose best from the existing
> > > parameters.  Attached draft patch implements returning oid of newly
> > > created relation as part of QueryCompletion.  Thoughts?
> >
> > I agree, returning the oid of the newly created relation is the best way
> > to solve the problem.
> > (Excuse me, I won't have access to a laptop for the next week - and
> > won't be able to look at the source code).
>
> Thank you for your feedback.  Although, I decided QueryCompletion is
> not a good place for this new field.  It looks more appropriate to
> place it to TableLikeClause, which already contains one relation oid
> inside.  The revised patch is attached.
>

I've looked at the patch v2. Remembering the OID of a relation newly
created with LIKE in TableLikeClause seems good to me.
Check-world passes sucessfully.

Shouldn't we also modify the TableLikeClause node in gram.y accordingly?

For the comments:
Put the Oid  -> Store the OID
so caller might use it -> for the caller to use it.
(Maybe also caller -> table create function)

Regards,
Pavel Borisov
Supabase

  151. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-21T11:55:04Z

    Hi, Pavel!

On Wed, Aug 21, 2024 at 1:48 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> On Mon, 19 Aug 2024 at 02:24, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>>
>> On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>> > > Probably
>> > > QueryCompletion struct fits this purpose best from the existing
>> > > parameters.  Attached draft patch implements returning oid of newly
>> > > created relation as part of QueryCompletion.  Thoughts?
>> >
>> > I agree, returning the oid of the newly created relation is the best way
>> > to solve the problem.
>> > (Excuse me, I won't have access to a laptop for the next week - and
>> > won't be able to look at the source code).
>>
>> Thank you for your feedback.  Although, I decided QueryCompletion is
>> not a good place for this new field.  It looks more appropriate to
>> place it to TableLikeClause, which already contains one relation oid
>> inside.  The revised patch is attached.
>
>
> I've looked at the patch v2. Remembering the OID of a relation newly created with LIKE in TableLikeClause seems good to me.
> Check-world passes sucessfully.

Thank you.

> Shouldn't we also modify the TableLikeClause node in gram.y accordingly?

On the one hand, makeNode() uses palloc0() and initializes all fields
with zero anyway.  On the other hand, there is already assignment of
relationOid.  So, yes I'll add assignment of newRelationOid for the
sake of uniformity.

> For the comments:
> Put the Oid  -> Store the OID
> so caller might use it -> for the caller to use it.

Accepted.

> (Maybe also caller -> table create function)

I'll prefer to leave it "caller" as more generic term, which could
also fit potential future usages.

The revised patch is attached.  I'm going to push it if no objections.

------
Regards,
Alexander Korotkov
Supabase

  152. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Pavel Borisov <pashkin.elfe@gmail.com> — 2024-08-21T12:06:31Z

    Hi, Alexander!

On Wed, 21 Aug 2024 at 15:55, Alexander Korotkov <aekorotkov@gmail.com>
wrote:

> Hi, Pavel!
>
> On Wed, Aug 21, 2024 at 1:48 PM Pavel Borisov <pashkin.elfe@gmail.com>
> wrote:
> > On Mon, 19 Aug 2024 at 02:24, Alexander Korotkov <aekorotkov@gmail.com>
> wrote:
> >>
> >> On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
> >> > > Probably
> >> > > QueryCompletion struct fits this purpose best from the existing
> >> > > parameters.  Attached draft patch implements returning oid of newly
> >> > > created relation as part of QueryCompletion.  Thoughts?
> >> >
> >> > I agree, returning the oid of the newly created relation is the best
> way
> >> > to solve the problem.
> >> > (Excuse me, I won't have access to a laptop for the next week - and
> >> > won't be able to look at the source code).
> >>
> >> Thank you for your feedback.  Although, I decided QueryCompletion is
> >> not a good place for this new field.  It looks more appropriate to
> >> place it to TableLikeClause, which already contains one relation oid
> >> inside.  The revised patch is attached.
> >
> >
> > I've looked at the patch v2. Remembering the OID of a relation newly
> created with LIKE in TableLikeClause seems good to me.
> > Check-world passes sucessfully.
>
> Thank you.
>
> > Shouldn't we also modify the TableLikeClause node in gram.y accordingly?
>
> On the one hand, makeNode() uses palloc0() and initializes all fields
> with zero anyway.  On the other hand, there is already assignment of
> relationOid.  So, yes I'll add assignment of newRelationOid for the
> sake of uniformity.
>
> > For the comments:
> > Put the Oid  -> Store the OID

> so caller might use it -> for the caller to use it.
>
> Accepted.
>
> > (Maybe also caller -> table create function)
>
> I'll prefer to leave it "caller" as more generic term, which could
> also fit potential future usages.
>
> The revised patch is attached.  I'm going to push it if no objections.
>
Looked at v3
All good except the patch has "Oid" and "OID" in two comments. I suppose
"OID" is preferred elsewhere in the PG comments.

Regards,
Pavel.

  153. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-21T12:40:44Z

    On Wed, Aug 21, 2024 at 3:06 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
> On Wed, 21 Aug 2024 at 15:55, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>>
>> Hi, Pavel!
>>
>> On Wed, Aug 21, 2024 at 1:48 PM Pavel Borisov <pashkin.elfe@gmail.com> wrote:
>> > On Mon, 19 Aug 2024 at 02:24, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>> >>
>> >> On Sat, Aug 10, 2024 at 6:57 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>> >> > > Probably
>> >> > > QueryCompletion struct fits this purpose best from the existing
>> >> > > parameters.  Attached draft patch implements returning oid of newly
>> >> > > created relation as part of QueryCompletion.  Thoughts?
>> >> >
>> >> > I agree, returning the oid of the newly created relation is the best way
>> >> > to solve the problem.
>> >> > (Excuse me, I won't have access to a laptop for the next week - and
>> >> > won't be able to look at the source code).
>> >>
>> >> Thank you for your feedback.  Although, I decided QueryCompletion is
>> >> not a good place for this new field.  It looks more appropriate to
>> >> place it to TableLikeClause, which already contains one relation oid
>> >> inside.  The revised patch is attached.
>> >
>> >
>> > I've looked at the patch v2. Remembering the OID of a relation newly created with LIKE in TableLikeClause seems good to me.
>> > Check-world passes sucessfully.
>>
>> Thank you.
>>
>> > Shouldn't we also modify the TableLikeClause node in gram.y accordingly?
>>
>> On the one hand, makeNode() uses palloc0() and initializes all fields
>> with zero anyway.  On the other hand, there is already assignment of
>> relationOid.  So, yes I'll add assignment of newRelationOid for the
>> sake of uniformity.
>>
>> > For the comments:
>> > Put the Oid  -> Store the OID
>>
>> > so caller might use it -> for the caller to use it.
>>
>> Accepted.
>>
>> > (Maybe also caller -> table create function)
>>
>> I'll prefer to leave it "caller" as more generic term, which could
>> also fit potential future usages.
>>
>> The revised patch is attached.  I'm going to push it if no objections.
>
> Looked at v3
> All good except the patch has "Oid" and "OID" in two comments. I suppose "OID" is preferred elsewhere in the PG comments.

Correct, the same file contains "OID" multiple times.  Revised version
is attached.

------
Regards,
Alexander Korotkov
Supabase

  154. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-08-22T16:33:27Z

    Hi,

In response to some concerns raised about this fix on the
pgsql-release list today, I spent some time investigating this patch.
Unfortunately, I think there are too many problems here to be
reasonably fixed before release, and I think all of SPLIT/MERGE
PARTITION needs to be reverted.

I focused my investigation on createPartitionTable(), which is a
helper for both SPLIT PARTITION and MERGE PARTITION, and it works by
consing up a CREATE TABLE AS statement and then feeding that back
through
ProcessUtility. I think it's bad design to use such a high-level
facility here; it is unlike what we do elsewhere in tablecmds.c and
opens us up to a variety of problems. The first thing that I
discovered is that this patch does not fix all of the repeated name
lookup problems. There is still this:

    tlc->relation =
makeRangeVar(get_namespace_name(RelationGetNamespace(modelRel)),
                                 RelationGetRelationName(modelRel), -1);

And also this:

    createStmt->tablespacename =
get_tablespace_name(modelRel->rd_rel->reltablespace);

In both cases, we do a reverse lookup on an OID to get a name which
the CREATE TABLE code will later turn back into an OID. If we don't
get the same value, that's at least a bug and probably a security
vulnerability, and there is no way to be certain that we will get the
same value. The only remedy is to not repeat the lookup in the first
place.

Then I got to looking at this:

    tlc->options = CREATE_TABLE_LIKE_ALL &
        ~(CREATE_TABLE_LIKE_INDEXES | CREATE_TABLE_LIKE_IDENTITY |
CREATE_TABLE_LIKE_STATISTICS);

It's not obvious at first glance that there is a critical problem
here, but there are reasons to be nervous. We're deploying a lot of
machinery here to copy a lot of stuff and, while that's efficient from
a coding perspective, it means that stuff you might not expect can
just kind of happen. For instance:

robert.haas=# \d+
                                            List of relations
 Schema | Name |       Type        |    Owner    | Persistence |
Access method |    Size    | Description
--------+------+-------------------+-------------+-------------+---------------+------------+-------------
 public | foo  | partitioned table | robert.haas | permanent   |
        | 0 bytes    |
 public | foo1 | table             | robert.haas | permanent   | heap
        | 8192 bytes |
 public | foo2 | table             | bob         | permanent   | heap
        | 8192 bytes |
(3 rows)
robert.haas=# alter table foo split partition foo2 into (partition
foo3 for values from (10) to (15), partition foo4 for values from (15)
to (20));
ALTER TABLE
robert.haas=# \d+
                                            List of relations
 Schema | Name |       Type        |    Owner    | Persistence |
Access method |    Size    | Description
--------+------+-------------------+-------------+-------------+---------------+------------+-------------
 public | foo  | partitioned table | robert.haas | permanent   |
        | 0 bytes    |
 public | foo1 | table             | robert.haas | permanent   | heap
        | 8192 bytes |
 public | foo3 | table             | robert.haas | permanent   | heap
        | 8192 bytes |
 public | foo4 | table             | robert.haas | permanent   | heap
        | 8192 bytes |
(4 rows)

I've split a partition owned by bob into two partitions owned by
robert.haas. That's rather surprising. It doesn't work to split a
partition that I don't own (and thus gain access to it) but if the
superuser splits a non-superuser's partition, the superuser ends
upowning the new partitions. I don't know if that's a vulnerability or
just unexpected. However, then I found this, which I'm pretty well
certain is a vulnerability:

robert.haas=# set role bob;
SET
robert.haas=> create table foo (a int, b text) partition by range (a);
CREATE TABLE
robert.haas=> create table foo1 partition of foo for values from (0) to (10);
CREATE TABLE
robert.haas=> create table foo2 partition of foo for values from (10) to (20);
CREATE TABLE
robert.haas=> insert into foo values (11, 'carrots'), (16, 'pineapple');
INSERT 0 2
robert.haas=> create or replace function run_me(integer) returns
integer as $$begin raise notice 'you are running me as %',
current_user; return $1; end$$ language plpgsql immutable;
CREATE FUNCTION
robert.haas=> create index on foo (run_me(a));
NOTICE:  you are running me as bob
NOTICE:  you are running me as bob
CREATE INDEX
robert.haas=> reset role;
RESET
robert.haas=# alter table foo split partition foo2 into (partition
foo3 for values from (10) to (15), partition foo4 for values from (15)
to (20));
NOTICE:  you are running me as robert.haas
NOTICE:  you are running me as robert.haas
ALTER TABLE

I think it is very unlikely that the problems mentioned above are the
only ones. They're just what I found in an hour or two of testing.
Even if they were, we're probably too close to release to be rushing
out last minute fixes to multiple unanticipated security problems. But
because of the design that was chosen here, I think there is probably
more stuff here that is not right, some of which is security relevant
and some of which is just a question of whether we're really getting
the behavior that we want. And I don't think we can fix all that
without either a very large number of grotty hacks similar to the one
installed by 04158e7fa37c2dda9c3421ca922d02807b86df19, or a complete
redesign of the feature. I believe the latter is probably a wiser
course of action.

...Robert

  155. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Jonathan S. Katz <jkatz@postgresql.org> — 2024-08-22T16:43:22Z

    On 8/22/24 12:33 PM, Robert Haas wrote:

> I think it is very unlikely that the problems mentioned above are the
> only ones. They're just what I found in an hour or two of testing.
> Even if they were, we're probably too close to release to be rushing
> out last minute fixes to multiple unanticipated security problems. But
> because of the design that was chosen here, I think there is probably
> more stuff here that is not right, some of which is security relevant
> and some of which is just a question of whether we're really getting
> the behavior that we want. And I don't think we can fix all that
> without either a very large number of grotty hacks similar to the one
> installed by 04158e7fa37c2dda9c3421ca922d02807b86df19, or a complete
> redesign of the feature. I believe the latter is probably a wiser
> course of action.

I can't comment on the design as much, but from a release standpoint, 
but security concerns this close to the RC/GA period do concern me.

Applying the lessons from PG15 + SQL/JSON where we (and I'll own that I 
was the one who pushed hard to include it) let it stay too long when it 
should have been reverted, I think we should take more time to work on 
this feature, revert it for PG17, and target it for PG18.

I understand it's disappointing to do a late revert of a feature, but I 
think it's better to be safer, particularly if we believe there's a an 
elevated risk of releasing something with vulnerabilities. As we saw 
with SQL/JSON, this we'll give us more time to come up with design we 
agree with, further test, and then promote as part of PG18.

Thanks,

Jonathan

  156. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-22T16:43:35Z

    Hi!

On Thu, Aug 22, 2024 at 7:33 PM Robert Haas <robertmhaas@gmail.com> wrote:
> In response to some concerns raised about this fix on the
> pgsql-release list today, I spent some time investigating this patch.
> Unfortunately, I think there are too many problems here to be
> reasonably fixed before release, and I think all of SPLIT/MERGE
> PARTITION needs to be reverted.
>
> I focused my investigation on createPartitionTable(), which is a
> helper for both SPLIT PARTITION and MERGE PARTITION, and it works by
> consing up a CREATE TABLE AS statement and then feeding that back
> through
> ProcessUtility. I think it's bad design to use such a high-level
> facility here; it is unlike what we do elsewhere in tablecmds.c and
> opens us up to a variety of problems. The first thing that I
> discovered is that this patch does not fix all of the repeated name
> lookup problems. There is still this:
>
>     tlc->relation =
> makeRangeVar(get_namespace_name(RelationGetNamespace(modelRel)),
>                                  RelationGetRelationName(modelRel), -1);
>
> And also this:
>
>     createStmt->tablespacename =
> get_tablespace_name(modelRel->rd_rel->reltablespace);
>
> In both cases, we do a reverse lookup on an OID to get a name which
> the CREATE TABLE code will later turn back into an OID. If we don't
> get the same value, that's at least a bug and probably a security
> vulnerability, and there is no way to be certain that we will get the
> same value. The only remedy is to not repeat the lookup in the first
> place.
>
> Then I got to looking at this:
>
>     tlc->options = CREATE_TABLE_LIKE_ALL &
>         ~(CREATE_TABLE_LIKE_INDEXES | CREATE_TABLE_LIKE_IDENTITY |
> CREATE_TABLE_LIKE_STATISTICS);
>
> It's not obvious at first glance that there is a critical problem
> here, but there are reasons to be nervous. We're deploying a lot of
> machinery here to copy a lot of stuff and, while that's efficient from
> a coding perspective, it means that stuff you might not expect can
> just kind of happen. For instance:
>
> robert.haas=# \d+
>                                             List of relations
>  Schema | Name |       Type        |    Owner    | Persistence |
> Access method |    Size    | Description
> --------+------+-------------------+-------------+-------------+---------------+------------+-------------
>  public | foo  | partitioned table | robert.haas | permanent   |
>         | 0 bytes    |
>  public | foo1 | table             | robert.haas | permanent   | heap
>         | 8192 bytes |
>  public | foo2 | table             | bob         | permanent   | heap
>         | 8192 bytes |
> (3 rows)
> robert.haas=# alter table foo split partition foo2 into (partition
> foo3 for values from (10) to (15), partition foo4 for values from (15)
> to (20));
> ALTER TABLE
> robert.haas=# \d+
>                                             List of relations
>  Schema | Name |       Type        |    Owner    | Persistence |
> Access method |    Size    | Description
> --------+------+-------------------+-------------+-------------+---------------+------------+-------------
>  public | foo  | partitioned table | robert.haas | permanent   |
>         | 0 bytes    |
>  public | foo1 | table             | robert.haas | permanent   | heap
>         | 8192 bytes |
>  public | foo3 | table             | robert.haas | permanent   | heap
>         | 8192 bytes |
>  public | foo4 | table             | robert.haas | permanent   | heap
>         | 8192 bytes |
> (4 rows)
>
> I've split a partition owned by bob into two partitions owned by
> robert.haas. That's rather surprising. It doesn't work to split a
> partition that I don't own (and thus gain access to it) but if the
> superuser splits a non-superuser's partition, the superuser ends
> upowning the new partitions. I don't know if that's a vulnerability or
> just unexpected. However, then I found this, which I'm pretty well
> certain is a vulnerability:
>
> robert.haas=# set role bob;
> SET
> robert.haas=> create table foo (a int, b text) partition by range (a);
> CREATE TABLE
> robert.haas=> create table foo1 partition of foo for values from (0) to (10);
> CREATE TABLE
> robert.haas=> create table foo2 partition of foo for values from (10) to (20);
> CREATE TABLE
> robert.haas=> insert into foo values (11, 'carrots'), (16, 'pineapple');
> INSERT 0 2
> robert.haas=> create or replace function run_me(integer) returns
> integer as $$begin raise notice 'you are running me as %',
> current_user; return $1; end$$ language plpgsql immutable;
> CREATE FUNCTION
> robert.haas=> create index on foo (run_me(a));
> NOTICE:  you are running me as bob
> NOTICE:  you are running me as bob
> CREATE INDEX
> robert.haas=> reset role;
> RESET
> robert.haas=# alter table foo split partition foo2 into (partition
> foo3 for values from (10) to (15), partition foo4 for values from (15)
> to (20));
> NOTICE:  you are running me as robert.haas
> NOTICE:  you are running me as robert.haas
> ALTER TABLE
>
> I think it is very unlikely that the problems mentioned above are the
> only ones. They're just what I found in an hour or two of testing.
> Even if they were, we're probably too close to release to be rushing
> out last minute fixes to multiple unanticipated security problems. But
> because of the design that was chosen here, I think there is probably
> more stuff here that is not right, some of which is security relevant
> and some of which is just a question of whether we're really getting
> the behavior that we want. And I don't think we can fix all that
> without either a very large number of grotty hacks similar to the one
> installed by 04158e7fa37c2dda9c3421ca922d02807b86df19, or a complete
> redesign of the feature. I believe the latter is probably a wiser
> course of action.

Thank you for your feedback.  Yes, it seems that there is not enough
time to even carefully analyze all the issues in these features.  The
rule of thumb I can get from this experience is "think multiple times
before accessing something already opened by its name".  I'm going to
revert these features during next couple days.

------
Regards,
Alexander Korotkov
Supabase

  157. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-08-22T17:25:07Z

    On Thu, Aug 22, 2024 at 12:43 PM Alexander Korotkov
<aekorotkov@gmail.com> wrote:
> Thank you for your feedback.  Yes, it seems that there is not enough
> time to even carefully analyze all the issues in these features.  The
> rule of thumb I can get from this experience is "think multiple times
> before accessing something already opened by its name".  I'm going to
> revert these features during next couple days.

Thanks, and sorry about that. I would say even "think multiple times"
is possibly not strong enough -- it might almost be "just don't ever
do it". Even if (in some particular case) the invalidation mechanism
seems to protect you from getting wrong answers, there are often holes
in that, specifically around search_path = foo, bar and you're
operating on an object in schema bar and an identically-named object
is created in schema foo at just the wrong time. Sometimes there are
problems even when search_path is not involved, but when it is, there
are more.

Here, aside from the name lookup issues, there are also problems with
expression evaluation: we can't split partitions without reindexing
rows that those partitions contain, and it is critical to think
through which is going to do the evaluation and make sure it's
properly sandboxed. I think we might need
SECURITY_RESTRICTED_OPERATION here.

Another thing I want to highlight if you do have another go at this
patch is that it's really critical to think about where every single
property of the newly-created tables comes from. The original patch
didn't consider relpersistence or tableam, and here I just discovered
that owner is also an issue that probably needs more consideration,
but it goes way beyond that. For example, I was surprised to discover
that if I put per-partition constraints or triggers on a partition and
then split it, they were not duplicated to the new partitions. Now,
maybe that's actually the behavior we want -- I'm not 100% positive --
but it sure wasn't what I was expecting. If we did duplicate them when
splitting, then what's supposed to happen when merging occurs? That is
not at all obvious, at least to me, but it needs careful thought. ACLs
and rules and default values and foreign keys (both outbond and
inbound) all need to be considered too, along with 27 other things
that I'm sure I'm not thinking about right now. Some of this behavior
should probably be explicitly documented, but all of it should be
considered carefully enough before commit to avoid surprises later. I
say that both from a security point of view and also just from a user
experience point of view. Even if things aren't insecure, they can
still be annoying, but it's not uncommon in cases like this for
annoying things to turn out to also be insecure.

Finally, if you do revisit this, I believe it would be a good idea to
think a bit harder about how data is moved around. My impression (and
please correct me if I am mistaken) is that currently, any split or
merge operation rewrites all the data in the source partition(s). If a
large partition is being split nearly equally, I think that has a good
chance of being optimal, but I think that might be the only case. If
we're merging partitions, wouldn't it be better to adjust the
constraints on the first partition -- or perhaps the largest partition
if we want to be clever -- and insert the data from all of the others
into it? Maybe that would even have syntax that puts the user in
control of which partition survives, e.g. ALTER TABLE tab1 MERGE
PARTITION part1 WITH part2, part3, .... That would also make it really
obvious to the user what all of the properties of part1 will be after
the merge: they will be exactly the same as they were before the
merge, except that the partition constraint will have been adjusted.
You basically dodge everything in the previous paragraph in one shot,
and it seems like it would also be faster. Splitting there's no
similar get-out-of-jail free card, at least not that I can see. Even
if you add syntax that splits a partition by using INSERT/DELETE to
move some rows to a newly-created partition, you still have to make at
least one new partition. But possibly that syntax is worth having
anyway, because it would be a lot quicker in the case of a highly
asymmetric split. On the other hand, maybe even splits are much more
likely and we don't really need it. I don't know.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  158. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-08-23T00:56:23Z

    On Thu, Aug 22, 2024 at 8:25 PM Robert Haas <robertmhaas@gmail.com> wrote:
>
> On Thu, Aug 22, 2024 at 12:43 PM Alexander Korotkov
> <aekorotkov@gmail.com> wrote:
> > Thank you for your feedback.  Yes, it seems that there is not enough
> > time to even carefully analyze all the issues in these features.  The
> > rule of thumb I can get from this experience is "think multiple times
> > before accessing something already opened by its name".  I'm going to
> > revert these features during next couple days.
>
> Thanks, and sorry about that. I would say even "think multiple times"
> is possibly not strong enough -- it might almost be "just don't ever
> do it". Even if (in some particular case) the invalidation mechanism
> seems to protect you from getting wrong answers, there are often holes
> in that, specifically around search_path = foo, bar and you're
> operating on an object in schema bar and an identically-named object
> is created in schema foo at just the wrong time. Sometimes there are
> problems even when search_path is not involved, but when it is, there
> are more.
>
> Here, aside from the name lookup issues, there are also problems with
> expression evaluation: we can't split partitions without reindexing
> rows that those partitions contain, and it is critical to think
> through which is going to do the evaluation and make sure it's
> properly sandboxed. I think we might need
> SECURITY_RESTRICTED_OPERATION here.
>
> Another thing I want to highlight if you do have another go at this
> patch is that it's really critical to think about where every single
> property of the newly-created tables comes from. The original patch
> didn't consider relpersistence or tableam, and here I just discovered
> that owner is also an issue that probably needs more consideration,
> but it goes way beyond that. For example, I was surprised to discover
> that if I put per-partition constraints or triggers on a partition and
> then split it, they were not duplicated to the new partitions. Now,
> maybe that's actually the behavior we want -- I'm not 100% positive --
> but it sure wasn't what I was expecting. If we did duplicate them when
> splitting, then what's supposed to happen when merging occurs? That is
> not at all obvious, at least to me, but it needs careful thought. ACLs
> and rules and default values and foreign keys (both outbond and
> inbound) all need to be considered too, along with 27 other things
> that I'm sure I'm not thinking about right now. Some of this behavior
> should probably be explicitly documented, but all of it should be
> considered carefully enough before commit to avoid surprises later. I
> say that both from a security point of view and also just from a user
> experience point of view. Even if things aren't insecure, they can
> still be annoying, but it's not uncommon in cases like this for
> annoying things to turn out to also be insecure.
>
> Finally, if you do revisit this, I believe it would be a good idea to
> think a bit harder about how data is moved around. My impression (and
> please correct me if I am mistaken) is that currently, any split or
> merge operation rewrites all the data in the source partition(s). If a
> large partition is being split nearly equally, I think that has a good
> chance of being optimal, but I think that might be the only case. If
> we're merging partitions, wouldn't it be better to adjust the
> constraints on the first partition -- or perhaps the largest partition
> if we want to be clever -- and insert the data from all of the others
> into it? Maybe that would even have syntax that puts the user in
> control of which partition survives, e.g. ALTER TABLE tab1 MERGE
> PARTITION part1 WITH part2, part3, .... That would also make it really
> obvious to the user what all of the properties of part1 will be after
> the merge: they will be exactly the same as they were before the
> merge, except that the partition constraint will have been adjusted.
> You basically dodge everything in the previous paragraph in one shot,
> and it seems like it would also be faster. Splitting there's no
> similar get-out-of-jail free card, at least not that I can see. Even
> if you add syntax that splits a partition by using INSERT/DELETE to
> move some rows to a newly-created partition, you still have to make at
> least one new partition. But possibly that syntax is worth having
> anyway, because it would be a lot quicker in the case of a highly
> asymmetric split. On the other hand, maybe even splits are much more
> likely and we don't really need it. I don't know.

Thank you for so valuable feedback!  When I have another go over this
patch I will ensure this is addressed.

------
Regards,
Alexander Korotkov
Supabase

  159. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-08-27T18:24:35Z

    Hi!

Alexander Korotkov, Robert Haas - thanks for fixes and feedbacks!

This email is a starting point for further work.
There are two files attached to this email:
v32-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch,
v32-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch.

They contains changes from reverted commits 1adf16b8fb, 87c21bb941, and
subsequent fixes and improvements including df64c81ca9, c99ef1811a,
9dfcac8e15, 885742b9f8, 842c9b2705, fcf80c5d5f, 96c7381c4c, f4fc7cb54b,
60ae37a8bc, 259c96fa8f, 449cdcd486, 3ca43dbbb6, 2a679ae94e, 3a82c689fd,
fbd4321fd5, d53a4286d7, c086896625, 4e5d6c4091.
I didn't include fix 04158e7fa3 into patches because Robert Haas 
objected to its use.

A short list of known issues and questions (see more details in [1] and 
[2]):

1. Function createPartitionTable() should be rewritten using partitioned 
table OID (not name) and without using ProcessUtility().

2. Should it be considered an error when we split a partition owned by 
another user and get partitions that owned by our user?
(I think this is not a problem. Perhaps disallow merging other users' 
partitions would be too strict a restriction.)

3. About the functional index "create index on foo (run_me(a));".
(Should we disallow merging of another user's partitions when 
partitioned table has functional indexes? SECURITY_RESTRICTED_OPERATION?)

4. Need to decide what is correct in case there are per-partition 
constraints or triggers on a split partition. They not duplicated to the 
new partitions now. (But might be in this case we should have an error 
or warning?)

5. "If we're merging partitions, wouldn't it be better to adjust the 
constraints on the first partition - or perhaps the largest partition if 
we want to be clever -- and insert the data from all of the others into 
it? Maybe that would even have syntax that puts the user in control of 
which partition survives, e.g. ALTER TABLE tab1 MERGE PARTITION part1 
WITH part2, part3, .... That would also make it really obvious to the 
user what all of the properties of part1 will be after the merge: they 
will be exactly the same as they were before the merge, except that the 
partition constraint will have been adjusted."
(Similar optimization was proposed in [3] but was rejected [4]).

Links.
[1] 
https://www.postgresql.org/message-id/CA%2BTgmobHYix%3DNn8D4RUHa6fhUVPR88KGAMq1pBfnGfOfEjRixA%40mail.gmail.com.
[2] 
https://www.postgresql.org/message-id/CA%2BTgmoY0%3DbT_xBP8csR%3DMFE%3DFxGE2n2-me2-31jBOgEcLvW7ug%40mail.gmail.com
[3] 
https://www.postgresql.org/message-id/c3730d78-6081-4c41-9715-d1d192734576%40postgrespro.ru, 
see v31-0003-Additional-patch-for-ALTER-TABLE-.-MERGE-PARTITI.patch
[4] 
https://www.postgresql.org/message-id/CAPpHfdtj7YsPaASoVPN%2BN3H4_Ct%2BkQw8QY1d_9u7FPnbghkicw%40mail.gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  160. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Robert Haas <robertmhaas@gmail.com> — 2024-08-28T13:45:36Z

    On Tue, Aug 27, 2024 at 2:24 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> They contains changes from reverted commits 1adf16b8fb, 87c21bb941, and
> subsequent fixes and improvements including df64c81ca9, c99ef1811a,
> 9dfcac8e15, 885742b9f8, 842c9b2705, fcf80c5d5f, 96c7381c4c, f4fc7cb54b,
> 60ae37a8bc, 259c96fa8f, 449cdcd486, 3ca43dbbb6, 2a679ae94e, 3a82c689fd,
> fbd4321fd5, d53a4286d7, c086896625, 4e5d6c4091.
> I didn't include fix 04158e7fa3 into patches because Robert Haas
> objected to its use.

To be clear, I'm not against 04158e7fa3. I just don't think it fixes everything.

> 1. Function createPartitionTable() should be rewritten using partitioned
> table OID (not name) and without using ProcessUtility().

Agree.

> 2. Should it be considered an error when we split a partition owned by
> another user and get partitions that owned by our user?
> (I think this is not a problem. Perhaps disallow merging other users'
> partitions would be too strict a restriction.)
>
> 3. About the functional index "create index on foo (run_me(a));".
> (Should we disallow merging of another user's partitions when
> partitioned table has functional indexes? SECURITY_RESTRICTED_OPERATION?)
>
> 4. Need to decide what is correct in case there are per-partition
> constraints or triggers on a split partition. They not duplicated to the
> new partitions now. (But might be in this case we should have an error
> or warning?)

I think we want to avoid giving errors or warnings.  For all of these
cases, and others, we need to consider what the expected behavior is,
and have test cases and documentation as appropriate. But we shouldn't
think of it as "let's make it fail if the user does something that's
not safe" but rather "let's figure out how to make it safe."

> 5. "If we're merging partitions, wouldn't it be better to adjust the
> constraints on the first partition - or perhaps the largest partition if
> we want to be clever -- and insert the data from all of the others into
> it? Maybe that would even have syntax that puts the user in control of
> which partition survives, e.g. ALTER TABLE tab1 MERGE PARTITION part1
> WITH part2, part3, .... That would also make it really obvious to the
> user what all of the properties of part1 will be after the merge: they
> will be exactly the same as they were before the merge, except that the
> partition constraint will have been adjusted."
> (Similar optimization was proposed in [3] but was rejected [4]).

Interesting. Maybe it would be a good idea to set up some test cases
to see which approach is better in different cases. Like try moving
data from foo1 to foo2 with DELETE..INSERT vs. creating a new table
with CTAS from foo1 UNION ALL foo2 and then indexing it. I think
Alexander has a good point there, but I think my point is good too so
I'm not sure which way wins.

-- 
Robert Haas
EDB: http://www.enterprisedb.com

  161. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-08-30T08:43:10Z

    Hi!

I plan to prepare fixes for issues from email [1] as separate commits 
(for better code readability). Attachment in this email is a variant of 
fix for the issue:

 > 1. Function createPartitionTable() should be rewritten using
 > partitioned table OID (not name) and without using ProcessUtility().

Patch "Refactor createPartitionTable to remove ProcessUtility call" 
contains code changes + test (see file 
v33-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch).

But I'm not sure that refactoring createPartitionTable is the best 
solution. PostgreSQL code has issue CVE-2014-0062 (commit 5f17304) - see 
relation_openrv() call in expandTableLikeClause() function [2] (opening 
relation by name after we got relation Oid).
Example for reproduce relation_openrv() call:

CREATE TABLE t (b bigint, i int DEFAULT 100);
CREATE TABLE t1 (LIKE t_bigint INCLUDING ALL);

Commit 04158e7fa3 [3] (by Alexander Korotkov) might be a good fix for 
this issue. But if we keep commit 04158e7fa3, do we need to refactor the 
createPartitionTable function (for removing ProcessUtility)?
Perhaps the existing code
1) v33-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch
2) v33-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch +
with patch 04158e7fa3 will look better.


I would be very grateful for comments and suggestions.

Links.
[1] 
https://www.postgresql.org/message-id/859476bf-3cb0-455e-b093-b8ab5ef17f0e%40postgrespro.ru
[2] 
https://github.com/postgres/postgres/blob/c39afc38cfec7c34b883095062a89a63b221521a/src/backend/parser/parse_utilcmd.c#L1171
[3] 
https://github.com/postgres/postgres/commit/04158e7fa37c2dda9c3421ca922d02807b86df19

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  162. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2024-12-09T06:36:25Z

    Hi!

On Fri, Aug 30, 2024 at 11:43 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> I plan to prepare fixes for issues from email [1] as separate commits
> (for better code readability). Attachment in this email is a variant of
> fix for the issue:
>
>  > 1. Function createPartitionTable() should be rewritten using
>  > partitioned table OID (not name) and without using ProcessUtility().
>
> Patch "Refactor createPartitionTable to remove ProcessUtility call"
> contains code changes + test (see file
> v33-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch).
>
> But I'm not sure that refactoring createPartitionTable is the best
> solution. PostgreSQL code has issue CVE-2014-0062 (commit 5f17304) - see
> relation_openrv() call in expandTableLikeClause() function [2] (opening
> relation by name after we got relation Oid).
> Example for reproduce relation_openrv() call:
>
> CREATE TABLE t (b bigint, i int DEFAULT 100);
> CREATE TABLE t1 (LIKE t_bigint INCLUDING ALL);
>
> Commit 04158e7fa3 [3] (by Alexander Korotkov) might be a good fix for
> this issue. But if we keep commit 04158e7fa3, do we need to refactor the
> createPartitionTable function (for removing ProcessUtility)?
> Perhaps the existing code
> 1) v33-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch
> 2) v33-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch +
> with patch 04158e7fa3 will look better.
>
>
> I would be very grateful for comments and suggestions.

Thank you for continuing your work on the subject.  The patches
currently doesn't apply cleanly.  Please, rebase.

I think getting away from expandTableLikeClause() is the right
direction to resolve the security problems.  That looks great, it
finally not as complex as I thought.  I think the code requires some
polishing: you need to revise the comments given its not part of LIKE
clause handling anymore.

I see fixes for the issues mentioned in [1] and [2] are still not
implemented.  Do you plan to do this in this release cycle?

Links.
1. https://www.postgresql.org/message-id/CA%2BTgmoY0%3DbT_xBP8csR%3DMFE%3DFxGE2n2-me2-31jBOgEcLvW7ug%40mail.gmail.com
2. https://www.postgresql.org/message-id/859476bf-3cb0-455e-b093-b8ab5ef17f0e%40postgrespro.ru

------
Regards,
Alexander Korotkov
Supabase

  163. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2024-12-09T21:01:43Z

    Thank you for email!


 >The patches currently doesn't apply cleanly.  Please, rebase.

I did rebase and made few fixes that were needed after the recent 
vanilla changes (added ATT_PARTITIONED_TABLE flag for 
ATSimplePermissions function + added creation of not-null constraints in 
the createTableConstraints function).
The patches are attached to the email.


 >I think the code requires some polishing: you need to revise
 > the comments given its not part of LIKE clause handling anymore.

I removed couple of lines from comment and a few lines from 
documentation, I hope that's enough ...


 >I see fixes for the issues mentioned in [1] and [2] are still not
 >implemented.  Do you plan to do this in this release cycle?

I would like to make some changes, but I think it would be appropriate 
to discuss these points first.
As far as I understand, there is currently no clear opinion on how to 
implement [1] and [2].


I would appreciate your opinions on what improvements are really needed 
and in what order they should be implemented.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  164. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2025-01-27T22:24:16Z

    Only patches v34-0001 and 2 has been tested.
Patch v34-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch do not apply anymore on src/backend/catalog/heap.c

The new status of this patch is: Waiting on Author

  165. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-01-27T23:14:52Z

     >Only patches v34-0001 and 2 has been tested.
 >Patch v34-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch
 >do not apply anymore on src/backend/catalog/heap.c

Thanks, rebased.
The patches are attached to the email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  166. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-01-31T09:07:17Z

    Hi, Dmitry!

On Tue, Jan 28, 2025 at 1:15 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
>  >Only patches v34-0001 and 2 has been tested.
>  >Patch v34-0003-Refactor-createPartitionTable-to-remove-ProcessU.patch
>  >do not apply anymore on src/backend/catalog/heap.c
>
> Thanks, rebased.
> The patches are attached to the email.

Thank you for the rebase.
I don't think we need a separate 0003 patch with refactoring.  It's
probably good idea to keep this functionality as a separate patch, but
let's make then it a 0001, which prepares functions used by 0002 and
0003.

------
Regards,
Alexander Korotkov
Supabase

  167. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-01-31T09:19:29Z

    Hi!

I'd like to share some thoughts on which particular way this patch could go.

On Thu, Aug 22, 2024 at 8:25 PM Robert Haas <robertmhaas@gmail.com> wrote:
> Here, aside from the name lookup issues, there are also problems with
> expression evaluation: we can't split partitions without reindexing
> rows that those partitions contain, and it is critical to think
> through which is going to do the evaluation and make sure it's
> properly sandboxed. I think we might need
> SECURITY_RESTRICTED_OPERATION here.

+1 for use SECURITY_RESTRICTED_OPERATION

> Another thing I want to highlight if you do have another go at this
> patch is that it's really critical to think about where every single
> property of the newly-created tables comes from. The original patch
> didn't consider relpersistence or tableam, and here I just discovered
> that owner is also an issue that probably needs more consideration,
> but it goes way beyond that. For example, I was surprised to discover
> that if I put per-partition constraints or triggers on a partition and
> then split it, they were not duplicated to the new partitions. Now,
> maybe that's actually the behavior we want -- I'm not 100% positive --
> but it sure wasn't what I was expecting. If we did duplicate them when
> splitting, then what's supposed to happen when merging occurs? That is
> not at all obvious, at least to me, but it needs careful thought. ACLs
> and rules and default values and foreign keys (both outbond and
> inbound) all need to be considered too, along with 27 other things
> that I'm sure I'm not thinking about right now. Some of this behavior
> should probably be explicitly documented, but all of it should be
> considered carefully enough before commit to avoid surprises later. I
> say that both from a security point of view and also just from a user
> experience point of view. Even if things aren't insecure, they can
> still be annoying, but it's not uncommon in cases like this for
> annoying things to turn out to also be insecure.

Yes, I think it's a good idea to duplicate dependent objects of split
partition to new partitions.  But it important to very carefully check
user have relevant permissions for all of them.  We could also provide
a syntax to exclude some of them (and even define new ones?), but I
strongly suspect that would overcomplicate patch for now and we need
to postpone this.

Regarding the merge, I think it would be good to provide a syntax to
let user choose a model partition between partitions to be merged.

> Finally, if you do revisit this, I believe it would be a good idea to
> think a bit harder about how data is moved around. My impression (and
> please correct me if I am mistaken) is that currently, any split or
> merge operation rewrites all the data in the source partition(s). If a
> large partition is being split nearly equally, I think that has a good
> chance of being optimal, but I think that might be the only case. If
> we're merging partitions, wouldn't it be better to adjust the
> constraints on the first partition -- or perhaps the largest partition
> if we want to be clever -- and insert the data from all of the others
> into it? Maybe that would even have syntax that puts the user in
> control of which partition survives, e.g. ALTER TABLE tab1 MERGE
> PARTITION part1 WITH part2, part3, .... That would also make it really
> obvious to the user what all of the properties of part1 will be after
> the merge: they will be exactly the same as they were before the
> merge, except that the partition constraint will have been adjusted.
> You basically dodge everything in the previous paragraph in one shot,
> and it seems like it would also be faster. Splitting there's no
> similar get-out-of-jail free card, at least not that I can see. Even
> if you add syntax that splits a partition by using INSERT/DELETE to
> move some rows to a newly-created partition, you still have to make at
> least one new partition. But possibly that syntax is worth having
> anyway, because it would be a lot quicker in the case of a highly
> asymmetric split. On the other hand, maybe even splits are much more
> likely and we don't really need it. I don't know.

Hmm... I think the important aspect for this DDL operation is to be
atomic and transactional.  And that seems to be extremely hard to
achieve if we move the data between existing relnodes.  How can we
rollback or recover after error?  So, it least for initial
implementation I would leave data movement as it is.

------
Regards,
Alexander Korotkov
Supabase

  168. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-01-31T09:22:51Z

    On Mon, Dec 9, 2024 at 11:01 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>  >I see fixes for the issues mentioned in [1] and [2] are still not
>  >implemented.  Do you plan to do this in this release cycle?
>
> I would like to make some changes, but I think it would be appropriate
> to discuss these points first.
> As far as I understand, there is currently no clear opinion on how to
> implement [1] and [2].
>
>
> I would appreciate your opinions on what improvements are really needed
> and in what order they should be implemented.

Please, check my thoughts on how this patch could be further
developed.  Given amount of work to be done, I doubt that'a a subject
for pg18.  But I think you could continue this work, and we could
consider it for early pg19 cycle.

Links.
1. https://www.postgresql.org/message-id/CAPpHfdtSxrcxQERO82cyQ2heN3%2BA7VC63k8SmL%3DEEiph-8rfHg%40mail.gmail.com
2. https://www.postgresql.org/message-id/CAPpHfdvVMdUX0DGSK3oAbt9C5TPup%3DBEq8QkmvDrMbuD4BR9Fw%40mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase

  169. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-02-03T15:37:47Z

    Hi, Alexander!
Thanks for your advices and recommendations!

 >I don't think we need a separate 0003 patch with refactoring.  It's
 >probably good idea to keep this functionality as a separate patch, but
 >let's make then it a 0001, which prepares functions used by 0002 and
 >0003.

Done. 0003 was created separately to better understand what changes were 
made after the verified changes 0001 and 0002.

 >Please, check my thoughts on how this patch could be further
 >developed.  Given amount of work to be done, I doubt that'a a subject
 >for pg18.  But I think you could continue this work, and we could
 >consider it for early pg19 cycle.

Good. I'll try to collect and summarize the opinions of colleagues on 
these issues [1], and then put them up for discussion in this thread.

Links.
[1] 
https://www.postgresql.org/message-id/CAPpHfdtSxrcxQERO82cyQ2heN3%2BA7VC63k8SmL%3DEEiph-8rfHg%40mail.gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  170. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alvaro Herrera <alvherre@alvh.no-ip.org> — 2025-02-03T16:40:18Z

    > --- a/src/backend/catalog/heap.c
> +++ b/src/backend/catalog/heap.c
> @@ -101,9 +101,6 @@ static ObjectAddress AddNewRelationType(const char *typeName,
>  										Oid new_row_type,
>  										Oid new_array_type);
>  static void RelationRemoveInheritance(Oid relid);
> -static Oid	StoreRelCheck(Relation rel, const char *ccname, Node *expr,
> -						  bool is_enforced, bool is_validated, bool is_local,
> -						  int16 inhcount, bool is_no_inherit, bool is_internal);
>  static void StoreConstraints(Relation rel, List *cooked_constraints,
>  							 bool is_internal);
>  static bool MergeWithExistingConstraint(Relation rel, const char *ccname, Node *expr,
> @@ -111,7 +108,6 @@ static bool MergeWithExistingConstraint(Relation rel, const char *ccname, Node *
>  										bool is_enforced,
>  										bool is_initially_valid,
>  										bool is_no_inherit);
> -static void SetRelationNumChecks(Relation rel, int numchecks);

I'm wary of taking some static functions and making them non-static
without further analysis, because we create a kind-of API that's
possible incoherent or at least incomplete and may cause trouble later.
Maybe we should make some effort to construct a real interface here.
For example, perhaps you don't need to expose both StoreRelCheck and
SetRelationNumChecks, but instead would be better served by exposing
StoreConstraints?  (Though I think we need some explicit check that that
function requires that the table is new and has no constraint, otherwise
SetRelationNumChecks stores the wrong thing.  No?)  Or maybe use
AddRelationNewConstraints() instead, which is an already exposed function.

What I'm saying is just that we should be more watchful of what we put
in our .h files.

Note: I didn't actually review the patch.

-- 
Álvaro Herrera         PostgreSQL Developer  —  https://www.EnterpriseDB.com/

  171. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    vignesh C <vignesh21@gmail.com> — 2025-03-17T06:55:16Z

    On Mon, 3 Feb 2025 at 21:08, Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi, Alexander!
> Thanks for your advices and recommendations!
>
>  >I don't think we need a separate 0003 patch with refactoring.  It's
>  >probably good idea to keep this functionality as a separate patch, but
>  >let's make then it a 0001, which prepares functions used by 0002 and
>  >0003.
>
> Done. 0003 was created separately to better understand what changes were
> made after the verified changes 0001 and 0002.
>
>  >Please, check my thoughts on how this patch could be further
>  >developed.  Given amount of work to be done, I doubt that'a a subject
>  >for pg18.  But I think you could continue this work, and we could
>  >consider it for early pg19 cycle.
>
> Good. I'll try to collect and summarize the opinions of colleagues on
> these issues [1], and then put them up for discussion in this thread.

I noticed that Alvaro's comments from [1] have not yet been addressed,
I have changed the status of commitfest entry to "Waiting on Author",
please address them and update it to "Needs review".
[1] - https://www.postgresql.org/message-id/202502031640.zem6orjmmxoz@alvherre.pgsql

Regards,
Vignesh

  172. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-03-17T13:36:56Z

     > I noticed that Alvaro's comments from [1] have not yet been addressed,

Thanks!
I'm sorry, I missed this comment.

 > I'm wary of taking some static functions and making them non-static
...
 > For example, perhaps you don't need to expose both StoreRelCheck and
 > SetRelationNumChecks, but instead would be better served by exposing
 > StoreConstraints? ... Or maybe use AddRelationNewConstraints() ...

I replaced explosion of functions StoreRelCheck + SetRelationNumChecks
to StoreConstraints explosion. Probably function 
AddRelationNewConstraints is not very suitable for this ...


[1] 
https://www.postgresql.org/message-id/202502031640.zem6orjmmxoz@alvherre.pgsql

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  173. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-05-12T08:31:04Z

    Hi!

We (with colleagues) discussed further improvements to SPLIT/MERGE 
PARTITION(S). As a result of the discussion, the following answers to 
the questions remained:

1. Who should be the owner of new partitions (SPLIT PARTITION command) 
if owner of the partition being split is not the same as the current user?
a) current user (since he is the one who creates new tables);
b) the owner of the partitioned partition (since it is the owner of the 
table and should continue to own it).

2. Who should be the owner of the new partition (MERGE PARTITIONS 
command) if the partitions being merged have different owners?
a) current user (since he is the one who creates new table);
b) merging of partitions should be forbidden if they have different owners.

Please, advise what seems to be the best solution for points 1 and 2.

With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  174. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-05-19T22:35:45Z

    Hi!

Changes in patches:

1) Added usage of SECURITY_RESTRICTED_OPERATION for SPLIT/MERGE 
PARTITION(S) commands.

2) For SPLIT PARTITION command: new partitions will have the same owner 
as the parent.

3) For MERGE PARTITIONS command: if merged partitions have different 
owners, an error will be generated.

Patches are attached to the email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  175. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-05-21T00:35:35Z

    On Mon, May 12, 2025 at 4:31 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
>
> We (with colleagues) discussed further improvements to SPLIT/MERGE
> PARTITION(S). As a result of the discussion, the following answers to
> the questions remained:
>
> 1. Who should be the owner of new partitions (SPLIT PARTITION command)
> if owner of the partition being split is not the same as the current user?
> a) current user (since he is the one who creates new tables);
> b) the owner of the partitioned partition (since it is the owner of the
> table and should continue to own it).

per https://www.postgresql.org/docs/current/sql-altertable.html
"You must own the table to use ALTER TABLE."

That means the current user must own the to be SPLITed partition.
the new partitions owner should be current user (the one who did ALTER TABLE)


> 2. Who should be the owner of the new partition (MERGE PARTITIONS
> command) if the partitions being merged have different owners?
> a) current user (since he is the one who creates new table);
> b) merging of partitions should be forbidden if they have different owners.
>
let say:

alter table whatever_range merge partitions
    ( whatever_range_c, whatever_range_de )
    into whatever_range_cde;

In this case, the current user must own
whatever_range_c and whatever_range_de to perform this operation.
and the current user will be the owner of whatever_range_cde.

  176. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-05-21T12:30:01Z

    Hi!

 >per https://www.postgresql.org/docs/current/sql-altertable.html
 >"You must own the table to use ALTER TABLE."
 >That means the current user must own the to be SPLITed partition.

Last statement may be incorrect (if the logic is different).
Current user can attach another user's partition. If current user can 
change (attach) another user's partition, why can't he split the 
partition? After attach, partition is part of current user's table, and 
current user can change his own table. (Moreover, the owner of the new 
partitions is the same as the owner of the split partition.)

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  177. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-05-21T14:54:48Z

    On Wed, May 21, 2025 at 8:30 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
>
>  >per https://www.postgresql.org/docs/current/sql-altertable.html
>  >"You must own the table to use ALTER TABLE."
>  >That means the current user must own the to be SPLITed partition.
>
> Last statement may be incorrect (if the logic is different).
> Current user can attach another user's partition. If current user can
> change (attach) another user's partition, why can't he split the
> partition? After attach, partition is part of current user's table, and
> current user can change his own table. (Moreover, the owner of the new
> partitions is the same as the owner of the split partition.)
>
> --

I think if he can attach another partition, then he can split the partition.
currently ALTER TABLE ATTACH PARTITION requires the current user
both own the partitioned table and the to be attached partition.

for example:

begin;
create role alice;
create role bob;
GRANT all privileges on schema public to alice;
GRANT all privileges on schema public to bob;
set role bob;
create table at_partitioned (a int, b text) partition by range (a);
set role alice;
create table at_part_2 (b text, a int);
set role bob;
alter table at_partitioned attach partition at_part_2 for values from
(1000) to (2000); --should error
rollback;

----------------
begin;
create role alice;
create role bob;
GRANT all privileges on schema public to alice;
GRANT all privileges on schema public to bob;
set role bob;
create table at_partitioned (a int, b text) partition by range (a);
set role alice;
create table at_part_2 (b text, a int);
alter table at_partitioned attach partition at_part_2 for values from
(1000) to (2000); --should error
rollback;


P.S. maybe using sql examples can illustrate the idea more intuitively,
sometimes words may cause confusion.

  178. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-05-21T17:01:06Z

    > for example:
> ...

If in both examples you replace

create role bob;

with

create role bob SUPERUSER;

and in the second example add "set role bob;" before "alter table ..." 
query, then no error will be occur.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  179. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-05-22T02:59:10Z

    On Thu, May 22, 2025 at 1:01 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> > for example:
> > ...
>
> If in both examples you replace
>
> create role bob;
>
> with
>
> create role bob SUPERUSER;
>
> and in the second example add "set role bob;" before "alter table ..."
> query, then no error will be occur.
>

That is fine.
superuser can bypass all permission checks.
superuser can attach any table to the partitioned table as he wants.

for example:

begin;
create role alice SUPERUSER;
create role bob SUPERUSER;
GRANT all privileges on schema public to alice;
GRANT all privileges on schema public to bob;
set role bob;
create table at_partitioned (a int, b text) partition by range (a);
set role alice;
create table at_part_2 (b text, a int);
set role bob;
alter table at_partitioned attach partition at_part_2 for values from
(1000) to (2000);
rollback;

Am I missing something?

  180. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-05-22T08:57:43Z

    > superuser can bypass all permission checks.
> superuser can attach any table to the partitioned table as he wants.

That's right.
Using SUPERUSER may be a rare situation, but it needs to be processed.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  181. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-06-03T09:58:24Z

    On Thu, May 22, 2025 at 11:57 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > superuser can bypass all permission checks.
> > superuser can attach any table to the partitioned table as he wants.
>
> That's right.
> Using SUPERUSER may be a rare situation, but it needs to be processed.

I suggest that owner of new partitions produced by ALTER TABLE ...
SPLIT should be the same as owner of original partition.  Even if this
operation is done by superuser.  Superuser may explicitly set he owner
if needed.  I think we need to explicitly document this.

------
Regards,
Alexander Korotkov
Supabase

  182. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-06-03T10:12:28Z

    Hi, Dmitry!

On Tue, May 20, 2025 at 1:36 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Changes in patches:
>
> 1) Added usage of SECURITY_RESTRICTED_OPERATION for SPLIT/MERGE
> PARTITION(S) commands.
>
> 2) For SPLIT PARTITION command: new partitions will have the same owner
> as the parent.
>
> 3) For MERGE PARTITIONS command: if merged partitions have different
> owners, an error will be generated.

Some notes to the patch.
1) I think we need explicitly document who is the owner of the new
partition(s).  The documentation for MERGE command says all the merged
partitions should have the same owner.  Good, but also we need to
state that the new partition should have the same owner.  Same to the
SPLIT command.
2) I think we also need to describe what happens not just with
ownership, but with ACLs.  We currently don't do anything to them, so
basically we discard existing ACLs.  It's probably OK, and we could
introduce options for ACL handling later.  But we need to state in the
docs that it's user responsibility to setup ACL on new partition(s).

------
Regards,
Alexander Korotkov
Supabase

  183. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-03T11:06:09Z

    > Some notes to the patch. ...

Thanks for the notes!
I'll try to make edits in the next few days.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  184. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-03T20:53:13Z

    Added some changes to documentation.
Patches are attached to the email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  185. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-04T10:45:32Z

    On Wed, Jun 4, 2025 at 4:53 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Added some changes to documentation.
> Patches are attached to the email.
>

hi.
I haven't touched v39-0002 yet.
The following are reviews of v39-0001.

+CREATE INDEX sales_range_sales_date_idx ON sales_range USING btree
(sales_date);
+INSERT INTO sales_range VALUES (1,  'May',      1000, '2022-01-31');
+INSERT INTO sales_range VALUES (2,  'Smirnoff', 500,  '2022-02-10');
+INSERT INTO sales_range VALUES (3,  'Ford',     2000, '2022-04-30');
you can put it into one INSERT. like
INSERT INTO sales_range VALUES (1,  'May',      1000, '2022-01-31'),
(1,  'May',      1000, '2022-01-31');
which can make the regress test faster.
(apply the logic to other places in src/test/regress/sql/partition_merge.sql)

+ key = RelationGetPartitionKey(parent);
+ strategy = get_partition_strategy(key);
+
+ if (strategy == PARTITION_STRATEGY_HASH)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("partition of hash-partitioned table cannot be merged")));
This error case doesn't seem to have a related test, and adding one
would be great.

per
https://www.postgresql.org/docs/current/error-message-reporting.html
"The extra parentheses were required before PostgreSQL version 12, but
are now optional."
so now you can remove the extra parentheses.
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("partition of hash-partitioned table cannot be merged"));


+ if (!partRel->rd_rel->relispartition)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("\"%s\" is not a partition",
+ RelationGetRelationName(partRel))));
+
+ if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_TABLE),
+ errmsg("relation \"%s\" is not a partition of relation \"%s\"",
+ RelationGetRelationName(partRel),
+ RelationGetRelationName(rel))));
we can make the first error message like the second one.
errmsg("\"%s\" is not a partition of \"%s\"....)


+ case AT_MergePartitions:
+ {
+ PartitionCmd *partcmd = (PartitionCmd *) cmd->def;
+
+ if (list_length(partcmd->partlist) < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("list of new partitions should contain at least two items")));
This also seems to have no tests.
adding a dummy one should be ok.


We added List *partlist into PartitionCmd
typedef struct PartitionCmd
{
    NodeTag        type;
    RangeVar   *name;            /* name of partition to attach/detach */
    PartitionBoundSpec *bound;    /* FOR VALUES, if attaching */
    List       *partlist;        /* list of partitions, for MERGE/SPLIT
                                 * PARTITION command */
    bool        concurrent;
} PartitionCmd;
then in src/backend/parser/gram.y
we should use
cmd->partlist = NIL;
instead of
cmd->partlist = NULL;
We also need comments explaining PartitionCmd.name
meaning for ALTER TABLE MERGE PARTITIONS INTO?


transformPartitionCmdForMerge
+ partOid = RangeVarGetRelid(name, NoLock, false);
here "NoLock" seems not right?
For example, after "RangeVarGetRelid(name, NoLock, false);"
before checkPartition, I drop the table test.t in another session.
i got the following error:
ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022, test.t) INTO sales_feb_mar_apr2022;
ERROR:  could not open relation with OID 18164

  186. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-04T19:44:26Z

    Hi!
Thank you very much for review.

1.
 >you can put it into one INSERT. like
 >INSERT INTO sales_range VALUES (1,  'May',      1000, '2022-01-31'),
 >(1,  'May',      1000, '2022-01-31');
 >which can make the regress test faster.
 >(apply the logic to other places in 
src/test/regress/sql/partition_merge.sql)

Test changed.


2.
 >+ errmsg("partition of hash-partitioned table cannot be merged")));
 >This error case doesn't seem to have a related test, and adding one
 >would be great.

Added test for hash partitioned table.


3.
 >per
 >https://www.postgresql.org/docs/current/error-message-reporting.html
 >"The extra parentheses were required before PostgreSQL version 12, but
 >are now optional."
 >so now you can remove the extra parentheses.

Extra parentheses removed.


4.
 >we can make the first error message like the second one.
 >errmsg("\"%s\" is not a partition of \"%s\"....)

Error message
errmsg("relation \"%s\" is not a partition of relation \"%s\""
occurs in two more places in the code.
I think it's better to keep this error message (for consistency).


5.
 >+ errmsg("list of new partitions should contain at least two items")));
 >This also seems to have no tests.
 >adding a dummy one should be ok.

Test added.


6.
 >We added List *partlist into PartitionCmd
 >typedef struct PartitionCmd
 >we should use
 >cmd->partlist = NIL;
 >instead of
 >cmd->partlist = NULL;
 >We also need comments explaining PartitionCmd.name
 >meaning for ALTER TABLE MERGE PARTITIONS INTO?

Fixed.


7.

 >transformPartitionCmdForMerge
 >+ partOid = RangeVarGetRelid(name, NoLock, false);
 >here "NoLock" seems not right?

AccessExclusiveLock on partitioned table protects only the DEFAULT 
partition. Fixed.


P.S. Similar changes were made for the second commit with SPLIT PARTITION.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  187. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-06-05T01:41:47Z

    Hi Dmitry!

On Wed, Jun 4, 2025 at 10:44 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Thank you very much for review.

Thank you for your work on this patch.  I have some additional notes on
this patch.

Why don't you use *existing_relation_id argument of
RangeVarGetAndCheckCreationNamespace(), when it is called from
createPartitionTable() and ATExecSplitPartition()?  This argument provide
an elegant way to find a duplicate table with the same name.

It also seems that 0002 patch has the following error message, which aren't
experienced in the regression tests.

+                   datum = list_nth(spec->upperdatums, abs(cmpval) - 1);
+                   ereport(ERROR,
+                           errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+                           errmsg("upper bound of partition \"%s\" is not
equal to upper bound of split partition",
+                                  relname),
+                           parser_errposition(pstate, datum->location));


+               ereport(ERROR,
+                       errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+                       errmsg("new partition \"%s\" cannot have this value
because split partition does not have",
+                              relname),
+                       parser_errposition(pstate, overlap_location));


+       ereport(ERROR,
+               errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+               errmsg("new partitions do not have value %s but split
partition does",
+                      searchNull ? "NULL" :
get_list_partvalue_string(notFoundVal)));


+                           ereport(ERROR,
+
errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+                                   errmsg("DEFAULT partition should be
one"),
+                                   parser_errposition(pstate,
sps->name->location));

+       ereport(ERROR,
+               errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+               errmsg("one partition in the list should be DEFAULT because
split partition is DEFAULT"),
+               parser_errposition(pstate, ((SinglePartitionSpec *)
linitial(partlist))->name->location));


+       ereport(ERROR,
+               errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+               errmsg("new partition cannot be DEFAULT because DEFAULT
partition already exists"),
+               parser_errposition(pstate, spsDef->name->location));

+               ereport(ERROR,
+                       errcode(ERRCODE_CHECK_VIOLATION),
+                       errmsg("can not find partition for split partition
row"),
+                       errtable(splitRel));

------
Regards,
Alexander Korotkov
Supabase

  188. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-05T04:22:40Z

    hi.

the following are review of
v40-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch

ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022) INTO sales_feb_mar_apr2022;
There are no tests when sales_feb2022 or sales_mar2022 have any constraints.
a partition can have its own constraint.

What should we do when any to be merged partition has constraints?
----------------------------------------------------------------
DROP TABLE IF EXISTS sales_range cascade;
CREATE TABLE sales_range (salesperson_id INT, salesperson_name text,
sales_amount INT generated always as (1) stored, sales_date DATE)
PARTITION BY RANGE (sales_date);
CREATE TABLE sales_feb2022 PARTITION OF sales_range FOR VALUES FROM
('2022-02-01') TO ('2022-03-01');
CREATE TABLE sales_mar2022 PARTITION OF sales_range FOR VALUES FROM
('2022-03-01') TO ('2022-04-01');
ALTER TABLE sales_feb2022 ALTER COLUMN sales_amount SET EXPRESSION AS (10);
ALTER TABLE sales_mar2022 ALTER COLUMN sales_amount SET EXPRESSION AS (20);

ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022) INTO sales_feb_mar2022;
with v40, sales_feb_mar2022 column sales_int generated expression is
(generated always as (1) stored)

maybe this is what we expected.
but we should have some tests on it.
----------------------------------------------------------------
DROP TABLE IF EXISTS sales_range cascade;
CREATE TABLE sales_range (salesperson_id INT, salesperson_name text,
sales_amount INT, sales_date DATE) PARTITION BY RANGE (sales_date);
CREATE TABLE sales_feb2022 PARTITION OF sales_range FOR VALUES FROM
('2022-02-01') TO ('2022-03-01');
CREATE TABLE sales_mar2022 PARTITION OF sales_range FOR VALUES FROM
('2022-03-01') TO ('2022-04-01');

CREATE VIEW x AS SELECT * FROM sales_mar2022;
ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022) INTO sales_feb_mar2022;
ERROR:  cannot drop table public.sales_mar2022 because other objects
depend on it
DETAIL:  view public.x depends on table public.sales_mar2022
HINT:  Use DROP ... CASCADE to drop the dependent objects too.

Maybe this is expected, but we need to mention it somewhere and have
some tests on it.
saying that MERGE PARTITIONS will effectively drop the partitions, so
if any object depends on that partition
then MERGE PARTITIONS can not be done.
----------------------------------------------------------------
+ */
+static Relation
+createPartitionTable(RangeVar *newPartName, Relation modelRel, Oid ownerId)
+{
+ Relation newRel;
+ Oid newRelId;
+ TupleDesc descriptor;
+ List   *colList = NIL;
+ Oid relamId;
+ Oid namespaceId;
+
+ /* If existing rel is temp, it must belong to this session */
+ if (modelRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP &&
+ !modelRel->rd_islocaltemp)
+ ereport(ERROR,
+ errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("cannot create as partition of temporary relation of another
session"));

Looking at it, modelRel is the partitioned table we called ALTER TABLE.
for example:
ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022) INTO sales_feb_mar2022;
modelRel is sales_range.

so this error check can be performed as early as the
transformPartitionCmdForMerge stage?
----------------------------------------------------------------
+ /* Look up the access method for new relation. */
+ relamId = (modelRel->rd_rel->relam != InvalidOid) ?
modelRel->rd_rel->relam : HEAP_TABLE_AM_OID;
looking at the output of "select * from pg_am;".

i think, we can do the following way:
if (modelRel->rd_rel->relam)
elog(ERROR, "error");
relamId = modelRel->rd_rel->relam;
----------------------------------------------------------------
Attached is some refactoring in moveMergedTablesRows, hope it's straightforward.

for example:
+ /* Extract data from old tuple. */
+ slot_getallattrs(srcslot);
+
+ if (tuple_map)
+ {
+ /* Need to use map to copy attributes. */
+ insertslot = execute_attr_map_slot(tuple_map->attrMap, srcslot, dstslot);
+ }
execute_attr_map_slot will call "slot_getallattrs(srcslot);" so the
first one is unncessary.

+ srcslot = MakeSingleTupleTableSlot(RelationGetDescr(mergingPartition),
+   table_slot_callbacks(mergingPartition));
can change to
srcslot = table_slot_create(mergingPartition, NULL);

  189. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-05T07:16:34Z

    hi.
bug in transformPartitionCmdForMerge "equal(name, name2))"

+static void
+transformPartitionCmdForMerge(CreateStmtContext *cxt, PartitionCmd *partcmd)
+{
+
+
+ foreach(listptr, partcmd->partlist)
+ {
+ RangeVar   *name = (RangeVar *) lfirst(listptr);
+
+ /* Partitions in the list should have different names. */
+ for_each_cell(listptr2, partcmd->partlist, lnext(partcmd->partlist, listptr))
+ {
+ RangeVar   *name2 = (RangeVar *) lfirst(listptr2);
+
+ if (equal(name, name2))
+ ereport(ERROR,
+ errcode(ERRCODE_DUPLICATE_TABLE),
+ errmsg("partition with name \"%s\" is already used", name->relname),
+ parser_errposition(cxt->pstate, name2->location));
+ }


ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
public.sales_feb2022) INTO sales_feb_mar2022;
ERROR:  lower bound of partition "sales_feb2022" conflicts with upper
bound of previous partition "sales_feb2022"
in this context. "sales_feb2022" is the same as "public.sales_feb2022".

  190. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-05T12:24:42Z

    hi.

When using ALTER TABLE ... MERGE PARTITIONS, some of the new
partition's properties will
not be inherited from to be merged partitions; instead, they will be directly
copied from the root partitioned table.
so we need to test this behavior.

The attached test file is for test table properties:
(COMMENTS, COMPRESSION, DEFAULTS, GENERATED, STATISTICS, STORAGE).

STATISTICS: to be merged partition's STATISTICS will be dropped.
COMMENTS: to be merged partition's COMMENTS will be dropped.

  191. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-05T16:40:12Z

    Hi Alexander!
Thanks for your notes!

1.
 >Why don't you use *existing_relation_id argument of
 >RangeVarGetAndCheckCreationNamespace(), when it is called from
 >createPartitionTable() and ATExecSplitPartition()?  This argument
 >provide an elegant way to find a duplicate table with the same name.

Code changed.

2.
 >It also seems that 0002 patch has the following error message, which
 >aren't experienced in the regression tests.

2a. Added tests for these error messages:
+errmsg("upper bound of partition \"%s\" is not equal to upper bound of 
split partition",
+errmsg("new partition \"%s\" cannot have this value because split 
partition does not have",
+errmsg("DEFAULT partition should be one"),
+errmsg("new partition cannot be DEFAULT because DEFAULT partition 
already exists"),

2b. Tests for these error messages already exists:
+errmsg("new partitions do not have value %s but split partition does",
+errmsg("one partition in the list should be DEFAULT because split 
partition is DEFAULT"),

2c. The error message
+errmsg("can not find partition for split partition row"),
cannot be reproduced using regression tests, because it is issued when
partition contains a record that should not be there (i.e. when the
database is corrupted).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  192. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-05T16:41:22Z

    Hi, jian he!

Thank you very much for your emails!
Unfortunately, due to urgent tasks at my work, I do not have time to 
look through your notes today and tomorrow.
But I will definitely do it at the beginning of next week.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  193. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-06T02:28:00Z

    hi.
one more patch for regress tests.

ALTER TABLE salespeople MERGE PARTITIONS (salespeople10_20,
salespeople20_30, salespeople30_40) INTO salespeople10_40;
the trigger on the merged partition  will be dropped.
For example, here, trigger on salespeople10_20 will be dropped.

I am surprised that partition_merge.sql doesn't have much \d+ command.
so I added two, which is necessary IMHO.

  194. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-06T07:13:22Z

    hi.

in createTableConstraints
+ /* Add a pre-cooked default expression. */
+ StoreAttrDefault(newRel, num, def, true);
+
+ /* Store CHECK constraints. */
+ StoreConstraints(newRel, cookedConstraints, false);
Here, StoreConstraints last argument should be set to true?
see also StoreAttrDefault.


+static void
+createTableConstraints(Relation modelRel, Relation newRel)
+ /*
+ * Construct a map from the LIKE relation's attnos to the child rel's.
+ * This re-checks type match etc, although it shouldn't be possible to
+ * have a failure since both tables are locked.
+ */
+ attmap = build_attrmap_by_name(RelationGetDescr(newRel),
+   tupleDesc,
+   false);
+
+ /* Cycle for default values. */
+ for (parent_attno = 1; parent_attno <= tupleDesc->natts; parent_attno++)
+ {
+ Form_pg_attribute attribute = TupleDescAttr(tupleDesc,
+ parent_attno - 1);
+
+ /* Ignore dropped columns in the parent. */
+ if (attribute->attisdropped)
+ continue;
+
+ /* Copy default, if present and it should be copied. */
+ if (attribute->atthasdef)
+ {
+ Node   *this_default = NULL;
+ AttrDefault *attrdef = constr->defval;
+ bool found_whole_row;
+ int16 num;
+ Node   *def;
+
+ /* Find default in constraint structure */
+ for (int i = 0; i < constr->num_defval; i++)
+ {
+ if (attrdef[i].adnum == parent_attno)
+ {
+ this_default = stringToNode(attrdef[i].adbin);
+ break;
+ }
+ }
+ if (this_default == NULL)
+ elog(ERROR, "default expression not found for attribute %d of
relation \"%s\"",
+ parent_attno, RelationGetRelationName(modelRel));
you can use TupleDescGetDefault, build_generation_expression
to simplify the above code.

The attached patch fixes the above issues.
it is based on v41-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch
----------------------
Do getAttributesList need to care about pg_attribute.attidentity?
currently MERGE PARTITION seems to work fine with identity columns,
this issue i didn't dig deeper.

I am wondering right after createPartitionTable,
do we need a CommandCounterIncrement?
because later moveMergedTablesRows will use the output of createPartitionTable.

  195. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-09T22:48:33Z

    Hi, Jian He!

Thanks for the suggestions and patches!
This email contains comments to three emails (05/06/2025).
I hope to read two emails (for 06/06/2025) tomorrow.

1.
 >What should we do when any to be merged partition has constraints?
 >...
 >Maybe this is expected, but we need to mention it somewhere and have
 >some tests on it saying that MERGE PARTITIONS will effectively drop
 >the partitions, so if any object depends on that partition
 >then MERGE PARTITIONS can not be done.

Added following phrases to the documentation (I hope this should be 
enough?):

If merged partitions have individual constraints, those constraints will 
be dropped because command uses partitioned table as a model to create 
the constraints.
If merged partitions have some objects dependent on them, the command 
can not be done (CASCADE is not used, an error will be returned).


2.
 > ... so this error check can be performed as early as the
 >transformPartitionCmdForMerge stage?

Function createPartitionTable will be used for various other cases 
besides MERGE PARTITIONS: for SPLIT PARTITION, for PARTITION BY 
REFERENCE (I hope).
So I think it's better to minimize the amount of code and not move the 
same one check into different functions (transformPartitionCmdForMerge, 
transformPartitionCmdForSplit, ...).


3.
 >i think, we can do the following way:
 >if (modelRel->rd_rel->relam)
 >  elog(ERROR, "error");
 >relamId = modelRel->rd_rel->relam;

Can you clarify what is reason to change the current AM-logic for 
creating a new partition?

+	/* Look up the access method for new relation. */
+	relamId = (modelRel->rd_rel->relam != InvalidOid) ? 
modelRel->rd_rel->relam : HEAP_TABLE_AM_OID;

(If AM is set for a partitioned table, then use it, otherwise use AM for 
heap tables.)


4.
 > Attached is some refactoring in moveMergedTablesRows, hope it's 
straightforward.

Thanks, these changes are useful.


5.
 >bug in transformPartitionCmdForMerge "equal(name, name2))"
 > ...
 >ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
 >public.sales_feb2022) INTO sales_feb_mar2022;
 >ERROR:  lower bound of partition "sales_feb2022" conflicts with upper
 >bound of previous partition "sales_feb2022"
 >in this context. "sales_feb2022" is the same as "public.sales_feb2022".

Added check and test for this case.


6.
 >When using ALTER TABLE ... MERGE PARTITIONS, some of the new
 >partition's properties will not be inherited from to be merged
 >partitions; instead, they will be directly copied from the root
 >partitioned table.
 >So we need to test this behavior.
 >The attached test file is for test table properties:
 >(COMMENTS, COMPRESSION, DEFAULTS, GENERATED, STATISTICS, STORAGE).

Some tests already exist (GENERATED, DEFAULTS) - see 
partition_merge.sql, lines after:

+-- Test for:
+--   * composite partition key;
+--   * GENERATED column;
+--   * column with DEFAULT value.
...

But the complex test is probably also interesting.
Test added.

--

Similar changes are made for the second commit (SPLIT PARTITION).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  196. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-10T05:50:12Z

    On Tue, Jun 10, 2025 at 6:48 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 3.
>  >i think, we can do the following way:
>  >if (modelRel->rd_rel->relam)
>  >  elog(ERROR, "error");
>  >relamId = modelRel->rd_rel->relam;
>
> Can you clarify what is reason to change the current AM-logic for
> creating a new partition?
>
> +       /* Look up the access method for new relation. */
> +       relamId = (modelRel->rd_rel->relam != InvalidOid) ?
> modelRel->rd_rel->relam : HEAP_TABLE_AM_OID;
>
> (If AM is set for a partitioned table, then use it, otherwise use AM for
> heap tables.)
>
I only want to allow HEAP_TABLE_AM_OID to be used
in the merge partition,
I guess that would avoid unintended consequences.

I proposed change was
+if (modelRel->rd_rel->relam != HEAP_TABLE_AM_OID)
+   elog(ERROR, "only heap table method is allowed");
+ relamId = modelRel->rd_rel->relam;


RangeVarGetAndCheckCreationNamespace
was called first on ATExecMergePartitions, then on createPartitionTable.
Maybe we can pass the first  ATExecMergePartitions call result
to createPartitionTable to avoid calling it twice.


CREATE TABLE pp (a int, b int) PARTITION BY LIST(a);
CREATE TABLE pp_p1 PARTITION OF pp FOR VALUES IN (1, 2);
CREATE TABLE pp_p2 PARTITION OF pp FOR VALUES IN (3, 4);
INSERT INTO pp(a, b) SELECT random(min=>1, max=>6),
random(min=>1::int, max=>10) FROM generate_series(0, 4) i;
alter table pp add constraint cc check(a < 0) not valid;

ALTER TABLE pp MERGE PARTITIONS (pp_p1,  pp_p2) INTO pp_p1_2;
src4=# \d+ pp_p1_2
                                         Table "public.pp_p1_2"
 Column |  Type   | Collation | Nullable | Default | Storage |
Compression | Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
 a      | integer |           |          |         | plain   |
    |              |
 b      | integer |           |          |         | plain   |
    |              |
Partition of: pp FOR VALUES IN (1, 2, 3, 4)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY[1, 2, 3, 4])))
Check constraints:
    "cc" CHECK (a < 0)
Access method: heap

constraint cc on pp_p1_2 should be NOT VALID.
also if the partitioned table has NOT ENFORCED CHECK constraint, it
will cause segfault.
attached is a possible fix, and related tests.(based on v42).


cosmetic changes:
many of the "forach" can change to "foreach_node".
for example in ATExecMergePartitions.
we can change
``foreach(listptr, cmd->partlist)``
to
``foreach_node(RangeVar, name, cmd->partlist)`

  197. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Junwang Zhao <zhjwpku@gmail.com> — 2025-06-10T15:58:04Z

    Hi Dmitry,

On Tue, Jun 10, 2025 at 6:48 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi, Jian He!
>
> Thanks for the suggestions and patches!
> This email contains comments to three emails (05/06/2025).
> I hope to read two emails (for 06/06/2025) tomorrow.
>
> 1.
>  >What should we do when any to be merged partition has constraints?
>  >...
>  >Maybe this is expected, but we need to mention it somewhere and have
>  >some tests on it saying that MERGE PARTITIONS will effectively drop
>  >the partitions, so if any object depends on that partition
>  >then MERGE PARTITIONS can not be done.
>
> Added following phrases to the documentation (I hope this should be
> enough?):
>
> If merged partitions have individual constraints, those constraints will
> be dropped because command uses partitioned table as a model to create
> the constraints.
> If merged partitions have some objects dependent on them, the command
> can not be done (CASCADE is not used, an error will be returned).
>
>
> 2.
>  > ... so this error check can be performed as early as the
>  >transformPartitionCmdForMerge stage?
>
> Function createPartitionTable will be used for various other cases
> besides MERGE PARTITIONS: for SPLIT PARTITION, for PARTITION BY
> REFERENCE (I hope).
> So I think it's better to minimize the amount of code and not move the
> same one check into different functions (transformPartitionCmdForMerge,
> transformPartitionCmdForSplit, ...).
>
>
> 3.
>  >i think, we can do the following way:
>  >if (modelRel->rd_rel->relam)
>  >  elog(ERROR, "error");
>  >relamId = modelRel->rd_rel->relam;
>
> Can you clarify what is reason to change the current AM-logic for
> creating a new partition?
>
> +       /* Look up the access method for new relation. */
> +       relamId = (modelRel->rd_rel->relam != InvalidOid) ?
> modelRel->rd_rel->relam : HEAP_TABLE_AM_OID;
>
> (If AM is set for a partitioned table, then use it, otherwise use AM for
> heap tables.)
>
>
> 4.
>  > Attached is some refactoring in moveMergedTablesRows, hope it's
> straightforward.
>
> Thanks, these changes are useful.
>
>
> 5.
>  >bug in transformPartitionCmdForMerge "equal(name, name2))"
>  > ...
>  >ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
>  >public.sales_feb2022) INTO sales_feb_mar2022;
>  >ERROR:  lower bound of partition "sales_feb2022" conflicts with upper
>  >bound of previous partition "sales_feb2022"
>  >in this context. "sales_feb2022" is the same as "public.sales_feb2022".
>
> Added check and test for this case.
>
>
> 6.
>  >When using ALTER TABLE ... MERGE PARTITIONS, some of the new
>  >partition's properties will not be inherited from to be merged
>  >partitions; instead, they will be directly copied from the root
>  >partitioned table.
>  >So we need to test this behavior.
>  >The attached test file is for test table properties:
>  >(COMMENTS, COMPRESSION, DEFAULTS, GENERATED, STATISTICS, STORAGE).
>
> Some tests already exist (GENERATED, DEFAULTS) - see
> partition_merge.sql, lines after:
>
> +-- Test for:
> +--   * composite partition key;
> +--   * GENERATED column;
> +--   * column with DEFAULT value.
> ...
>
> But the complex test is probably also interesting.
> Test added.
>
> --
>
> Similar changes are made for the second commit (SPLIT PARTITION).
>
> --
> With best regards,
> Dmitry Koval
>
> Postgres Professional: http://postgrespro.com

I had one trivial comment and a question if you don't mind.

+ /* If existing rel is temp, it must belong to this session */
+ if (modelRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP &&
+ !modelRel->rd_islocaltemp)
+ ereport(ERROR,
+ errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("cannot create as partition of temporary relation of another
session"));

Would it be better to use RELATION_IS_OTHER_TEMP in this case?
I noticed that while other parts of tablecmds.c don’t use the macro,
all other files consistently use RELATION_IS_OTHER_TEMP.

+ /*
+ * We intended to create the partition with the same persistence as the
+ * parent table, but we still need to recheck because that might be
+ * affected by the search_path.  If the parent is permanent, so must be
+ * all of its partitions.
+ */

I have trouble understanding how this is possible, can you kindly
give me some guidance on this logic?

-- 
Regards
Junwang Zhao

  198. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-11T00:06:27Z

    Hi!

This email contains comments to three emails
(2 x 06.06.2025 + 1 x 10.06.2025).

1.
 >I am surprised that partition_merge.sql doesn't have much \d+ command.
 >so I added two, which is necessary IMHO.

I think that a lot of \d+ commands can make the out-files difficult to
read. But I agree, that test for triggers is useful. Test for triggers
added to test of partition properties.


2.
 >Here, StoreConstraints last argument should be set to true?
 >see also StoreAttrDefault.

Thanks, this is correct.


 >you can use TupleDescGetDefault, build_generation_expression
 >to simplify the above code.

Should we use ATTRIBUTE_GENERATED_VIRTUAL [1] and
build_generation_expression [2] here? Function expandTableLikeClause
("CREATE TABLE ... LIKE ..." clause) does not use GENERATED VIRTUAL
yet ...
  


 >Do getAttributesList need to care about pg_attribute.attidentity?
 >currently MERGE PARTITION seems to work fine with identity columns,
 >this issue i didn't dig deeper.

Probably after commit [3] partition's identity columns shares the
identity space (i.e. underlying sequence) as the corresponding
columns of the partitioned table. So call BuildDescForRelation in
createPartitionTable function should copy pg_attribute.attidentity
for new partition.


 >I am wondering right after createPartitionTable,
 >do we need a CommandCounterIncrement?
 >because later moveMergedTablesRows will use the output of
 >createPartitionTable.

We call CommandCounterIncrement in createPartitionTable function right
after heap_create_with_catalog (same code in create_toast_table,
make_new_heap, DefineRelation functions). We need an additional
CommandCounterIncrement call in case we use objects created after this
point. But we probably don't use these objects (in function
moveMergedTablesRows too).


3.
 >I only want to allow HEAP_TABLE_AM_OID to be used
 >in the merge partition,
 >I guess that would avoid unintended consequences.

Thanks for the clarification. Isn't this limitation too strong?
It is very likely that the user will create an AM based on
HEAP_TABLE_AM_OID, in which case the code should work.


 >RangeVarGetAndCheckCreationNamespace
 >was called first on ATExecMergePartitions, then on
 >createPartitionTable. Maybe we can pass the first
 >ATExecMergePartitions call result to createPartitionTable to avoid
 >calling it twice.

Unfortunately, this is not the case for SPLIT PARTITION. I will think
about it, but I am concerned that the createPartitionTable function
will be passed two related arguments - newPartName and namespaceId
(result of RangeVarGetAndCheckCreationNamespace).

Thanks for 
v42-0001-fix-MERGE-PARTITION-with-partitioned-table-not-enforc.no-cfbot! 
I forgot about not valid or not enforced constraints.


 >cosmetic changes:
 >many of the "forach" can change to "foreach_node".
 >for example in ATExecMergePartitions.
 >we can change
 >``foreach(listptr, cmd->partlist)``
 >to
 >``foreach_node(RangeVar, name, cmd->partlist)`

Changed.


Links.
------
[1] Virtual generated columns, 
https://github.com/postgres/postgres/commit/83ea6c54025bea67bcd4949a6d58d3fc11c3e21b
[2] Expand virtual generated columns in the planner, 
https://github.com/postgres/postgres/commit/1e4351af329f2949c679a215f63c51d663ecd715
[3] Support identity columns in partitioned tables, 
https://github.com/postgres/postgres/commit/699586315704a8268808e3bdba4cb5924a038c49


P.S. 2Junwang Zhao: sorry, I'll write an answer a little later.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  199. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-11T04:45:17Z

    hi.

we generally no need to worry about the partitioned table check constraint,
generated column does not apply to newly merged partitions.
since partitioned table constraints apply to each individual partition,
including newly created partitions.

However, there are corner cases:
constraints include tableoid column reference.
say pk_1, pk_2 is the partition of pk.
alter table pk add constraint cc check(tableoid <> 18259);

Here, Constraint cc will check whether the tableoid of pk_1 or pk_2 meets the
specified criteria. Similarly, if partitions are merged, the newly merged
partition must also be evaluated against the constraint cc to ensure
it is satisfied.

----------------
The attached patch ensures that the newly merged partition is evaluated against
all of its check constraints and that all stored generated columns are
recomputed, i guess this would be more safe.

Alternatively, we can use pull_varattnos to check whether a constraint or
generated expression contains a reference to the tableoid column.

  200. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-11T07:28:36Z

    On Wed, Jun 11, 2025 at 8:06 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
>  >Do getAttributesList need to care about pg_attribute.attidentity?
>  >currently MERGE PARTITION seems to work fine with identity columns,
>  >this issue i didn't dig deeper.
>
> Probably after commit [3] partition's identity columns shares the
> identity space (i.e. underlying sequence) as the corresponding
> columns of the partitioned table. So call BuildDescForRelation in
> createPartitionTable function should copy pg_attribute.attidentity
> for new partition.
>
but BuildDescForRelation is based on getAttributesList,
in getAttributesList, assign pg_attribute.attidentity to def->identity
should be safe, IMHO.

+     <para>
+      If merged partitions have different owners, an error will be generated.
+      The owner of the merged partitions will be the owner of the new
partition.
+     </para>
+     <para>
+      It is the user's responsibility to setup <acronym>ACL</acronym> on the
+      new partition.
+     </para>
since they <para> are related, these two can be one <para>?


+     <para>
+      If merged partitions have individual constraints, those constraints will
+      be dropped because command uses partitioned table as a model to create
+      the constraints.
+     </para>

I feel like it's not fully accurate, the following is what I can come up with:
+     <para>
+ When partitions are merged, any individual objects belonging to those
+ partitions, such as constraints or statistics will be dropped. This occurs
+ because ALTER TABLE MERGE PARTITIONS uses the partitioned table itself as the
+ template to define these objects.
+     </para>


>
>  >I am wondering right after createPartitionTable,
>  >do we need a CommandCounterIncrement?
>  >because later moveMergedTablesRows will use the output of
>  >createPartitionTable.
>
> We call CommandCounterIncrement in createPartitionTable function right
> after heap_create_with_catalog (same code in create_toast_table,
> make_new_heap, DefineRelation functions). We need an additional
> CommandCounterIncrement call in case we use objects created after this
> point. But we probably don't use these objects (in function
> moveMergedTablesRows too).
>
As mentioned in the previous thread [1], moveMergedTablesRows need
latest relcache entry for newPartRel. so I guess, put one
CommandCounterIncrement at the end of createPartitionTable
should be fine, which I already did in [1].
[1]: https://postgr.es/m/CACJufxH3mfNYfHy9+dCUZPhOsmVRtJUJbWU1vH248Lg0eZjhzQ@mail.gmail.com

> 3.
>  >I only want to allow HEAP_TABLE_AM_OID to be used
>  >in the merge partition,
>  >I guess that would avoid unintended consequences.
>
> Thanks for the clarification. Isn't this limitation too strong?
> It is very likely that the user will create an AM based on
> HEAP_TABLE_AM_OID, in which case the code should work.
>
ok.


if you looking at ATExecDetachPartition, we have:
    /*
     * Detaching the partition might involve TOAST table access, so ensure we
     * have a valid snapshot.
     */
    PushActiveSnapshot(GetTransactionSnapshot());
    /* Do the final part of detaching */
    DetachPartitionFinalize(rel, partRel, concurrent, defaultPartOid);
    PopActiveSnapshot();

do we need do the same to the following DetachPartitionFinalize:
    foreach_ptr(RelationData, mergingPartition, mergingPartitionsList)
    {
        /* Remove the pg_inherits row first. */
        RemoveInheritance(mergingPartition, rel, false);
        /* Do the final part of detaching. */
        DetachPartitionFinalize(rel, mergingPartition, false, defaultPartOid);
    }

  201. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-11T13:10:00Z

    Hi, Junwang Zhao!

Thank you for note.

1.
 >Would it be better to use RELATION_IS_OTHER_TEMP in this case?
 >I noticed that while other parts of tablecmds.c don’t use the macro,
 >all other files consistently use RELATION_IS_OTHER_TEMP.

Agreed, RELATION_IS_OTHER_TEMP is better. Changed.
The fix will be in the next patch.


2.
 >+/*
 >+* We intended to create the partition with the same persistence as the
 >+* parent table, but we still need to recheck because that might be
 >+* affected by the search_path.  If the parent is permanent, so must be
 >+* all of its partitions.
 >+*/

 >I have trouble understanding how this is possible, can you kindly
 >give me some guidance on this logic?

Perhaps this is best explained with an example.
(see src/test/regress/sql/partition_merge.sql).

(a) Create permanent table "t":

SET search_path = partitions_merge_schema, pg_temp, public;
CREATE TABLE t (i int) PARTITION BY RANGE (i);
CREATE TABLE tp_0_1 PARTITION OF t FOR VALUES FROM (0) TO (1);
CREATE TABLE tp_1_2 PARTITION OF t FOR VALUES FROM (1) TO (2);

(b) Attempt to merge persistent partitions tp_0_1, tp_1_2 into
temporary partition tp_0_2:

SET search_path = pg_temp, partitions_merge_schema, public;
-- Can't merge persistent partitions into a temporary partition
ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO tp_0_2;

(c) "ALTER TABLE ... MERGE PARTITIONS ..." will return an error:

ERROR:  cannot create a temporary relation as partition of permanent 
relation "t"

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  202. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-11T13:22:22Z

    hi.

+ /* Copy data from merged partitions to new partition. */
+ moveMergedTablesRows(rel, mergingPartitionsList, newPartRel);
+
+ /* Drop the current partitions before attaching the new one. */
+ foreach_ptr(RelationData, mergingPartition, mergingPartitionsList)
+ {
+ ObjectAddress object;
+
+ /* Get relation id before table_close() call. */
+ object.objectId = RelationGetRelid(mergingPartition);
+ object.classId = RelationRelationId;
+ object.objectSubId = 0;
+
+ /* Keep the lock until commit. */
+ table_close(mergingPartition, NoLock);
+
+ performDeletion(&object, DROP_RESTRICT, 0);
+ }
+ list_free(mergingPartitionsList);
In here, should performDeletion last flags have PERFORM_DELETION_INTERNAL?

also this is not ideal, imagine you first did all the main work in
moveMergedTablesRows,
then suddenly error out, saying:

ERROR:  cannot drop table public.pk_1 because other objects depend on it
DETAIL:  view public.v1 depends on table public.pk_1
HINT:  Use DROP ... CASCADE to drop the dependent objects too.
(this errhint, seems not ideal in this context)

We can perform a preliminary check to determine whether dropping a partition is
allowed, and raise an error if it's not. To do it, I invented a new
function, performDeletionCheck to verify whether an object can be
safely dropped.

please check attached, it was based on v43.

  203. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-12T07:31:04Z

    hi.

+/*
+ * check_two_partitions_bounds_range
+ *
+ * (function for BY RANGE partitioning)
+ *
+ * This is a helper function for check_partitions_for_split() and
+ * calculate_partition_bound_for_merge().
check_partitions_for_split does not exist in v43-0001.


+ /*
+ * Rename the existing partition with a temporary name, leaving it
+ * free for the new partition.  We don't need to care about this
+ * in the future because we're going to eventually drop the
+ * existing partition anyway.
+ */
+ RenameRelationInternal(RelationGetRelid(sameNamePartition),
+   tmpRelName, false, false);
the third argument, is_internal should set to true?


+ /* Cycle for CHECK constraints. */
+ for (ccnum = 0; ccnum < constr->num_check; ccnum++)
+ {
+ char   *ccname = constr->check[ccnum].ccname;
+ char   *ccbin = constr->check[ccnum].ccbin;
+ bool ccenforced = constr->check[ccnum].ccenforced;
+ bool ccnoinherit = constr->check[ccnum].ccnoinherit;

a partitioned table can not have NO INHERIT check constraint,
you may see StoreRelCheck.
we can add an Assert: Assert(!ccnoinherit);

+ /* Reproduce not-null constraints. */
+ if (constr->has_not_null)
+ {
+ List   *nnconstraints;
+
+ nnconstraints = RelationGetNotNullConstraints(RelationGetRelid(modelRel),
+  false, true);
as mentioned in above, partitioned table cannot have NO INHERIT constraint,
maybe we should set RelationGetNotNullConstraints last argument to false


/*
 * calculate_partition_bound_for_merge
 *
 * Calculates the bound of merged partition "spec" by using the bounds of
 * partitions to be merged.
 *
 * parent:            partitioned table
 * partNames:        names of partitions to be merged
 * partOids:        Oids of partitions to be merged
 * spec (out):        bounds specification of the merged partition
 * pstate:            pointer to ParseState struct for determine error position
 */
void
calculate_partition_bound_for_merge(Relation parent,
                                    List *partNames,
                                    List *partOids,
                                    PartitionBoundSpec *spec,
                                    ParseState *pstate)

if we are within calculate_partition_bound_for_merge,
then we at least hold AccessShareLock for all the partOids
(see transformPartitionCmdForMerge)
partNames is a list of RangeVar one to one corresponding to partOids,
then we really do not need partNames at all for error messages handling, we can
just use get_rel_name.
so we don't need to pass partNames to calculate_partition_bound_for_merge
The attached patch is a rewrite for
calculate_partition_bound_for_merge and callees.
please let me know whether this improves code readability


in RelationBuildPartitionDesc
we have the following code pattern:
    foreach(cell, inhoids)
    {
        Oid            inhrelid = lfirst_oid(cell);
        HeapTuple    tuple;
        PartitionBoundSpec *boundspec = NULL;
        /* Try fetching the tuple from the catcache, for speed. */
        tuple = SearchSysCache1(RELOID, ObjectIdGetDatum(inhrelid));
        if (HeapTupleIsValid(tuple))
        {
            datum = SysCacheGetAttr(RELOID, tuple,
                                    Anum_pg_class_relpartbound,
                                    &isnull);
            if (!isnull)
                boundspec = stringToNode(TextDatumGetCString(datum));
            ReleaseSysCache(tuple);
        }
        if (boundspec == NULL)
        {
            pg_class = table_open(RelationRelationId, AccessShareLock);
            ScanKeyInit(&key[0],
                        Anum_pg_class_oid,
                        BTEqualStrategyNumber, F_OIDEQ,
                        ObjectIdGetDatum(inhrelid));
            scan = systable_beginscan(pg_class, ClassOidIndexId, true,
                                      NULL, 1, key);
....
}
I wonder if we should do the same in get_partition_bound_spec?
you may also see comments in RelationBuildPartitionDesc, partdesc.c line 203.

  204. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-12T11:03:13Z

    hi.
one more minor issue.

+ * defaultPart: true if one of split partitions is DEFAULT
+ * pstate: pointer to ParseState struct for determining error position
+ */
+static void
+check_two_partitions_bounds_range(Relation parent,
+  RangeVar *first_name,
+  PartitionBoundSpec *first_bound,
+  RangeVar *second_name,
+  PartitionBoundSpec *second_bound,
+  bool defaultPart,
+  ParseState *pstate)

v43-0001 doesn't have the SPLIT PARTITION feature.
maybe we need to remove the argument (bool defaultPart)
from check_two_partitions_bounds_range, aslo remove the comments.
then we can add it on 0002 SPLIT PARTITION patch.

  205. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-12T20:36:25Z

    Hi, Jian He!

Thanks for the notes and patches (again).
I read a part of emails, I hope to read the rest emails tomorrow.


1.
 >The attached patch ensures that the newly merged partition is
 >evaluated against all of its check constraints and that all stored
 >generated columns are recomputed, i guess this would be more safe.
 >v43-0001-MERGE-PARTITIONS-constraint-revalidation.no-cfbot

I modified the patch to apply it to the SPLIT PARTITION command too.


2.
 >but BuildDescForRelation is based on getAttributesList,
 >in getAttributesList, assign pg_attribute.attidentity to def->identity
 >should be safe, IMHO.

You are right. Corrected.


3.
+<para>
+If merged partitions have different owners, an error will be generated
 >since they <para> are related, these two can be one <para>?

Changed.


4.
 >I feel like it's not fully accurate, the following is what I can come
 >up with:
 >+<para>
 >+ When partitions are merged, any individual objects belonging to

Changed.


5.
 > /*
 >  * Detaching the partition might involve TOAST table access, so ensure
 >  * we have a valid snapshot.
 >  */
 > PushActiveSnapshot(GetTransactionSnapshot());
 > /* Do the final part of detaching */
 > DetachPartitionFinalize(rel, partRel, concurrent, defaultPartOid);
 > PopActiveSnapshot();
 >do we need do the same to the following DetachPartitionFinalize:
 >...

Thanks. This needs to be done, especially after the recent commit [1].
Fixed.


Links.
------
[1] Ensure we have a snapshot when updating various system catalogs, 
https://github.com/postgres/postgres/commit/706054b11b959c865c0c7935c34d92370d7168d4

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  206. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-13T06:29:23Z

    On Fri, Jun 13, 2025 at 4:36 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi, Jian He!
>
> Thanks for the notes and patches (again).
> I read a part of emails, I hope to read the rest emails tomorrow.
>

hi.

in doc/src/sgml/ref/alter_table.sgml
<title>Parameters</title> section,
we also need explain
<replaceable class="parameter">partition_name1</replaceable>
and
<replaceable class="parameter">partition_name2</replaceable>
?


+     <para>
+      This form merges several partitions into the one partition of
the target table.
+      Hash-partitioning is not supported.
maybe we can change to
+    <para>
+   This form merges several partitions of the target table into a new
one partition.
+   Hash-partitioned target table is not supported


+      <para>
+       This command acquires an <literal>ACCESS EXCLUSIVE</literal> lock.
+       This is a significant limitation, which limits the usage of this
+       command with large partitioned tables under a high load.
+      </para>
would be better mentioning that the parent table and to be merged
partition will all take
<literal>ACCESS EXCLUSIVE</literal> lock.
for example, other places we have word like:

     <para>
      Attaching a partition acquires a
      <literal>SHARE UPDATE EXCLUSIVE</literal> lock on the parent table,
      in addition to the <literal>ACCESS EXCLUSIVE</literal> locks on the table
      being attached and on the default partition (if any).
     </para>
---------------------------------------------------------------------------------
+        <para>
+         For range- and list-partitioned tables the ranges and lists of values
+         of the merged partitions can be any.
+        </para>
we can change it to
<para>
For range-partitioned and list-partitioned tables, the partition
bounds specification can be arbitrary.
</para>


+        <para>
+         For range-partitioned tables it is necessary that the ranges
+         of the partitions <replaceable
class="parameter">partition_name1</replaceable>,
+         <replaceable class="parameter">partition_name2</replaceable>
[, ...] can
+         be merged into one range without spaces and overlaps
(otherwise an error
+         will be generated).
"without spaces and overlaps" seems not ideal, I think I found out the
perfect word for it: adjacent.
in [1], we have description like:

anyrange -|- anyrange → boolean
Are the ranges adjacent?
[1] https://www.postgresql.org/docs/current/functions-range.html
so we can change the above to

For range-partitioned tables, the ranges of the partitions partition_name1,
partition_name2, [...] must be adjacent in order to be merged. Otherwise, an
error will be raised. The resulting combined range will be the new
partition bound for the partition partition_name.

+      The new partition will have the same table access method as the parent.
+      If the parent table is persistent then the new partition is created
+      persistent.  If the parent table is temporary then the new partition
+      is also created temporary.  The new partition will also be created in
+      the same tablespace as the parent.
+     </para>
we can simplify the above as
" The new partition will inherit the same table access method, persistence
 type, and tablespace as the parent table.
"

+     <para>
+      If merged partitions have different owners, an error will be generated.
+      The owner of the merged partitions will be the owner of the new
partition.
+      It is the user's responsibility to setup <acronym>ACL</acronym> on the
+      new partition.
+     </para>
I think we also need to mention that individual ACL on the merged
partition will be dropped.

We have function signature
static void RemoveInheritance(Relation child_rel, Relation parent_rel,
bool expect_detached)
I found that "child_rel", "parent_rel" naming improves code
readability a lot for partitioned table contexts.
so we can change
+static void
+detachPartitionTable(Relation partRel, Relation rel, Oid defaultPartOid)
to
+static void
+detachPartitionTable(Relation partRel, Relation rel, Oid defaultPartOid)

We can also rename the ATExecMergePartitions argument (Relation rel)
to (Relation parent_rel), I think it will improve code reliability.

The attached patch is mainly documentation refactoring
and renaming function detachPartitionTable argument,
it is based on v44.

  207. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-13T20:06:55Z

    Hi!

Additional changes (in attached patch):
a. Added using pull_varattnos to check whether a CHECK constraint 
contains a reference to the tableoid column (only such CHECKs are 
recalculated).
b. Added test for recomputation of stored generated columns.


1.
 >We can perform a preliminary check to determine whether dropping a
 >partition is allowed, and raise an error if it's not. To do it, I
 >invented a new function, performDeletionCheck to verify whether an
 >object can be safely dropped.

Applied. I moved calls performDeletionCheck a bit earlier, right after
detachPartitionTable. Is it ok?


2.
 >check_partitions_for_split does not exist in v43-0001.

Fixed.


 >+ RenameRelationInternal(RelationGetRelid(sameNamePartition),
 >+   tmpRelName, false, false);
 >the third argument, is_internal should set to true?

Ok.


 >a partitioned table can not have NO INHERIT check constraint,
 >you may see StoreRelCheck.
 >we can add an Assert: Assert(!ccnoinherit);

Ok.


 >+ nnconstraints =
 >+     RelationGetNotNullConstraints(RelationGetRelid(modelRel),
 >+  false, true);
 >as mentioned in above, partitioned table cannot have NO INHERIT
 >constraint, maybe we should set RelationGetNotNullConstraints last
 >argument to false

Ok.


 >The attached patch is a rewrite for
 >calculate_partition_bound_for_merge and callees.
 >please let me know whether this improves code readability

I think these changes can be taken partially.
MERGE PARTITIONS and SPLIT PARTITION commands use the same function
check_two_partitions_bounds_range. For MERGE PARTITIONS we can pass
Oids instead of RangeVars (first_name/second_name arguments).
But for SPLIT PARTITION we cannot do this, because at this stage the
new partitions have not yet been created (there are only their names).
Different realizations of check_two_partitions_bounds_range functions
for MERGE PARTITIONS and SPLIT PARTITION are not very good. I think it's
better not change the check_two_partitions_bounds_range function.


3.
 >v43-0001 doesn't have the SPLIT PARTITION feature.
 >maybe we need to remove the argument (bool defaultPart)
 >from check_two_partitions_bounds_range, aslo remove the comments.
 >then we can add it on 0002 SPLIT PARTITION patch.

Changed.


4. I don't know English very well, so it's difficult for me to correct
the documentation. Thank you very much for the corrections!

 >would be better mentioning that the parent table and to be merged
 >partition will all take <literal>ACCESS EXCLUSIVE</literal> lock.

Ok, updated.


Other email notes fixed in patch
 >v44-0001-documentation-refactoring-based-on-v44.no-cfbot

Patch applied.


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  208. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-16T05:52:45Z

    hi.

static void checkPartition(Relation rel, Oid partRelOid)
function name checkPartition is not ideal, maybe we can change it to
CheckPartitionForMerge or MergePartitionCheck.
(attached v45-002 is error message refactoring for checkPartition,
I didn't change the name though.)


For the command:
ALTER TABLE pk MERGE PARTITIONS (pk_1, pk_2) INTO pk_1;
Acquiring AccessExclusiveLock on the partitions to be merged
(pk_1, pk_2) during transformPartitionCmdForMerge should be fine, IMHO.
Here’s why:

* The merged partitions (pk_1, pk_2) will be dropped in the end, so acquiring
AccessExclusiveLock is unavoidable for ALTER TABLE MERGE PARTITIONS.
* Taking an AccessShareLock first, then later acquiring AccessExclusiveLock
in ATExecMergePartitions unnecessarily wastes resources.
(acquire two locks, one stronger should be enough)
* Acquiring AccessExclusiveLock first helps avoid potential anomalies
caused by concurrent operations.

The attached patch refactors transformPartitionCmdForMerge and
ATExecMergePartitions based on the idea of acquiring AccessExclusiveLock on the
to be merged partitions during transformPartitionCmdForMerge


+ * Callback allows caller to check permissions or acquire additional locks
+ * prior to grabbing the relation lock.
Please see the above comments in RangeVarGetRelidExtended.

+ /*
+ * Search DEFAULT partition in the list. Lock partitions before
+ * calculating the boundary for resulting partition.
+ */
+ partOid = RangeVarGetRelid(name, AccessShareLock, false);
so the above transformPartitionCmdForMerge does not check if the
currently user have permission
or not, directly take a lock on RangeVar, name, which is a bug, we should
first do permission check then acquire a lock.

  209. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-16T09:33:52Z

    for v45.

+ foreach_ptr(CookedConstraint, ccon, cookedConstraints)
+ {
+ if (!ccon->skip_validation && ccon->contype == CONSTR_CHECK)
+ {
+ Bitmapset  *attnums = NULL;
+
+ pull_varattnos((Node *) ccon->expr, 1, &attnums);
+
+ /*
+ * Add check only if it contains tableoid
+ * (TableOidAttributeNumber).
+ */
+ if (bms_is_member(TableOidAttributeNumber -
FirstLowInvalidHeapAttributeNumber,
+  attnums))
+ {
+ NewConstraint *newcon;
+
+ newcon = (NewConstraint *) palloc0(sizeof(NewConstraint));
+ newcon->name = ccon->name;
+ newcon->contype = ccon->contype;
+ newcon->qual = ccon->expr;
+
+ tab->constraints = lappend(tab->constraints, newcon);
+ }
+ }
+ }

we need to expand the virtual generated column here,
otherwise, bms_is_member would  be not correct.
consider case like:

CREATE TABLE pp (f1 INT, f2 INT generated always as (f1 +
tableoid::int)) PARTITION BY RANGE (f1);
CREATE TABLE pp_1 (f2 INT generated always as (f1 + tableoid::int), f1 int);
ALTER TABLE pp ATTACH PARTITION pp_1 FOR VALUES FROM (-1) TO (10);
CREATE TABLE pp_2 (f2 INT generated always as (f1 + tableoid::int), f1 int);
ALTER TABLE pp ATTACH PARTITION pp_2 FOR VALUES FROM (10) TO (20);
ALTER TABLE PP add check (f2 > 0);

ALTER TABLE pp MERGE PARTITIONS (pp_1, pp_2) INTO pp_12;

In this context, the merge partition command  needs to evaluate the constraint
"pp_f2_check" again on pp_12.

attach minor diff fix this problem.

  210. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-16T20:15:41Z

    Hi!

1.
 >function name checkPartition is not ideal, maybe we can change it to
 >CheckPartitionForMerge or MergePartitionCheck.

I agree that this name is not ideal. But the "checkPartition" function
is used for both MERGE/SPLIT commands, so the word 'Merge' in the
prefix or suffix of the function name would not be correct ...
May be "checkPartitionForModification" or something like that?


 >The attached patch refactors transformPartitionCmdForMerge and
 >ATExecMergePartitions based on the idea of acquiring
 >AccessExclusiveLock on the to be merged partitions during
 >transformPartitionCmdForMerge

Thanks, applied.


2.
 >we need to expand the virtual generated column here,
 >otherwise, bms_is_member would  be not correct.
 >...
 >attach minor diff fix this problem.

Added patch and a bit modified test.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  211. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-17T07:51:50Z

    On Tue, Jun 17, 2025 at 4:15 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Added patch and a bit modified test.
>

hi.
Please check the attached patch for addressing the following issues.

+ else
+ {
+ ereport(ERROR,
+ errcode(ERRCODE_DUPLICATE_TABLE),
+ errmsg("relation \"%s\" already exists", cmd->name->relname));
+ }
There are no regress tests for this, I have added a simple test for it.


SELECT c.oid::pg_catalog.regclass, c.relkind, inhdetachpending,
pg_catalog.pg_get_expr(c.relpartbound, c.oid)
  FROM pg_catalog.pg_class c, pg_catalog.pg_inherits i
  WHERE c.oid = i.inhrelid AND i.inhparent = 'sales_range'::regclass
  ORDER BY pg_catalog.pg_get_expr(c.relpartbound, c.oid) = 'DEFAULT',
c.oid::pg_catalog.regclass::pg_catalog.text;

I found about 10 similar queries in partition_merge. We can replace them with
prepared statement—parsing once and executing multiple times. This not only
improves the readability of the test code but also slightly speeds up test
execution.


there are some test code pattern like:
+SELECT * FROM sales_range;
+SELECT * FROM sales_jan2022;
+SELECT * FROM partitions_merge_schema2.sales_feb_mar_apr2022;
+SELECT * FROM sales_others;
since sales_jan2022, sales_feb_mar_apr2022, sales_others
are partition of sales_range, we can simply use
+SELECT tableoid::regclass, * FROM sales_range ORDER BY tableoid,
salesperson_id;

After applying the attached patch, partition_merge.out size is 61379
without the size is: 66145.

The attached patch also addressed issues raised from Álvaro Herrera in [1].
instead of making function StoreConstraints external, using
AddRelationNewConstraints to install check constraints for new relation
in createTableConstraints

[1] https://postgr.es/m/202502031640.zem6orjmmxoz@alvherre.pgsql

  212. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-17T21:45:24Z

    Hi!

 >v46-0001-regress-test-refactoring-and-others.no-cfbot
Thanks, applied. Two comments:

 >+ errmsg("relation \"%s\" already exists", cmd->name->relname));
 >There are no regress tests for this, I have added a simple test for it.

Test for this is in the second commit (SPLIT PARTITION) but
it's okay to have test here.

 >+SELECT tableoid::regclass, * FROM sales_range ORDER BY ...

Similar changes are also made for SPLIT PARTITION.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  213. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-18T02:55:38Z

    hi.

The following are changes I made based on v47.
mainly comments refactoring, variable/argument renaming.
please see the attached patch.

+
+ /* Create the relation. */
+ newRelId = heap_create_with_catalog(newPartName->relname,
+ namespaceId,
+ modelRel->rd_rel->reltablespace,
+....
+ allowSystemTableMods,
+ false,
+ InvalidOid,
+ NULL);
heap_create_with_catalog parameter is_internal should be set to true.


+ /*
+ * Construct a map from the LIKE relation's attnos to the child rel's.
+ * This re-checks type match etc, although it shouldn't be possible to
+ * have a failure since both tables are locked.
+ */
+ attmap = build_attrmap_by_name(RelationGetDescr(newRel),
+   tupleDesc,
+   false);
this comment in createTableConstraints is confusing, especially the word "LIKE".
I didn' change it though.


+/*
+ * moveMergedTablesRows: scan partitions to be merged (mergingPartitions)
+ * of the partitioned table (rel) and move rows into the new partition
+ * (newPartRel). We also reevaulate check constraints against these rows.
+ */
+static void
+moveMergedTablesRows(List **wqueue, Relation rel,
+ List *mergingPartitions, Relation newPartRel)
the argument (Relation rel) never used in moveMergedTablesRows
we can remove it, or rename it as "parent_rel".
I didn' change it though.


moveMergedTablesRows was never used in SPLIT PARTITION,
so maybe we can rename it to
ATMergePartitionMoveTablesRows
or
ATMergePartitionMoveRows
or
ATMergePartitionRows
what do you think?


check_two_partitions_bounds_range
we can name it to
range_partition_bounds_check
I don't have a huge opinion though.


createTableConstraints(List **wqueue, AlteredTableInfo *tab,
                       Relation modelRel, Relation newRel)
rename argument (Relation modelRel) to parent_rel,
I think it will improve readability, since "parent_rel" can tell you it's a
partitioned table.

ATExecMergePartitions some comments position adjusted.

minor change src/test/regress/sql/partition_merge.sql
-SELECT * FROM sales_list;
-SELECT * FROM sales_nord;
-SELECT * FROM sales_all;
+SELECT tableoid::regclass, * FROM sales_list ORDER BY tableoid, salesperson_id;

we need an error case for
ERROR:  cannot drop table \"%s\" because other objects depend on it
so I added a test for it. as you can see below, the error HINT message
is not great in this
context.

CREATE VIEW jan2022v as SELECT * FROM sales_jan2022;
ALTER TABLE sales_range MERGE PARTITIONS (sales_jan2022,
sales_feb2022) INTO sales_dec_jan_feb2022;
ERROR:  cannot drop table sales_jan2022 because other objects depend on it
DETAIL:  view jan2022v depends on table sales_jan2022
HINT:  Use DROP ... CASCADE to drop the dependent objects too.
DROP VIEW jan2022v;

  214. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-18T23:12:38Z

    Hi!

1.
 >v47-0001-rename-function-argument-and-minor-refactor.no-cfbot

Thanks, applied.


2.
 >+ * Construct a map from the LIKE relation's attnos to the child rel's
 >this comment in createTableConstraints is confusing, especially the
 > word "LIKE". I didn' change it though.

It is copy from expandTableLikeClause function. Changed.


3.
 >the argument (Relation rel) never used in moveMergedTablesRows
 >we can remove it, or rename it as "parent_rel".
 >I didn' change it though.

Removed.


4.
 >moveMergedTablesRows was never used in SPLIT PARTITION,
 >so maybe we can rename it to
 >ATMergePartitionMoveTablesRows
 >or
 >ATMergePartitionMoveRows
 >or
 >ATMergePartitionRows
 >what do you think?

I like the name "MergePartitionsMoveRows" (without prefix "AT" - "ALTER 
TABLE", because this function is not called from ATExecCmd function).
Is it ok?


5.
 >so I added a test for it. as you can see below, the error HINT message
 >is not great in this context.
...
 >HINT:  Use DROP ... CASCADE to drop the dependent objects too.

Maybe a special flag (DEPFLAG_NOHINT?) should be added to skip hints for 
the performDeletionCheck function?

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  215. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-06-24T08:06:56Z

    hi.

+ /* Sort array of lower bounds. */
+ qsort_arg(lower_bounds, nparts, sizeof(PartitionRangeBound *),
+  qsort_partition_rbound_cmp, (void *) key);
here, we don't need ``(void *)``


+ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
+    MERGE PARTITIONS (<replaceable
class="parameter">partition_name1</replaceable>, <replaceable
class="parameter">partition_name2</replaceable> [, ...])
+        INTO <replaceable class="parameter">partition_name</replaceable>
In the synopsis section, we can combine the last two lines into one
for better formatting.

after
<varlistentry id="sql-altertable-parms-partition-name">
we can add the following to briefly explain parameters: partition_name1,
partition_name2


     <varlistentry id="sql-altertable-parms-partition-name1">
      <term><replaceable class="parameter">partition_name1</replaceable></term>
      <term><replaceable class="parameter">partition_name2</replaceable></term>
      <listitem>
       <para>
        The names of the tables being merged into the new partition.
       </para>
      </listitem>
     </varlistentry>

What do you think about alternative syntax:
ALTER TABLE tab1 MERGE PARTITION part1 WITH (part2, part3) mentioned in [1].
I think we need to settle this issue before moving forward.
If the current MERGE PARTITION design is finalized, then v48-0001 looks solid.

[1] https://postgr.es/m/CA+TgmoY0=bT_xBP8csR=MFE=FxGE2n2-me2-31jBOgEcLvW7ug@mail.gmail.com

  216. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-06-24T21:28:28Z

    Hi!
Thanks for notes!

1.
 >here, we don't need ``(void *)``

Corrected.


2.
 >In the synopsis section, we can combine the last two lines into one
 >for better formatting.

Changed.


3.
 > after ...
 >we can add the following to briefly explain parameters: partition_name1,
 >partition_name2

Added.


4.
 >What do you think about alternative syntax:
 >ALTER TABLE tab1 MERGE PARTITION part1 WITH (part2, part3) mentioned 
in [1].
 >....

Is it additional syntax (to the existing one) with functionality: 
partition part1 survives and data from partitions part2, part3 is moved 
into part1?
And (if "yes") should we delete the part1 indexes (or other constraints) 
before moving the data?


[1] 
https://postgr.es/m/CA+TgmoY0=bT_xBP8csR=MFE=FxGE2n2-me2-31jBOgEcLvW7ug@mail.gmail.com

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  217. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-07-12T14:01:22Z

    On Wed, Jun 25, 2025 at 5:28 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
> Thanks for notes!
>
hi.

+static void
+transformPartitionCmdForMerge(CreateStmtContext *cxt, PartitionCmd *partcmd)
+{
+
+ /* Is current partition a DEFAULT partition? */
+ defaultPartOid =
get_default_oid_from_partdesc(RelationGetPartitionDesc(parent, true));
+
this comment seems wrong?
it should be "does this partitioned table (parent) have a default partition?


+ case AT_MergePartitions:
+ {
+ PartitionCmd *partcmd = (PartitionCmd *) cmd->def;
+
+ if (list_length(partcmd->partlist) < 2)
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("list of new partitions should contain at least two items"));
This error message is not good, IMHO. I don't really have any good ideas though.
if we polish this message later, now we can add a comment: "FIXME
improve this error message".


in ATExecMergePartitions we have:
    RangeVarGetAndCheckCreationNamespace(cmd->name, NoLock, &existingRelid);
it will call RangeVarAdjustRelationPersistence
after it, the cmd->name->relpersistence will be set properly.

so the following in createPartitionTable can be checked way earlier.
    /*
     * We intended to create the partition with the same persistence as the
     * parent table, but we still need to recheck because that might be
     * affected by the search_path.  If the parent is permanent, so must be
     * all of its partitions.
     */
    if (parent_rel->rd_rel->relpersistence != RELPERSISTENCE_TEMP &&
        newRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
        ereport(ERROR,
                errcode(ERRCODE_WRONG_OBJECT_TYPE),
                errmsg("cannot create a temporary relation as
partition of permanent relation \"%s\"",
                       RelationGetRelationName(parent_rel)));
    /* Permanent rels cannot be partitions belonging to temporary parent */
    if (newRel->rd_rel->relpersistence != RELPERSISTENCE_TEMP &&
        parent_rel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
        ereport(ERROR,
                errcode(ERRCODE_WRONG_OBJECT_TYPE),
                errmsg("cannot create a permanent relation as
partition of temporary relation \"%s\"",
                       RelationGetRelationName(parent_rel)));

after createPartitionTable->heap_create_with_catalog do the error check seems
unintuitive to me.
it can be checked right after RangeVarGetAndCheckCreationNamespace.

  218. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-07-14T08:41:30Z

    Hi!

1.
 >this comment seems wrong?
 >it should be "does this partitioned table (parent) have a default
 >partition?

Agreed, this comment belongs to different code position. Corrected.

2.
 >This error message is not good, IMHO. I don't really have any good
 >ideas though.
 >if we polish this message later, now we can add a comment: "FIXME
 >improve this error message".

Probably it won't be possible to make the error message correct for both 
MERGE PARTITIONS and SPLIT PARTITION.
I split code for MERGE PARTITIONS and SPLIT PARTITION and now error 
messages are different for them.

3.
 >after createPartitionTable->heap_create_with_catalog do the error check
 >seems unintuitive to me.
 >it can be checked right after RangeVarGetAndCheckCreationNamespace.

I agree that these error checks are unintuitive.
But the createPartitionTable function, which contains checks, will be 
used in many places - not only for MERGE PARTITIONS, but also for SPLIT 
PARTITION. I think it's not a good idea to do the same checks in 
different places in code. Maybe it's better to do these checks in one 
place (even though it's unintuitive)?


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  219. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-07-16T11:42:38Z

    bug:

begin;
drop table if exists pks cascade;
create table pks(i int primary key, b int) partition by range (i);
create table pks_34 partition of pks for values from (3) to (6);
create table pks_d partition of pks default;
insert into pks values (0), (1), (3), (4), (5);
commit;
alter table pks_d add constraint cc check(i <> 5);
ALTER TABLE pks SPLIT PARTITION pks_34 INTO
    (PARTITION pks_3 FOR VALUES FROM (3) TO (4),
      PARTITION pks_4 FOR VALUES FROM (4) TO (5));

now pks_d have values(5) for column i, which would violate the
cc check constraint.
------------------------
/*
 * PartitionCmd - info for ALTER TABLE/INDEX ATTACH/DETACH PARTITION commands
 */
typedef struct PartitionCmd
{
    NodeTag        type;
    RangeVar   *name;            /* name of partition to
attach/detach/merge/split */
    PartitionBoundSpec *bound;    /* FOR VALUES, if attaching */
    List       *partlist;        /* list of partitions, for MERGE/SPLIT
                                 * PARTITION command */
    bool        concurrent;
} PartitionCmd;
"ALTER TABLE/INDEX ATTACH/DETACH PARTITION" comments need updated.


SplitPartitionMoveRows
                ereport(ERROR,
                        errcode(ERRCODE_CHECK_VIOLATION),
                        errmsg("can not find partition for split
partition row"),
                        errtable(splitRel));
will this ever be reachable?

in SplitPartitionMoveRows
+ if (sps->bound->is_default)
+ {
+ /* We should not create constraint for detached DEFAULT partition. */
+ defaultPartCtx = pc;
+ }
I am not sure the comment "detached DEFAULT partition" is correct.
the "constraint" is not ideal, better word would be "partition
constraint" or "partition qual".

In ATExecSplitPartition,
we can first create the new partitions and then detach the original partition.
It makes more sense, IMHO.
For example:
ALTER TABLE pks SPLIT PARTITION pks_34 INTO
    (PARTITION pks_3 FOR VALUES FROM (3) TO (4),
     PARTITION pks_4 FOR VALUES FROM (4) TO (6));
In this case, we could first create the relations pks_3 and pks_4, then detach
the partition pks_34

should the newly created partitions be based on the split partition or on the
partitioned table?
In the current v50 implementation, they are based on the partitioned table, but
these newly created partitions based on the split partition also make sense.


+ econtext->ecxt_scantuple = srcslot;
+
+ /* Search partition for current slot srcslot. */
+ foreach(listptr, partContexts)
+ {
+ pc = (SplitPartitionContext *) lfirst(listptr);
+
+ if (pc->partqualstate /* skip DEFAULT partition */ &&
+ ExecCheck(pc->partqualstate, econtext))
+ {
+ found = true;
+ break;
+ }
+ ResetExprContext(econtext);
this ResetExprContext is not needed, if you are evaluate different
ExprState again the same slot. See ExecRelCheck also.


see execute_extension_script
                     control->trusted
                     ? errhint("Must have CREATE privilege on current
database to create this extension.")
                     : errhint("Must be superuser to create this extension.")));
in v50-0002, we can refactor it as like the following
checkPartition(Relation rel, Oid partRelOid, bool is_merge)
{
    Relation    partRel;
    partRel = table_open(partRelOid, NoLock);
    if (partRel->rd_rel->relkind != RELKIND_RELATION)
        ereport(ERROR,
                errcode(ERRCODE_WRONG_OBJECT_TYPE),
                errmsg("\"%s\" is not a table",
RelationGetRelationName(partRel)),
                is_merge
                ? errhint("ALTER TABLE ... MERGE PARTITIONS can only
merge partitions don't have sub-partitions")
                : errhint("ALTER TABLE ... SPLIT PARTITIONS can only
split partitions don't have sub-partitions"));

I did some regress test refactoring to reduce test size.
-SELECT * FROM sales_range;
+SELECT tableoid, * FROM sales_range ORDER BY salesperson_id;

other miscellaneous refactoring is also attached.

  220. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-07-17T07:50:53Z

    Hi!

1.
 > bug:
 > ...
 > now pks_d have values(5) for column i, which would violate the
 > cc check constraint.

Probably we shouldn't use a DEFAULT-partition if it wasn't created by
SPLIT PARTITION command, because DEFAULT-partition can has other
restrictions besides CHECK (e.g. FOREIGN KEY).
Corrected.


2.
 > "ALTER TABLE/INDEX ATTACH/DETACH PARTITION" comments need updated.

Updated.


3.
 > SplitPartitionMoveRows
 >   ereport(ERROR,
 >           errcode(ERRCODE_CHECK_VIOLATION),
 >           errmsg("can not find partition for split partition row"),
 >           errtable(splitRel));
 >will this ever be reachable?

In case correct work, there should be no errors. This is insurance.


4.
 >I am not sure the comment "detached DEFAULT partition" is correct.
 >the "constraint" is not ideal, better word would be "partition
 >constraint" or "partition qual".

Corrected.


5.
 >In this case, we could first create the relations pks_3 and pks_4,
 >then detach the partition pks_34


I think it's better not to change the code, because detachPartitionTable
function call should be before performDeletionCheck.
And performDeletionCheck function should be used as soon as possible.


6.
 >should the newly created partitions be based on the split partition
 >or on the partitioned table?
 >In the current v50 implementation, they are based on the partitioned
 >table, but these newly created partitions based on the split partition
 >also make sense.

Yes, now newly created partitions are based on the partitioned.
I think this is more correct: user after SPLIT can modify some of new
partitions as he wants.


7.
 >this ResetExprContext is not needed, if you are evaluate different
 >ExprState again the same slot. See ExecRelCheck also.

Thanks!


8.
 >I did some regress test refactoring to reduce test size.
 >other miscellaneous refactoring is also attached.

Thanks!


P.S. Excuse me, but I can't answer emails for the next three weeks.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  221. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-07-29T09:38:22Z

    Hi, Dmitry!

I went through the patches. Both of them applied with a small conflict in
the parallel_schedule, which is easy to resolve.

+               ExecClearTuple(insertslot);
+
+               memcpy(insertslot->tts_values, srcslot->tts_values,
+                      sizeof(Datum) * srcslot->tts_nvalid);
+               memcpy(insertslot->tts_isnull, srcslot->tts_isnull,
+                      sizeof(bool) * srcslot->tts_nvalid);
+
+               ExecStoreVirtualTuple(insertslot);

Similar fragments are present in SplitPartitionMoveRows()
and MergePartitionsMoveRows().  Do you think we could use ExecCopySlot()
(or similar) instead?

The presence of the same name in the split partition list is checked with
equal() (similar concern was already raised about the merge case [1]).  If
one of the names is schema-qualified, the error message is different.
Could we make them the same?

# alter table part_test split partition part_test_1_2 into (partition
part_test_1 for values in (1), partition part_test_1 for values in (2));
ERROR:  name "part_test_1" is already used
LINE 1: ...artition part_test_1 for values in (1), partition part_test_...
                                                             ^
Time: 1.194 ms
# alter table part_test split partition part_test_1_2 into (partition
part_test_1 for values in (1), partition public.part_test_1 for values in
(2));
ERROR:  relation "part_test_1" already exists
Time: 6.187 ms

+       /* Search partition for current slot srcslot. */
+       foreach(listptr, partContexts)
+       {
+           pc = (SplitPartitionContext *) lfirst(listptr);
+
+           /* skip DEFAULT partition */
+           if (pc->partqualstate && ExecCheck(pc->partqualstate, econtext))
+           {
+               found = true;
+               break;
+           }
+       }

I see we're searching for a partition to place each row using the
sequential application of partition constraints.  I concerned if this could
be exhausting when the number of new partitions is large.  Could we use
something like binary search here?

New split/drop commands do the full reorganization of the involved
partitions.  As Robert previously stated [2], there are other possible
strategies.  While they are hard to implement, I don't think we need them
in the initial version.  But I think it's worth mentioning in the docs that
we're completely rewriting the involved partitions.  And this is not
recommended to use for merging very big partitions with small ones, and
splitting a small fraction of rows out of a very big partition.

Both patches could use pgindent run.

Links
1.
https://www.postgresql.org/message-id/CACJufxHHnJm6Jb2YQpuRU1RX__tO%3DJJNJ5%3DEUMuzif_KNxGd9A%40mail.gmail.com
2.
https://www.postgresql.org/message-id/CA%2BTgmoY0%3DbT_xBP8csR%3DMFE%3DFxGE2n2-me2-31jBOgEcLvW7ug%40mail.gmail.com

------
Regards,
Alexander Korotkov
Supabase

  222. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-08-12T13:09:37Z

    Hi, Alexander!

Thank you for notes.

1.
 >+ ExecClearTuple(insertslot);
 >+
 >+ memcpy(insertslot->tts_values, srcslot->tts_values,
 >+        sizeof(Datum) * srcslot->tts_nvalid);
 >+ memcpy(insertslot->tts_isnull, srcslot->tts_isnull,
 >+        sizeof(bool) * srcslot->tts_nvalid);
 >+
 >+ ExecStoreVirtualTuple(insertslot);
 >
 > Similar fragments are present in SplitPartitionMoveRows() and
 > MergePartitionsMoveRows().  Do you think we could use ExecCopySlot()
 > (or similar) instead?

I think ExecCopySlot function is not suitable here because of 
tts_virtual_materialize function
(ExecCopySlot -> tts_virtual_copyslot -> tts_virtual_materialize).
It is good to use the same data in insertslot as in srcslot, without 
materialization.
Therefore, it seems to me that it is better not to change the existing 
code, especially since it is similar to the code of the ATRewriteTable 
function [1] and is understandable.


2.
 > The presence of the same name in the split partition list is checked
 > with equal() (similar concern was already raised about the merge case
 > [1]).  If one of the names is schema-qualified, the error message is
 > different.  Could we make them the same?

Corrected. Added namespace comparison for this case.


3.
 >+ /* Search partition for current slot srcslot. */
 >+ foreach(listptr, partContexts)
 >+ {
 >+     pc = (SplitPartitionContext *) lfirst(listptr);
 >+
 >+     /* skip DEFAULT partition */
 >+     if (pc->partqualstate && ExecCheck(pc->partqualstate, econtext))
 >+     {
 >+         found = true;
 >+         break;
 >+     }
 >+ }
 > I see we're searching for a partition to place each row using the
 > sequential application of partition constraints.  I concerned if this
 > could be exhausting when the number of new partitions is large.
 > Could we use something like binary search here?

I think binary search for this case would make the code much more
complicated. And is binary search really needed for real cases?
I assume that the main use of SPLIT PARTITION will be split into a
small number of partitions, for example:

  * for extracting partition from the DEFAULT partition;
  * splitting partition for big intreval into partitions for smaller
    intervals (for example, splitting a partition for quarter/year into
    partitions for months).
  * ...

In these cases, we highly likely won't need binary search.


4.
 > New split/drop commands do the full reorganization of the involved
 > partitions.  As Robert previously stated, there are other
 > possible strategies.  While they are hard to implement, I don't think
 > we need them in the initial version.  But I think it's worth
 > mentioning in the docs that we're completely rewriting the involved
 > partitions.  And this is not recommended to use for merging very big
 > partitions with small ones, and splitting a small fraction of rows out
 > of a very big partition.

Added notes to documentation (sorry my poor English).


5.
 > Both patches could use pgindent run.

Fixed.

Links
-----
[1] 
https://github.com/postgres/postgres/blob/71ea0d6795438f95f4ee6e35867058c44b270d51/src/backend/commands/tablecmds.c#L6361

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  223. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-08-20T09:22:47Z

    hi.
this time, I only checked
v52-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch

typedef struct PartitionCmd
{
    NodeTag        type;
    RangeVar   *name;            /* name of partition to attach/detach/merge */
    PartitionBoundSpec *bound;    /* FOR VALUES, if attaching */
    List       *partlist;        /* list of partitions, for MERGE PARTITION
                                 * command */
    bool        concurrent;
} PartitionCmd;
The field "partlist" comments are not very helpful, IMO.
I think the following is more descriptive.
/* list of partitions to be merged, used only in ALTER TABLE MERGE PARTITION */


+ /*
+ * Search DEFAULT partition in the list. Open and lock partitions
+ * before calculating the boundary for resulting partition, we also
+ * check for ownership along the way.  We need to use
+ * AccessExclusiveLock here, because these merged partitions will be
+ * detached then dropped in ATExecMergePartitions.
+ */
+ partOid = RangeVarGetRelidExtended(name,
+   AccessExclusiveLock,
+   false,
+   RangeVarCallbackOwnsRelation,
+   NULL);
here "false" should be "0"?


+ /* Ranges of partitions should not overlap. */
+ for (i = 1; i < nparts; i++)
+ {
+ int index = lower_bounds[i]->index;
+ int prev_index = lower_bounds[i - 1]->index;
+ check_two_partitions_bounds_range(parent,
+  (RangeVar *) list_nth(partNames, prev_index),
+  (PartitionBoundSpec *) list_nth(bounds, prev_index),
+  (RangeVar *) list_nth(partNames, index),
+  (PartitionBoundSpec *) list_nth(bounds, index),
+  pstate);
+ }
the comment should be
+ /* Ranges of partitions should be adjacent */


--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -102,11 +102,11 @@ static ObjectAddress AddNewRelationType(const
char *typeName,
  Oid new_row_type,
  Oid new_array_type);
 static void RelationRemoveInheritance(Oid relid);
+static void StoreConstraints(Relation rel, List *cooked_constraints,
+ bool is_internal);
-static void StoreConstraints(Relation rel, List *cooked_constraints,
- bool is_internal);

Is this change necessary?


in  createPartitionTable
+ /* Create the relation. */
+ newRelId = heap_create_with_catalog(newPartName->relname,
+ namespaceId,
+ parent_rel->rd_rel->reltablespace,
....
+ newPartName->relpersistence,
....
+
+ /*
+ * We intended to create the partition with the same persistence as the
+ * parent table, but we still need to recheck because that might be
+ * affected by the search_path.  If the parent is permanent, so must be
+ * all of its partitions.
+ */
+ if (parent_rel->rd_rel->relpersistence != RELPERSISTENCE_TEMP &&
+ newRel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
+ ereport(ERROR,
+ errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("cannot create a temporary relation as partition of permanent
relation \"%s\"",
+   RelationGetRelationName(parent_rel)));
+
+ /* Permanent rels cannot be partitions belonging to temporary parent */
+ if (newRel->rd_rel->relpersistence != RELPERSISTENCE_TEMP &&
+ parent_rel->rd_rel->relpersistence == RELPERSISTENCE_TEMP)
+ ereport(ERROR,
+ errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("cannot create a permanent relation as partition of temporary
relation \"%s\"",
+   RelationGetRelationName(parent_rel)));

i raised this question in [1], you replied at [2].
I still think it's not intuitive.
parent->relpersistence is fixed. and newRel->relpersistence is the same as
the same as newPartName->relpersistence, see heap_create_with_catalog.
These relpersistence error checks can happen before heap_create_with_catalog.

I added a regress test for src/test/modules/test_ddl_deparse.
I refactored regress to make
src/test/regress/expected/partition_merge.out less verbose.

[1]  https://postgr.es/m/CACJufxHM0sD8opy2hUxXLcdY3CQOCaMfsQtJs7yF3TS2YxSqKg@mail.gmail.com
[2] https://postgr.es/m/195b67ee-ef41-4451-9396-844442eef1a4@postgrespro.ru

  224. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-08-21T02:53:27Z

    On Wed, Aug 20, 2025 at 5:22 PM jian he <jian.universality@gmail.com> wrote:
>
> this time, I only checked
> v52-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch
>
> typedef struct PartitionCmd
> {
>     NodeTag        type;
>     RangeVar   *name;            /* name of partition to attach/detach/merge */
>     PartitionBoundSpec *bound;    /* FOR VALUES, if attaching */
>     List       *partlist;        /* list of partitions, for MERGE PARTITION
>                                  * command */
>     bool        concurrent;
> } PartitionCmd;

/*
 * PartitionCmd - info for ALTER TABLE/INDEX ATTACH/DETACH PARTITION commands
 */
typedef struct PartitionCmd
the above comments also need to be updated?


+-- Use indexscan for testing indexes after merging partitions
+SET enable_seqscan = OFF;
+
+SELECT * FROM sales_all WHERE sales_state = 'Warsaw';
+SELECT * FROM sales_list WHERE sales_state = 'Warsaw';
+SELECT * FROM sales_list WHERE salesperson_name = 'Ivanov';
+
+RESET enable_seqscan;

``+SELECT * FROM sales_all WHERE sales_state = 'Warsaw';``
may ultimately fall back to using  seqscan?
so we need to use
``explain(costs off)`` to see if it use indexscan or not.


+ /*
+ * We reject whole-row variables because the whole point of LIKE is
+ * that the new table's rowtype might later diverge from the parent's.
+ * So, while translation might be possible right now, it wouldn't be
+ * possible to guarantee it would work in future.
+ */
+ if (found_whole_row)
+ ereport(ERROR,
+ errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot convert whole-row table reference"),
+ errdetail("Constraint \"%s\" contains a whole-row reference to table \"%s\".",
+  ccname,
+  RelationGetRelationName(parent_rel)));
this error is unlikely to happen, we can simply use elog(ERROR, ....),
rather than  ereport.

evaluateGeneratedExpressionsAndCheckConstraints seem not necessary?
we should make the MergePartitionsMoveRows code pattern aligned with
ATRewriteTable.
by comparing these two function, i found that before call table_scan_getnextslot
we need to switch memory context to EState->ecxt_per_tuple_memor
please check the attached changes.

  225. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-08-21T06:45:12Z

    On Thu, Aug 21, 2025 at 10:53 AM jian he <jian.universality@gmail.com> wrote:
>
> > this time, I only checked
> > v52-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch
> >


hi.
we may need to change checkPartition.

+-- ERROR:  "sales_apr2022" is not a table
+ALTER TABLE sales_range MERGE PARTITIONS (sales_feb2022,
sales_mar2022, sales_apr2022) INTO sales_feb_mar_apr2022;
+ERROR:  "sales_apr2022" is not a table
+HINT:  ALTER TABLE ... MERGE PARTITIONS can only merge partitions
don't have sub-partitions

+ERROR:  "sales_apr2022" is not a table
the above error message seems not intuitive to me.
IMV, the error message pattern should be something like:
ERROR: can not merge relation \"%s\"  with other partitions
DETAIL:  "sales_apr2022" is not a table
HINT:  ALTER TABLE ... MERGE PARTITIONS can only merge partitions
don't have sub-partition


+/*
+ * checkPartition
+ * Check whether partRelOid is a leaf partition of the parent table (rel).
+ * Partition with OID partRelOid must be locked before function call.
+ */
+static void
+checkPartition(Relation rel, Oid partRelOid)
"Partition with OID partRelOid must be locked before function call."
we can remove this sentence.
otherwise, "function call" seems confusing?


+SET search_path = pg_temp, partitions_merge_schema, public;
+
+BEGIN;
+CREATE TABLE t (i int) PARTITION BY RANGE (i);
+CREATE TABLE tp_0_1 PARTITION OF t FOR VALUES FROM (0) TO (1);
+CREATE TABLE tp_1_2 PARTITION OF t FOR VALUES FROM (1) TO (2);
+SELECT c.oid::pg_catalog.regclass, c.relpersistence FROM
pg_catalog.pg_class c WHERE c.oid = 't'::regclass;
+EXECUTE get_partition_info('{t}');
+DEALLOCATE get_partition_info;
+SET search_path = partitions_merge_schema, pg_temp, public;
+
+-- Can't merge temporary partitions into a persistent partition
+ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO tp_0_2;
+ROLLBACK;

"+-- Can't merge temporary partitions into a persistent partition"
means
+ALTER TABLE t MERGE PARTITIONS (tp_0_1, tp_1_2) INTO tp_0_2;
should error out, but it didn't.
then I found out:

+/*
+ * ALTER TABLE <name> MERGE PARTITIONS <partition-list> INTO <partition-name>
+ */
+static void
+ATExecMergePartitions(List **wqueue, AlteredTableInfo *tab, Relation rel,
+  PartitionCmd *cmd, AlterTableUtilityContext *context)
+{
....
+ /*
+ * Look up existing relation by new partition name, check we have
+ * permission to create there, lock it against concurrent drop, and mark
+ * stmt->relation as RELPERSISTENCE_TEMP if a temporary namespace is
+ * selected.
+ */
+ cmd->name->relpersistence = rel->rd_rel->relpersistence;
+ RangeVarGetAndCheckCreationNamespace(cmd->name, NoLock, &existingRelid);

``cmd->name->relpersistence`` will be adjusted in
RangeVarGetAndCheckCreationNamespace->RangeVarAdjustRelationPersistence.

+ cmd->name->relpersistence = rel->rd_rel->relpersistence;
seems wrong?
comments "stmt->relation" not sure what it refers to?


attached patch did following the changes:
* remove line  ``cmd->name->relpersistence = rel->rd_rel->relpersistence;``
* refactor relpersistence, temp schema related regress tests.

  226. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-08-22T03:36:45Z

    On Thu, Aug 21, 2025 at 2:45 PM jian he <jian.universality@gmail.com> wrote:
>
> On Thu, Aug 21, 2025 at 10:53 AM jian he <jian.universality@gmail.com> wrote:
> >
> > > this time, I only checked
> > > v52-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch

hi.

+static void
+check_two_partitions_bounds_range(Relation parent,
+{
....
+
+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("lower bound of partition \"%s\" is not equal to the upper
bound of partition \"%s\"",
+   second_name->relname, first_name->relname),
+ errhint("ALTER TABLE ... MERGE PARTITIONS requires the partition
bounds to be adjacent."),
+ parser_errposition(pstate, datum->location));
+ }

I propose change it to:
        ereport(ERROR,
                errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
                errmsg("can not merge partition \"%s\" together with
partition \"%s\"",
                       second_name->relname, first_name->relname),
                errdetail("lower bound of partition \"%s\" is not
equal to the upper bound of partition \"%s\"",
                          second_name->relname, first_name->relname),
                errhint("ALTER TABLE ... MERGE PARTITIONS requires the
partition bounds to be adjacent."),
                parser_errposition(pstate, datum->location));


    <para>
+     There is also an option for merging multiple table partitions into
+     a single partition using the
+     <link linkend="sql-altertable-merge-partitions"><command>ALTER
TABLE ... MERGE PARTITIONS</command></link>.
+     This feature simplifies the management of partitioned tables by allowing
+     users to combine partitions that are no longer needed as
+     separate entities.  It's important to note that this operation is not
+     supported for hash-partitioned tables and acquires an
+     <literal>ACCESS EXCLUSIVE</literal> lock, which could impact high-load
+     systems due to the lock's restrictive nature.  For example, we can
+     merge three monthly partitions into one quarter partition:
I am not sure last sentence "merge three monthly partitions into one
quarter partition:"
is correct.


buildExpressionExecutionStates seems not needed, same reason as
mentioned before,
code pattern aligned with ATRewriteTable.
while at it, also did some minor changes.

  227. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-08-26T08:05:47Z

    Hi!
Thanks for the notes and patches!

1.
 >/* list of partitions, for MERGE PARTITION command */
 > ...
 >The field "partlist" comments are not very helpful, IMO.
 >I think the following is more descriptive.
 >/* list of partitions to be merged, used only in ALTER TABLE MERGE
 >PARTITION */

Corrected.


2.
 >+ partOid = RangeVarGetRelidExtended(name,
 >+   AccessExclusiveLock,
 >+   false,
 >+   RangeVarCallbackOwnsRelation,
 >+   NULL);
 >here "false" should be "0"?

Corrected.


3.
 >the comment should be
 >+ /* Ranges of partitions should be adjacent */

Corrected.


4.
 >+static void StoreConstraints(Relation rel, List *cooked_constraints,
 >+ bool is_internal);
 >-static void StoreConstraints(Relation rel, List *cooked_constraints,
 >- bool is_internal);
 >
 >Is this change necessary?

Corrected.


5.
 >i raised this question in [1], you replied at [2].
 >I still think it's not intuitive.
 >parent->relpersistence is fixed. and newRel->relpersistence is the
 >same as the same as newPartName->relpersistence, see
 >heap_create_with_catalog. These relpersistence error checks can
 >happen before heap_create_with_catalog.
 >I added a regress test for src/test/modules/test_ddl_deparse.
 >I refactored regress to make
 >src/test/regress/expected/partition_merge.out less verbose.

I'm sorry, I misunderstood the point.
Applied.


6.
 >/*
 > * PartitionCmd - info for ALTER TABLE/INDEX ATTACH/DETACH PARTITION
 > commands
 > */
 >typedef struct PartitionCmd
 >the above comments also need to be updated?

Updated.
(Previously the comment was updated for SPLIT PARTITION command only.)


7.
 >``+SELECT * FROM sales_all WHERE sales_state = 'Warsaw';``
 >may ultimately fall back to using  seqscan?
 >so we need to use
 >``explain(costs off)`` to see if it use indexscan or not.

Corrected.


8.
 >...
 >+  RelationGetRelationName(parent_rel)));
 >this error is unlikely to happen, we can simply use elog(ERROR, ....),
 >rather than  ereport.

Applied.


9.
 >evaluateGeneratedExpressionsAndCheckConstraints seem not necessary?

The "evaluateGeneratedExpressionsAndCheckConstraints" function is used
for both commands (SPLIT and MERGE), so I prefer to keep it
(probably, code duplication is worse).


10.
 >we should make the MergePartitionsMoveRows code pattern aligned with
 >ATRewriteTable.
 >by comparing these two function, i found that before call
 >table_scan_getnextslot we need to switch memory context to
 >EState->ecxt_per_tuple_memor

Thanks, that is correct. Applied.


11.
 >we may need to change checkPartition.
 > ...
 >IMV, the error message pattern should be something like:
 >ERROR: can not merge relation \"%s\"  with other partitions
 >DETAIL:  "sales_apr2022" is not a table
 >HINT:  ALTER TABLE ... MERGE PARTITIONS can only merge partitions
 >don't have sub-partition

This error is generated if the condition
"if (partRel->rd_rel->relkind!= RELKIND_RELATION)"
is true. But error message pattern
"can not merge relation ... with other partitions"
is correct for RELKIND_PARTITIONED_TABLE only. Separate messages for 
each of the relation types (RELKIND_VIEW, RELKIND_FOREIGN_TABLE, ...) 
looks a bit complicated (see example [1]).
Might be we can replace error message "... is not a table" to
"... is not a simple partition"?


12.
 >+checkPartition(Relation rel, Oid partRelOid)
 >"Partition with OID partRelOid must be locked before function call."
 >we can remove this sentence.
 >otherwise, "function call" seems confusing?

Removed.


13.
 >+ cmd->name->relpersistence = rel->rd_rel->relpersistence;
 >seems wrong?
 >comments "stmt->relation" not sure what it refers to?

Applied and corrected the same for SPLIT PARTITION.


14.
 > ...
 >I propose change it to:
 >ereport(ERROR,
 >        errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
 >        errmsg("can not merge partition \"%s\" together with partition
 >\"%s\"",
 >                second_name->relname, first_name->relname),
 >        errdetail("lower bound of partition \"%s\" is not equal to the
 >upper bound of partition \"%s\"",
 >                  second_name->relname, first_name->relname),
 >        errhint("ALTER TABLE ... MERGE PARTITIONS requires the
 >partition bounds to be adjacent."),
 >        parser_errposition(pstate, datum->location));


Changed.


15.
 >buildExpressionExecutionStates seems not needed, same reason as
 >mentioned before,
 >code pattern aligned with ATRewriteTable.

"buildExpressionExecutionStates" function is used for both commands
(SPLIT and MERGE). Probably, is better to keep this function?


16.
 >while at it, also did some minor changes.

Applied.


Links
-----
[1] 
https://github.com/postgres/postgres/blob/989b2e4d5c95f6b183e76f3eb06d2d360651ccf2/src/backend/commands/copyto.c#L649

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  228. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-08-29T10:31:58Z

    On Tue, Aug 26, 2025 at 4:05 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi!
> Thanks for the notes and patches!
>
hi.

ORDER BY regclass seems not stable, see
https://git.postgresql.org/cgit/postgresql.git/commit/?id=17bcf4f5450430f67b744c225566c9e0e6413e95
some of the SQL tests seem not necessary, so I refactored tests.

+static List *
+getAttributesList(Relation parent_rel)
+{
.....
+
+ for (parent_attno = 1; parent_attno <= modelDesc->natts;
+ parent_attno++)
+ {
....
+
+ /* Add to column list */
+ colList = lappend(colList, def);
+
+ /*
+ * Although we don't transfer the column's default/generation
+ * expression now, we need to mark it GENERATED if appropriate.
+ */
+ if (attribute->atthasdef && attribute->attgenerated)
+ def->generated = attribute->attgenerated;
+
+ def->storage = attribute->attstorage;
+
+ /* Likewise, copy compression if requested */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+ }
+
+ return colList;

the last part seems intuitive?
"colList = lappend(colList, def);" should be at the end of the for loop?


+-- Indexname values should be 'tp_1_2_pkey' and 'tp_1_2_i_idx'.
+-- Not-null constraint name should be 'tp_1_2_i_not_null'.
+\d+ tp_1_2
+                          Table "partitions_merge_schema.tp_1_2"
+ Column |  Type   | Collation | Nullable | Default | Storage | Stats
target | Description
+--------+---------+-----------+----------+---------+---------+--------------+-------------
+ i      | integer |           | not null |         | plain   |              |
+Partition of: t FOR VALUES FROM (0) TO (2)
+Partition constraint: ((i IS NOT NULL) AND (i >= 0) AND (i < 2))
+Indexes:
+    "tp_1_2_pkey" PRIMARY KEY, btree (i)
+    "tp_1_2_i_idx" btree (i)
+Not-null constraints:
+    "t_i_not_null" NOT NULL "i" (inherited)
+
"-- Not-null constraint name should be 'tp_1_2_i_not_null'.
Comments conflict with the result.


+ /*
+ * We reject whole-row variables because the whole point of LIKE is
+ * that the new table's rowtype might later diverge from the parent's.
+ * So, while translation might be possible right now, it wouldn't be
+ * possible to guarantee it would work in future.
+ */
+ if (found_whole_row)
+ elog(ERROR, "Constraint \"%s\" contains a whole-row reference to
table \"%s\".",
+ ccname,
+ RelationGetRelationName(parent_rel));
the above comment needs change, since LIKE is not related to here.


+ foreach_ptr(NewConstraint, con, tab->constraints)
+ {
+ switch (con->contype)
+ {
+ case CONSTR_CHECK:
+
+ /*
+ * We already expanded virtual expression in
+ * createTableConstraints.
+ */
+ con->qualstate = ExecPrepareExpr((Expr *)
expand_generated_columns_in_expr(con->qual, newPartRel, 1), estate);
+ break;
here, we don't need expand_generated_columns_in_expr, the comment also
explained it.

the attached patch is the changes for the above comments.

  229. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-01T11:03:58Z

    Hi!
Thank you for the notes and patch!

1.
 >ORDER BY regclass seems not stable, see
 >https://git.postgresql.org/cgit/postgresql.git/commit/
 >?id=17bcf4f5450430f67b744c225566c9e0e6413e95
 >some of the SQL tests seem not necessary, so I refactored tests.

Thanks.
Also changed tests for SPLIT PARTITION.


2.
 >the last part seems intuitive?
 >"colList = lappend(colList, def);" should be at the end of the for loop?

I agree, it's better.


3.
 >"-- Not-null constraint name should be 'tp_1_2_i_not_null'.
 >Comments conflict with the result.

Thanks, this was correct for older versions.


4.
 >* We reject whole-row variables because the whole point of LIKE is
 >* that the new table's rowtype might later diverge from the parent's.
...
 >the above comment needs change, since LIKE is not related to here.

Corrected.


5.
 >here, we don't need expand_generated_columns_in_expr, the comment also
 >explained it.
 >the attached patch is the changes for the above comments.

Applied.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  230. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-09-15T08:03:54Z

    Hi Dmitry.

On Mon, Sep 1, 2025 at 2:04 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Hi!
> Thank you for the notes and patch!

Some additional notes from me.

1) src/backend/parser/parse_utilcmd.c includes are not alphabetically
ordered here
+#include "partitioning/partdesc.h"
+#include "partitioning/partbounds.h"

2) There is unicode dash in the comment of ATExecMergePartitions() here.  I
suggest we should stick to ascii.

+   /*
+    * Check ownership of merged partitions — partitions with different
+    * owners cannot be merged. Also, collect the OIDs of these partitions
+    * during the check.
+    */

3) Regarding 17bcf4f545, I see btnamecmp() is collation-aware.  Should we
also specify COLLATE "C" every time we do "ORDER BY relname"?

4) This comment sounds misleading.  Probably it should say "are contained".

+/*
+ * check_parent_values_in_new_partitions
+ *
+ * (function for BY LIST partitioning)
+ *
+ * Checks that all values of split partition (with Oid partOid) contains
in new
+ * partitions.
+ *

5) Given what latter items say, I think the 1. should say "The DEFAULT
partition must be at most one."

/*
 * check_partitions_for_split
 *
 * Checks new partitions for SPLIT PARTITIONS command:
 * 1. DEFAULT partition should be one.

6) Regarding the isolation tests.  I see we are exercising INSERTs and
intra-partition UPDATEs.  Should we also try some cross-partition UPDATEs?

------
Regards,
Alexander Korotkov
Supabase

  231. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-09-15T17:47:15Z

    On Mon, Sep 15, 2025 at 11:03 AM Alexander Korotkov
<aekorotkov@gmail.com> wrote:
>
> On Mon, Sep 1, 2025 at 2:04 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > Hi!
> > Thank you for the notes and patch!
>
> Some additional notes from me.
>
> 1) src/backend/parser/parse_utilcmd.c includes are not alphabetically ordered here
> +#include "partitioning/partdesc.h"
> +#include "partitioning/partbounds.h"
>
> 2) There is unicode dash in the comment of ATExecMergePartitions() here.  I suggest we should stick to ascii.
>
> +   /*
> +    * Check ownership of merged partitions — partitions with different
> +    * owners cannot be merged. Also, collect the OIDs of these partitions
> +    * during the check.
> +    */
>
> 3) Regarding 17bcf4f545, I see btnamecmp() is collation-aware.  Should we also specify COLLATE "C" every time we do "ORDER BY relname"?
>
> 4) This comment sounds misleading.  Probably it should say "are contained".
>
> +/*
> + * check_parent_values_in_new_partitions
> + *
> + * (function for BY LIST partitioning)
> + *
> + * Checks that all values of split partition (with Oid partOid) contains in new
> + * partitions.
> + *
>
> 5) Given what latter items say, I think the 1. should say "The DEFAULT partition must be at most one."
>
> /*
>  * check_partitions_for_split
>  *
>  * Checks new partitions for SPLIT PARTITIONS command:
>  * 1. DEFAULT partition should be one.
>
> 6) Regarding the isolation tests.  I see we are exercising INSERTs and intra-partition UPDATEs.  Should we also try some cross-partition UPDATEs?

Additionally, I've made a numerous and small fixes for grammar to the
docs directly to the patchset.

------
Regards,
Alexander Korotkov
Supabase

  232. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-15T22:11:19Z

    Hi Alexander.

Thanks for the notes and corrections!


1) src/backend/parser/parse_utilcmd.c includes are not alphabetically 
ordered here
+#include "partitioning/partdesc.h"
+#include "partitioning/partbounds.h"

Fixed.


2) There is unicode dash in the comment of ATExecMergePartitions() here. 
  I suggest we should stick to ascii.

+   /*
+    * Check ownership of merged partitions — partitions with different
+    * owners cannot be merged. Also, collect the OIDs of these partitions
+    * during the check.
+    */

Fixed.


3) Regarding 17bcf4f545, I see btnamecmp() is collation-aware.  Should 
we also specify COLLATE "C" every time we do "ORDER BY relname"?

Queries changed.


4) This comment sounds misleading.  Probably it should say "are contained".

+/*
+ * check_parent_values_in_new_partitions
+ *
+ * (function for BY LIST partitioning)
+ *
+ * Checks that all values of split partition (with Oid partOid) 
contains in new
+ * partitions.
+ *

Changed.


5) Given what latter items say, I think the 1. should say "The DEFAULT 
partition must be at most one."

/*
  * check_partitions_for_split
  *
  * Checks new partitions for SPLIT PARTITIONS command:
  * 1. DEFAULT partition should be one.

Corrected.


6) Regarding the isolation tests.  I see we are exercising INSERTs and 
intra-partition UPDATEs.  Should we also try some cross-partition UPDATEs?

Added simple tests for cross-partition UPDATE.


7) Additionally, I've made a numerous and small fixes for grammar to the
docs directly to the patchset.

Thanks, applied.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  233. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-09-16T08:52:02Z

    On Tue, Sep 16, 2025 at 6:11 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> 7) Additionally, I've made a numerous and small fixes for grammar to the
> docs directly to the patchset.
>

v56-0002, SPLIT PARTITION check_partitions_for_split is way too overwhelming.
Similar to transformPartitionCmdForMerge, we can put some error handling code
to transformPartitionCmdForSplit.
please check the attached refactoring.

in v56-0001
+ Oid newPartitionOid = InvalidOid;
+
+ foreach_oid(mergingPartitionOid, mergingPartitions)
+ {
+ if (mergingPartitionOid == existingRelid)
+ {
+ newPartitionOid = mergingPartitionOid;
+ break;
+ }
+ }

can simplified to
        if (list_member_oid(mergingPartitions, existingRelid))
            newPartitionOid = existingRelid;

  234. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-16T22:08:17Z

    Hi, Jiah He!

Thank you for the patch!

1.
 >can simplified to
 >        if (list_member_oid(mergingPartitions, existingRelid))
 >            newPartitionOid = existingRelid;

Applied (with small additional changes).


2. Patch v56-0001-refactor-v56-check_partitions_for_split.no-cfbot 
applied with cosmetic changes.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  235. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-09-17T07:35:34Z

    On Wed, Sep 17, 2025 at 6:08 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> 2. Patch v56-0001-refactor-v56-check_partitions_for_split.no-cfbot
> applied with cosmetic changes.
>

hi.

check_two_partitions_bounds_range
        ereport(ERROR,
                errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
                merge_split
                ? errmsg("can not merge partition \"%s\" together with
partition \"%s\"",
                         second_name->relname, first_name->relname)
                : errmsg("can not split to partition \"%s\" together
with partition \"%s\"",
                         second_name->relname, first_name->relname),
                errdetail("lower bound of partition \"%s\" is not
equal to the upper bound of partition \"%s\"",
                          second_name->relname, first_name->relname),
                merge_split
                ? errhint("ALTER TABLE ... MERGE PARTITIONS requires
the partition bounds to be adjacent.")
                : errhint("ALTER TABLE ... SPLIT PARTITIONS requires
the partition bounds to be adjacent."),
                parser_errposition(pstate, datum->location));

This is too much...., also bad for translation, so I refactored this.
I also refactored check_partition_bounds_for_split_range.

In ATExecSplitPartition,
"char        relname[NAMEDATALEN];" is not necessary?

  236. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Chao Li <li.evan.chao@gmail.com> — 2025-09-17T07:41:40Z

    
> On Sep 17, 2025, at 06:08, Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 
> 
> Postgres Professional: http://postgrespro.com<v57-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch><v57-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch>


1 - 0001
```
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -319,6 +319,56 @@ performDeletion(const ObjectAddress *object,
 	table_close(depRel, RowExclusiveLock);
 }
 
+/*
+ * performDeletionCheck: Check whether a specific object can be safely deleted.
+ * This function does not perform any deletion; instead, it raises an error
+ * if the object cannot be deleted due to existing dependencies.
+ *
+ * It can be useful when you need delete some objects later.  See comments in

+ * The behavior must specified as DROP_RESTRICT.


+	/*
+	 * Construct a list of objects we want delete later (ie, the given object
```

Nit: “when you need delete” => “when you need to delete"

“Must specified” => “must be specified"

“We want delete” => “we want to delete"

2 - 0001
```
+void
+performDeletionCheck(const ObjectAddress *object,
+					 DropBehavior behavior, int flags)
+{
+	Relation	depRel;
+	ObjectAddresses *targetObjects;
+
+	Assert(behavior == DROP_RESTRICT);
+
+	depRel = table_open(DependRelationId, RowExclusiveLock);
```

This function looks only performing read-only checks, why do we need RowExclusiveLock? Is AccessShareLock good enough?

3 - 0001
```
@@ -5272,6 +5278,11 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
 			/* No command-specific prep needed */
 			pass = AT_PASS_MISC;
 			break;
+		case AT_MergePartitions:
+			ATSimplePermissions(cmd->subtype, rel, ATT_PARTITIONED_TABLE);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
```

AT_MergePartitions, AT_DetachPartitionFinalize and AT_DetachPartition do the same thing, why don’t combine them together?

I see the previous code combine multiple cases if they do the same thing right in this switch:

        case AT_EnableRule:     /* ENABLE/DISABLE RULE variants */
        case AT_EnableAlwaysRule:
        case AT_EnableReplicaRule:
        case AT_DisableRule:
        case AT_AddOf:          /* OF */
        case AT_DropOf:         /* NOT OF */
        case AT_EnableRowSecurity:
        case AT_DisableRowSecurity:
        case AT_ForceRowSecurity:
        case AT_NoForceRowSecurity:
            ATSimplePermissions(cmd->subtype, rel,
                                ATT_TABLE | ATT_PARTITIONED_TABLE);
            /* These commands never recurse */
            /* No command-specific prep needed */
            pass = AT_PASS_MISC;
            break;

4 - 0001
```
+	/* Attach a new partition to the partitioned table. */
+	attachPartitionTable(wqueue, rel, attachrel, cmd->bound);
```

I think the comment can be removed as the function name has clearly described what it is doing.

5 - 0001
```
+static void
+attachPartitionTable(List **wqueue, Relation rel, Relation attachrel, PartitionBoundSpec *bound)
```

I think bound can be const: const PartitionBoundSpec *bound, to indicate read-only on bound.

Of course, you need to also update StorePartitionBound() to make bound also const, as StorePartitionBound() doesn’t update bound as well.

I tried to make them const in my local, and the build passed.

6 - 0001
```
+
+
+/*
+ * buildExpressionExecutionStates: build the needed expression execution states
```

Here seems an extra empty line is added.

6 - 0001
```
+			case CONSTR_CHECK:
+
+				/*
+				 * We already expanded virtual expression in
+				 * createTableConstraints.
+				 */
```

Nit: an unneeded empty line.

7 - 0001
```
+static void
+createTableConstraints(List **wqueue, AlteredTableInfo *tab,
+					   Relation parent_rel, Relation newRel)
+{
+	TupleDesc	tupleDesc;
+	TupleConstr *constr;
+	AttrMap    *attmap;
+	AttrNumber	parent_attno;
+	int			ccnum;
+	List	   *Constraints = NIL;
```

Why this local variable starts with a capital character “C”?

8 - 0001
```
+	/* Look up the access method for new relation. */
+	relamId = (parent_rel->rd_rel->relam != InvalidOid) ? parent_rel->rd_rel->relam : HEAP_TABLE_AM_OID;
```

In this function, “parent_rel->rd_rel” is used in many places, maybe we can cache it to a local variable.

9 - 0001
```
+		/* Create tuple slot for new partition. */
+		srcslot = table_slot_create(mergingPartition, NULL);
```

This comment is quite confusing. Can you rewording to something like:

```
/* Create a source tuple slot for the partition being merged. */
```

10 - 0001
```
+	/*
+	 * We don't need process this newPartRel since we already processed in
+	 * here, so delete the ALTER TABLE queue of it.
+	 */
+	foreach(ltab, *wqueue)
+	{
+		tab = (AlteredTableInfo *) lfirst(ltab);
+		if (tab->relid == RelationGetRelid(newPartRel))
+			*wqueue = list_delete_cell(*wqueue, ltab);
+	}
```

Based on the comment of “foreach”, deleting cell while interacting is unsafe.

And nit: “need process” => “need to process"

11 - 0001
```
+	/* Detach all merged partitions */
+	foreach_oid(mergingPartitionOid, mergingPartitions)
```

Should it be “all merging partitions”?

12 - 0001
```
+static void
+checkPartition(Relation rel, Oid partRelOid)
+{
+	Relation	partRel;
+
+	partRel = table_open(partRelOid, NoLock);
+
+	if (partRel->rd_rel->relkind != RELKIND_RELATION)
+		ereport(ERROR,
+				errcode(ERRCODE_WRONG_OBJECT_TYPE),
+				errmsg("\"%s\" is not a table", RelationGetRelationName(partRel)),
+				errhint("ALTER TABLE ... MERGE PARTITIONS can only merge partitions don't have sub-partitions"));
+
+	if (!partRel->rd_rel->relispartition)
+		ereport(ERROR,
+				errcode(ERRCODE_WRONG_OBJECT_TYPE),
+				errmsg("\"%s\" is not a partition of partitioned table \"%s\"",
+					   RelationGetRelationName(partRel), RelationGetRelationName(rel)),
+				errhint("ALTER TABLE ... MERGE PARTITIONS can only merge partitions don't have sub-partitions"));
```

I think the first two “if” can be combined. We are trying to check If “partRel” is a partition, when a relation is a partition, its relkind is “r” and “relispartition” is true. Instead, “xx is not a table” is a quite confusing message. I would suggest:

```
if (partRel->rd_rel->relkind != RELKIND_RELATION || !partRel->rd_rel->relispartition)
		ereport(ERROR,
				errcode(ERRCODE_WRONG_OBJECT_TYPE),
				errmsg("\"%s\" is not a partition of partitioned table \"%s\"",
					   RelationGetRelationName(partRel), RelationGetRelationName(rel)),
				errhint("ALTER TABLE ... MERGE PARTITIONS can only merge partitions don't have sub-partitions"));
```

13 - 0001
```
+static void
+checkPartition(Relation rel, Oid partRelOid)
+{
+	Relation	partRel;
+
+	partRel = table_open(partRelOid, NoLock);
```

We can immediately close the table, as data is stored in partRel already, we don’t have to defer table close.

14 - 0002
```
@@ -5283,6 +5290,11 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
 			/* No command-specific prep needed */
 			pass = AT_PASS_MISC;
 			break;
+		case AT_SplitPartition:
+			ATSimplePermissions(cmd->subtype, rel, ATT_PARTITIONED_TABLE);
+			/* No command-specific prep needed */
+			pass = AT_PASS_MISC;
+			break;
```

Same as comment 3.

14 - 0002
```
+	foreach(ltab, *wqueue)
+	{
+		AlteredTableInfo *tab = (AlteredTableInfo *) lfirst(ltab);
+
+		if (tab->relid == RelationGetRelid(pc->partRel))
+			*wqueue = list_delete_cell(*wqueue, ltab);
+	}
```

Same as comment 10.

15 - 0002
```
+	/* Scan through the rows. */
+	snapshot = RegisterSnapshot(GetLatestSnapshot());
+	scan = table_beginscan(splitRel, snapshot, 0, NULL);
+
+	/*
+	 * Switch to per-tuple memory context and reset it for each tuple
+	 * produced, so we don't leak memory.
+	 */
+	oldCxt = MemoryContextSwitchTo(GetPerTupleMemoryContext(estate));
+
+	while (table_scan_getnextslot(scan, ForwardScanDirection, srcslot))
+	{
+		bool		found = false;
+		TupleTableSlot *insertslot;
```

For move rows, the logic of merge partitions and split partition are quite similar. Only difference is that merge partitions takes a fixed dest partition, but split partition use a logic to determine target partition. Maybe we can add a common function to reduce the duplicate code.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

  237. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-09-17T08:35:31Z

    On Wed, Sep 17, 2025 at 3:35 PM jian he <jian.universality@gmail.com> wrote:
>
> On Wed, Sep 17, 2025 at 6:08 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> >
> > 2. Patch v56-0001-refactor-v56-check_partitions_for_split.no-cfbot
> > applied with cosmetic changes.
> >

In the previous thread
(https://postgr.es/m/CACJufxGWCVf5r9kE-z6MyR2b+wkaU15Q5m2tKz4cvBhYX3-x1g@mail.gmail.com)
I refactored src/test/regress/sql/partition_merge.sql permission related tests.

attached is refactor for src/test/regress/sql/partition_split.sql
permission related tests.
I grouped the permission-related tests together and
removed unnecessary CREATE ROLE commands.
overall readability improved, i think.

  238. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-17T19:41:22Z

    Hi, Jiah He!


1. v57-0001-refactor-for-v57.no-cfbot
Thanks, the patch make the code clearer.
I added small correction in function check_partition_bounds_for_split_range.
Before the patch, the old code contained the condition:
---------------------------------
if (!defaultPart)
{
	if (cmpval != 0)
...
}
else
{
	if (cmpval < 0)
...
}
---------------------------------
The patch changed this to:
---------------------------------
if (!defaultPart && cmpval != 0)
...
else if (cmpval < 0)
...
---------------------------------
This change is not equivalent to the old code, so it replaced with:
---------------------------------
if (!defaultPart)
{
	if (cmpval != 0)
....
}
else if (cmpval < 0)
...
---------------------------------


2. v57-0001-partition_split.sql-test-refactor-based-on-v57.no-cfbot
 >I grouped the permission-related tests together and
 >removed unnecessary CREATE ROLE commands.
 >overall readability improved, i think.


Applied (I agree, readability improved).

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  239. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-18T00:25:47Z

    Hi Chao Li!

Thanks for reporting the issues!

1.
 >Nit: "when you need delete" => "when you need to delete"
 >"Must specified" => "must be specified"
 >"We want delete" => "we want to delete"

Replaced.


2.
 >+	depRel = table_open(DependRelationId, RowExclusiveLock);
 >This function looks only performing read-only checks, why do we need
 >RowExclusiveLock? Is AccessShareLock good enough?

performDeletionCheck function is not required for the SPLIT/MERGE 
PARTITION(S) functionality.
Its main goal is perform a preliminary check to determine whether it's 
safe to drop split partition before we actually do so later.
For this purpose, it would be better if the lock is the same as in the 
performDeletion function.


3.
 >AT_MergePartitions, AT_DetachPartitionFinalize and AT_DetachPartition
 >do the same thing, why don't combine them together?

I think AT_MergePartitions and AT_SplitPartitions can be combine.
But I prefer not to change of other conditions as possible 
(AT_DetachPartitionFinalize, AT_DetachPartition, etc.) to avoid rebase 
conflicts.
Changed.


4.
 >+	/* Attach a new partition to the partitioned table. */
 >+	attachPartitionTable(wqueue, rel, attachrel, cmd->bound);
 >I think the comment can be removed as the function name has clearly 
described what it is doing.

Deleted.


5.
 >+static void
 >+attachPartitionTable(List **wqueue, Relation rel, Relation attachrel, 
PartitionBoundSpec *bound)
 >I think bound can be const: const PartitionBoundSpec *bound, to 
indicate read-only on bound.
 >Of course, you need to also update StorePartitionBound() to make bound 
also const ...

I agree, this correction could be done.
But I think it's better to change the argument of the 
StorePartitionBound function in a separate commit, since this fix is not 
related to SPLIT/MERGE PARTITION(S).
And then we can change attachPartitionTable function.


6.
 > Nit: an unneeded empty line.

Empty lines (this and similar ones) removed.


7.
 >+	case CONSTR_CHECK:
 >+
 >+		/*
 >+		 * We already expanded virtual expression in
 >+		 * createTableConstraints.
 >+		 */
 >Nit: an unneeded empty line.

This line was added by pgindent...


8.
 >+	List	  *Constraints = NIL;
 >Why this local variable starts with a capital character "C"?

I think it was because of my carelessness. Renamed.


9.
 >+	/* Look up the access method for new relation. */
 >+	relamId = (parent_rel->rd_rel->relam != InvalidOid) ? 
parent_rel->rd_rel->relam : HEAP_TABLE_AM_OID;
 >In this function, "parent_rel->rd_rel" is used in many places, maybe 
we can cache it to a local variable.```

Changed.


10.
 >+		/* Create tuple slot for new partition. */
 >+		srcslot = table_slot_create(mergingPartition, NULL);
 >This comment is quite confusing. Can you rewording to something like:
 >/* Create a source tuple slot for the partition being merged. */

Renamed.


11.
 >Based on the comment of "foreach", deleting cell while interacting is 
unsafe.
 >And nit: "need process" => "need to process"

Corrected.


12.
 >+	/* Detach all merged partitions */
 >+	foreach_oid(mergingPartitionOid, mergingPartitions)
 >Should it be "all merging partitions"?

Corrected.


13.
 >+	if (partRel->rd_rel->relkind != RELKIND_RELATION)
...
 >+	if (!partRel->rd_rel->relispartition)
...
 >I think the first two "if" can be combined. We are trying to check If
 >"partRel" is a partition, when a relation is a partition, its relkind
 >is "r" and "relispartition" is true. Instead, "xx is not a table" is
 >a quite confusing message. ...
 >if (partRel->rd_rel->relkind != RELKIND_RELATION ||
 >  !partRel->rd_rel->relispartition)
 >     ereport(ERROR,
 >       errcode(ERRCODE_WRONG_OBJECT_TYPE),
 >       errmsg("\"%s\" is not a partition of partitioned table \"%s\"",
 > ...

Probably, in this case we will generate a strange error if relkind='f' 
(RELKIND_FOREIGN_TABLE) since it's a partition, but error message will 
be "... is not a partition ...".
Perhaps we should change the error message from "... is not a table" to 
"... is not a ordinary table"?
See comment:
#define	  RELKIND_RELATION	  'r'	/* ordinary table */


14.
 >+checkPartition(Relation rel, Oid partRelOid)
 >...
 >+	partRel = table_open(partRelOid, NoLock);
 >We can immediately close the table, as data is stored in partRel
 >already, we don't have to defer table close.

It can be done, but I think this violates the style used in PostgreSQL 
(for example, [1], [2], ...).


15.
 > For move rows, the logic of merge partitions and split partition are
 >quite similar. Only difference is that merge partitions takes a fixed
 >dest partition, but split partition use a logic to determine target
 >partition. Maybe we can add a common function to reduce the duplicate
 >code.

The problem is that MERGE PARTITIONS is planned to be optimized further 
to use the merging partition with the maximum number of rows as the new 
merged partition (we minimize the number of moved rows).
This will complicate the logic, and we will have to revert to situation 
with different code for SPLIT and MERGE.


Links.
------
[1] 
https://github.com/postgres/postgres/blob/b0cc0a71e0a0a760f54c72edb8cd000e4555442b/src/backend/commands/cluster.c#L1197
[2] 
https://github.com/postgres/postgres/blob/b0cc0a71e0a0a760f54c72edb8cd000e4555442b/src/backend/commands/cluster.c#L1590

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  240. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-09-19T14:53:35Z

    hi. about v59.

check_partitions_not_overlap_list
            overlap = list_intersection(sps1->bound->listdatums,
                                        sps2->bound->listdatums);
            if (list_length(overlap) > 0)
            {
                Const       *val = (Const *) lfirst(list_head(overlap));
                ereport(ERROR,
                        errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
                        errmsg("new partition \"%s\" would overlap
with another new partition \"%s\"",
                               sps1->name->relname, sps2->name->relname),
                        parser_errposition(pstate, exprLocation((Node *) val)));
            }

list_intersection seems not right, how can we be sure it deals with
collation correctly?

It failed to deal with numeric special value (0.0).
demo:
CREATE TABLE t (a numeric) PARTITION BY LIST (a);
CREATE TABLE t1 PARTITION OF t FOR VALUES in  ('0', '1');
ALTER TABLE t SPLIT PARTITION t1 INTO
  (PARTITION x  FOR VALUES in  ('0'),
  PARTITION x1 FOR VALUES IN ('0.0', '1'));

I’ll think about the solution later; for now, I just wanted to point
out this problem.

  241. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-19T20:06:44Z

    Hi, Jiah He!

Thanks!

 >list_intersection seems not right, how can we be sure it deals with
 >collation correctly?

list_intersection function replaced by new partitions_lists_intersection 
function.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  242. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-09-22T07:47:49Z

    On Sat, Sep 20, 2025 at 4:06 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi, Jiah He!
>
> Thanks!
>
>  >list_intersection seems not right, how can we be sure it deals with
>  >collation correctly?
>
> list_intersection function replaced by new partitions_lists_intersection
> function.
>

hi, more about v60.

+ /*
+ * If new partition has the same name as split partition then we should
+ * rename split partition for reusing name.
+ */
+ if (isSameName)
+ {
+ /*
+ * We must bump the command counter to make the split partition tuple
+ * visible for renaming.
+ */
+ CommandCounterIncrement();
+ /* Rename partition. */
+ sprintf(tmpRelName, "split-%u-%X-tmp", RelationGetRelid(rel), MyProcPid);
+ RenameRelationInternal(splitRelOid, tmpRelName, true, false);
+
+ /*
+ * We must bump the command counter to make the split partition tuple
+ * visible after renaming.
+ */
+ CommandCounterIncrement();
+ }
duplicated CommandCounterIncrement call?


+
+/*
+ * get_list_partvalue_string
+ * A C string representation of one list partition value
+ */
+char *
+get_list_partvalue_string(Const *val)
+{
+ deparse_context context;
+ StringInfo buf = makeStringInfo();
+
+ memset(&context, 0, sizeof(deparse_context));
+ context.buf = buf;
+
+ get_const_expr(val, &context, -1);
+
+ return buf->data;
+}

+ ereport(ERROR,
+ errcode(ERRCODE_INVALID_OBJECT_DEFINITION),
+ errmsg("new partitions do not have value %s but split partition does",
+   searchNull ? "NULL" : get_list_partvalue_string(notFoundVal)));

get_list_partvalue_string seems not quite right, because it won't print Const
node collation info.  However, sometimes we do really need to print the
collation, i think.
give that we can do:
CREATE TABLE x(a text collate "C") PARTITION BY LIST (a collate
case_insensitive);

we can simply call deparse_expression. so I removed the
get_list_partvalue_string.


partitions_lists_intersection
will get all the common Const nodes in two PartitionBoundSpec->listdatums.
but that's expensive, and we only need the first common Const location
for error reporting, so I refactored it too.


I propose to change the function name check_partitions_not_overlap_list
to check_list_partitions_overlap,
what do you think?

I also heavily refactor the test again.
(arrange some error check test code together, combine several tests together,
remove/reduce unnecessary tests).
For example, the below code can be merged together to make it more readable.
SELECT attname, attidentity, attgenerated FROM pg_attribute WHERE
attnum > 0 AND attrelid = 'salespeople2_3'::regclass::oid ORDER BY
attnum;
SELECT attname, attidentity, attgenerated FROM pg_attribute WHERE
attnum > 0 AND attrelid = 'salespeople3_4'::regclass::oid ORDER BY
attnum;
SELECT attname, attidentity, attgenerated FROM pg_attribute WHERE
attnum > 0 AND attrelid = 'salespeople4_5'::regclass::oid ORDER BY
attnum;

please check the attached v60 refactor for SPLIT PARTITION.

  243. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-09-22T20:12:20Z

    Hi, Jiah He!

1.
 >duplicated CommandCounterIncrement call?

Probably this duplication is necessary to rename partition correctly ...


2.
 >we can simply call deparse_expression. so I removed the
 >get_list_partvalue_string.

I agree. This looks better.


3.
 >partitions_lists_intersection
 >will get all the common Const nodes in two
 >PartitionBoundSpec->listdatums.
 >but that's expensive, and we only need the first common Const location
 >for error reporting, so I refactored it too.


Ok. If a universal function is needed in the future, 
partitions_lists_intersection can be modified.


4.
 > I also heavily refactor the test again....

Thanks! Applied with minor changes.


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  244. Re: Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-10-01T05:01:09Z

    hi.

+ /*
+ * For simplify check for ranges of new partitions need to sort all
+ * partitions in ascending order of them bounds (we compare upper
+ * bound only).
+ */
+ lower_bounds = (PartitionRangeBound **)
+ palloc0(nparts * sizeof(PartitionRangeBound *));
+
+ /* Create array of lower bounds. */
+ for (i = 0; i < nparts; i++)
+ {
+ lower_bounds[i] = make_one_partition_rbound(key, i,
+ new_parts[i]->bound->lowerdatums, true);
+ }
+
I am confused by the above comments "we compare upper bound only".

some of the function partition_rbound_cmp can be replaced by marco
compare_range_bounds,
for example in check_two_partitions_bounds_range we can use
compare_range_bounds instead of partition_rbound_cmp
not sure if it's worth it or not.


doc:
<varlistentry id="sql-altertable-split-partition">
should come after
<varlistentry id="sql-altertable-merge-partitions">
I’ve refactored the SPLIT PARTITION docs quite a bit—let me know if they make
sense.
Also tweaked the regression tests a little again.

  245. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-10-02T13:26:00Z

    Hi, Jiah He!

Thanks!

1.
 >I am confused by the above comments "we compare upper bound only".

Replaced: upper -> lower.


2.
 >some of the function partition_rbound_cmp can be replaced by marco
 >compare_range_bounds,
 >for example in check_two_partitions_bounds_range we can use
 >compare_range_bounds instead of partition_rbound_cmp
 >not sure if it's worth it or not.

I think would be better keep partition_rbound_cmp in this function 
because we should use "false" instead of "second_lower->kind".


3.
 ><varlistentry id="sql-altertable-split-partition">
 >should come after
 ><varlistentry id="sql-altertable-merge-partitions">
 >I’ve refactored the SPLIT PARTITION docs quite a bit—let me know if
 >they make sense.
 >Also tweaked the regression tests a little again.

Applied.
Unfortunately, I don't know English well enough to spot the inaccuracies 
in doc.
It looks good in translation.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  246. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-10-15T11:34:37Z

    hi.

please check the attach doc refactor for v62-0001.

            if (found_whole_row && attribute->attgenerated != '\0')
                ereport(ERROR,
                        errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
                        errmsg("cannot convert whole-row table reference"),
                        errdetail("Generation expression for column
\"%s\" contains a whole-row reference to table \"%s\".",
                                  NameStr(attribute->attname),
                                  RelationGetRelationName(parent_rel)));

here ereport should be elog(ERROR...).
since this error should be unreachable. see check_nested_generated, cookDefault.

        /*
         * For the moment we have to reject whole-row variables (as for LIKE
         * and inheritances).
         */
        if (found_whole_row)
            elog(ERROR, "Constraint \"%s\" contains a whole-row
reference to table \"%s\".",
                 ccname,
                 RelationGetRelationName(parent_rel));

"table \"%s\".", we don't need that extra period.

"(as for LIKE and inheritances)":
I think you meant, “CREATE TABLE LIKE and table inheritance reject
whole-row check constraint, here we will do the same”.
maybe change to ""(as for CREATE TABLE LIKE and inheritances)".

Overall, I found v62-0001 code makes sense to me.

  247. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-10-15T16:59:01Z

    Hi, Jiah He!

1.
 >please check the attach doc refactor for v62-0001.

Thanks for the changes, they look good (in translation).


2.
 >here ereport should be elog(ERROR...).
 >since this error should be unreachable. see check_nested_generated,
 >cookDefault.

Changed.


3.
 >"(as for LIKE and inheritances)":
 >I think you meant, “CREATE TABLE LIKE and table inheritance reject
 >whole-row check constraint, here we will do the same”.
 >maybe change to ""(as for CREATE TABLE LIKE and inheritances)".

Changed.


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  248. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-10-27T12:13:36Z

    On Mon, Sep 22, 2025 at 11:12 PM Dmitry Koval <d.koval@postgrespro.ru>
wrote:

> Hi, Jiah He!
>
> 1.
>  >duplicated CommandCounterIncrement call?
>
> Probably this duplication is necessary to rename partition correctly ...
>

The second CommandCounterIncrement() is needed to make the renamed relation
visible within our transaction.  Why do we need the first one?  I see tests
pass without it.

Also, I doubt this is correct in the partitions_listdatum_intersection()
function.

    foreach_node(Const, val1, list1)
    {
        if (val1->constisnull)
        {
            if (isnull2)
            {
                result = lappend(result, val1);
                return result;
            }
            isnull1 = true;
            continue;
        }

The branch handling null value in the outer loop, uses null2 flag from the
inner loop.  I think for the null value of the outer loop we still need to
run inner loop to search for the matching null value.

------
Regards,
Alexander Korotkov
Supabase

  249. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-10-27T23:12:29Z

    Hi Alexander!

1.
 > The second CommandCounterIncrement() is needed to make the renamed
 > relation visible within our transaction.  Why do we need the first
 >one?  I see tests pass without it.

It's strange. If I comment the first "CommandCounterIncrement();", in block
-----------------------------------------------------------------------
/*
  * We must bump the command counter to make the split partition tuple
  * visible for renaming.
  */
CommandCounterIncrement();
/* Rename partition. */
sprintf(tmpRelName, "split-%u-%X-tmp", RelationGetRelid(rel), ...);
RenameRelationInternal(splitRelOid, tmpRelName, true, false);
/*
  * We must bump the command counter to make the split partition tuple
  * visible after renaming.
  */
CommandCounterIncrement();
-----------------------------------------------------------------------
, I got the error "ERROR: tuple already updated by self" in the
partition_split.sql test (Ubuntu). If I comment the second
"CommandCounterIncrement();", I got the error "ERROR:  relation
"sales_others" already exists" in the same test.


2.
 >The branch handling null value in the outer loop, uses null2 flag from
 >the inner loop.  I think for the null value of the outer loop we still
 >need to run inner loop to search for the matching null value.

This code looks a little confusing, but it probably works correctly.
This can be verified using two typical examples:
-----------------------------------------------------------------------
list1: (NULL, 1)
list2: (2, NULL)

(1) isnull1 = true, (2) "if (isnull1) lappend(NULL)"
-----------------------------------------------------------------------
list1: (1, NULL)
list2: (NULL, 2)

(1) isnull2 = true, (2) "if (isnull2) lappend(NULL)"
-----------------------------------------------------------------------
In both cases, we return from the function immediately after lappend.
This works because we need to find exactly one repeating value.


-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  250. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-10-30T23:48:50Z

    On Tue, Oct 28, 2025 at 1:12 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> 1.
>  > The second CommandCounterIncrement() is needed to make the renamed
>  > relation visible within our transaction.  Why do we need the first
>  >one?  I see tests pass without it.
>
> It's strange. If I comment the first "CommandCounterIncrement();", in block
> -----------------------------------------------------------------------
> /*
>   * We must bump the command counter to make the split partition tuple
>   * visible for renaming.
>   */
> CommandCounterIncrement();
> /* Rename partition. */
> sprintf(tmpRelName, "split-%u-%X-tmp", RelationGetRelid(rel), ...);
> RenameRelationInternal(splitRelOid, tmpRelName, true, false);
> /*
>   * We must bump the command counter to make the split partition tuple
>   * visible after renaming.
>   */
> CommandCounterIncrement();
> -----------------------------------------------------------------------
> , I got the error "ERROR: tuple already updated by self" in the
> partition_split.sql test (Ubuntu). If I comment the second
> "CommandCounterIncrement();", I got the error "ERROR:  relation
> "sales_others" already exists" in the same test.

Sorry, actually it fails.  It appears that the first
CommandCounterIncrement() is needed to see the result of
detachPartitionTable().

> 2.
>  >The branch handling null value in the outer loop, uses null2 flag from
>  >the inner loop.  I think for the null value of the outer loop we still
>  >need to run inner loop to search for the matching null value.
>
> This code looks a little confusing, but it probably works correctly.
> This can be verified using two typical examples:
> -----------------------------------------------------------------------
> list1: (NULL, 1)
> list2: (2, NULL)
>
> (1) isnull1 = true, (2) "if (isnull1) lappend(NULL)"
> -----------------------------------------------------------------------
> list1: (1, NULL)
> list2: (NULL, 2)
>
> (1) isnull2 = true, (2) "if (isnull2) lappend(NULL)"
> -----------------------------------------------------------------------
> In both cases, we return from the function immediately after lappend.
> This works because we need to find exactly one repeating value.

How this function could handle this case?

list1: (NULL)
list2: whatever containing NULL

The outer loop will just quit after iterating the only element of
list1 without even iterating list2.

------
Regards,
Alexander Korotkov
Supabase

  251. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-10-31T15:41:14Z

    Hi Alexander!

 >How this function could handle this case?
 >
 >list1: (NULL)
 >list2: whatever containing NULL
 >
 >The outer loop will just quit after iterating the only element of
 >list1 without even iterating list2.

Thanks for the case!
Patches with the fix are attached to email.

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  252. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-09T15:14:44Z

    Hi!

On Fri, Oct 31, 2025 at 5:41 PM Dmitry Koval <d.koval@postgrespro.ru> wrote:
>
> Hi Alexander!
>
>  >How this function could handle this case?
>  >
>  >list1: (NULL)
>  >list2: whatever containing NULL
>  >
>  >The outer loop will just quit after iterating the only element of
>  >list1 without even iterating list2.
>
> Thanks for the case!
> Patches with the fix are attached to email.

I went though the patchset and did a lot of cleaning and grammar
corrections for comments and commit messages.  During the previous
attempt for this patch, I had to revert it mainly because of the
security issue caused by repeated table lookup by its name.  The
present version doesn't have this problem, because it doesn't do the
repeated lookup.  Additionally, the patchset went through many series
of detailed review.  I dare give this patchset another chance.  I'm
going to push this if no objections.

------
Regards,
Alexander Korotkov
Supabase

  253. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Dmitry Koval <d.koval@postgrespro.ru> — 2025-12-09T23:22:53Z

    Hi, Alexander!

Thank you for your corrections!
Two questions:
----

1) "v65-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch"
  patch, "src/backend/partitioning/partbounds.c" file, 
"check_two_partitions_bounds_range" function.

Old comment (v64)

+	/*
+	 * lower1=false (the second to last argument) for correct comparison of
+	 * lower and upper bounds.
+	 */

was changed to (v65)

+	/*
+	 * cmpval == false for the correct comparison result of the lower and
+	 * upper bounds.
+	 */

Maybe it's better to keep the old comment (or keep its meaning)?
----

2) "v65-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch"
patch, "src/backend/parser/parse_utilcmd.c" file, "checkPartition" function.

The third argument of function was changed from "is_merge" (v64) to
"isMerge" (v65). Maybe the function description should be changed in
the same way?

+ * is_merge: true indicates the operation is "ALTER TABLE ... MERGE 
PARTITIONS";

-- 
With best regards,
Dmitry Koval

Postgres Professional: http://postgrespro.com

  254. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-09T23:54:00Z

    On Wed, Dec 10, 2025 at 1:22 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> Thank you for your corrections!
> Two questions:
> ----
>
> 1) "v65-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch"
>   patch, "src/backend/partitioning/partbounds.c" file,
> "check_two_partitions_bounds_range" function.
>
> Old comment (v64)
>
> +       /*
> +        * lower1=false (the second to last argument) for correct comparison of
> +        * lower and upper bounds.
> +        */
>
> was changed to (v65)
>
> +       /*
> +        * cmpval == false for the correct comparison result of the lower and
> +        * upper bounds.
> +        */
>
> Maybe it's better to keep the old comment (or keep its meaning)?
> ----
>
> 2) "v65-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch"
> patch, "src/backend/parser/parse_utilcmd.c" file, "checkPartition" function.
>
> The third argument of function was changed from "is_merge" (v64) to
> "isMerge" (v65). Maybe the function description should be changed in
> the same way?
>
> + * is_merge: true indicates the operation is "ALTER TABLE ... MERGE
> PARTITIONS";

Thank you for catching this.  Both accepted.

------
Regards,
Alexander Korotkov
Supabase

  255. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Stéphane Tachoires <stephane.tachoires@gmail.com> — 2025-12-10T20:04:11Z

    Hi,

Patches don't apply anymore.
Could you rebase it please ?

Thank you so much to all of you for your persistence.
Stéphane.
Back.

Le mer. 10 déc. 2025 à 00:54, Alexander Korotkov
<aekorotkov@gmail.com> a écrit :
>
> On Wed, Dec 10, 2025 at 1:22 AM Dmitry Koval <d.koval@postgrespro.ru> wrote:
> > Thank you for your corrections!
> > Two questions:
> > ----
> >
> > 1) "v65-0001-Implement-ALTER-TABLE-.-MERGE-PARTITIONS-.-comma.patch"
> >   patch, "src/backend/partitioning/partbounds.c" file,
> > "check_two_partitions_bounds_range" function.
> >
> > Old comment (v64)
> >
> > +       /*
> > +        * lower1=false (the second to last argument) for correct comparison of
> > +        * lower and upper bounds.
> > +        */
> >
> > was changed to (v65)
> >
> > +       /*
> > +        * cmpval == false for the correct comparison result of the lower and
> > +        * upper bounds.
> > +        */
> >
> > Maybe it's better to keep the old comment (or keep its meaning)?
> > ----
> >
> > 2) "v65-0002-Implement-ALTER-TABLE-.-SPLIT-PARTITION-.-comman.patch"
> > patch, "src/backend/parser/parse_utilcmd.c" file, "checkPartition" function.
> >
> > The third argument of function was changed from "is_merge" (v64) to
> > "isMerge" (v65). Maybe the function description should be changed in
> > the same way?
> >
> > + * is_merge: true indicates the operation is "ALTER TABLE ... MERGE
> > PARTITIONS";
>
> Thank you for catching this.  Both accepted.
>
> ------
> Regards,
> Alexander Korotkov
> Supabase

  256. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-11T18:56:37Z

    On Wed, Dec 10, 2025 at 10:04 PM Stéphane Tachoires
<stephane.tachoires@gmail.com> wrote:
> Patches don't apply anymore.
> Could you rebase it please ?
>
> Thank you so much to all of you for your persistence.

Thank you for pointing this.
Rebased.

------
Regards,
Alexander Korotkov
Supabase

  257. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-14T10:44:13Z

    On Thu, Dec 11, 2025 at 8:56 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> On Wed, Dec 10, 2025 at 10:04 PM Stéphane Tachoires
> <stephane.tachoires@gmail.com> wrote:
> > Patches don't apply anymore.
> > Could you rebase it please ?
> >
> > Thank you so much to all of you for your persistence.
>
> Thank you for pointing this.
> Rebased.

I found that I didn't rebase properly regression tests in
src/test/modules/test_ddl_deparse.  The corrected version is attached.

------
Regards,
Alexander Korotkov
Supabase

  258. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    zengman <zengman@halodbtech.com> — 2025-12-14T12:51:22Z

    Hi everyone,

I just noticed two relatively large commits. I recall that not long ago we started attempting to adopt `palloc_object` and `palloc_array` in many places, 
yet in these two commits, some sections have used the new approach while others have not. 
Perhaps we could standardize the coding style for consistency.

src/backend/commands/tablecmds.c
src/backend/partitioning/partbounds.c

Regards,
Man Zeng

  259. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-14T13:33:45Z

    Hi Man,

On Sun, Dec 14, 2025 at 2:51 PM zengman <zengman@halodbtech.com> wrote:
> I just noticed two relatively large commits. I recall that not long ago we started attempting to adopt `palloc_object` and `palloc_array` in many places,
> yet in these two commits, some sections have used the new approach while others have not.
> Perhaps we could standardize the coding style for consistency.
>
> src/backend/commands/tablecmds.c
> src/backend/partitioning/partbounds.c

Thank you for noticing this.  I'm going to prepare patches to adopt
`palloc_object` and `palloc_array` in my recent commits right away.

------
Regards,
Alexander Korotkov
Supabase

  260. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-14T13:47:34Z

    Hi Man,

On Sun, Dec 14, 2025 at 2:51 PM zengman <zengman@halodbtech.com> wrote:
> I just noticed two relatively large commits. I recall that not long ago we started attempting to adopt `palloc_object` and `palloc_array` in many places,
> yet in these two commits, some sections have used the new approach while others have not.
> Perhaps we could standardize the coding style for consistency.
>
> src/backend/commands/tablecmds.c
> src/backend/partitioning/partbounds.c

Could you, please, check the attached patch?

------
Regards,
Alexander Korotkov
Supabase

  261. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    zengman <zengman@halodbtech.com> — 2025-12-14T14:06:31Z

    Looks great! Thanks for your hard work – that’s all I’ve noticed for now.

Regards,
Man Zeng

  262. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-14T14:12:15Z

    On Sun, Dec 14, 2025 at 4:06 PM zengman <zengman@halodbtech.com> wrote:
>
> Looks great! Thanks for your hard work – that’s all I’ve noticed for now.

Pushed, thank you!

------
Regards,
Alexander Korotkov
Supabase

  263. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Tender Wang <tndrwang@gmail.com> — 2025-12-20T03:18:16Z

     Hi Alexander,

I found this feature merged; thanks for this work.
I tested it and found that one place in the error errcode may need to be
changed.
In checkPartition():
...
if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
    ereport(ERROR,
    errcode(ERRCODE_UNDEFINED_TABLE),
    errmsg("relation \"%s\" is not a partition of relation \"%s\"",
...

ERRCODE_UNDEFINED_TABLE usually means "table does not exist."
When entering here, the table should exist, otherwise table_open() already
reports an error.
I found another two errcode in checkPartition()
use ERRCODE_WRONG_OBJECT_TYPE,
In the attached patch, I replace ERRCODE_UNDEFINED_TABLE with
ERRCODE_WRONG_OBJECT_TYPE.

-- 
Thanks,
Tender Wang

  264. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Kirill Reshke <reshkekirill@gmail.com> — 2025-12-20T08:58:21Z

    hi!

I have been looking though git log, noticed this commit, and did small tests.

This is what I found:

```
reshke=# create table z(i int) partition by range(i);
CREATE TABLE
reshke=# create table z_1 partition of z for values from (0)to(1);
CREATE TABLE
reshke=# create table z_2 partition of z for values from (1)to(2);
CREATE TABLE
reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
ALTER TABLE
reshke=#

```

IMO "alter table only ... merge partitions" does not make perfect
sense and should be rejected rather than executed. WDYT?

  265. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-20T10:41:39Z

    Hi Kirill!

On Sat, Dec 20, 2025 at 10:58 AM Kirill Reshke <reshkekirill@gmail.com> wrote:
> I have been looking though git log, noticed this commit, and did small tests.
>
> This is what I found:
>
> ```
> reshke=# create table z(i int) partition by range(i);
> CREATE TABLE
> reshke=# create table z_1 partition of z for values from (0)to(1);
> CREATE TABLE
> reshke=# create table z_2 partition of z for values from (1)to(2);
> CREATE TABLE
> reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> ALTER TABLE
> reshke=#
>
> ```
>
> IMO "alter table only ... merge partitions" does not make perfect
> sense and should be rejected rather than executed. WDYT?

Could you, please, clarify your point? I didn't quite get it.  It
looks like pretty basic example of merging two adjacent partitions.

------
Regards,
Alexander Korotkov
Supabase

  266. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-20T11:07:51Z

    Hi Tender,

On Sat, Dec 20, 2025 at 5:18 AM Tender Wang <tndrwang@gmail.com> wrote:
> I found this feature merged; thanks for this work.
> I tested it and found that one place in the error errcode may need to be changed.
> In checkPartition():
> ...
> if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
>     ereport(ERROR,
>     errcode(ERRCODE_UNDEFINED_TABLE),
>     errmsg("relation \"%s\" is not a partition of relation \"%s\"",
> ...
>
> ERRCODE_UNDEFINED_TABLE usually means "table does not exist."
> When entering here, the table should exist, otherwise table_open() already reports an error.
> I found another two errcode in checkPartition() use ERRCODE_WRONG_OBJECT_TYPE,
> In the attached patch, I replace ERRCODE_UNDEFINED_TABLE with ERRCODE_WRONG_OBJECT_TYPE.

I agree with you that ERRCODE_UNDEFINED_TABLE is certainly wrong error
code because the table actually exists.  ERRCODE_WRONG_OBJECT_TYPE is
better.  For example, we throw it when trying to attach a partition to
non-partitioned table.  So, the parent table type is wrong.  However,
are objects in the situation under consideration really have wrong
type?  The problem is that one table is not partition of another.
However, it's possibly that they could be attached without changing of
their types.  So, I think about
ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE.  What do you think?

------
Regards,
Alexander Korotkov
Supabase

  267. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    jian he <jian.universality@gmail.com> — 2025-12-20T11:14:53Z

    On Sat, Dec 20, 2025 at 6:42 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
>
> Hi Kirill!

> > reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> > ALTER TABLE
> > reshke=#
> >
> > ```
> >
> > IMO "alter table only ... merge partitions" does not make perfect
> > sense and should be rejected rather than executed. WDYT?
>
> Could you, please, clarify your point? I didn't quite get it.  It
> looks like pretty basic example of merging two adjacent partitions.
>

> reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
I think it should fail, because we are not applying to table "z" itself,
For Split/Merge partitions, we are processing the whole partitioned
table z hierarchy.

alter table z merge partitions (z_1,z_2) into z_12;
should work.

I guess the attached maybe is what Krill wants.

--
jian
https://www.enterprisedb.com

  268. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-20T11:26:59Z

    On Sat, Dec 20, 2025 at 1:15 PM jian he <jian.universality@gmail.com> wrote:
> On Sat, Dec 20, 2025 at 6:42 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> >
> > Hi Kirill!
>
> > > reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> > > ALTER TABLE
> > > reshke=#
> > >
> > > ```
> > >
> > > IMO "alter table only ... merge partitions" does not make perfect
> > > sense and should be rejected rather than executed. WDYT?
> >
> > Could you, please, clarify your point? I didn't quite get it.  It
> > looks like pretty basic example of merging two adjacent partitions.
> >
>
> > reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> I think it should fail, because we are not applying to table "z" itself,
> For Split/Merge partitions, we are processing the whole partitioned
> table z hierarchy.
>
> alter table z merge partitions (z_1,z_2) into z_12;
> should work.
>
> I guess the attached maybe is what Krill wants.

Thank you.  I missed there is an ONLY keyword.  But I'm not sure about
the error message.  I don't think the problem with ONLY keyword is
that MERGE/SPLIT must be always recursive.  I think opposite, it's
always non-recursive and this is why ONLY is meaningless.  Otherwise,
we may decide to just leave it as it allowing ONLY.

------
Regards,
Alexander Korotkov
Supabase

  269. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Kirill Reshke <reshkekirill@gmail.com> — 2025-12-20T11:37:57Z

    On Sat, 20 Dec 2025 at 16:27, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>
> On Sat, Dec 20, 2025 at 1:15 PM jian he <jian.universality@gmail.com> wrote:
> > On Sat, Dec 20, 2025 at 6:42 PM Alexander Korotkov <aekorotkov@gmail.com> wrote:
> > >
> > > Hi Kirill!
> >
> > > > reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> > > > ALTER TABLE
> > > > reshke=#
> > > >
> > > > ```
> > > >
> > > > IMO "alter table only ... merge partitions" does not make perfect
> > > > sense and should be rejected rather than executed. WDYT?
> > >
> > > Could you, please, clarify your point? I didn't quite get it.  It
> > > looks like pretty basic example of merging two adjacent partitions.
> > >
> >
> > > reshke=# alter table only z merge partitions (z_1,z_2) into z_12;
> > I think it should fail, because we are not applying to table "z" itself,
> > For Split/Merge partitions, we are processing the whole partitioned
> > table z hierarchy.
> >
> > alter table z merge partitions (z_1,z_2) into z_12;
> > should work.
> >
> > I guess the attached maybe is what Krill wants.


Jian, Thank, you got me right. Your patch is addressing the problem I
talk about, yes. The only issue about your patch is the actual error
message (error hint is exactly on point.)

So, instead of

```
+ERROR:  ALTER TABLE MERGE PARTITIONS must apply to child tables too
+HINT:  Do not specify the ONLY keyword.
``

I would prefer (something like)

```
+ERROR:  ALTER TABLE MERGE PARTITIONS is a non-recursive command.
+HINT:  Do not specify the ONLY keyword.
``

On Sat, 20 Dec 2025 at 16:27, Alexander Korotkov <aekorotkov@gmail.com> wrote:

>  I don't think the problem with ONLY keyword is
> that MERGE/SPLIT must be always recursive.  I think opposite, it's
> always non-recursive and this is why ONLY is meaningless.  Otherwise,
> we may decide to just leave it as it allowing ONLY.

+1


-- 
Best regards,
Kirill Reshke

  270. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Tender Wang <tndrwang@gmail.com> — 2025-12-21T07:14:01Z

    Alexander Korotkov <aekorotkov@gmail.com> 于2025年12月20日周六 19:08写道：

> Hi Tender,
>
> On Sat, Dec 20, 2025 at 5:18 AM Tender Wang <tndrwang@gmail.com> wrote:
> > I found this feature merged; thanks for this work.
> > I tested it and found that one place in the error errcode may need to be
> changed.
> > In checkPartition():
> > ...
> > if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
> >     ereport(ERROR,
> >     errcode(ERRCODE_UNDEFINED_TABLE),
> >     errmsg("relation \"%s\" is not a partition of relation \"%s\"",
> > ...
> >
> > ERRCODE_UNDEFINED_TABLE usually means "table does not exist."
> > When entering here, the table should exist, otherwise table_open()
> already reports an error.
> > I found another two errcode in checkPartition() use
> ERRCODE_WRONG_OBJECT_TYPE,
> > In the attached patch, I replace ERRCODE_UNDEFINED_TABLE with
> ERRCODE_WRONG_OBJECT_TYPE.
>
> I agree with you that ERRCODE_UNDEFINED_TABLE is certainly wrong error
> code because the table actually exists.  ERRCODE_WRONG_OBJECT_TYPE is
> better.  For example, we throw it when trying to attach a partition to
> non-partitioned table.  So, the parent table type is wrong.  However,
> are objects in the situation under consideration really have wrong
> type?  The problem is that one table is not partition of another.
> However, it's possibly that they could be attached without changing of
> their types.  So, I think about
> ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE.  What do you think?
>

It's ok for me. Please check the v2 patch.

-- 
Thanks,
Tender Wang

  271. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Kirill Reshke <reshkekirill@gmail.com> — 2025-12-21T07:42:36Z

    On Sun, 21 Dec 2025, 12:14 Tender Wang, <tndrwang@gmail.com> wrote:

>
>
> Alexander Korotkov <aekorotkov@gmail.com> 于2025年12月20日周六 19:08写道：
>
>> Hi Tender,
>>
>> On Sat, Dec 20, 2025 at 5:18 AM Tender Wang <tndrwang@gmail.com> wrote:
>> > I found this feature merged; thanks for this work.
>> > I tested it and found that one place in the error errcode may need to
>> be changed.
>> > In checkPartition():
>> > ...
>> > if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
>> >     ereport(ERROR,
>> >     errcode(ERRCODE_UNDEFINED_TABLE),
>> >     errmsg("relation \"%s\" is not a partition of relation \"%s\"",
>> > ...
>> >
>> > ERRCODE_UNDEFINED_TABLE usually means "table does not exist."
>> > When entering here, the table should exist, otherwise table_open()
>> already reports an error.
>> > I found another two errcode in checkPartition() use
>> ERRCODE_WRONG_OBJECT_TYPE,
>> > In the attached patch, I replace ERRCODE_UNDEFINED_TABLE with
>> ERRCODE_WRONG_OBJECT_TYPE.
>>
>> I agree with you that ERRCODE_UNDEFINED_TABLE is certainly wrong error
>> code because the table actually exists.  ERRCODE_WRONG_OBJECT_TYPE is
>> better.  For example, we throw it when trying to attach a partition to
>> non-partitioned table.  So, the parent table type is wrong.  However,
>> are objects in the situation under consideration really have wrong
>> type?  The problem is that one table is not partition of another.
>> However, it's possibly that they could be attached without changing of
>> their types.  So, I think about
>> ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE.  What do you think?
>>
>
> It's ok for me. Please check the v2 patch.
>
> --
> Thanks,
> Tender Wang
>


On Sun, 21 Dec 2025, 12:14 Tender Wang, <tndrwang@gmail.com> wrote:

>
>
> Alexander Korotkov <aekorotkov@gmail.com> 于2025年12月20日周六 19:08写道：
>
>> Hi Tender,
>>
>> On Sat, Dec 20, 2025 at 5:18 AM Tender Wang <tndrwang@gmail.com> wrote:
>> > I found this feature merged; thanks for this work.
>> > I tested it and found that one place in the error errcode may need to
>> be changed.
>> > In checkPartition():
>> > ...
>> > if (get_partition_parent(partRelOid, false) != RelationGetRelid(rel))
>> >     ereport(ERROR,
>> >     errcode(ERRCODE_UNDEFINED_TABLE),
>> >     errmsg("relation \"%s\" is not a partition of relation \"%s\"",
>> > ...
>> >
>> > ERRCODE_UNDEFINED_TABLE usually means "table does not exist."
>> > When entering here, the table should exist, otherwise table_open()
>> already reports an error.
>> > I found another two errcode in checkPartition() use
>> ERRCODE_WRONG_OBJECT_TYPE,
>> > In the attached patch, I replace ERRCODE_UNDEFINED_TABLE with
>> ERRCODE_WRONG_OBJECT_TYPE.
>>
>> I agree with you that ERRCODE_UNDEFINED_TABLE is certainly wrong error
>> code because the table actually exists.  ERRCODE_WRONG_OBJECT_TYPE is
>> better.  For example, we throw it when trying to attach a partition to
>> non-partitioned table.  So, the parent table type is wrong.  However,
>> are objects in the situation under consideration really have wrong
>> type?  The problem is that one table is not partition of another.
>> However, it's possibly that they could be attached without changing of
>> their types.  So, I think about
>> ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE.  What do you think?
>>
>
> It's ok for me. Please check the v2 patch.
>
> --
> Thanks,
> Tender Wang
>


Hi! Your v2 looks fine
The only question for me is, should we add any regression test to exercise
this code, or it is not worth the troubles?

>

  272. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Alexander Korotkov <aekorotkov@gmail.com> — 2025-12-21T22:02:22Z

    On Sun, Dec 21, 2025 at 9:42 AM Kirill Reshke <reshkekirill@gmail.com> wrote:
> On Sun, 21 Dec 2025, 12:14 Tender Wang, <tndrwang@gmail.com> wrote:
>> Alexander Korotkov <aekorotkov@gmail.com> 于2025年12月20日周六 19:08写道：
>>> I agree with you that ERRCODE_UNDEFINED_TABLE is certainly wrong error
>>> code because the table actually exists.  ERRCODE_WRONG_OBJECT_TYPE is
>>> better.  For example, we throw it when trying to attach a partition to
>>> non-partitioned table.  So, the parent table type is wrong.  However,
>>> are objects in the situation under consideration really have wrong
>>> type?  The problem is that one table is not partition of another.
>>> However, it's possibly that they could be attached without changing of
>>> their types.  So, I think about
>>> ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE.  What do you think?
>>
>>
>> It's ok for me. Please check the v2 patch.
>
>
> Hi! Your v2 looks fine
> The only question for me is, should we add any regression test to exercise this code, or it is not worth the troubles?

I've checked contents of out regression tests.  I see we very rarely
include SQLSTATE there, mostly in psql and plpgsql tests.  Thus, I
think we should just fix the SQLSTATE without dedicating a test for
that.  So, I'm going to push the patch from Tender Wang if no
objections.

------
Regards,
Alexander Korotkov
Supabase

  273. Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

    Kirill Reshke <reshkekirill@gmail.com> — 2025-12-22T05:47:37Z

    On Mon, 22 Dec 2025 at 03:02, Alexander Korotkov <aekorotkov@gmail.com> wrote:
>
> On Sun, Dec 21, 2025 at 9:42 AM Kirill Reshke <reshkekirill@gmail.com> wrote:

> > Hi! Your v2 looks fine
> > The only question for me is, should we add any regression test to exercise this code, or it is not worth the troubles?
>
> I've checked contents of out regression tests.  I see we very rarely
> include SQLSTATE there, mostly in psql and plpgsql tests.  Thus, I
> think we should just fix the SQLSTATE without dedicating a test for
> that.  So, I'm going to push the patch from Tender Wang if no
> objections.


Ok, no objections.


-- 
Best regards,
Kirill Reshke