Horton

Re: Add SPLIT PARTITION/MERGE PARTITIONS commands

Zhihong Yu <zyu@yugabyte.com>

View thread
From: Zhihong Yu <zyu@yugabyte.com>
To: Dmitry Koval <d.koval@postgrespro.ru>
Cc: Matthias van de Meent <boekewurm+postgres@gmail.com>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>
Date: 2022-05-31T22:14:25Z
Lists: pgsql-hackers

Commits

Same data as JSON: GET /api/v1/messages/:b64id/commits the thread's linked commits as JSON, with link sources. API reference →

  1. Adjust errcode in checkPartition()

  2. Fix usage of palloc() in MERGE/SPLIT PARTITION(s) code

  3. Implement ALTER TABLE ... SPLIT PARTITION ... command

  4. Implement ALTER TABLE ... MERGE PARTITIONS ... command

  5. Calculate agglevelsup correctly when Aggref contains a CTE.

  6. Use PqMsg_* macros in applyparallelworker.c.

  7. Restrict psql meta-commands in plain-text dumps.

  8. Ensure we have a snapshot when updating various system catalogs.

  9. Use specific collation where needed in new test

  10. Expand virtual generated columns in the planner

  11. Virtual generated columns

  12. Define PG_LOGICAL_DIR for path pg_logical/ in data folder

  13. Revert support for ALTER TABLE ... MERGE/SPLIT PARTITION(S) commands

  14. Avoid repeated table name lookups in createPartitionTable()

  15. Provide deterministic order for catalog queries in partition_split.sql

  16. Don't copy extended statistics during MERGE/SPLIT partition operations

  17. Fix the name collision detection in MERGE/SPLIT partition operations

  18. Fix regression tests conflict in 3ca43dbbb6

  19. Add permission check for MERGE/SPLIT partition operations

  20. Fix one more portability shortcoming in new test_pg_dump test.

  21. Inherit parent's AM for partition MERGE/SPLIT operations

  22. Add tab completion for partition MERGE/SPLIT operations

  23. Rename tables in tests of partition MERGE/SPLIT operations

  24. Make new partitions with parent's persistence during MERGE/SPLIT

  25. Document the way partition MERGE/SPLIT operations create new partitions

  26. Change the way ATExecMergePartitions() handles the name collision

  27. Grammar fixes for split/merge partitions code

  28. Checks for ALTER TABLE ... SPLIT/MERGE PARTITIONS ... commands

  29. Fix some grammer errors from error messages and codes comments

  30. Support TZ and OF format codes in to_timestamp().

  31. Support identity columns in partitioned tables

  32. Fix indentation in twophase.c

  33. Fix corner-case planner failure for MERGE.

  34. Doc: fix documentation example for bytea hex output format.

  35. Avoid repeated name lookups during table and index DDL.

On Tue, May 31, 2022 at 1:43 PM Zhihong Yu <zyu@yugabyte.com> wrote:

>
>
> On Tue, May 31, 2022 at 12:43 PM Dmitry Koval <d.koval@postgrespro.ru>
> wrote:
>
>>  >Just out of curiosity, why is SPLIT / MERGE support not included for
>>  >HASH partitions? Because sibling partitions can have a different
>>  >modulus, you should be able to e.g. split a partition with (modulus,
>>  >remainder) of (3, 1) into two partitions with (mod, rem) of (6, 1) and
>>  >(6, 4) respectively, with the reverse being true for merge operations,
>>  >right?
>>
>> You are right, SPLIT/MERGE operations can be added for HASH-partitioning
>> in the future. But HASH-partitioning is rarer than RANGE- and
>> LIST-partitioning and I decided to skip it in the first step.
>> Maybe community will say that SPLIT/MERGE commands are not needed... (At
>> first step I would like to make sure that it is no true)
>>
>> P.S. I attached patch with 1-line warning fix (for cfbot).
>> --
>> With best regards,
>> Dmitry Koval
>>
>> Postgres Professional: http://postgrespro.com
>
>
> Hi,
> For attachPartTable, the parameter wqueue is missing from comment.
> The parameters of CloneRowTriggersToPartition are called parent
> and partition. I think it is better to name the parameters to
> attachPartTable in a similar manner.
>
> For struct SplitPartContext, SplitPartitionContext would be better name.
>
> +       /* Store partition contect into list. */
> contect -> context
>
> Cheers
>
Hi,
For transformPartitionCmdForMerge(), nested loop is used to detect
duplicate names.
If the number of partitions in partcmd->partlist, we should utilize map to
speed up the check.

For check_parent_values_in_new_partitions():

+           if (!find_value_in_new_partitions(&key->partsupfunc[0],
+                                             key->partcollation, parts,
nparts, datum, false))
+               found = false;

It seems we can break out of the loop when found is false.

Cheers