Re: ATTACH/DETACH PARTITION CONCURRENTLY

Alvaro Herrera <alvherre@2ndquadrant.com>

From: Alvaro Herrera <alvherre@2ndquadrant.com>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Michael Paquier <michael@paquier.xyz>, Sergei Kornilov <sk@zsrv.org>, Amit Langote <langote_amit_f8@lab.ntt.co.jp>, David Rowley <david.rowley@2ndquadrant.com>, Andres Freund <andres@anarazel.de>, PostgreSQL Hackers <pgsql-hackers@lists.postgresql.org>, Simon Riggs <simon@2ndquadrant.com>
Date: 2018-12-20T20:58:02Z
Lists: pgsql-hackers
Thanks for this work!  I like the name "partition directory".

On 2018-Dec-20, Robert Haas wrote:

> 0002 introduces the concept of a partition directory.  The idea is
> that the planner will create a partition directory, and so will the
> executor, and all calls which occur in those places to
> RelationGetPartitionDesc() will instead call
> PartitionDirectoryLookup().  Every lookup for the same relation in the
> same partition directory is guaranteed to produce the same answer.  I
> believe this patch still has a number of weaknesses.  More on that
> below.

The commit message for this one also points out another potential
problem,

> Introduce the concept of a partition directory.
>
> Teach the optimizer and executor to use it, so that a single planning
> cycle or query execution gets the same PartitionDesc for the same table
> every time it looks it up.  This does not prevent changes between
> planning and execution, nor does it guarantee that all tables are
> expanded according to the same snapshot.

Namely: how does this handle the case of partition pruning structure
being passed from planner to executor, if an attach happens in the
middle of it and puts a partition in between existing partitions?  Array
indexes of any partitions that appear later in the partition descriptor
will change.

This is the reason I used the query snapshot rather than EState.

-- 
Álvaro Herrera                https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


Commits

  1. Allow ATTACH PARTITION with only ShareUpdateExclusiveLock.

  2. Change lock acquisition order in expand_inherited_rtentry.

  3. Move code for managing PartitionDescs into a new file, partdesc.c

  4. Remove more redundant relation locking during executor startup.

  5. Add assertions that we hold some relevant lock during relation open.

  6. Try to acquire relation locks in RangeVarGetRelid.