Re: executor relation handling

On 2018-10-04 12:34:44 -0700, Andres Freund wrote:
> Hi,
>
> On 2018-10-04 15:27:59 -0400, Tom Lane wrote:
> > Andres Freund <andres@anarazel.de> writes:
> > > I've not really followed this thread, and just caught up to here.  It
> > > seems entirely unacceptable to not acquire locks on workers to me.
> > > Maybe I'm missing something, but why do/did the patches in this thread
> > > require that / introduce that? We didn't have that kind of concept
> > > before, no?  The group locking stuff should rely / require that kind of
> > > thing, no?
> >
> > I'm possibly confused, but I thought that the design of parallel query
> > involved an expectation that workers didn't need to get their own
> > locks.
>
> Not as far as I'm aware of - but I'm not exactly the expert
> there. There's an exception that some lock classes don't conflict
> between the leader and the workers - that's group locking
> (a1c1af2a1f60). But the locks still have to be acquired, and I think
> it's quite dangerous not to do so.  The group locking logic is required
> because otherwise it'd be trivial to get into deadlocks, and some of the
> restrictions around parallel query are required to make that safe.

Re-read docs + code just to make sure.  Here's the relevant readme parts:

src/backend/access/transam/README.parallel

To prevent unprincipled deadlocks when running in parallel mode, this code
also arranges for the leader and all workers to participate in group
locking.  See src/backend/storage/lmgr/README for more details.


src/backend/storage/lmgr/README:

Group Locking
-------------

As if all of that weren't already complicated enough, PostgreSQL now supports
parallelism (see src/backend/access/transam/README.parallel), which means that
we might need to resolve deadlocks that occur between gangs of related
processes rather than individual processes.  This doesn't change the basic
deadlock detection algorithm very much, but it makes the bookkeeping more
complicated.

We choose to regard locks held by processes in the same parallel group as
non-conflicting.  This means that two processes in a parallel group can hold a
self-exclusive lock on the same relation at the same time, or one process can
acquire an AccessShareLock while the other already holds AccessExclusiveLock.
This might seem dangerous and could be in some cases (more on that below), but
if we didn't do this then parallel query would be extremely prone to
self-deadlock.  For example, a parallel query against a relation on which the
leader already had AccessExclusiveLock would hang, because the workers would
try to lock the same relation and be blocked by the leader; yet the leader
can't finish until it receives completion indications from all workers.  An
undetected deadlock results.  This is far from the only scenario where such a
problem happens.  The same thing will occur if the leader holds only
AccessShareLock, the worker seeks AccessShareLock, but between the time the
leader attempts to acquire the lock and the time the worker attempts to
acquire it, some other process queues up waiting for an AccessExclusiveLock.
In this case, too, an indefinite hang results.

...


So yes, locks are expected to be acquired in workers.

Greetings,

Andres Freund