Horton

Re: generic plans and "initial" pruning

Amit Langote <amitlangote09@gmail.com>

View thread View on postgr.es
From: Amit Langote <amitlangote09@gmail.com>
To: Junwang Zhao <zhjwpku@gmail.com>
Cc: Robert Haas <robertmhaas@gmail.com>, Alvaro Herrera <alvherre@alvh.no-ip.org>, Andres Freund <andres@anarazel.de>, Daniel Gustafsson <daniel@yesql.se>, David Rowley <dgrowleyml@gmail.com>, PostgreSQL Hackers <pgsql-hackers@postgresql.org>, Thom Brown <thom@linux.com>, Tom Lane <tgl@sss.pgh.pa.us>
Date: 2024-09-05T09:55:47Z
Lists: pgsql-hackers

Attachments

On Mon, Sep 2, 2024 at 5:19 PM Amit Langote <amitlangote09@gmail.com> wrote:
> On Sat, Aug 31, 2024 at 9:30 PM Junwang Zhao <zhjwpku@gmail.com> wrote:
> > @@ -1241,7 +1244,7 @@ GetCachedPlan(CachedPlanSource *plansource,
> > ParamListInfo boundParams,
> >   if (customplan)
> >   {
> >   /* Build a custom plan */
> > - plan = BuildCachedPlan(plansource, qlist, boundParams, queryEnv);
> > + plan = BuildCachedPlan(plansource, qlist, boundParams, queryEnv, true);
> >
> > Is the *true* here a typo? Seems it should be *false* for custom plan?
>
> That's correct, thanks for catching that.  Will fix.

Done.

I've also rewritten the new GetSingleCachedPlan() function in 0003.
The most glaring bug in the previous version was that the transient
CachedPlan it creates cannot be seen by PlanCacheRelCallback() et al
functions because it was intentionally not linked to the
CachedPlanSource, so if the CachedPlan would not be invalidated even
if some prunable relation got changed before it is locked during
ExecutorStart().  I've added a new list standalone_plan_list to add
these to and changed the inval callback functions to invalidate any
plans contained in them.

Another thing I found out through testing is that CachedPlanSource can
have become invalid since leaving GetCachedPlan() (actually even
before returning from that function) because of
PlanCacheSysCallback(), which drops/invalidates *all* plans when a
syscache is invalidated.  There are comments in plancache.c (see
BuildCachedPlan()) saying that such invalidations are, in theory,
false positives, but that gave me a pause nonetheless.

Finally, instead of calling GetCachedPlan() from GetSingleCachedPlan()
to create a plan for only the query whose plan got invalidated, which
required a bunch of care to ensure that the CachedPlanSource is not
overwritten with the information about this single-query planning,
I've made GetSingleCachedPlan() create the PlannedStmt and the
detached CachedPlan object on its own, borrowing the minimal necessary
code from BuildCachedPlan() to do so.

-- 
Thanks, Amit Langote

Commits

  1. Stamp 19beta1.

  2. Revert "Don't lock partitions pruned by initial pruning"

  3. Ensure first ModifyTable rel initialized if all are pruned

  4. Fix bug in cbc127917 to handle nested Append correctly

  5. Remove unstable test suite added by 525392d57

  6. Don't lock partitions pruned by initial pruning

  7. Fix an oversight in cbc127917 to handle MERGE correctly

  8. Track unpruned relids to avoid processing pruned relations

  9. Perform runtime initial pruning outside ExecInitNode()

  10. Move PartitionPruneInfo out of plan nodes into PlannedStmt

  11. Fix setrefs.c's failure to do expression processing on prune steps.

  12. Remove obsolete executor cleanup code

  13. Revert "Move PartitionPruneInfo out of plan nodes into PlannedStmt"

  14. Move PartitioPruneInfo out of plan nodes into PlannedStmt

  15. Refactor and cleanup runtime partition prune code a little

  16. Remove some unnecessary fields from Plan trees.

  17. Remove more redundant relation locking during executor startup.

  18. Shut down Gather's children before shutting down Gather itself.