Re: Reference to - BUG #18349: ERROR: invalid DSA memory alloc request size 1811939328, CONTEXT: parallel worker
Craig Milhiser <craig@milhiser.com>
From: Craig Milhiser <craig@milhiser.com>
To: Thomas Munro <thomas.munro@gmail.com>
Cc: Tom Lane <tgl@sss.pgh.pa.us>, pgsql-bugs@lists.postgresql.org
Date: 2024-09-24T01:43:50Z
Lists: pgsql-bugs
On Sun, Sep 22, 2024 at 10:23 PM Thomas Munro <thomas.munro@gmail.com> wrote: > On Mon, Sep 23, 2024 at 1:46 PM Thomas Munro <thomas.munro@gmail.com> > wrote: > > 432 bytes > > Oh, as Tomas pointed out in the referenced thread, Thanks for working on it and the detailed explanation. I tested set max_parallel_workers_per_gather = 0 from the original thread and it was working. We are putting that into the application, for our largest customers. Set to 0 before the query then back to 2 after. Your explanation also shows why rewriting of the query works. I reduced the number of rows being processed much earlier in the query. The query was written with 1 set of many joins which worked on millions of rows then reduced to a handful. I broke this into a materialized CTE that forced Postgres to reduce the rows early then do the joins. Rewriting the query is better regardless of this issue. I am working on getting a stock Postgres in our production protected enclave with our production database. Probably a full day of work that I need to splice in. We have a similar mechanism in our development environment. Once working I can help test and debug any changes. I can also work on a reproducible example. > >
Commits
-
Fix extreme skew detection in Parallel Hash Join.
- 98c7c7152d2d 18.0 landed
- 4ac5d33a8b08 17.1 landed
- 53edc948580f 16.5 landed
- 1831545ca186 15.9 landed
- 20d94899414a 14.14 landed
- 45329466fdc9 13.17 landed
- 4fa80a6d7d4b 12.21 landed