Thread

1. How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?

   Shaozhong SHI <shishaozhong@gmail.com> — 2022-04-10T13:50:32Z

   There is a plpgsql script that have a loop to carry out the same recursive
   queries.
   
   The estimation of iteration is in the order of tens of thousands.
   
   What is the best way of making using parallel query strategy.
   
   Any examples?
   
   Regards,
   
   David
   

2. Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?

   Steve Midgley <science@misuse.org> — 2022-04-10T16:03:04Z

   On Sun, Apr 10, 2022, 6:50 AM Shaozhong SHI <shishaozhong@gmail.com> wrote:
   
   > There is a plpgsql script that have a loop to carry out the same recursive
   > queries.
   >
   > The estimation of iteration is in the order of tens of thousands.
   >
   > What is the best way of making using parallel query strategy.
   >
   > Any examples?
   >
   
   Can you provide some sample ddl, data and sql to illustrate your question?
   
   You mention in one place "iteration" and "parallel" but in another you
   mention recursion.. Recursion (by definition?) involves pushing state data
   forward into each successive iteration.. Therefore you can't run recursion
   in parallel.
   
   If you can provide examples maybe we can see which if parallelism is a
   possibility..
   
   Steve
   

3. Re: How best to do parallel query given tens of thousands of iteration of a loop of recursive queries?

   David G. Johnston <david.g.johnston@gmail.com> — 2022-04-10T16:13:14Z

   On Sun, Apr 10, 2022 at 6:50 AM Shaozhong SHI <shishaozhong@gmail.com>
   wrote:
   
   > There is a plpgsql script that have a loop to carry out the same recursive
   > queries.
   >
   > The estimation of iteration is in the order of tens of thousands.
   >
   > What is the best way of making using parallel query strategy.
   >
   
   An example would be helpful.
   
   However, as a general guideline, since parallelism is done at the per-row
   scope, removing looping logic from the script and turning the main script
   logic into one or more functions that operate on a single row, while
   obeying the rules such functions need to abide by in order to be marked
   parallel safe, will open up the possibility for the server to process
   different rows using different workers and then appending their results
   together for the next node to consume.
   
   David J.