Re: Append with naive multiplexing of FDWs

Hello Kyotaro,



>"Parallel scan" at the moment means multiple workers fetch unique 

>blocks from *one* table in an arbitrated manner. In this sense 

>"parallel FDW scan" means multiple local workers fetch unique bundles 

>of tuples from *one* foreign table, which means it is running on a 

>single session.  That doesn't offer an advantage. 



It maybe not "parallel FDW scan", it can be "parallel shards scan"

the local workers will pick every foreign partition to scan. I have ever 

draw a picture about that you can see it in the link below.

https://www.highgo.ca/2019/08/22/parallel-foreign-scan-of-postgresql/

I think the "parallel shards scan" make sence in this way.



>If parallel query processing worked in worker-per-table mode, 

>especially on partitioned tables, maybe the current FDW would work 

>without much of modification. But I believe asynchronous append on 

>foreign tables on a single process is far resource-effective and 

>moderately faster than parallel append. 



As the test result, current patch can not gain more performance when 

it returns a huge number of tuples. By "parallel shards scan" method,

it can work well, because the 'parallel' can take full use of CPUs while 

'asynchronous' can't. 






Highgo Software (Canada/China/Pakistan) 

URL : http://www.highgo.ca/ 

EMAIL: mailto:movead(dot)li(at)highgo(dot)ca