Re: BUG #16583: merge join on tables with different DB collation behind postgres_fdw fails
Tom Lane <tgl@sss.pgh.pa.us>
From: Tom Lane <tgl@sss.pgh.pa.us>
To: pgsql-bugs@lists.postgresql.org
Cc: jurafejfar@gmail.com, Peter Eisentraut <peter.eisentraut@2ndquadrant.com>
Date: 2020-08-17T22:37:47Z
Lists: pgsql-bugs, pgsql-hackers
Attachments
- dont-import-collations-as-default-1.patch (text/x-diff) patch
I wrote: > So I think what is happening here is that postgres_fdw's version of > IMPORT FOREIGN SCHEMA translates "COLLATE default" on the remote > server to "COLLATE default" on the local one, which of course is > a big fail if the defaults don't match. That allows the local > planner to believe that remote ORDER BYs on the two foreign tables > will give compatible results, causing the merge join to not work > very well at all. > We probably need to figure out some way of substituting the remote > database's actual lc_collate setting when we see "COLLATE default". Here's a draft patch for that part. There's a few things to quibble about: * It tests for "COLLATE default" by checking whether pg_collation.oid is DEFAULT_COLLATION_OID, thus assuming that that OID will never change. I think this is safer than checking the collation name, but maybe somebody else would have a different opinion? Another idea is to check whether collprovider is 'd', but that only works with v10 and up. * It might not be able to find a remote collation matching the database's datcollate/datctype. As coded, we'll end up creating the local column with "COLLATE default", putting us back in the same hurt we're in now. I think this is okay given the other planned change to interpret "COLLATE default" as "we don't know what collation this is". In any case it's hard to see what else we could do, other than fail entirely. * Alternatively, it might find more than one such remote collation; indeed that's the norm, eg we'd typically find both "en_US" and "en_US.utf8", or the like. I made it choose the shortest collation name in such cases, but maybe there is a case for the longest? I don't much want it to pick "ucs_basic" over "C", though. * The whole thing is certain to fall over whenever we find a way to allow ICU collations as database defaults. While we can presumably fix the query when we make that change, existing postgres_fdw releases would not work against a newer server. Probably there's little to be done about this, either. * As shown by the expected-output changes, there are some test cases that expose that we're not picking the default collation anymore. That creates a testing problem: this can't be committed as-is because it'll fail with any other locale environment than what the expected file was made with. We could lobotomize the test cases to not print the column collation, but then we're not really verifying that this code does what it's supposed to. Not sure what the best compromise is. Comments? regards, tom lane
Commits
-
Doc: extend warnings about collation-mismatch hazards in postgres_fdw.
- 2aa255588756 9.6.24 landed
- 1d7e8e772453 10.19 landed
- b274c982db67 11.14 landed
- 97c5651cea0f 12.9 landed
- 7b0be9fb2ddd 15.0 landed
- 739b872f6c98 13.5 landed
- 02c4e3533926 14.0 landed
-
Add support for asynchronous execution.
- 27e1f14563cf 14.0 cited
-
Suppress unnecessary RelabelType nodes in yet more cases.
- 814a57065ec9 12.5 landed
- 69ffc2f83899 13.0 landed
- 207293240780 14.0 landed