Re: pg_trgm comparison bug on cross-architecture replication due to different char implementation

Joe Conway <mail@joeconway.com>

From: Joe Conway <mail@joeconway.com>
To: Peter Eisentraut <peter@eisentraut.org>, Tom Lane <tgl@sss.pgh.pa.us>
Cc: Alexander Korotkov <aekorotkov@gmail.com>, "Guo, Adam" <adamguo@amazon.com>, "pgsql-hackers@lists.postgresql.org" <pgsql-hackers@lists.postgresql.org>, Nathan Bossart <nathandbossart@gmail.com>, Masahiko Sawada <sawada.mshk@gmail.com>, Jim Mlodgenski <jimmy76@gmail.com>
Date: 2024-05-03T22:36:05Z
Lists: pgsql-hackers
On 5/3/24 11:44, Peter Eisentraut wrote:
> On 03.05.24 16:13, Tom Lane wrote:
>> Peter Eisentraut <peter@eisentraut.org> writes:
>>> On 30.04.24 19:29, Tom Lane wrote:
>>>> Also, the bigger picture here is the seeming assumption that "if
>>>> we change pg_trgm then it will be safe to replicate from x86 to
>>>> arm".  I don't believe that that's a good idea and I'm unwilling
>>>> to promise that it will work, regardless of what we do about
>>>> char signedness.  That being the case, I don't want to invest a
>>>> lot of effort in the signedness issue.  Option (1) is clearly
>>>> a small change with little if any risk of future breakage.
>>
>>> But note that option 1 would prevent some replication that is currently
>>> working.
>>
>> The point of this thread though is that it's working only for small
>> values of "work".  People are rightfully unhappy if it seems to work
>> and then later they get bitten by compatibility problems.
>>
>> Treating char signedness as a machine property in pg_control would
>> signal that we don't intend to make it work, and would ensure that
>> even the most minimal testing would find out that it doesn't work.
>>
>> If we do not do that, it seems to me we have to buy into making
>> it work.  That would mean dealing with the consequences of an
>> incompatible change in pg_trgm indexes, and then going through
>> the same dance again the next time(s) similar problems are found.
> 
> Yes, that is understood.  But anecdotally, replicating between x86-64 arm64 is 
> occasionally used for upgrades or migrations.  In practice, this appears to have 
> mostly worked.  If we now discover that it won't work with certain index 
> extension modules, it's usable for most users. Even if we say, you have to 
> reindex everything afterwards, it's probably still useful for these scenarios.

+1

I have heard similar anecdotes, and the reported experience goes even further -- 
many such upgrade/migration uses, with exceedingly rare reported failures.

-- 
Joe Conway
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com




Commits

  1. pg_upgrade: Check for the expected error message in TAP tests.

  2. Fix a typo in 005_char_signedness.pl test.

  3. Add test 005_char_signedness.pl to meson.build.

  4. Fix an issue with index scan using pg_trgm due to char signedness on different architectures.

  5. pg_upgrade: Add --set-char-signedness to set the default char signedness of new cluster.

  6. pg_upgrade: Preserve default char signedness value from old cluster.

  7. pg_resetwal: Add --char-signedness option to change the default char signedness.

  8. Add default_char_signedness field to ControlFileData.

  9. Remove unneeded nbtree array preprocessing assert.