Re: Hash Functions

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Robert Haas <robertmhaas@gmail.com>
Cc: Andres Freund <andres@anarazel.de>, "pgsql-hackers@postgresql.org" <pgsql-hackers@postgresql.org>, Jeff Davis <pgsql@j-davis.com>, Yugo Nagata <nagata@sraoss.co.jp>, amul sul <sulamul@gmail.com>
Date: 2017-05-12T17:34:52Z
Lists: pgsql-hackers
Robert Haas <robertmhaas@gmail.com> writes:
> On Fri, May 12, 2017 at 1:12 PM, Andres Freund <andres@anarazel.de> wrote:
>> Given that a lot of data types have a architecture dependent representation, it seems somewhat unrealistic and expensive to have a hard rule to keep them architecture agnostic.   And if that's not guaranteed, then I'm doubtful it makes sense as a soft rule either.

> That's a good point, but the flip side is that, if we don't have such
> a rule, a pg_dump of a hash-partitioned table on one architecture
> might fail to restore on another architecture.  Today, I believe that,
> while the actual database cluster is architecture-dependent, a pg_dump
> is architecture-independent.  Is it OK to lose that property?

I'd vote that it's not, which means that this whole approach to hash
partitioning is unworkable.  I agree with Andres that demanding hash
functions produce architecture-independent values will not fly.

Maintaining such a property for float8 (and the types that depend on it)
might be possible if you believe that nobody ever uses anything but IEEE
floats, but we've never allowed that as a hard assumption before.

Even architecture dependence isn't the whole scope of the problem.
Consider for example dumping a LATIN1-encoded database and trying
to reload it into a UTF8-encoded database.  People will certainly
expect that to be possible, and do you want to guarantee that the
hash of a text value is encoding-independent?

			regards, tom lane


Commits

  1. Introduce 64-bit hash functions with a 64-bit seed.

  2. Create a "sort support" interface API for faster sorting.

  3. Add seven kanji characters defined in the Windows 950 codepage to our