General purpose hashing func in pgbench
Ildar Musin <i.musin@postgrespro.ru>
From: Ildar Musin <i.musin@postgrespro.ru>
To: pgsql-hackers <pgsql-hackers@postgresql.org>
Cc: Fabien COELHO <coelho@cri.ensmp.fr>
Date: 2017-12-20T05:58:18Z
Lists: pgsql-hackers
Attachments
- zipfian.png (image/png)
- hashed_zipfian.png (image/png)
- pgbench_hash.patch (text/plain) patch
Hi hackers, Following up the recent discussion on zipfian distribution I was trying to reproduce some YCSB-like workloads. As this paper [1] describes, YCSB uses zipfian distribution to generate keys in order simulate intensive load on small number of records as it happens in real world applications (e.g. blogs). One problem is that most popular records keys are clustered together. To scatter them across the keyspace authors use hashing, the FNV-1a hash function in particular [2]. I've read Fabien Coelho's thread on additional operators and functions. Generally it could be possible to implement some basic hashing algorithms right in a pgbench script using just bitwise and arithmetic operators. But should we probably provide users with some general purpose hash function? The attached patch introduces hash() function which implements FNV-1a as an example of such hashing algorithm. There are also couple of images in the attachement that I have got from visualizing original zipfian distribution and the hashed one. Usage example: In psql: create table abc as select generate_series(0, 999) as a, 0 as b; pgbench script: \set rnd random_zipfian(0, 1000000, 0.99) \set key abs(hash(:rnd)) % 1000 begin; update abc set b = b + 1 where a = :key; end; Any thoughts or suggestions? [1] http://www.brianfrankcooper.net/home/publications/ycsb.pdf [2] https://en.wikipedia.org/wiki/Fowler–Noll–Vo_hash_function Thanks! -- Ildar Musin Postgres Professional: http://www.postgrespro.com Russian Postgres Company
Commits
-
Add general purpose hasing functions to pgbench.
- e51a04840a1c 11.0 landed