Re: pgbench - implement strict TPC-B benchmark

On Thu, Aug 1, 2019 at 2:53 AM Fabien COELHO <coelho@cri.ensmp.fr> wrote:
> All in all, pgbench overheads are small compared to postgres processing
> times and representative of a reasonably optimized client application.

It's pretty easy to devise tests where pgbench is client-limited --
just try running it with threads = clients/4, sometimes even
clients/2. So I don't buy the idea that this is true in general.

> To try to salvage my illustration idea: I could change the name to "demo",
> i.e. quite far from "TPC-B", do some extensions to make it differ, eg use
> a non-uniform random generator, and then explicitly say that it is a
> vaguely inspired by "TPC-B" and intended as a demo script susceptible to
> be updated to illustrate new features (eg if using a non-uniform generator
> I'd really like to add a permutation layer if available some day).
>
> This way, the "demo" real intention would be very clear.

I do not like this idea at all; "demo" is about as generic a name as
imaginable.  But I have another idea: how about including this script
in the documentation with some explanatory text that describes (a) the
ways in which it is more faithful to TPC-B than what the normal
pgbench thing and (b) the problems that it doesn't solve, as
enumerated by Fabien upthread:

"table creation and data types, performance data collection, database
configuration, pricing of hardware used in the tests, post-benchmark run
checks, auditing constraints, whatever…"

Perhaps that idea still won't attract any votes, but I throw it out
there for consideration.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company