Re: Ryu floating point output patch
Andrew Gierth <andrew@tao11.riddles.org.uk>
From: Andrew Gierth <andrew@tao11.riddles.org.uk>
To: Andres Freund <andres@anarazel.de>
Cc: Tom Lane <tgl@sss.pgh.pa.us>,
Thomas Munro <thomas.munro@enterprisedb.com>,
Pg Hackers <pgsql-hackers@postgresql.org>
Date: 2018-12-14T20:37:46Z
Lists: pgsql-hackers
>>>>> "Andres" == Andres Freund <andres@anarazel.de> writes:
>> Is this a path we really want to go down? I'm not convinced the
>> cost/benefit ratio is attractive.
Andres> float->text conversion is one of the major bottlenecks when
Andres> backing up postgres, it's definitely a pain-point in practice.
Also an issue with queries. I got into this whole area after diagnosing
a problem for a user on IRC who was seeing a 4x slowdown for PG as
compared to MSSQL, from 30 seconds to 120 seconds. We determined that
the _entire_ difference was accounted for by float conversion.
Now that was an unusual case, downloading a large dataset of floats at
once into a statistics program, but it's very easy to show
proportionally large overheads from float output:
(using this table:
create table flttst4 as
select i a, i b, i c, i d, i::float8 e, random() f
from generate_series(1::bigint, 10000000::bigint) i;
)
postgres=# copy flttst4(d) to '/dev/null'; -- bigint column
COPY 10000000
Time: 2166.001 ms (00:02.166)
postgres=# copy flttst4(e) to '/dev/null'; -- float8 col, integer values
COPY 10000000
Time: 4233.005 ms (00:04.233)
postgres=# copy flttst4(f) to '/dev/null'; -- float8 col, random()
COPY 10000000
Time: 7261.421 ms (00:07.261)
-- vs. timings with Ryu:
postgres=# copy flttst4(e) to '/dev/null'; -- float8 col, integer values
COPY 10000000
Time: 2391.725 ms (00:02.392)
postgres=# copy flttst4(f) to '/dev/null'; -- float8 col, random()
COPY 10000000
Time: 2245.699 ms (00:02.246)
--
Andrew (irc:RhodiumToad)
Commits
-
Provide an extra-float-digits setting for pg_dump / pg_dumpall
- af25bc03e17e 12.0 landed
-
Change floating-point output format for improved performance.
- 02ddd499322a 12.0 landed
-
Use strtof() and not strtod() for float4 input.
- f397e08599a3 12.0 landed