Re: Exponentiation confusion

Tom Lane <tgl@sss.pgh.pa.us>

From: Tom Lane <tgl@sss.pgh.pa.us>
To: Dean Rasheed <dean.a.rasheed@gmail.com>
Cc: Erik Wienhold <ewie@ewie.name>, Adrian Klaver <adrian.klaver@aklaver.com>, pgsql-general <pgsql-general@postgresql.org>
Date: 2022-10-13T19:12:31Z
Lists: pgsql-hackers, pgsql-general
Dean Rasheed <dean.a.rasheed@gmail.com> writes:
> The most obvious thing to do is to try to make power_var_int() choose
> the same result rscale as power_var() so that the results are
> consistent regardless of whether the exponent is an integer.

Yeah, I think we should try to end up with that.

> It's worth noting, however, that that will cause in a *reduction* in
> the output rscale rather than an increase in some cases, since the
> power_var_int() code path currently always chooses an rscale of at
> least 16, whereas the other code path in power_var() uses the rscales
> of the 2 inputs, and produces a minimum of 16 significant digits,
> rather than 16 digits after the decimal point.

Right.  I think this is not bad though.  In a lot of cases (such
as the example here) the current behavior is just plastering on
useless zeroes.

			regards, tom lane



Commits

  1. Improve the accuracy of numeric power() for integer exponents.