Thread

1. modulo operator bug?

   Kemal Bayram <kemal@corenetworks.co.uk> — 2001-03-13T22:29:10Z

   from psql:
   
   kemal=# select 0.99 % 0.50;
    ?column? 
   ----------
        0.04
   (1 row)
   
   Am I missing something fundemental here or is this indeed a bug?
   
   
   

2. Re: modulo operator bug?

   Tom Lane <tgl@sss.pgh.pa.us> — 2001-03-14T15:41:24Z

   Kemal Bayram <kemal@corenetworks.co.uk> writes:
   > from psql:
   > kemal=# select 0.99 % 0.50;
   >  ?column? 
   > ----------
   >      0.04
   > (1 row)
   
   > Am I missing something fundemental here or is this indeed a bug?
   
   Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
   I think.
   
   			regards, tom lane
   
   

3. Re: modulo operator bug?

   Tom Lane <tgl@sss.pgh.pa.us> — 2001-03-14T16:56:44Z

   > Kemal Bayram <kemal@corenetworks.co.uk> writes:
   >> from psql:
   >> kemal=# select 0.99 % 0.50;
   >> ?column? 
   >> ----------
   >> 0.04
   >> (1 row)
   
   >> Am I missing something fundemental here or is this indeed a bug?
   
   > Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
   > I think.
   
   OK, fixed for 7.1.  If you need a fix in 7.0.*, the attached patch should
   apply to 7.0.*, although the line numbers will be different.
   
   			regards, tom lane
   
   
   *** src/backend/utils/adt/numeric.c.orig	Wed Dec  6 21:47:35 2000
   --- src/backend/utils/adt/numeric.c	Wed Mar 14 11:50:37 2001
   ***************
   *** 3355,3370 ****
     	init_var(&tmp);
     
     	/* ----------
   ! 	 * We do it by fiddling around with global_rscale and truncating
   ! 	 * the result of the division.
     	 * ----------
     	 */
     	save_global_rscale = global_rscale;
     	global_rscale = var2->rscale + 2;
     
     	div_var(var1, var2, &tmp);
     	tmp.rscale = var2->rscale;
   - 	tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));
     
     	global_rscale = var2->rscale;
     	mul_var(var2, &tmp, &tmp);
   --- 3355,3373 ----
     	init_var(&tmp);
     
     	/* ----------
   ! 	 * We do this using the equation
   ! 	 *		mod(x,y) = x - trunc(x/y)*y
   ! 	 * We fiddle a bit with global_rscale to control result precision.
     	 * ----------
     	 */
     	save_global_rscale = global_rscale;
     	global_rscale = var2->rscale + 2;
     
     	div_var(var1, var2, &tmp);
   + 
   + 	/* do trunc() by forgetting digits to the right of the decimal point */
   + 	tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
     	tmp.rscale = var2->rscale;
     
     	global_rscale = var2->rscale;
     	mul_var(var2, &tmp, &tmp);