diff mbox

fpu: Correct edgecase in float64_muladd

Message ID 1365781072-24979-1-git-send-email-peter.maydell@linaro.org
State Accepted
Commit e3d142d073d02f0a3a4aad79eb838c15b6f99c01
Headers show

Commit Message

Peter Maydell April 12, 2013, 3:37 p.m. UTC
In handling float64_muladd, if we end up doing a subtraction of the
product and c, and the 128 bit result of this subtraction happens to
have its most significant bit in bit 63, we weren't handling this
correctly when attempting to normalize to put the most significant
bit into bit 126.  We would end up doing a right shift by a negative
number (undefined behaviour in C) so at best we would return an
incorrect result to the guest.  MSB in bit 63 has to be handled as a
special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
in 127 is not possible.)

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
---
Specific test vector which triggers this:
 a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000

Also tested with my usual set of random test vectors.

 fpu/softfloat.c |   12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

Comments

Aurelien Jarno April 13, 2013, 1:57 p.m. UTC | #1
On Fri, Apr 12, 2013 at 04:37:52PM +0100, Peter Maydell wrote:
> In handling float64_muladd, if we end up doing a subtraction of the
> product and c, and the 128 bit result of this subtraction happens to
> have its most significant bit in bit 63, we weren't handling this
> correctly when attempting to normalize to put the most significant
> bit into bit 126.  We would end up doing a right shift by a negative
> number (undefined behaviour in C) so at best we would return an
> incorrect result to the guest.  MSB in bit 63 has to be handled as a
> special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
> in 127 is not possible.)
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Specific test vector which triggers this:
>  a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000
> 
> Also tested with my usual set of random test vectors.
> 
>  fpu/softfloat.c |   12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/fpu/softfloat.c b/fpu/softfloat.c
> index 83ccc4b..7ba51b6 100644
> --- a/fpu/softfloat.c
> +++ b/fpu/softfloat.c
> @@ -3898,9 +3898,15 @@ float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
>              }
>              zExp -= shiftcount;
>          } else {
> -            shiftcount = countLeadingZeros64(zSig1) - 1;
> -            zSig0 = zSig1 << shiftcount;
> -            zExp -= (shiftcount + 64);
> +            shiftcount = countLeadingZeros64(zSig1);
> +            if (shiftcount == 0) {
> +                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
> +                zExp -= 63;
> +            } else {
> +                shiftcount--;
> +                zSig0 = zSig1 << shiftcount;
> +                zExp -= (shiftcount + 64);
> +            }
>          }
>          return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
>      }
> 
Reviewed-by: Aurelien Jarno <aurelien@aurel32.net>
Aurelien Jarno April 15, 2013, 2:25 p.m. UTC | #2
On Fri, Apr 12, 2013 at 04:37:52PM +0100, Peter Maydell wrote:
> In handling float64_muladd, if we end up doing a subtraction of the
> product and c, and the 128 bit result of this subtraction happens to
> have its most significant bit in bit 63, we weren't handling this
> correctly when attempting to normalize to put the most significant
> bit into bit 126.  We would end up doing a right shift by a negative
> number (undefined behaviour in C) so at best we would return an
> incorrect result to the guest.  MSB in bit 63 has to be handled as a
> special case separately from MSB in 0..62 and MSB in 63..126.  (MSB
> in 127 is not possible.)
> 
> Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
> ---
> Specific test vector which triggers this:
>  a = 3fffffffffe00000 b = 3fffffffffe00000 c = c00fffffffc00000
> 
> Also tested with my usual set of random test vectors.
> 
>  fpu/softfloat.c |   12 +++++++++---
>  1 file changed, 9 insertions(+), 3 deletions(-)
> 
> diff --git a/fpu/softfloat.c b/fpu/softfloat.c
> index 83ccc4b..7ba51b6 100644
> --- a/fpu/softfloat.c
> +++ b/fpu/softfloat.c
> @@ -3898,9 +3898,15 @@ float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
>              }
>              zExp -= shiftcount;
>          } else {
> -            shiftcount = countLeadingZeros64(zSig1) - 1;
> -            zSig0 = zSig1 << shiftcount;
> -            zExp -= (shiftcount + 64);
> +            shiftcount = countLeadingZeros64(zSig1);
> +            if (shiftcount == 0) {
> +                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
> +                zExp -= 63;
> +            } else {
> +                shiftcount--;
> +                zSig0 = zSig1 << shiftcount;
> +                zExp -= (shiftcount + 64);
> +            }
>          }
>          return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
>      }

Thanks, applied.
diff mbox

Patch

diff --git a/fpu/softfloat.c b/fpu/softfloat.c
index 83ccc4b..7ba51b6 100644
--- a/fpu/softfloat.c
+++ b/fpu/softfloat.c
@@ -3898,9 +3898,15 @@  float64 float64_muladd(float64 a, float64 b, float64 c, int flags STATUS_PARAM)
             }
             zExp -= shiftcount;
         } else {
-            shiftcount = countLeadingZeros64(zSig1) - 1;
-            zSig0 = zSig1 << shiftcount;
-            zExp -= (shiftcount + 64);
+            shiftcount = countLeadingZeros64(zSig1);
+            if (shiftcount == 0) {
+                zSig0 = (zSig1 >> 1) | (zSig1 & 1);
+                zExp -= 63;
+            } else {
+                shiftcount--;
+                zSig0 = zSig1 << shiftcount;
+                zExp -= (shiftcount + 64);
+            }
         }
         return roundAndPackFloat64(zSign, zExp, zSig0 STATUS_VAR);
     }