Fix some issues found by ubsan.

* lib.h (num_fast): Under HAVE_UBSAN, avoid left shifting a
negative value, which is diagnosed by ubsan (by default), but
is otherwise a documented GCC extension that behaves sanely in
the manner one would expect on two's complement machines.

* mpi/mpi.c (s_mp_mul_2d): Fix instance of undefined shift
equal to the width of the type. This occurs in the case when
d == 0.  In that case, we try to calculate a 32 or 64 bit full
mask by shifting 1 to the left that many times and then
subtracting 1. However, the entire code for the fractional
shift is not necessary in that case and may be skipped. I'm
also removing the unnecesary s_mp_clamp call. If the number is
clamped on entry into the function, it stays clamped.

1 files changed, 4 insertions, 0 deletions

diff --git a/lib.h b/lib.h
index 5d095108..d2d7a486 100644
--- a/lib.h
+++ b/lib.h
@@ -491,7 +491,11 @@ INLINE wchar_t *litptr(val obj)
 
 INLINE val num_fast(cnum n)
 {
+#if HAVE_UBSAN
+  return coerce(val, (n * (1 << TAG_SHIFT)) | TAG_NUM);
+#else
   return coerce(val, (n << TAG_SHIFT) | TAG_NUM);
+#endif
 }
 
 INLINE mp_int *mp(val bign)