bugfix: fixnum crackdown.

The purpose of this commit is to address certain situations in
which code is wrongly relying on a cnum value being in the
fixnum range (NUM_MIN to NUM_MAX), so that num_fast can safely
be used on it.

One wrong pattern is that c_num is applied to some Lisp value,
and that value (or one derived from it arithmetically) is then
passed to num_fast. The problem is that c_num succeeds on
integers outside of the fixnum range.  Some bignum values
convert to a cnum successfully. Thus either num has to be used
instead of num_fast, or else the original c_num attempt must
be replaced with something that will fail if the original
value isn't a fixnum. (In the latter case, any arithmetic on
the fixnum cannot produce value outside of that range).

* buf.c (buf_put_bytes): The size argument here is not
guaranteed to be in fixnum range: use num.

* combi.c (perm_init_common): Throw if the sequence length
isn't a fixnum. Thus the num_fast in perm_while_fun is
correct, since the ci value is bounded by k, which is bounded
by n.

* hash.c (hash_grow): Remove dubious assertion which aborts
the run-time if the hash table doubling overflows.  Simply
don't allow the modulus to grow beyond NUM_MAX.  If doubling
it makes it larger than NUM_MAX, then just don't grow the
table. We need the modulus to be in fixnum range, so that
uses of num_fast on the modulus value elsewhere are correct.
(group_by, group_reduce): Use c_fixnum rather than c_num to
extract a value that is later assumed to be a fixnum.

* lib.c (c_fixnum): New function.
(nreverse, reverse, remove_if, less, window_map_list,
sort_vec, unique): Use c_fixnum rather than c_num to extract a
value that is later assumed to be a fixnum.
(string_extend): Use c_fixnum rather than c_num to extract a
value that is later assumed to be a fixnum. Cap the string
allocation size to fixnum range rather than INT_PTR_MAX.
(cmp_str): The wcscmp function could return values outside
of the fixnum range, so we must use num, not num_fast.

* lib.h (c_fixnum): Declared.

1 files changed, 1 insertions, 1 deletions

diff --git a/buf.c b/buf.c
index b22ff2b2..6a5ed15b 100644
--- a/buf.c
+++ b/buf.c
@@ -228,7 +228,7 @@ static void buf_put_bytes(val buf, val pos, mem_t *ptr, cnum size, val self)
 {
   struct buf *b = buf_handle(buf, self);
   cnum p = buf_check_index(pos, self);
-  val req_len = plus(pos, num_fast(size));
+  val req_len = plus(pos, num(size));
   if (gt(req_len, b->len))
     buf_do_set_len(buf, b, req_len, nil, self);
   memcpy(b->data + p, ptr, size);