ffi: fix broken char handling in undimensioned arrays.

The undimensioned (array <type>) and (zarray <type>) types are
not doing UTF-8 conversion when <type> is char or zchar,
or doing what they are supposed to with the FFI character
types, which is inconsistent from their dimensioned
counterparts.

* ffi.c (ffi_varray_dynsize): if the element type is marked
for character conversion, then do the size calculation for
char and zchar by measuring the UTF-8 coded size.
(ffi_varray_alloc): Call ffi_varray_dynsize to get the size,
to benefit from the char handling. Thus when FFI allocates
buffers for a variable length array, it will allocate correct
size required for the UTF-8 encoded string.
(ffi_varray_put, ffi_varray_in): Here we must call
ffi_varray_dynsize and divide by the element type to get the
proper numer of elements. Then we must check for character
conversion and handle the cases.
(ffi_varray_null_term_in): Check for character conversion
cases and route those through ffi_varray_in, which handles
null-terminated strings.

* tests/017/ffi-misc.tl: New file.

* tests/017/ffi-misc.expected: New file.

2 files changed, 11 insertions, 0 deletions

diff --git a/tests/017/ffi-misc.expected b/tests/017/ffi-misc.expected
new file mode 100644
index 00000000..e69de29b
--- /dev/null
+++ b/tests/017/ffi-misc.expected
diff --git a/tests/017/ffi-misc.tl b/tests/017/ffi-misc.tl
new file mode 100644
index 00000000..1578cd2c
--- /dev/null
+++ b/tests/017/ffi-misc.tl
@@ -0,0 +1,11 @@
+(load "../common")
+
+(defvarl ar (ffi (array char)))
+
+(defvarl zar (ffi (zarray char)))
+
+(test (ffi-put "\x1234@@@" ar) #b'e188b4404040')
+
+(test (ffi-put "\x1234@@@" zar) #b'e188b440404000')
+
+(test (ffi-get (ffi-put "\x1234@@@" zar) zar) "\x1234@@@")