TXR 275 2022-05-10 Features - Architecture support: RISC-V and Loongarch (64 bit). - Hashing: new group-map function: group-reduce with built-in map pass. - Lib: new isecp function: test whether two sequences intersect without calculating intersectin. - FFI: - intmax-t and uintmax-t types. - new str-s, bstr-s and wstr-s types for receiving foreign string without freeing its memory. - after a FFI call, the arguments are processed for reverse data flows and memory clean-up in reverse order. - with above two features, strtol can be wrapped in FFI, including the error-reporting char **end pointer. - Loading: new *load-search-dirs* variable. - default search directory list includes sysrooted lib dir Bugs - configure: minor escaping corner cases in in production of ./reconfigure script. - compiler: package-related bug in file-compilation, reported by Paul A. Patience. - load: regression: do not try adding suffixes to a path which exists; try the given path before anything else - Reported by Paul A. Patience. - sh, run, open-command and open-process now flush *stdout* in situations when it makes sense, standard output is ordered between the subprocesses and the parent. - listener: Ctrl-Z issue when txr is one of multiple processes in a job control process group. TXR 274 2022-02-24 Features - Configure/Build: - 'make clean-c" now cleans the C object files without removing .tlo files. - complementary to "make clean-tlo". - experimental, not tested support for configuring 64 bit time_t on 32 bit Glibc. - CPPFLAGS (C preprocessor flags) variable noticed and used now. - TXR now supports building with -fsanitize=undefined option. - you must specify it yourself via platform-flags, etc. - configure detects it and puts #define HAVE_UBSAN 1 into config.h - FFI: - new feature: enumed bitfield type combination now works. - Doc: - numerous documentation fixes. - Lib: - cptr-int: allow full unsigned range, so pointers can be specified as unsigned integers, or using negative signed values also. - New copy-cptr function; copy copies cptr objects. - New nandf and norf functions. - New function random-sample for one-pass reservoir sampling of a sequence. - load: supports catenated .tlo files now - cat-files: new function for catenating files, like POSIX cat. - find-max uses generic iteration. - new find-max-key function. - new partition-if function. - new list-builder method oust - also local function in build macro - Getopts: - various improvements. - opt-help function/method split up into several. - Macros: - New etypecase macro. - New nand and nor macros and functions. - opip now allows embedded (ap ...) and so on. - Compiler: - new optimizations. - TXR Pattern Language: - new function match-fboundp for testing whether a symbol has a binding as a pattern function. - Expander: - new @,expr hack: quasiquote generates (sys:var ...) or (sys:expr ...) based on type of substituted value. - macro-time is no longer a special operator, but a macro. - Listener: - Hack: Ctrl-V Ctrl-J now inserts CR (i.e. new line in multiline mode) rather than a LF. - Good for people used to inserting line breaks in GNU Readline. - Improvement in method completion. - getopts: - Numerous improvements, mainly in area of help generation. - Autoloading: - More nuanced implementation with multiple symbol namespaces, reduces spurious loading of modules not actually used. Bugs - TXR Pattern Language: - bug fixed in @(freeform) - involves bugfix in lazy-str-get-trailing-list function. - filtering now throws when there is an invalid filter, due to a fix in the filter-string-tree function. - Parser: - bug: carriage returns in JSON not tolerated. - Configure/Build - fixed broken file offset bits detection, resulting in no large file support on 32 bit Glibc platforms (regression since 244). - fixed broken syntax in unwind.h causing build to break if CONFIG_DEBUG_SUPPORT disabled. - Macros: - sum-each, mul-each: handle no vars case. - typecase: return nil from formless clauses. - fix broken :key parameters. - Lib: - carray: allow t and floating 0 in sub and replace. - carray-replace: two overrun bugs. - separate: wrong return value when seq is nil. - time structure: added missing wday and yday slots. - Listener: - bug handling comments in plain mode. - issue handling Ctrl-C in plain mode. - Structural Pattern Matching: - quasiliteral match wrongly allowing loose prefix matching. - `@{nil #/regex/}` wrongly throwing exception. - Command Line: - -Dvar now binds var to empty string rather than t. - this t was some thing inadvertently introduced in 2014. - -Dvar=foo=bar (value containing equal sign) works. - -Dx,y,z now diagnosed. - Vim Syntax Files: - improvement in handling multi-line string literals. - Search trees: - fixed two array underruns found by ubsan. - both situations work reliably by fluke in unfixed code due the memory cell below the array reliably being zero bits. - PRNG: - undefined behavior (32 bit shift of 32 bit value) in random function. - termios: - variables cmspar and crtscts had wrong values on 32 bits due to overflow in initialization. - General: - numerous numeric conversion issues identified by ubsan were addressed in various places in the code base. - Missing autoload for *in-compilation-unit* caused loading problem for compiled code making use of with-compilation-unit. TXR 273 2021-12-28 Features - compiler: - new jump optimizations. - register compacting optimization: greatly reduces stack use, especially of complex functions, and improves cache locality. - pattern language: - @{var /regex/} changes: - regex no longer ignored when var already has binding - text is extracted with regex, then compared to variable - @{var (fun ..)} changes: - (fun ...) now processed in vertical mode if sole item in line. - variable captures lines skipped over in vertical processing. - (fun ...) no longer ignored when var already has binding. - fun executed like in unbound case. - text that would be bound to variable is compared to existing value. - structural pattern matching: - in quasiliteral patterns, @{var #/regex/} can specify bound variable now: - matches text in same way as unbound case - matched text must then match content of var - lib: - new functions: tuples*, rot, nrot, subq, subq, subqual, subst, pairlis. - hash tables: - use 64 bit hash on 64 bit platforms, rather than 32 bit. - search trees: - new function tree-count, and length/len works on trees. - duplicate keys supported: - tree, tree-insert, tree-insert-node have optional argument for allowing duplicates. - tree-delete-specific-node allows specific node to be removed, when removing by key is ambiguous. - priority queue support: - tree-min, tree-min-node, tree-del-min, tree-del-min-node - oop/structs: - new feature: application-defined struct clause macros. - new: :delegate and :mass-delegate clause macros for generating delegate method boilerplate with minimal code. Bugs - compiler: - fix non-working (compile '(lambda ...)). - buffers: - file-get-buf and command-get-buf use unbuffered I/O to read the exact number of bytes into the buffer, avoiding reading more bytes than requested. - case macros (mainly affecting casequal): - fixed 2017 regression causing a key value like ((a b c)), which is the single key (a b c), to be wrongly converted into a list of three keys. - each-match, each-prod, sum-each family of macros: - documented and added missing anonymous block - maprodo: spurious non-nil return value issue. - interpreter: bug in interpreting optional parameters, present in original implementation from 2014 (absent in compiler). - iteration: gc stability problem in iter-begin and iter-reset. - define-accessor: broken argument handling. - less/greater: gaping bug, vectors not supported. TXR 272 2021-11-11 Features: - path manipulation: - new path-equal function for comparing paths. - pic macro: - support for digit-separating commas. - support for (...) notation for negative values. - FFI: - internal improvements and minor optimizations. - more ergonomic handling of carray, cptr passed by pointer. - compiler: - now diagnoses constant expressions that throw. - improved elimination of wasteful jmp instructions. - minor new optimization eliminating a wasteful register copy. - PRNG: - new random-float-incl function: like random-float but the range is [0, 1] rather than [0, 1). - syntactic places: - new read-once accessor for caching a place so that it is read only once even by place mutating operators which access it more than once. - ifa macro semantics adjusted to take advantage of read-once. Bugs: - FFI: - broken range checks in enum types. - bad format calls in enum error handling code. - math: - bad edge cases in 64 bit conversion (affecting 32-bit platforms). - path manipulation: - rel-path bugfixes for native Windows. - printer: - cases where fallback package syms are wrongly printed without package prefix. - compiler: - ordering issue in load-time. - incorrect algebraic transformation of (- a b c ...) minus forms. - incorrect code generation when compiling catch forms. - top-level lambdas no longer captured into D registers: - not strictly a bug, but undesirable behavior that crept in when lambda lifting by load-time was introduced. - syntax: - broken #; syntax for first element of list. - listener: - bug causing incomplete auto-loading of modules during Tab completion. - structural pattern matching: - unquoted quasiliteral patterns now work. - less function: - crash when arguments are symbolic and the right one is nil. - other inconsistent, incorrect behavior for some combinations of symbolic arguments. TXR 271 2021-10-05 Features: - load: - new *load-hooks*: defer exeucution to load finish time. - used via push-after-load and pop-after-load - libtags.txr: script for generating extra tags to jump to TXR's C code using Lisp symbols. - lib: - delcons function: destructively remove indicated cons from list. - improvements in string-extend. - set-mask and clear-mask macros: shorter code working with masks. - new module for quantile estimation: see quantile function. - summing and producing variations of each operator: - sum-each-prod, mul-each-prod, sum-each-prod*, mul-each-prod* - path-search: semantics changes. - path access test functions now use read uid/gid rather than effective uid/gid. - new replace-env function. - new *child-env* variable for specifying environment for executed process images. - FFI: - socklen-t type now defined. - ffi macro now generates load-time form to avoid repeated invocation of FFI type compiler. - new cptr-carray function, inverse of carray-cptr. - exceptions: - system exceptions now store errno in exception message. - see string-get-code in doc. - sockets: - socket options now supported via new sock-opt and set-sock-opt. - exception now thrown if socket call fails. - compiler: - one small optimization improvement and internal improvements. - awk: - new :fields feature to give fields names and type conversions. Bugs: - compiler: - regression in calculation of output path of compiled files. - random perturbation in code generation due to dependency on hash table order in an optimization routine. - poll: array from alloca passed to free. - sockets: bug in sock-peer assignment place. - hash: gc problem in copy-hash function. - sequence iteration: gc problem. - maprodo: problm with single-list argument. TXR 270 2021-08-30 Features - open-file now supports "x" mode for exclusive create, contributed by Paul A. Patience. Bugs - sequence iteration: - garbage collection corruption was discovered via experimentation with string ranges (new feature in 269). - caused by not properly doing the counter-generation object mutations. - Bug is not new: affects bignum ranges, oop sequences. - iterating ranges that go from fixnum to bignum now allowed. - open-file: - "+" mode was behaving like "r" and not "r+" (Paul A. Patience) - "w+", "m+" and "a+" refused to create file (Paul A. Patience) TXR 269 2021-08-28 Twelfth Anniversary Edition Features - networking: - getaddrinfo now implements ai-canonname flag. - system interface: - mmap function now supported, integrated with carray. - mprotect, msync, madvise are there. - structural pattern matching: - new match and match-ecase macros for irrefutable matching. - basic Lisp: - new ecase family of macros. - sequences/iterables: - string ranges like "AAA".."ZZZ" are now iterable. - iterators (objects from iter-begin) are iterable. - sub function allows iterables. - FFI: - carray-pun function allows displacement - improved support for big/little endian types: - more efficient when matches local endian - aligned reads and writes transfer word at a time. - PRNG: - new random-buf function for obtaining a block of pseudo-random bytes. - doc improvements. - build/port: - builds on FreeBSD; test cases pass. - new --no-full-repl option builds trimmed-down listener that supports only plain mode editing, not requiring termios. Bugs - int-str: bug stripping 0x unconditionally regardess of radix argument. - format: leading sign state leaking into subsequent conversions. - ffi: deffi: broken support for variadic functions. - random: bug with modulus that is multiple of 32 bits, found on PPC64. - open-file: "+" mode must be equivalent to "r+" not "r". TXR 268 2021-08-07 Features - subtypep: arguments can now be struct type objects returned by find-struct-type, not only type symbols. - JSON: - new *read-bad-json* dynamic variable, enabling tolerance for trailing commas in JSON arrays and objects. - OOP: - syntactic infelicity in new* and lnew* operators addressed. - streams: - close-stream now replays return value if called redudnantly. - get-lines/lazy-stream-cons now have optional parameter controlling whether the implicit close-stream can throw. - listener: plain mode - handles multi-line expressions - prints prompts if stdin is tty - prompts can be turned on with :prompt-on - banner is suppressed when stdin isn't tty; more usable in pipes. - TXR Pattern Language: - @(eof) now takes an optional argument which can bind the exit status of the input source. Useful for process pipes. Bugs - gc: aborts caused by incorrectness in several object-copying functions. - correct diagnostic name in remql function. - listener: plain mode (txr -n, or input is not a terminal) now handles multi-line expressions. - build: musl fix for socket.c: need . - streams: incorrect argument defaulting of second arg of close-stream TXR 267 2021-07-26 Features - system interface: - new getrlimit, setrlimit functions - random numbers: - buffer objects can be used as random seeds now - build: - txr.c now recompiled if build_id changes. - PDF build is now reproducible even if ghostscript and groff don't have patches for this. - tags.tl: - now supports --emacs argument for Emacs-style tags, thanks to Paul A. Patience. - other improvements - hashes: - Hashes now support both and-semantics and or-semantics for tables that have both weak keys and values. - and-semantics means both key and value must be unreachable for the hash entry to disappear. - or-semantics means the entry disappears if just the key or the value is unreachable. Bugs - compat: - glaring bug fixed going back more than 150 versions. - certain effects of the -C compatibility option not having their documented effect. - caused by referencing the opt_compat variable in global initialization functions, at which time opt_compat is always zero due to -C not having been processed yet. - Test case fixes for missing /bin/sh situation exemplified by Guix build environment. - op: - weirdness in handling nested do (do do do ...) fixed - hashes: - fixed TXR 235 regression in weak processing, causing entries to spuriously disappear from weak hash tables that are only referenced by other weak hash tables. - fixed incorrect recalculation of hash table counts of weak hash tables during garbage collection. - carray: - missing type checking in a couple of functions, creating opportunity for trivial crash. TXR 266 2021-07-12 Features - built-in macros and special operators are now subject to more rigorous syntax checking during the macro-expansion walk. - improvements in error reporting - built-in macros use compiler-like error reporting now. - Lisp files executed from command line rather than loaded, ditto. - running make tests out of an editor now takes you to the error line. - improvements in doc function, and OpenBSD support. - type system overhauled to disallow structs that clash with built-in types. - new function called separate contributed by Paul A. Patience. - combines keep-if and remove-if semantics - new path-manipulation-related functions trim-short-suffix, trim-long-suffix and add-suffix. - new build-id feature: optional string that can be inserted into TXR at build time, displayed by txr --build-id. Bugs - non-functional chmod.tl test case fixed, thanks to Paul A. Patience. - regex: argument defaulting problem in regex-compile. - *stderr* stream is now sanely reset during unhandled exception processing. - new steps taken to prevent runaway recursion in exception processing. - streams: - close-stream function refuses to close stderr. (previously refused only stdin and stdout.) - put-char, put-line: lack of type checking on stream argument. - bug in with-resources problem fixed, reported by Paul A. Patience. - doc ignores BROWSER variable if it is empty. TXR 265 2021-07-04 Features - requirements change in new long-suffix and short-suffix functions: - dot is now part of suffix. - leading dot is not a suffix delimiter: e.g. .bashrc is not a suffix. - trailing path separators ignored, like in base-name. - regex: optimization function exposed. - constantp function now recognizes more kinds of expressions: - (+ 1 (* 3 4)) is constantp, as is (symacrolet ((a (+ 2 2))) (* b 3)). - doc function - now handles situations in which xdg-open blocks until browser exits. - now reacts to BROWSER variable, and if xdg-open is not found, falls back on the first of a long list of possible browsers. - filesystem interface: - path-cat function is now variadic: (path-cat "a" "b" "c" ...). - new path-search function, searches for an executable by name in path, defaulting to the system's PATH. - sequences: - new find-true function; like find-if, but returns the true value that the predicate produces, rather than the item from the sequence. - I/O streams: - argument defaulting tightened; functions no longer treat a nil value for the stream argument as a missing argument. - stack limit: - minimum limit now imposed when the system's stack limit is too low, rather than disabling the mechanism. - stack limit is now always on, even if we don't obtain a value from the system or that value indicates that there is no limit. - documentation infrastructure: - improvements from Paul A. Patience integrated. - doc workflow catches more kinds of problems. - listener: empty EDITOR variable now treated as missing. Bugs - build: regression in separate-directory build. - parser: regression: not working with byacc. - compiler: a number of bugs in inline lambda implementation. - op: subtle bug in do operator; code refactored. - base-name function: problem with empty suffix. - listener: end-of-line/buffer visual glitch in selection. - trie: bugs in regex-from-trie function, now covered by tests. - regex: print/read consistency problem printing n-ary operators. - doc: *doc-url* variable not special, as documented. - getopts: throwing sys:opt-error instead of usr:opt-error. - command line: lack of robustness in -b option fixed. - documentation: numerous fixes. - packages: find-symbol was behaving identically to find-symbol-fb. - signals: itimer-prof variable misspelled as itimer-prov. - search trees: documented tnodep function now actually exists. - stack limit: fix crash when system stack limit is RLIM_INFINITY. TXR 264 2021-06-25 Features - system interface: - TXR no longer relies on popen for open-command. - glob function accepts multiple pattern arguments and uses multiple calls to the C function with GLOB_APPEND. - parser: - parsing Lisp or JSON from a string now produces error if there is any trailing material in the string. - paths: - new functions short-suffix and long-suffix for robustly extracting the suffixes/extensions of path names. - lib: - new functions cxr and cyr for traversing cons-cell structures using a car/cdr path binary-coded in an integer. - mismatch/rmismatch better optimized for strings - starts-with and ends-with use these. - structural pattern matching: - new looping macros while-match, while-match-case, while-true-match-case. - parser: - no longer wastefully allocates dynamic string when scanning a floating-point token. - tests: - target-installable test cases are now relocatable (can be installed at any path) due to a small improvement in the run.sh script. - program-wide: - share/txr/stdlib moved to stdlib. - type mismatches when a string is expected now give function name in error diagnostic. - stack overflow protection is introduced: - in key places, TXR detects whether the stack pointer is over a predetermined limit and throws a stack-overflow exception. - controlled by set-stack-limit function. Bugs - fixed wrong result from (rmismatch #() ()) and (rmismatch "" ()). TXR 263 2021-06-17 Features - New macro named flow, providing the syntactic sugar for using an opip function on a value. - I/O: - the *stdnull* stream lazily attaches to /dev/null if fileno is invoked on it - formatted printing: - format: new precision modifier - for zero instead of plus sign. - pic macro: takes advantage of format work to generate better code. - subprocesses: - some file descriptor saving-restoring manipulations moved into child process (in open-process, open-subprocess, run) - diagnostic for situation when *stdout*, *stdin* or *stderr* are redirected to something that cannot produce a file descriptor. - match-fun/txr-if - documented that input can be a stream - documented that input can be a single string - txr-case: - if input is a stream, it is now converted to a lazy list of lines, so that the txr-case construct effectively backtracks over the data as it tries successive cases. - command-line - new --noprofile option to invoke listener without processing ~/.txr_profile file. Bugs - format: numeric handling maintenance - poor behaviors identified and revised. - requirements clarified. - cemented in test cases. - exceptions: - unwind dynamic environment when tracing unhandled exception - solves problem when exception goes off while *stderr* is redirected. - subprocesses: - diagnostic for situation when *stdout*, *stdin* or *stderr* are redirected to something that cannot produce a file descriptor. - macros: - fixed TXR 191 regression in defsymacro: expanding the replacement form before associating it with the symbol, rather than taking as-is. - quasiliterals: - fixed issue arising when a macro invoked as a @(...) expression in a quasiliteral expands to a non-string atom. - math: - forbid dubious inequality comparisons like (< 1 "abc") which became unintentionally allowed due to numbers being iterable. TXR 262 2021-06-11 Features - structural pattern matching: - new feature: quasiquote matching. - JSON: - improved escaping of JSON output for safe embedding in