TXR 71 2013-12-07 Features - New functions countqual, countql, countq and count-if. - New regex-parse function: parse regex from a string at run-time. - New *stdnull* stream for discarding data. - New function: seek-stream for positioning within a stream. - New function open-tail for opening a new kind of stream called a tail stream. This stream follows rotating log files. - New functions for converting a time expressed as individual fields into numeric time: make-time, make-time-utc. - New function daemon (where supported). Calls the BSD-derived daemon function to daemonize the process in one step. - New errno function for accessing C errno. - Real-time input support in pattern matching language, via "simple" lazy text stream line lists which sacrifice accuracy of representation for timely delivery. Bugs - Fixed missing recognition for hexadecimal integers in Vim syntax higlighting file txr.vim. - Fixed missing check for null handle in stdio back-end of flush-stream. - Grammar and spelling in HACKING guide. TXR 70 2013-11-22 Features - Fixed a nasty bug that affects all versions; I was not able to find a revision in git that works right with gcc 4.6.3. Invoking an @(accept) in the middle of a @(collect) was found not work at all as documented, and this was root-caused to my neglect to use volatile on some local variables. - Fixed a more recent regression in the op syntax, caused this October. - Fixed an amazing regression introduced in TXR 22, back in November 2009, easily reproduced using: echo ":x" | ./txr -c "@x:@x" - - Fixed all unintentional deviations from 1990 ISO C. GCC had not been diagnosing them for us due to my neglect to use the --pedantic flag. - Revamped and up-to-date txr.vim syntax file, for you Vim users. TXR 69 2013-10-23 Features - Multiple @(output) blocks can now continue on the same output stream, which is particularly useful if it is a pipe. Bugs - Fixed errors in documentation for the functions time-string-local, time-string-utc and expt. TXR 68 2013-10-07 Features - @(repeat) inside @(output) supports a :vars parameter, making it possible to declare the existence of iteration variables that are otherwise hidden within Lisp code. Bugs - Out-of-sequence numeric parameters can be used in the op operator now, like (op foo @2) where @1 is absent. - The implicit ". @rest" is now added to op forms only if they do not mention @rest or any numeric parameters, to prevent unintentional passing of extra parameters. - @rest or a numeric param like @1 can now be specified in the dot position of the op form, as in (op foo . @rest). TXR 67 2013-07-13 Features - New functions tok-str and string-cmp. - Internal lazy string functions exposed as intrinsics. - New pattern language directive: @(require ...). Bugs - Unrecognized backslash escapes in string literals and string quasiliterals are diagnosed. - Backslash-space escape recognized in string literals and string quasiliterals. TXR 66 2013-05-16 Features - Documentation completely filled in. - Added specfile for RPM builds. - @(repeat) introduced as shorthand for @(collect :vars nil). - New open-command and open-process functions. The open-pipe function becomes deprecated. - New multi-sort function for sorting two or more lists at the same time as if they were columns of a table. - assq and aconsq functions are renamed to assql and aconsql becuse they use eql equality, not eq. - New prop function for property list lookup. - New stat function for information about files. - New functions in hashing library: - Copying hashes: make-similar-hash, copy-hash - Set operations: hash-uni, hash-diff, hash-isec. - Gapingly missing hexadecimal integer constants have been implemented. - New bit operation functions that work with TXR's arbitrary precision integers: loand, logior, loxor, lognot, logtest, ash. - Test form in for loop can be omitted. - New package-related functions. - New functions for obtaining time of day and converting to text. Bugs - Fixed broken (+ ) case in addition. This fixes the range and range* functions too. - Fixed nonworking building in a separate directory. - Fixed broken eval function. - Bugfix in form expander's handling of regular expression syntax, causing the (set ...) notation for character sets being mistaken for the (set ...) assignment operator. - Bugfix in format: apply field formatting to argument not only if a nonzero with has been specified, but also if a precision has been specified. - Bugfix in format: ~s on a floating point number now always shows .0 except when a precision is given, and it is zero. - Fixed broken @(last) clause processing of @(collect), int he case when the last material matches at the end of a stream. - Fixed certain function applications not being able to call functions that have optional arguments with fewer than TXR 65 2012-04-20 Features - New ways to iterate over hash tables using lazy lists. - Regular expression compiler expressed to user code, allowing TXR programs to build regex abstract syntax trees and turn them into regular expression machines. - The popular regular expression tokens \s, \d, \w and \S, \D, \W are now supported (the first three of these in character classes also). - Low level time-of-day functions added for retrieving epoch seconds, and optionally microseconds. - New remove-if and keep-if functions, plus lazy counterparts remove-if* and keep-if*. - Lazy variants remq*, remql* and remqual* added. - New function find-if. - Improved seeding of pseudo-random-number generator. - Configure script now diagnoses specification of nonexistent config variables. - Documentation improvements. - Portability to Mac OS/X 10.7 (Lion), NetBSD 5.1 and FreeBSD 9.0. Bugs - Bugfix in @(next) directive, which showed up when using @(next :args) more than once to scan arguments more than once. Quiet Changes - First two arguments to the find function were reversed. TXR 64 2012-04-06 Features - In the pattern language, binding the symbol nil as a variable is permitted now and has a special meaning of matching and discarding input. - Caching optimization in hash tables. - Ephemeral GC implemented. This is experimental and currently must be enabled at compile time with the --gen-gc configure option. Bugs - Interaction between @(trailer) and @(accept) has been clarified. An @(accept) return from the forms of a @(trailer) will still roll back the input position to the starting point for the @(trailer). TXR 63 2012-03-30 Features - Significant improvement in the performance of the garbage collector, which can drastically cut the run-time of some programs. - match-str and match-str-tree functions exposed. - new @(rebind) directive provides for a one step dynamic shadowing of a variable with a local variable, whose value can be derived from the one being shadowed. - filters now work on nested lists of strings, not just strings. - floating point formatting now produces the same exponential notation on different platforms, by post-filtering the sprintf output: a leading + and leading zeros are removed: e+023 becomes e23. - new functions: num-str, tan, asin, acos, =, - min and max return the left operand rather than the right one, when two operands are equal. Bugs - search-str optional argument was not working when omitted. - Fixed situations involving an incorrect behavior: partial horizontal matches (not matching the full line) were succeeding. - It was impossible to define a pattern function called @(text) due to the use of the symbol text as an internal symbol, rather than a symbol in the private system package. - error checking code in funcall had an improperly terminated argument list, causing an exception. - @(output) blocks now flush their stream at the @(end), without which output is not correctly interleaved between output produced by external commands. - fixed some misleading exception wordings in the numeric library. - fixed sign semantics of floating-point mod to be like the integer mod. - gcd function allows zeros. - fixed broken exptmod: it was not normalizing bignum values that fall into the fixnum range into fixnum objects, resulting in undersized bignums being returned. TXR 62 2012-03-23 Features - Floating-point support has been added. - TXR programs can specify floating-point constants. - Printing floting points is supported in the format function. - New specifiers: ~f and ~e. - Arithmetic on floating points, with implicit conversion. - New / function which produces a floating-point result. - New functions like sin, cos, exp, and log. - Functions for converting between floating and integer, and parsing a float from a string. - New combinators for easier programming with higher order functions: chain, andf, orf, iff. - url_decode and url_encode functions take an optional parameter which determines whether + is used as the code for space, distinguishing URL encoding from percent encoding. Separate named filters are introduced: :frompercent, :topercent distinct from :fromurl and :tourl. Bugs - Buggy quicksort routine repaired. This is a recently added feature which allows vectors and strings to be sorted. List sorting, done by merge sort, is unaffected. - Breakpoints can now be set by line number and file name, allowing breakpoints in loaded modules, not just the main module. TXR 61 2012-03-15 Features - URL encoding and decoding functions added, and :tourl/:fromurl filters implemented for output substitution. - split-str function works with regex objects now. - new regsub function for regex substitution in strings; matched text can be filtered through a function. - new *stddebug* stream used by debugger. - put-byte works on string output streams, and does right thing with mixtures of bytes (which are taken as UTF-8) and characters written with put-char. - Hash table literals implemented; hashes can now be notated rather than constructed. - Vectors can now be quasiquoted with unquote and splicing, and so can the new hash table literals. - @(block) syntax change: blocks do not extend to the end of the surrounding scope but introduce their own, delimited by @(end). Fixes - Fixed memory leak in string byte input stream. - Fixed wrong parsing of tokens in cases like @(collect-foo) where this was treated as @(collect -foo) rather than a single token. TXR 60 2012-03-04 Features - List element removing functions: remq, remql, remqual. - Memory use optimizations in @(freeform) horizontal scanning. - Improved hashing function for bignums. Bugs - Fixed incorrect processing within long lines, caused by earlier optimizations in 57. - Missing 'd' command for deleting a breakpoint implemented in the debugger. - numberp function no longer blows up on nil. - Fixed broken @(load) directive: was not allowing directives which follow it to be processed. - Fixed incorrectly formatted line number information in some exceptions. - Fixed missing support for optional and trailing parameters in defun. - Fixed showstopper bug in plus function. TXR 59 2012-02-29 Features - Implemented put-byte function on streams. - Regex syntax in TXR Lisp changes from /re/ to #/re/, allowing symbols containing slashes. - New -B command line option forces TXR to dump the bindings even if the program produced output. - Revised and improved txr.vim syntax highlighting file for greater lexical accuracy and superior flagging of errors. Fixes - Regression that broke [n..t] and [n..nil] sequence ranges. - Broken hex-escaped character literals. - Old bug from database: filter functions now see bindings in all contexts in which they are invoked. - Output clauses consisting of one empty line were being treated as empty. - Implemented more sane rule for deciding when output has taken place and suppress the printing fo bindings. TXR 58 2012-02-25 Features - Exception handling exposed in TXR Lisp. - range* function similar to range, but for generating a range with the endpoint excluded. - TXR Lisp can invoke a function in the pattern language now using the new match-fun function. - Braced variable syntax in @(output) extended to support arbitrary expressions in place of the variable name. The expressions are converted to text and then treated the same way as a variable substitution. Indexing is allowed, and field-formatting takes place. Moreover, these expressions are code-walked by @(repeat) and @(rep) to look for embedded variables. - New TXR Lisp functions ref, refset, sub and replace. - Indexing and range extraction in brace substitution in @(output) clauses is now generic over lists, vectors and strings. - Optional arguments introduced into TXR Lisp, in defun and lambdas, with a simple syntax involving a colon. Numerous intrinsic functions have been converted such that some previously required arguments are now optional. - Sort function that was previously for lists only is now generic over vectors and strings. - New del operator in TXR Lisp can delete elements of sequences or hashes which are denoted using the bracket syntax. - Range indexing has "floating zero" behavior now so that for example [seq -2 0] means the "last two elements": when the start of the range is negative, then a zero end is treated as length plus one. - TXR programs can now be split into multiple modules, using the load directive to load and run code. Bugs - range function terminates lazy list when the sequence overshoots the final value. - Variable that holds nil is treated as a list in (set [var x..y] ...) range assignment. - Vestigial (t . obj) representation in exception handling code removed. - TXR now does not not dump bindings if any output occured on a stream connected to the C stdout stream. Previously, it suppressed printing of bindings if @(output) was carried out on the *std-output* stream. - Function remhash was found to be buggy and rewritten. TXR 57 2012-02-14 Features - Operations under the @(freeform) directive can now scan in constant memory, allowing large files to be processed. (Scanning a single regex still requires the data to be all in memory: an experimental patch for this exists.) - Improved printing of character position context in debugger when lines are long. - Metanums (@1, etc) can be used in a quasiliteral, which is useful for quasiliterals occuring inside the op notation. Bugs - lazy-flatten function did not handle atoms. This broke @(next :list expr) also, for the case where expr evaluates to a string atom. - In format, the ~s directive was found to be printing strings in the same way as ~a. - Hex and octal character constants did not work. - Control characters in strings and characters are printed as hex now rather than octal. A semicolon is added if the next character would be interpreted as part of the escape. - Hash indexing via the [] notation was still requiring the default value argument. TXR 56 2012-02-06 Features - Hex and octal escapes work in strings and quasilterals now: the documentation has stopped lying. - Escapes can be followed by a semicolon which terminates them and is removed, which is useful if an escape is followed by characters that would otherwise be interpreted as part of the escape. - More color categories and more accurate syntax in Vim syntax highlighting file. Highlights @[...] syntax properly inside quasiquote. - The third argument (the default value if a key is not found) can be omitted when indexing hashes with the [hash key] syntax. It defaults to nil. - The dwim operator (and thus [] syntax) is even more Lisp-1 like. It now has Lisp-1 namespace semantics for evaluating arguments that are symbols. - A new operator called "op" as been added. This allows simple lambda functions to be written as partial evaluatios of functions, with implicit arguments as well as numbered/rest arguments appearing in the body. Bugs - Fixed missing type check in hash module that allows bad code to crash interpreter instead of getting an excepion. - Fixed regression in TXR 55 which broke computed field widths in output variables. - Fixed incorrect UTF-8 decoding of some code points. - Closed several security holes in UTF-8 decoder by recognizing all invalid UTF-8 sequences, and mapping invalid bytes in such a way that any byte sequence processed by the decoder into Unicode code points will be recovered exactly when encoded back into UTF-8. TXR 55 2012-01-26 Features - New square bracket syntax for Lisp-1 like invocation and array-indexing over lists, vectors, strings and hashes. - New a .. b pair syntactic sugar for (cons a b). Works with array indexing brackets to extract ranges (slices) from lists, vectors and strings. - Indexed elements and slices are assignable. - In the pattern language, variables in output templates (output variables) can be indexed and sliced. - Output variables that hold lists are now interpolated with spaces between, and this can be overridden with any separator string. TXR 54 2012-01-21 Features - Improved debugger: - step into - step over - finish - backtrace for pattern and TXR Lisp functions. - dump more complete environment. - Debugging support can be disabled at compile time: configure with --no-debug-support. - New lazy append function (append*). TXR 53 2012-01-11 Features - In variable substitutions in output and quasiliterals, the field width expression is now evaluated. - TXR Lisp: - New operators and functions for generating lazy lists more conveniently. - lazy mapcar and mappend: return lazy list. - delay and force operators. - parallel iteration/collection over lists. - list-str function for splitting string into list of characters. Bugs - Fixed global buitin variables, broken in 52. Properly implemented intended fix. TXR 52 2012-01-07 Features - @(rep) and @(repeat) now take an keyword argument, :counter. This specifies the name of a variable which holds the repetition count, thus making the count available within the repeated body. - New @(mod) and @(modlast) clauses in repeat/rep, allowing special behaviors to be coded based on whether the repetition count is a particular value modulo some number. - @(gather) directive now supports an @(until)/@(last) clause, so the search range can be restricted. - New directive @(fuzz) for doing an imprecise match over a range of lines, similar to the context line fuzz logic in the patch utility. - gensym function added to TXR Lisp, along with a *gensym-counter* global variable. Bugs - Fixed a regression in repeat/rep triggered when multiple clauses of the same type appear (like two occurences of @(last)). - Built-in global variables in TXR Lisp now properly denote the underlying C variable locations, rather than holding copies of the values of those variables. If you assign to *stdout*, it will now really replace the stdout stream stored in the corresponding C variable (std_output), so it has the proper global effect. Previously, this action would just replace *stdout* with a new value, without any effect on std_output. TXR 51 2011-12-28 Features - Better algorithm in the random function for generating evenly-distributed pseudo-random numbers of arbitrary precision. - PRNG startup behavior improved. - New lazy-flatten function for flattening lists lazily. Used within @(next :list ...) implementation, making it possible to scan through Lisp-generated lazy lists. Bugs - Fixed showstopper bug introduced in a patch for the MPI library, affecting 32 bit platforms. - Fixed bug in Lisp expression interpolation in quasiliterals. - Fixed fatal exception being thrown in verbose mode, by one of the formatted print statements. TXR 50 2011-12-23 Features - Dropped silly leading 0 from version number. :) - New vector functions: copy-vec, sub-vec, cat-vec. - New API for pseudo-random-number generation, complete with independent random state objects that can be seeded and copied. - Vim syntax highlighting definition improvements. - In the format function, left-adjustment is specified using < rathr than the - character. (However a negative width specified as an argument using * still means left adjust.) The leading zero for zero padding as well as the sign character (space or +) are specified in the precision field, not the width field. - More complete documentation. - Slight return value convention change in some and all functions. - Added memql function. Bugs - Critical flaw in hashing fixed that could crash on some platforms. - Exception handling fix: if a catch clause performs a non-local exit, the finally clause is still executed. - "make distclean" fixed. - Fix for differences in syntax error diagnosis between Byacc and Bison. - Fixed a memory leak in a division-by-zero case in the bignum mod function. - Fixed a terrible bug in one of the MPI patches affecting the correctness of various operations on numbers having a 1 in the most significant bit position of the most significant digit word. - Fixes in format function. All objects obey field width and left/right alignment. Numeric precision, zero padding and optional sign all works. - Lisp code evaluated in @(repeat)/@(rep) clauses can now see all variables, not just the ones subject to the repeat. (But whether or not a repeat executes, or which clauses, is still independent of what variables are accessed by the embedded Lisp, taking into account only the variables directly embedded in the clause.) TXR 049 2011-12-19 Features - New functions for converting between characters and integers. - Some arithmetic and relational operations are generic over characters in a way that makes sense. - dohash establishes anonymous block. - Improvements in Vim syntax highlighting file. - Lazy cons semantics cleaned up making lazy list programming easier. - Some API renaming and restructuring in the area of vectors. - Semicolon comments supported in Lisp code and @; comments in the pattern matching language. @# becoming obsolescent. - Not function, synonym for null. - Some progress in TXR Lisp documentation. - Hashing functions improved for fixnums, literals and characters. - API for introspecting over interpreted functions added, in anticipation of doing some compiler work. - Quasiliteral strings supported in TXR Lisp. Bugs - Broken abs function fixed for bignums. - mappend semantics clarified and fixed: behaves like append for improper lists and atoms. - Bugfix in code walker for let/let* forms, which resulted in quasiquotes not being expanded. - Fixed incorrect format arguments in some error messages, resulting in aborts in some error cases, instead of the intended diagnostics. TXR 048 2011-12-13 Features - New functions: expt, exptmod, sqrt, numberp, evenp, oddp, abs, gcd reduce-left, reduce-right. - Replaced horribly slow square root in MPI with a less horribly slow one. Bugs - Fixed numerous instances, in the MPI library, of coding broken for mpi_digit wider than 16 bits, leading to incorrect results and crashes. - Fixed mpi_int for 32 bit platforms so that obj_t stays 4 pointers wide. (The sign becomes a 1 bit wide bitfield). TXR 047 2011-12-12 Features - Transparent bignum arithmetic: when operations on machine word (fixnum) integers overflow, multi-precision (bignum) integers are produced. - TXR Lisp: - New operators: progn, flip. - Vector functions added, and vecref is an assignment place. - Character manipulation functions. - Association list functions. - Implicit anonymous block around forms for loop. - Implicit named block around forms for loop. - Nump renamed to fixnump. - Push arguments reversed: (push obj list). - Syntax highlighting definition update for all new operators. Bugs - Another bugfix to character literals, allowing non-alphanumeric constants like #\$. - Fix in rplacd to make lazy list programming feasible. - Reversed assoc and assq arguments throughout codebase. - Debugger: repeating last command by pressing Enter works again. TXR 046 2011-12-06 Features - Vector-related functions exposed in Lisp interpreter. - Syntax added for specifying vector literals in code. - Length function made generic over strings, lists and vectors. Bugs - Broken get_line function over string input streams fixed. - Some kinds of character literals were not being recognized properly. - bugfixes in configure script, affecting 64 bit Mac OS X. Thanks to Andy Wildenberg for reporting issue and confirming root cause. TXR 045 2011-12-05 Features - New functions exposed in Lisp interpreter: strings, characters, symbols and lazy lists. Bugs - Flaws in some string-splitting functions identified and fixed. - Bugs in quasiquote. - Handling of singleton clauses in cond operator. TXR 044 2011-12-01 Features - Lisp interpreter added. - Relaxed rules for what constitutes symbol names. Bugs - Regression in source location tracking (line numbers shown in debugger and trace output). - Regression in vertical skip directive caused it to ignore its arguments. - Fixed :vars () in collect/coll not working as intended. This should prevent any bindings from being collected, and allows iteration with collect without accumulating memory. - Two-year-old bug in weak hash tables. TXR 043 2011-11-23 Bugs - Buggy @(eol) directive fixed. - Semantics change for text and regular expressions in "negative match": - a variable is considered to be followed by a run of text which consists of any mixture of regexes and literal text - thus @foo bar behaves properly once again; it is not treated as foo followed by the regex / +/, ignoring the text bar. - Infinite looping bug in negative match with longest-match semantics. - Bug in the overflow detection in the lib.c plus and minus function. TXR 042 2011-11-20 Features - Access to environment via @(next :env) - New @(gather) directive for matching out-of-order material. - Horizontal functions: - TXR can now parse grammars. - Variable syntax allows capture of text spanned by function calls: e.g. @{var (func x y)}: text spanned by (func x y) goes into var. - Horizontal modes for numerous directives such as @(bind), @(local), ... - Lisp-style bindings output. - Interactive breakpoint/step debugger added. This is an experimental prototype, but already quite useful. - Directives following a variable match have searching semantics now, as do function calls. @a@(foo) will scan for a match for @(foo), and the text skipped over is captured by a. - New :resolve keyword in @(some) directive allowing conflicting variable bindings to be set in separate clauses and resolved. - deffilter is more powerful: arguments are evaluated so filters can be computed by the query. Bugs - Horizontal @(some) directive hooked in. - @(freeform) with no data fails to match instead of throwing strange error. - Setting non-local variables from a function works now. - Stability fix: several long-time uninitialized variable bugs found, and some faulty object initialization orders. - :vars in @(collect)/@(coll) does not fire an exception about missing required variables if no variable bindings are produced at all, allowing strict collects to have explicit matches for unwanted material without triggering this nuisance error. - @(repeat)/@(rep) allow empty body. Internal - New infrastructure for matching location info to source forms. Location info (currently just the line number) is associated with source forms via a weak hash table. - Character literals are now Lispy (hash-backslash syntax), based on Scheme. - Added quote, unquote and splicing syntax similar to Lisp. Not used for anything yet. - Improved Valgrind integration: more accurate, timely detection of uninitialized fields in heap objects. Misc. - A TXR syntax highlighting file for the Vim editor exists now. See the txr.vim file. TXR 041 2011-10-30 Features - New :append keyword in @(output) to append instead of overwriting. - Variable contents can be treated as input sources using :string and :list keywords in @(next). Variables can be treated as output destinations using :into keyword in @(output). - New @(set) directive for destructive assignment to a variable. - New filters: :upcase and :downcase. - @(bind) can now compare left and right objects through filters. - Filters can now be chained into compound filters. - Pattern matching functions can be used as filters. - Shorthand notation in @(deffilter) when multiple strings map to the same replacement string. - @(cat) directive changes syntax. - Error handling improvements in parser: no more reams and reams of errors. Bugs - Runaway recursion in @(block) directive, introduced in 040. - Fixed bug in matching list variable against text, at the same time clarifying semantics to longest-match. - Fixed potential excessive memory use caused by refactoring in 040.