diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/ChangeLog | 275 | ||||
| -rw-r--r-- | doc/Makefile.in | 33 | ||||
| -rw-r--r-- | doc/awkcard.in | 6 | ||||
| -rw-r--r-- | doc/gawk.1 | 66 | ||||
| -rw-r--r-- | doc/gawk.info | 7590 | ||||
| -rw-r--r-- | doc/gawk.texi | 6178 | ||||
| -rw-r--r-- | doc/gawkinet.info | 6 | ||||
| -rw-r--r-- | doc/gawkinet.texi | 2 | ||||
| -rw-r--r-- | doc/gawktexi.in | 6071 | ||||
| -rw-r--r-- | doc/texinfo.tex | 271 |
10 files changed, 11237 insertions, 9261 deletions
diff --git a/doc/ChangeLog b/doc/ChangeLog index ed2a2761..0e88c869 100644 --- a/doc/ChangeLog +++ b/doc/ChangeLog @@ -1,3 +1,278 @@ +2015-04-02 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in, gawk.1, awkcard.in: Name change: div() --> intdiv(). + +2015-03-31 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Update discussion of calling built-in functions + indirectly. Small additional fix relating to rand(). Thanks + to Antonio Colombo. + +2015-03-27 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor edits. + +2015-03-24 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor fixes from Antonio Colombo and new exercise + in chapter 16. + * gawk.1: Minor edits. + * gawktexi.in: Edits in material on errno and retryable and get_file + API. + +2015-03-17 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawktexi.in: Modify inplace.awk to call inplace_end in BEGINFILE + and END instead of in ENDFILE. This way, actions in ENDFILE rules + will be redirected as expected. + +2015-03-17 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Turn "positive" into non-negative as appropriate. + Thanks to Nicholas Mills <nlmills@clemson.edu> for pointing out + the issue. + +2015-03-08 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Briefly describe that nonfatal I/O overrides + GAWK_SOCK_RETRIES, in the env var part and in the nonfatal I/O + part. + +2015-03-01 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Change quotes to @dfn for pseudorandom. + A last-minute O'Reilly fix. + +2015-02-27 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Update UPDATE-MONTH and copyright year. + Note that "the guide is definitive" quote is really + from "The Restaurant at the End of the Universe". Thanks + to Antonio Colombo for pointing this out. + +2015-02-24 Arnold D. Robbins <arnold@skeeve.com> + + * texinfo.tex: Update to most current version. + * gawktexi.in: Minor edit to match an O'Reilly fix. + Add some FIXMEs to one day use @sup. + +2015-02-22 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Change 'div' to 'divisor' in some examples. + This future-proofs against a new function in master. + Thanks to Antonio Giovanni Colombo for the report. + +2015-02-20 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: More O'Reilly fixes. I think it's done! + +2015-02-19 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: More O'Reilly fixes. + +2015-02-17 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: A few minor formatting fixes to sync with O'Reilly + version. + +2015-02-13 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. Through QC1 review. + +2015-02-11 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + +2015-02-10 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor fixes, O'Reilly fixes. + +2015-02-09 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Restore a lost sentence. O'Reilly fixes. + +2015-02-08 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + Make non-fatal i/o use "NONFATAL". + +2015-02-06 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + +2015-02-04 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + * gawktexi.in: Update various version-related bits of info. + +2015-02-02 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + +2015-02-01 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: POSIX requirement that function parameters cannot + have the same name as a function is now --posix. + Restore indirectcall example. + + More O'Reilly fixes. + +2015-01-30 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Document POSIX requirement that function parameters + cannot have the same name as a function. Fix indirectcall example. + +2015-01-27 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + And still more. Also, fix @code --> @command in a number of places. + +2015-01-26 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + +2015-01-25 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Fix a bad URL. And another one. + More O'Reilly fixes. + +2015-01-23 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + (Glossary): Many new entries from Antonio Giovanni Colombo. + +2015-01-21 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + Remove obsolete start/end of range indexing comments. + +2015-01-20 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: O'Reilly fixes. + +2015-01-19 Arnold D. Robbins <arnold@skeeve.com> + + * gawkinet.texi: Fix capitalization in document title. + * gawktexi.in: Here we go again: Starting on more O'Reilly fixes. + +2014-12-27 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Add info that nonfatal I/O works with stdout and + stderr. Revise version info and what was added when. + +2015-01-05 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawktexi.in: Improve get_file documentation. + +2015-01-05 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawktexi.in: Replace "Retrying I/O" with "Retrying Input", since this + feature pertains to input, not output. + +2015-01-04 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawktexi.in: Document the get_file API function. + +2015-01-04 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawk.1: Document new features PROCINFO["errno"] and + PROCINFO["input", "RETRY"], and new getline return value of -2. + * gawktexi.in: Ditto. + +2014-12-26 Antonio Giovanni Colombo <azc100@gmail.com> + + * gawktexi.in (Glossary): Really sort the items. + +2014-12-24 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Start documenting nonfatal output. + +2014-12-24 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Add one more paragraph to new foreword. + * gawktexi.in: Fix exponentiation in TeX mode. Thanks to + Marco Curreli by way of Antonio Giovanni Colombo. + + * texinfo.tex: Updated. + +2014-12-12 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor fix. + Thanks to Teri Price <tjp212@lehigh.edu>. + +2014-12-10 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: More minor fixes. + +2014-12-09 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: More minor fixes. + +2014-12-07 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor fixes. + +2014-12-06 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: A minor fix. + +2014-12-05 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Various minor fixes and updates. + +2014-11-23 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Update that TZ env. var can influnce mktime + in running program. Thanks to Hermann Peifer. + +2014-11-19 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Update that RFC 4180 documents CSV data. + +2014-11-17 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Copyedits applied. + +2014-11-02 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Comment out that I need an owner for awk.info. + I may have found one or two people. + +2014-10-29 Andrew J. Schorr <aschorr@telemetry-investments.com> + + * gawktexi.in: Document new extras directory containing shell startup + files to manipulate AWKPATH and AWKLIBPATH environment variables. + +2014-10-28 Arnold D. Robbins <arnold@skeeve.com> + + * gawk.1: Clarification that debugger reads stdin. + * gawktexi.in: Ditto, and correctly place the "Braces" entry in + the Glossary. Thanks to Antonio Colombo for that. + + Unrelated: + + * gawktexi.in: Restore use of @sc. Karl fixed makeinfo. :-) + +2014-10-25 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Minor typo fixes. + Fix discussion of \x, per note from Antonio Colombo. + +2014-10-17 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Fix date in docbook attribution for new Foreword; + thanks to Antonio Colombo for the catch. Update latest version + of gettext. + +2014-10-15 Arnold D. Robbins <arnold@skeeve.com> + + * gawk.1: Fix default value for AWKLIBPATH. + * gawktexi.in: Revised text for AWKPATH and AWKLIBPATH. + +2014-10-14 Arnold D. Robbins <arnold@skeeve.com> + + * gawktexi.in: Add new Foreword from Mike Brennan. + 2014-10-13 Arnold D. Robbins <arnold@skeeve.com> * gawktexi.in: Fix example outputs in chapter 2. diff --git a/doc/Makefile.in b/doc/Makefile.in index ba3ee735..3599e42f 100644 --- a/doc/Makefile.in +++ b/doc/Makefile.in @@ -1,7 +1,7 @@ -# Makefile.in generated by automake 1.13.4 from Makefile.am. +# Makefile.in generated by automake 1.15 from Makefile.am. # @configure_input@ -# Copyright (C) 1994-2013 Free Software Foundation, Inc. +# Copyright (C) 1994-2014 Free Software Foundation, Inc. # This Makefile.in is free software; the Free Software Foundation # gives unlimited permission to copy and/or distribute it, @@ -38,7 +38,17 @@ # Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA # VPATH = @srcdir@ -am__is_gnu_make = test -n '$(MAKEFILE_LIST)' && test -n '$(MAKELEVEL)' +am__is_gnu_make = { \ + if test -z '$(MAKELEVEL)'; then \ + false; \ + elif test -n '$(MAKE_HOST)'; then \ + true; \ + elif test -n '$(MAKE_VERSION)' && test -n '$(CURDIR)'; then \ + true; \ + else \ + false; \ + fi; \ +} am__make_running_with_option = \ case $${target_option-} in \ ?) ;; \ @@ -102,8 +112,6 @@ POST_UNINSTALL = : build_triplet = @build@ host_triplet = @host@ subdir = doc -DIST_COMMON = $(srcdir)/Makefile.in $(srcdir)/Makefile.am \ - $(top_srcdir)/mkinstalldirs texinfo.tex ChangeLog ACLOCAL_M4 = $(top_srcdir)/aclocal.m4 am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \ $(top_srcdir)/m4/codeset.m4 $(top_srcdir)/m4/gettext.m4 \ @@ -118,6 +126,7 @@ am__aclocal_m4_deps = $(top_srcdir)/m4/arch.m4 \ $(top_srcdir)/m4/ulonglong.m4 $(top_srcdir)/configure.ac am__configure_deps = $(am__aclocal_m4_deps) $(CONFIGURE_DEPENDENCIES) \ $(ACLOCAL_M4) +DIST_COMMON = $(srcdir)/Makefile.am $(am__DIST_COMMON) mkinstalldirs = $(SHELL) $(top_srcdir)/mkinstalldirs CONFIG_HEADER = $(top_builddir)/config.h CONFIG_CLEAN_FILES = @@ -214,6 +223,8 @@ man1dir = $(mandir)/man1 NROFF = nroff MANS = $(man_MANS) am__tagged_files = $(HEADERS) $(SOURCES) $(TAGS_FILES) $(LISP) +am__DIST_COMMON = $(srcdir)/Makefile.in $(top_srcdir)/mkinstalldirs \ + ChangeLog texinfo.tex DISTFILES = $(DIST_COMMON) $(DIST_SOURCES) $(TEXINFOS) $(EXTRA_DIST) ACLOCAL = @ACLOCAL@ AMTAR = @AMTAR@ @@ -277,6 +288,7 @@ PACKAGE_URL = @PACKAGE_URL@ PACKAGE_VERSION = @PACKAGE_VERSION@ PATH_SEPARATOR = @PATH_SEPARATOR@ POSUB = @POSUB@ +SED = @SED@ SET_MAKE = @SET_MAKE@ SHELL = @SHELL@ SOCKET_LIBS = @SOCKET_LIBS@ @@ -394,7 +406,6 @@ $(srcdir)/Makefile.in: $(srcdir)/Makefile.am $(am__configure_deps) echo ' cd $(top_srcdir) && $(AUTOMAKE) --gnu doc/Makefile'; \ $(am__cd) $(top_srcdir) && \ $(AUTOMAKE) --gnu doc/Makefile -.PRECIOUS: Makefile Makefile: $(srcdir)/Makefile.in $(top_builddir)/config.status @case '$?' in \ *config.status*) \ @@ -452,13 +463,9 @@ $(am__aclocal_m4_deps): $(AM_V_at)if $(MAKEINFOHTML) $(AM_MAKEINFOHTMLFLAGS) $(MAKEINFOFLAGS) -I $(srcdir) \ -o $(@:.html=.htp) $<; \ then \ - rm -rf $@; \ - if test ! -d $(@:.html=.htp) && test -d $(@:.html=); then \ - mv $(@:.html=) $@; else mv $(@:.html=.htp) $@; fi; \ + rm -rf $@ && mv $(@:.html=.htp) $@; \ else \ - if test ! -d $(@:.html=.htp) && test -d $(@:.html=); then \ - rm -rf $(@:.html=); else rm -Rf $(@:.html=.htp) $@; fi; \ - exit 1; \ + rm -rf $(@:.html=.htp); exit 1; \ fi $(srcdir)/gawk.info: gawk.texi gawk.dvi: gawk.texi @@ -867,6 +874,8 @@ uninstall-man: uninstall-man1 uninstall-info-am uninstall-man uninstall-man1 \ uninstall-pdf-am uninstall-ps-am +.PRECIOUS: Makefile + # Uncomment the following definition of AWKCARD if your troff can produce # Postscript but still has troubles with macros from 'colors'. As this diff --git a/doc/awkcard.in b/doc/awkcard.in index 556bdc1e..2e455b2d 100644 --- a/doc/awkcard.in +++ b/doc/awkcard.in @@ -1609,11 +1609,11 @@ expand; l lw(2i). \*(CD\*(FCatan2(\*(FIy\*(FC, \*(FIx\*(FC)\*(FR The arctangent of \*(FIy/x\fP in radians. \*(FCcos(\*(FIexpr\*(FC)\*(FR The cosine of \*(FIexpr\fP, which is in radians. -\*(CB\*(FCdiv(\*(FIn\*(FR\*(FC,\*(FI d\*(FR\*(FC,\*(FI res\*(FR\*(FC)\*(FR T{ -Return the result of integer division in \*(FIres\*(FR.\*(CD -T} \*(FCexp(\*(FIexpr\*(FC)\*(FR The exponential function (\*(FIe \*(FC^ \*(FIx\*(FR). \*(FCint(\*(FIexpr\*(FC)\*(FR Truncate to integer. +\*(CB\*(FCintdiv(\*(FIn\*(FR\*(FC,\*(FI d\*(FR\*(FC,\*(FI res\*(FR\*(FC)\*(FR T{ +Return the result of integer division in \*(FIres\*(FR.\*(CD +T} \*(FClog(\*(FIexpr\*(FC)\*(FR The natural logarithm function (base \*(FIe\^\*(FR). \*(FCrand()\fP A random number \*(FIN\fP such that 0 \(<= \*(FIN\fP < 1. \*(FCsin(\*(FIexpr\*(FC)\*(FR The sine of \*(FIexpr\fP, which is in radians. @@ -13,7 +13,7 @@ . if \w'\(rq' .ds rq "\(rq . \} .\} -.TH GAWK 1 "Aug 03 2014" "Free Software Foundation" "Utility Commands" +.TH GAWK 1 "Apr 02 2015" "Free Software Foundation" "Utility Commands" .SH NAME gawk \- pattern scanning and processing language .SH SYNOPSIS @@ -231,7 +231,8 @@ and so on.) .PD \fB\-\^\-debug\fR[\fB=\fIfile\fR] Enable debugging of \*(AK programs. -By default, the debugger reads commands interactively from the terminal. +By default, the debugger reads commands interactively from the keyboard +(standard input). The optional .IR file argument specifies a file with a list @@ -626,7 +627,7 @@ specifies a search path to use when finding source files named with the .B \-l option. If this variable does not exist, the default path is -\fB".:/usr/local/lib/gawk"\fR. +\fB"/usr/local/lib/gawk"\fR. (The actual directory may vary, depending upon how .I gawk was built and installed.) @@ -941,6 +942,15 @@ then will contain a string describing the error. The value is subject to translation in non-English locales. +If the string in +.B ERRNO +corresponds to a system error in the +.IR errno (3) +variable, then the numeric value can be found in +.B PROCINFO["errno"]. +For non-system errors, +.B PROCINFO["errno"] +will be zero. .TP .B FIELDWIDTHS A whitespace separated list of field widths. When set, @@ -1102,6 +1112,13 @@ system call. The default time format string for .BR strftime() . .TP +\fBPROCINFO["errno"]\fP +The value of +.IR errno (3) +when +.BR ERRNO +is set to the associated error message. +.TP \fBPROCINFO["euid"]\fP The value of the .IR geteuid (2) @@ -1220,6 +1237,28 @@ where is a redirection string or a filename. A value of zero or less than zero means no timeout. .TP +\fBPROCINFO["input", "RETRY"]\fP +If an I/O error that may be retried occurs when reading data from +.IR input , +and this array entry exists, then +.B getline +will return \-2 instead of following the default behavior of returning \-1 +and configuring +.IR input +to return no further data. +An I/O error that may be retried is one where +.IR errno (3) +has the value +.IR EAGAIN , +.IR EWOULDBLOCK , +.IR EINTR , +or +.IR ETIMEDOUT . +This may be useful in conjunction with +\fBPROCINFO["input", "READ_TIMEOUT"]\fP +or situations where a file descriptor has been configured to behave in a +non-blocking fashion. +.TP \fBPROCINFO["mpfr_version"]\fP The version of the GNU MPFR library used for arbitrary precision number support in @@ -2288,6 +2327,13 @@ below.) The .B getline command returns 1 on success, 0 on end of file, and \-1 on an error. +If the +.IR errno (3) +value indicates that the I/O operation may be retried, +and \fBPROCINFO["input", "RETRY"]\fP +is set, then \-2 will be returned instead of \-1, and further calls to +.B getline +may be attempted. Upon an error, .B ERRNO is set to a string describing the problem. @@ -2634,7 +2680,13 @@ Return the cosine of .IR expr , which is in radians. .TP -.BI div( num ", " denom ", " result ) +.BI exp( expr ) +The exponential function. +.TP +.BI int( expr ) +Truncate to integer. +.TP +.BI intdiv( num ", " denom ", " result ) Truncate .I num and @@ -2651,12 +2703,6 @@ This is a extension, primarily of value when working with arbitrarily large integers. .TP -.BI exp( expr ) -The exponential function. -.TP -.BI int( expr ) -Truncate to integer. -.TP .BI log( expr ) The natural logarithm function. .TP diff --git a/doc/gawk.info b/doc/gawk.info index 4e8b653a..685df45b 100644 --- a/doc/gawk.info +++ b/doc/gawk.info @@ -9,7 +9,7 @@ START-INFO-DIR-ENTRY * awk: (gawk)Invoking gawk. Text scanning and processing. END-INFO-DIR-ENTRY - Copyright (C) 1989, 1991, 1992, 1993, 1996-2005, 2007, 2009-2014 + Copyright (C) 1989, 1991, 1992, 1993, 1996-2005, 2007, 2009-2015 Free Software Foundation, Inc. @@ -29,7 +29,7 @@ entitled "GNU Free Documentation License". modify this GNU manual." -File: gawk.info, Node: Top, Next: Foreword, Up: (dir) +File: gawk.info, Node: Top, Next: Foreword3, Up: (dir) General Introduction ******************** @@ -37,7 +37,7 @@ General Introduction This file documents `awk', a program that you can use to select particular records in a file and perform operations upon them. - Copyright (C) 1989, 1991, 1992, 1993, 1996-2005, 2007, 2009-2014 + Copyright (C) 1989, 1991, 1992, 1993, 1996-2005, 2007, 2009-2015 Free Software Foundation, Inc. @@ -58,8 +58,9 @@ entitled "GNU Free Documentation License". * Menu: -* Foreword:: Some nice words about this +* Foreword3:: Some nice words about this Info file. +* Foreword4:: More nice words. * Preface:: What this Info file is about; brief history and acknowledgments. * Getting Started:: A basic introduction to using @@ -216,6 +217,7 @@ entitled "GNU Free Documentation License". `getline'. * Getline Summary:: Summary of `getline' Variants. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -245,6 +247,7 @@ entitled "GNU Free Documentation License". * Special Caveats:: Things to watch out for. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. * Values:: Constants, Variables, and Regular @@ -556,6 +559,7 @@ entitled "GNU Free Documentation License". * Array Functions:: Functions for working with arrays. * Flattening Arrays:: How to flatten arrays. * Creating Arrays:: How to create and populate arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension Versioning:: API Version information. * Extension API Informational Variables:: Variables providing information about @@ -614,6 +618,7 @@ entitled "GNU Free Documentation License". * Unix Installation:: Installing `gawk' under various versions of Unix. * Quick Installation:: Compiling `gawk' under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. * Non-Unix Installation:: Installation on Other Operating @@ -679,34 +684,34 @@ your life together with me. lives in innumerable ways. -File: gawk.info, Node: Foreword, Next: Preface, Prev: Top, Up: Top +File: gawk.info, Node: Foreword3, Next: Foreword4, Prev: Top, Up: Top -Foreword -******** +Foreword to the Third Edition +***************************** Arnold Robbins and I are good friends. We were introduced in 1990 by circumstances--and our favorite programming language, AWK. The circumstances started a couple of years earlier. I was working at a new job and noticed an unplugged Unix computer sitting in the corner. No one knew how to use it, and neither did I. However, a couple of days -later it was running, and I was `root' and the one-and-only user. That -day, I began the transition from statistician to Unix programmer. +later, it was running, and I was `root' and the one-and-only user. +That day, I began the transition from statistician to Unix programmer. On one of many trips to the library or bookstore in search of books -on Unix, I found the gray AWK book, a.k.a. Aho, Kernighan and -Weinberger, `The AWK Programming Language', Addison-Wesley, 1988. -AWK's simple programming paradigm--find a pattern in the input and then -perform an action--often reduced complex or tedious data manipulations -to a few lines of code. I was excited to try my hand at programming in -AWK. +on Unix, I found the gray AWK book, a.k.a. Alfred V. Aho, Brian W. +Kernighan, and Peter J. Weinberger's `The AWK Programming Language' +(Addison-Wesley, 1988). `awk''s simple programming paradigm--find a +pattern in the input and then perform an action--often reduced complex +or tedious data manipulations to a few lines of code. I was excited to +try my hand at programming in AWK. Alas, the `awk' on my computer was a limited version of the -language described in the AWK book. I discovered that my computer had -"old `awk'" and the AWK book described "new `awk'." I learned that -this was typical; the old version refused to step aside or relinquish -its name. If a system had a new `awk', it was invariably called -`nawk', and few systems had it. The best way to get a new `awk' was to -`ftp' the source code for `gawk' from `prep.ai.mit.edu'. `gawk' was a +language described in the gray book. I discovered that my computer had +"old `awk'" and the book described "new `awk'." I learned that this +was typical; the old version refused to step aside or relinquish its +name. If a system had a new `awk', it was invariably called `nawk', +and few systems had it. The best way to get a new `awk' was to `ftp' +the source code for `gawk' from `prep.ai.mit.edu'. `gawk' was a version of new `awk' written by David Trueman and Arnold, and available under the GNU General Public License. @@ -717,14 +722,15 @@ almost any system; my wife uses `gawk' on her VMS box.) My Unix system started out unplugged from the wall; it certainly was not plugged into a network. So, oblivious to the existence of `gawk' and the Unix community in general, and desiring a new `awk', I wrote my -own, called `mawk'. Before I was finished I knew about `gawk', but it +own, called `mawk'. Before I was finished, I knew about `gawk', but it was too late to stop, so I eventually posted to a `comp.sources' newsgroup. A few days after my posting, I got a friendly email from Arnold introducing himself. He suggested we share design and algorithms and attached a draft of the POSIX standard so that I could update `mawk' to -support language extensions added after publication of the AWK book. +support language extensions added after publication of `The AWK +Programming Language'. Frankly, if our roles had been reversed, I would not have been so open and we probably would have never met. I'm glad we did meet. He @@ -738,17 +744,17 @@ a definitive reference to the AWK language as defined by the 1987 Bell Laboratories release and codified in the 1992 POSIX Utilities standard. On the other hand, the novice AWK programmer can study a wealth of -practical programs that emphasize the power of AWK's basic idioms: data -driven control-flow, pattern matching with regular expressions, and -associative arrays. Those looking for something new can try out +practical programs that emphasize the power of AWK's basic idioms: +data-driven control flow, pattern matching with regular expressions, +and associative arrays. Those looking for something new can try out `gawk''s interface to network protocols via special `/inet' files. The programs in this book make clear that an AWK program is typically much smaller and faster to develop than a counterpart written -in C. Consequently, there is often a payoff to prototype an algorithm -or design in AWK to get it running quickly and expose problems early. -Often, the interpreted performance is adequate and the AWK prototype -becomes the product. +in C. Consequently, there is often a payoff to prototyping an +algorithm or design in AWK to get it running quickly and expose +problems early. Often, the interpreted performance is adequate and the +AWK prototype becomes the product. The new `pgawk' (profiling `gawk'), produces program execution counts. I recently experimented with an algorithm that for n lines of @@ -763,10 +769,47 @@ want to learn how, then read this book. Michael Brennan Author of `mawk' - March, 2001 + March 2001 + + +File: gawk.info, Node: Foreword4, Next: Preface, Prev: Foreword3, Up: Top + +Foreword to the Fourth Edition +****************************** + +Some things don't change. Thirteen years ago I wrote: "If you use AWK +or want to learn how, then read this book." True then, and still true +today. + + Learning to use a programming language is about more than mastering +the syntax. One needs to acquire an understanding of how to use the +features of the language to solve practical programming problems. A +focus of this book is many examples that show how to use AWK. + + Some things do change. Our computers are much faster and have more +memory. Consequently, speed and storage inefficiencies of a high-level +language matter less. Prototyping in AWK and then rewriting in C for +performance reasons happens less, because more often the prototype is +fast enough. + + Of course, there are computing operations that are best done in C or +C++. With `gawk' 4.1 and later, you do not have to choose between +writing your program in AWK or in C/C++. You can write most of your +program in AWK and the aspects that require C/C++ capabilities can be +written in C/C++, and then the pieces glued together when the `gawk' +module loads the C/C++ module as a dynamic plug-in. *note Dynamic +Extensions::, has all the details, and, as expected, many examples to +help you learn the ins and outs. + + I enjoy programming in AWK and had fun (re)reading this book. I +think you will too. + + Michael Brennan + Author of `mawk' + October 2014 -File: gawk.info, Node: Preface, Next: Getting Started, Prev: Foreword, Up: Top +File: gawk.info, Node: Preface, Next: Getting Started, Prev: Foreword4, Up: Top Preface ******* @@ -779,12 +822,12 @@ rest of the file alone. Such jobs are often easy with `awk'. The makes it easy to handle simple data-reformatting jobs. The GNU implementation of `awk' is called `gawk'; if you invoke it -with the proper options or environment variables it is fully compatible -with the POSIX(1) specification of the `awk' language and with the Unix -version of `awk' maintained by Brian Kernighan. This means that all -properly written `awk' programs should work with `gawk'. So most of -the time, we don't distinguish between `gawk' and other `awk' -implementations. +with the proper options or environment variables, it is fully +compatible with the POSIX(1) specification of the `awk' language and +with the Unix version of `awk' maintained by Brian Kernighan. This +means that all properly written `awk' programs should work with `gawk'. +So most of the time, we don't distinguish between `gawk' and other +`awk' implementations. Using `awk' you can: @@ -794,7 +837,7 @@ implementations. * Validate data - * Produce indexes and perform other document preparation tasks + * Produce indexes and perform other document-preparation tasks * Experiment with algorithms that you can adapt later to other computer languages @@ -857,7 +900,7 @@ File: gawk.info, Node: History, Next: Names, Up: Preface History of `awk' and `gawk' =========================== - Recipe For A Programming Language + Recipe for a Programming Language 1 part `egrep' 1 part `snobol' 2 parts `ed' 3 parts C @@ -869,7 +912,7 @@ release. Document very well and release. The name `awk' comes from the initials of its designers: Alfred V. -Aho, Peter J. Weinberger and Brian W. Kernighan. The original version +Aho, Peter J. Weinberger, and Brian W. Kernighan. The original version of `awk' was written in 1977 at AT&T Bell Laboratories. In 1985, a new version made the programming language more powerful, introducing user-defined functions, multiple input streams, and computed regular @@ -886,10 +929,10 @@ advice from Richard Stallman. John Woods contributed parts of the code as well. In 1988 and 1989, David Trueman, with help from me, thoroughly reworked `gawk' for compatibility with the newer `awk'. Circa 1994, I became the primary maintainer. Current development -focuses on bug fixes, performance improvements, standards compliance +focuses on bug fixes, performance improvements, standards compliance, and, occasionally, new features. - In May of 1997, Ju"rgen Kahrs felt the need for network access from + In May 1997, Ju"rgen Kahrs felt the need for network access from `awk', and with a little help from me, set about adding features to do this for `gawk'. At that time, he also wrote the bulk of `TCP/IP Internetworking with `gawk'' (a separate document, available as part of @@ -898,10 +941,10 @@ the `gawk' distribution). His code finally became part of the main John Haque rewrote the `gawk' internals, in the process providing an `awk'-level debugger. This version became available as `gawk' version -4.0, in 2011. +4.0 in 2011. - *Note Contributors::, for a full list of those who made important -contributions to `gawk'. + *Note Contributors::, for a full list of those who have made +important contributions to `gawk'. File: gawk.info, Node: Names, Next: This Manual, Prev: History, Up: Preface @@ -911,10 +954,10 @@ A Rose by Any Other Name The `awk' language has evolved over the years. Full details are provided in *note Language History::. The language described in this -Info file is often referred to as "new `awk'". By analogy, the +Info file is often referred to as "new `awk'." By analogy, the original version of `awk' is referred to as "old `awk'." - Today, on most systems, when you run the `awk' utility, you get some + On most current systems, when you run the `awk' utility you get some version of new `awk'.(1) If your system's standard `awk' is the old one, you will see something like this if you try the test program: @@ -966,110 +1009,120 @@ heading "sidebar." the more advanced sections show only the part of the `awk' program that illustrates the concept being described. - While this Info file is aimed principally at people who have not been -exposed to `awk', there is a lot of information here that even the `awk' -expert should find useful. In particular, the description of POSIX -`awk' and the example programs in *note Library Functions::, and in -*note Sample Programs::, should be of interest. + Although this Info file is aimed principally at people who have not +been exposed to `awk', there is a lot of information here that even the +`awk' expert should find useful. In particular, the description of +POSIX `awk' and the example programs in *note Library Functions::, and +in *note Sample Programs::, should be of interest. This Info file is split into several parts, as follows: - Part I describes the `awk' language and `gawk' program in detail. -It starts with the basics, and continues through all of the features of -`awk'. It contains the following chapters: + * Part I describes the `awk' language and the `gawk' program in + detail. It starts with the basics, and continues through all of + the features of `awk'. It contains the following chapters: - *note Getting Started::, provides the essentials you need to know to -begin using `awk'. + - *note Getting Started::, provides the essentials you need to + know to begin using `awk'. - *note Invoking Gawk::, describes how to run `gawk', the meaning of -its command-line options, and how it finds `awk' program source files. + - *note Invoking Gawk::, describes how to run `gawk', the + meaning of its command-line options, and how it finds `awk' + program source files. - *note Regexp::, introduces regular expressions in general, and in -particular the flavors supported by POSIX `awk' and `gawk'. + - *note Regexp::, introduces regular expressions in general, + and in particular the flavors supported by POSIX `awk' and + `gawk'. - *note Reading Files::, describes how `awk' reads your data. It -introduces the concepts of records and fields, as well as the `getline' -command. I/O redirection is first described here. Network I/O is also -briefly introduced here. + - *note Reading Files::, describes how `awk' reads your data. + It introduces the concepts of records and fields, as well as + the `getline' command. I/O redirection is first described + here. Network I/O is also briefly introduced here. - *note Printing::, describes how `awk' programs can produce output -with `print' and `printf'. + - *note Printing::, describes how `awk' programs can produce + output with `print' and `printf'. - *note Expressions::, describes expressions, which are the basic -building blocks for getting most things done in a program. + - *note Expressions::, describes expressions, which are the + basic building blocks for getting most things done in a + program. - *note Patterns and Actions::, describes how to write patterns for -matching records, actions for doing something when a record is matched, -and the predefined variables `awk' and `gawk' use. + - *note Patterns and Actions::, describes how to write patterns + for matching records, actions for doing something when a + record is matched, and the predefined variables `awk' and + `gawk' use. - *note Arrays::, covers `awk''s one-and-only data structure: -associative arrays. Deleting array elements and whole arrays is also -described, as well as sorting arrays in `gawk'. It also describes how -`gawk' provides arrays of arrays. + - *note Arrays::, covers `awk''s one-and-only data structure: + the associative array. Deleting array elements and whole + arrays is described, as well as sorting arrays in `gawk'. + The major node also describes how `gawk' provides arrays of + arrays. - *note Functions::, describes the built-in functions `awk' and `gawk' -provide, as well as how to define your own functions. It also -discusses how `gawk' lets you call functions indirectly. + - *note Functions::, describes the built-in functions `awk' and + `gawk' provide, as well as how to define your own functions. + It also discusses how `gawk' lets you call functions + indirectly. - Part II shows how to use `awk' and `gawk' for problem solving. -There is lots of code here for you to read and learn from. It contains -the following chapters: + * Part II shows how to use `awk' and `gawk' for problem solving. + There is lots of code here for you to read and learn from. This + part contains the following chapters: - *note Library Functions::, which provides a number of functions -meant to be used from main `awk' programs. + - *note Library Functions::, provides a number of functions + meant to be used from main `awk' programs. - *note Sample Programs::, which provides many sample `awk' programs. + - *note Sample Programs::, provides many sample `awk' programs. - Reading these two chapters allows you to see `awk' solving real -problems. + Reading these two chapters allows you to see `awk' solving real + problems. - Part III focuses on features specific to `gawk'. It contains the -following chapters: + * Part III focuses on features specific to `gawk'. It contains the + following chapters: - *note Advanced Features::, describes a number of advanced features. -Of particular note are the abilities to control the order of array -traversal, have two-way communications with another process, perform -TCP/IP networking, and profile your `awk' programs. + - *note Advanced Features::, describes a number of advanced + features. Of particular note are the abilities to control + the order of array traversal, have two-way communications + with another process, perform TCP/IP networking, and profile + your `awk' programs. - *note Internationalization::, describes special features for -translating program messages into different languages at runtime. + - *note Internationalization::, describes special features for + translating program messages into different languages at + runtime. - *note Debugger::, describes the `gawk' debugger. + - *note Debugger::, describes the `gawk' debugger. - *note Arbitrary Precision Arithmetic::, describes advanced -arithmetic facilities. + - *note Arbitrary Precision Arithmetic::, describes advanced + arithmetic facilities. - *note Dynamic Extensions::, describes how to add new variables and -functions to `gawk' by writing extensions in C or C++. + - *note Dynamic Extensions::, describes how to add new + variables and functions to `gawk' by writing extensions in C + or C++. - Part IV provides the appendices, the Glossary, and two licenses that -cover the `gawk' source code and this Info file, respectively. It -contains the following appendices: + * Part IV provides the appendices, the Glossary, and two licenses + that cover the `gawk' source code and this Info file, respectively. + It contains the following appendices: - *note Language History::, describes how the `awk' language has -evolved since its first release to present. It also describes how -`gawk' has acquired features over time. + - *note Language History::, describes how the `awk' language + has evolved since its first release to the present. It also + describes how `gawk' has acquired features over time. - *note Installation::, describes how to get `gawk', how to compile it -on POSIX-compatible systems, and how to compile and use it on different -non-POSIX systems. It also describes how to report bugs in `gawk' and -where to get other freely available `awk' implementations. + - *note Installation::, describes how to get `gawk', how to + compile it on POSIX-compatible systems, and how to compile + and use it on different non-POSIX systems. It also describes + how to report bugs in `gawk' and where to get other freely + available `awk' implementations. - *note Notes::, describes how to disable `gawk''s extensions, as well -as how to contribute new code to `gawk', and some possible future -directions for `gawk' development. + - *note Notes::, describes how to disable `gawk''s extensions, + as well as how to contribute new code to `gawk', and some + possible future directions for `gawk' development. - *note Basic Concepts::, provides some very cursory background -material for those who are completely unfamiliar with computer -programming. + - *note Basic Concepts::, provides some very cursory background + material for those who are completely unfamiliar with + computer programming. - The *note Glossary::, defines most, if not all, the significant -terms used throughout the Info file. If you find terms that you aren't -familiar with, try looking them up here. + The *note Glossary::, defines most, if not all, of the + significant terms used throughout the Info file. If you find + terms that you aren't familiar with, try looking them up here. - *note Copying::, and *note GNU Free Documentation License::, present -the licenses that cover the `gawk' source code and this Info file, -respectively. + - *note Copying::, and *note GNU Free Documentation License::, + present the licenses that cover the `gawk' source code and + this Info file, respectively. ---------- Footnotes ---------- @@ -1092,8 +1145,8 @@ node briefly documents the typographical conventions used in Texinfo. common shell primary and secondary prompts, `$' and `>'. Input that you type is shown `like this'. Output from the command is preceded by the glyph "-|". This typically represents the command's standard -output. Error messages, and other output on the command's standard -error, are preceded by the glyph "error-->". For example: +output. Error messages and other output on the command's standard +error are preceded by the glyph "error-->". For example: $ echo hi on stdout -| hi on stdout @@ -1105,7 +1158,7 @@ particular, there are special characters called "control characters." These are characters that you type by holding down both the `CONTROL' key and another key, at the same time. For example, a `Ctrl-d' is typed by first pressing and holding the `CONTROL' key, next pressing the `d' -key and finally releasing both keys. +key, and finally releasing both keys. For the sake of brevity, throughout this Info file, we refer to Brian Kernighan's version of `awk' as "BWK `awk'." (*Note Other @@ -1114,14 +1167,14 @@ Versions::, for information on his and other versions.) Dark Corners ------------ - Dark corners are basically fractal -- no matter how much you + Dark corners are basically fractal--no matter how much you illuminate, there's always a smaller but darker one. -- Brian Kernighan Until the POSIX standard (and `GAWK: Effective AWK Programming'), many features of `awk' were either poorly documented or not documented at all. Descriptions of such features (often called "dark corners") -are noted in this Info file with "(d.c.)". They also appear in the +are noted in this Info file with "(d.c.)." They also appear in the index under the heading "dark corner." But, as noted by the opening quote, any coverage of dark corners is @@ -1144,13 +1197,13 @@ editor. GNU Emacs is the most widely used version of Emacs today. The GNU(1) Project is an ongoing effort on the part of the Free Software Foundation to create a complete, freely distributable, -POSIX-compliant computing environment. The FSF uses the "GNU General -Public License" (GPL) to ensure that their software's source code is +POSIX-compliant computing environment. The FSF uses the GNU General +Public License (GPL) to ensure that its software's source code is always available to the end user. A copy of the GPL is included for your reference (*note Copying::). The GPL applies to the C language source code for `gawk'. To find out more about the FSF and the GNU Project online, see the GNU Project's home page (http://www.gnu.org). -This Info file may also be read from their web site +This Info file may also be read from GNU's website (http://www.gnu.org/software/gawk/manual/). A shell, an editor (Emacs), highly portable optimizing C, C++, and @@ -1167,32 +1220,32 @@ from the Internet. The Info file itself has gone through multiple previous editions. Paul Rubin wrote the very first draft of `The GAWK Manual'; it was -around 40 pages in size. Diane Close and Richard Stallman improved it, -yielding a version that was around 90 pages long and barely described -the original, "old" version of `awk'. +around 40 pages long. Diane Close and Richard Stallman improved it, +yielding a version that was around 90 pages and barely described the +original, "old" version of `awk'. I started working with that version in the fall of 1988. As work on it progressed, the FSF published several preliminary versions (numbered -0.X). In 1996, Edition 1.0 was released with `gawk' 3.0.0. The FSF +0.X). In 1996, edition 1.0 was released with `gawk' 3.0.0. The FSF published the first two editions under the title `The GNU Awk User's Guide'. This edition maintains the basic structure of the previous editions. For FSF edition 4.0, the content was thoroughly reviewed and updated. All references to `gawk' versions prior to 4.0 were removed. Of -significant note for that edition was *note Debugger::. +significant note for that edition was the addition of *note Debugger::. For FSF edition 4.1, the content has been reorganized into parts, and the major new additions are *note Arbitrary Precision Arithmetic::, and *note Dynamic Extensions::. This Info file will undoubtedly continue to evolve. If you find an -error in this Info file, please report it! *Note Bugs::, for +error in the Info file, please report it! *Note Bugs::, for information on submitting problem reports electronically. ---------- Footnotes ---------- - (1) GNU stands for "GNU's not Unix." + (1) GNU stands for "GNU's Not Unix." (2) The terminology "GNU/Linux" is explained in the *note Glossary::. @@ -1216,15 +1269,12 @@ contributed code: the archive did not grow and the domain went unused for several years. Late in 2008, a volunteer took on the task of setting up an -`awk'-related web site--`http://awk.info'--and did a very nice job. +`awk'-related website--`http://awk.info'--and did a very nice job. If you have written an interesting `awk' program, or have written a `gawk' extension that you would like to share with the rest of the world, please see `http://awk.info/?contribute' for how to contribute -it to the web site. - - As of this writing, this website is in search of a maintainer; please -contact me if you are interested. +it to the website. File: gawk.info, Node: Acknowledgments, Prev: How To Contribute, Up: Preface @@ -1239,7 +1289,7 @@ acknowledgments: this manual. Jay Fenlason contributed many ideas and sample programs. Richard Mlynarik and Robert Chassell gave helpful comments on drafts of this manual. The paper `A Supplemental - Document for `awk'' by John W. Pierce of the Chemistry Department + Document for AWK' by John W. Pierce of the Chemistry Department at UC San Diego, pinpointed several issues relevant both to `awk' implementation and to this manual, that would otherwise have escaped us. @@ -1252,7 +1302,7 @@ GNU Project. acknowledgements: The following people (in alphabetical order) provided helpful - comments on various versions of this book, Rick Adams, Dr. Nelson + comments on various versions of this book: Rick Adams, Dr. Nelson H.F. Beebe, Karl Berry, Dr. Michael Brennan, Rich Burridge, Claire Cloutier, Diane Close, Scott Deifik, Christopher ("Topher") Eliot, Jeffrey Friedl, Dr. Darrel Hankerson, Michal Jaegermann, Dr. @@ -1261,7 +1311,7 @@ acknowledgements: Robert J. Chassell provided much valuable advice on the use of Texinfo. He also deserves special thanks for convincing me _not_ - to title this Info file `How To Gawk Politely'. Karl Berry helped + to title this Info file `How to Gawk Politely'. Karl Berry helped significantly with the TeX part of Texinfo. I would like to thank Marshall and Elaine Hartholz of Seattle and @@ -1298,25 +1348,25 @@ continues to be a pleasure working with this team of fine people. Notable code and documentation contributions were made by a number of people. *Note Contributors::, for the full list. - Thanks to Michael Brennan for the Foreword. + Thanks to Michael Brennan for the Forewords. Thanks to Patrice Dumas for the new `makeinfo' program. Thanks to -Karl Berry who continues to work to keep the Texinfo markup language +Karl Berry, who continues to work to keep the Texinfo markup language sane. - Robert P.J. Day, Michael Brennan and Brian Kernighan kindly acted as + Robert P.J. Day, Michael Brennan, and Brian Kernighan kindly acted as reviewers for the 2015 edition of this Info file. Their feedback helped improve the final work. - I would like to thank Brian Kernighan for invaluable assistance -during the testing and debugging of `gawk', and for ongoing help and -advice in clarifying numerous points about the language. We could not -have done nearly as good a job on either `gawk' or its documentation -without his help. + I would also like to thank Brian Kernighan for his invaluable +assistance during the testing and debugging of `gawk', and for his +ongoing help and advice in clarifying numerous points about the +language. We could not have done nearly as good a job on either `gawk' +or its documentation without his help. Brian is in a class by himself as a programmer and technical author. -I have to thank him (yet again) for his ongoing friendship and the role -model he has been for me for close to 30 years! Having him as a +I have to thank him (yet again) for his ongoing friendship and for +being a role model to me for close to 30 years! Having him as a reviewer is an exciting privilege. It has also been extremely humbling... @@ -1328,6 +1378,12 @@ also must acknowledge my gratitude to G-d, for the many opportunities He has sent my way, as well as for the gifts He has given me with which to take advantage of those opportunities. + +Arnold Robbins +Nof Ayalon +Israel +February 2015 + File: gawk.info, Node: Getting Started, Next: Invoking Gawk, Prev: Preface, Up: Top @@ -1337,28 +1393,28 @@ File: gawk.info, Node: Getting Started, Next: Invoking Gawk, Prev: Preface, The basic function of `awk' is to search files for lines (or other units of text) that contain certain patterns. When a line matches one of the patterns, `awk' performs specified actions on that line. `awk' -keeps processing input lines in this way until it reaches the end of -the input files. +continues to process input lines in this way until it reaches the end +of the input files. Programs in `awk' are different from programs in most other -languages, because `awk' programs are "data-driven"; that is, you -describe the data you want to work with and then what to do when you -find it. Most other languages are "procedural"; you have to describe, -in great detail, every step the program is to take. When working with +languages, because `awk' programs are "data driven" (i.e., you describe +the data you want to work with and then what to do when you find it). +Most other languages are "procedural"; you have to describe, in great +detail, every step the program should take. When working with procedural languages, it is usually much harder to clearly describe the data your program will process. For this reason, `awk' programs are often refreshingly easy to read and write. When you run `awk', you specify an `awk' "program" that tells `awk' -what to do. The program consists of a series of "rules". (It may also +what to do. The program consists of a series of "rules" (it may also contain "function definitions", an advanced feature that we will ignore -for now. *Note User-defined::.) Each rule specifies one pattern to +for now; *note User-defined::). Each rule specifies one pattern to search for and one action to perform upon finding the pattern. - Syntactically, a rule consists of a pattern followed by an action. -The action is enclosed in braces to separate it from the pattern. -Newlines usually separate rules. Therefore, an `awk' program looks -like this: + Syntactically, a rule consists of a "pattern" followed by an +"action". The action is enclosed in braces to separate it from the +pattern. Newlines usually separate rules. Therefore, an `awk' program +looks like this: PATTERN { ACTION } PATTERN { ACTION } @@ -1426,7 +1482,7 @@ program as the first argument of the `awk' command, like this: awk 'PROGRAM' INPUT-FILE1 INPUT-FILE2 ... -where PROGRAM consists of a series of PATTERNS and ACTIONS, as +where PROGRAM consists of a series of patterns and actions, as described earlier. This command format instructs the "shell", or command interpreter, @@ -1441,7 +1497,8 @@ programs from shell scripts, because it avoids the need for a separate file for the `awk' program. A self-contained shell script is more reliable because there are no other files to misplace. - *note Very Simple::, presents several short, self-contained programs. + Later in this chapter, in *note Very Simple::, we'll see examples of +several short, self-contained programs. File: gawk.info, Node: Read Terminal, Next: Long, Prev: One-shot, Up: Running gawk @@ -1456,7 +1513,7 @@ following command line: `awk' applies the PROGRAM to the "standard input", which usually means whatever you type on the keyboard. This continues until you indicate -end-of-file by typing `Ctrl-d'. (On other operating systems, the +end-of-file by typing `Ctrl-d'. (On non-POSIX operating systems, the end-of-file character may be different. For example, on OS/2, it is `Ctrl-z'.) @@ -1483,7 +1540,7 @@ ugly shell quoting tricks. This next simple `awk' program emulates the `cat' utility; it copies whatever you type on the keyboard to its standard output (why this -works is explained shortly). +works is explained shortly): $ awk '{ print }' Now is the time for all good men @@ -1531,10 +1588,9 @@ that are provided on the `awk' command line. (Also, placing the program in a file allows us to use a literal single quote in the program text, instead of the magic `\47'.) - If you want to clearly identify your `awk' program files as such, -you can add the extension `.awk' to the file name. This doesn't affect -the execution of the `awk' program but it does make "housekeeping" -easier. + If you want to clearly identify an `awk' program file as such, you +can add the extension `.awk' to the file name. This doesn't affect the +execution of the `awk' program but it does make "housekeeping" easier. File: gawk.info, Node: Executable Scripts, Next: Comments, Prev: Long, Up: Running gawk @@ -1575,7 +1631,7 @@ the instructions in your program. (This is different from a "compiled" language such as C, where your program is first compiled into machine code that is executed directly by your system's processor.) The `awk' utility is thus termed an "interpreter". Many modern languages are -interperted. +interpreted. The line beginning with `#!' lists the full file name of an interpreter to run and a single optional initial command-line argument @@ -1603,8 +1659,8 @@ the name of your script (`advice'). (d.c.) Don't rely on the value of ---------- Footnotes ---------- - (1) The `#!' mechanism works on GNU/Linux systems, BSD-based systems -and commercial Unix systems. + (1) The `#!' mechanism works on GNU/Linux systems, BSD-based +systems, and commercial Unix systems. File: gawk.info, Node: Comments, Next: Quoting, Prev: Executable Scripts, Up: Running gawk @@ -1618,13 +1674,13 @@ Comments can explain what the program does and how it works. Nearly all programming languages have provisions for comments, as programs are typically hard to understand without them. - In the `awk' language, a comment starts with the sharp sign + In the `awk' language, a comment starts with the number sign character (`#') and continues to the end of the line. The `#' does not have to be the first character on the line. The `awk' language ignores -the rest of a line following a sharp sign. For example, we could have +the rest of a line following a number sign. For example, we could have put the following into `advice': - # This program prints a nice friendly message. It helps + # This program prints a nice, friendly message. It helps # keep novice users from being afraid of the computer. BEGIN { print "Don't Panic!" } @@ -1633,15 +1689,15 @@ programs, but this usually isn't very useful; the purpose of a comment is to help you or another person understand the program when reading it at a later time. - CAUTION: As mentioned in *note One-shot::, you can enclose small - to medium programs in single quotes, in order to keep your shell - scripts self-contained. When doing so, _don't_ put an apostrophe - (i.e., a single quote) into a comment (or anywhere else in your - program). The shell interprets the quote as the closing quote for - the entire program. As a result, usually the shell prints a - message about mismatched quotes, and if `awk' actually runs, it - will probably print strange messages about syntax errors. For - example, look at the following: + CAUTION: As mentioned in *note One-shot::, you can enclose short + to medium-sized programs in single quotes, in order to keep your + shell scripts self-contained. When doing so, _don't_ put an + apostrophe (i.e., a single quote) into a comment (or anywhere else + in your program). The shell interprets the quote as the closing + quote for the entire program. As a result, usually the shell + prints a message about mismatched quotes, and if `awk' actually + runs, it will probably print strange messages about syntax errors. + For example, look at the following: $ awk 'BEGIN { print "hello" } # let's be cute' > @@ -1657,20 +1713,20 @@ at a later time. error--> source line number 1 Putting a backslash before the single quote in `let's' wouldn't - help, since backslashes are not special inside single quotes. The - next node describes the shell's quoting rules. + help, because backslashes are not special inside single quotes. + The next node describes the shell's quoting rules. File: gawk.info, Node: Quoting, Prev: Comments, Up: Running gawk -1.1.6 Shell-Quoting Issues +1.1.6 Shell Quoting Issues -------------------------- * Menu: * DOS Quoting:: Quoting in Windows Batch Files. - For short to medium length `awk' programs, it is most convenient to + For short to medium-length `awk' programs, it is most convenient to enter the program on the `awk' command line. This is best done by enclosing the entire program in single quotes. This is true whether you are entering the program interactively at the shell prompt, or @@ -1690,15 +1746,15 @@ string. The null string is character data that has no value. In other words, it is empty. It is written in `awk' programs like this: `""'. In the shell, it can be written using single or double quotes: `""' or -`'''. While the null string has no characters in it, it does exist. -Consider this command: +`'''. Although the null string has no characters in it, it does exist. +For example, consider this command: $ echo "" Here, the `echo' utility receives a single argument, even though that argument has no characters in it. In the rest of this Info file, we use the terms "null string" and "empty string" interchangeably. Now, on to -the quoting rules. +the quoting rules: * Quoted items can be concatenated with nonquoted items as well as with other quoted items. The shell turns everything into one @@ -1719,7 +1775,7 @@ the quoting rules. on the quoted text. Different shells may do additional kinds of processing on double-quoted text. - Since certain characters within double-quoted text are processed + Because certain characters within double-quoted text are processed by the shell, they must be "escaped" within the text. Of note are the characters `$', ``', `\', and `"', all of which must be preceded by a backslash within double-quoted text if they are to @@ -1758,7 +1814,7 @@ shell quoting tricks, like this: -| Here is a single quote <'> This program consists of three concatenated quoted strings. The first -and the third are single-quoted, the second is double-quoted. +and the third are single-quoted, and the second is double-quoted. This can be "simplified" to: @@ -1785,8 +1841,7 @@ like so: $ awk 'BEGIN { print "Here is a double quote <\42>" }' -| Here is a double quote <"> -This works nicely, except that you should comment clearly what the -escapes mean. +This works nicely, but you should comment clearly what the escapes mean. A fourth option is to use command-line variable assignment, like this: @@ -1795,11 +1850,11 @@ this: -| Here is a single quote <'> (Here, the two string constants and the value of `sq' are -concatenated into a single string which is printed by `print'.) +concatenated into a single string that is printed by `print'.) If you really need both single and double quotes in your `awk' program, it is probably best to move it into a separate file, where the -shell won't be part of the picture, and you can say what you mean. +shell won't be part of the picture and you can say what you mean. File: gawk.info, Node: DOS Quoting, Up: Quoting @@ -1822,7 +1877,7 @@ file surrounded by double quotes: File: gawk.info, Node: Sample Data Files, Next: Very Simple, Prev: Running gawk, Up: Getting Started -1.2 Data Files for the Examples +1.2 Data files for the Examples =============================== Many of the examples in this Info file take their input from two sample @@ -1833,7 +1888,7 @@ about monthly shipments. In both files, each line is considered to be one "record". In `mail-list', each record contains the name of a person, his/her -phone number, his/her email-address, and a code for their relationship +phone number, his/her email address, and a code for his/her relationship with the author of the list. The columns are aligned using spaces. An `A' in the last column means that the person is an acquaintance. An `F' in the last column means that the person is a friend. An `R' means @@ -1857,7 +1912,7 @@ of green crates shipped, the number of red boxes shipped, the number of orange bags shipped, and the number of blue packages shipped, respectively. There are 16 entries, covering the 12 months of last year and the first four months of the current year. An empty line separates -the data for the two years. +the data for the two years: Jan 13 25 15 115 Feb 15 32 24 226 @@ -1889,7 +1944,7 @@ File: gawk.info, Node: Very Simple, Next: Two Rules, Prev: Sample Data Files, The following command runs a simple `awk' program that searches the input file `mail-list' for the character string `li' (a grouping of characters is usually called a "string"; the term "string" is based on -similar usage in English, such as "a string of pearls," or "a string of +similar usage in English, such as "a string of pearls" or "a string of cars in a train"): awk '/li/ { print $0 }' mail-list @@ -1925,24 +1980,25 @@ prints all lines matching the pattern `li'. By comparison, omitting the `print' statement but retaining the braces makes an empty action that does nothing (i.e., no lines are printed). - Many practical `awk' programs are just a line or two. Following is a -collection of useful, short programs to get you started. Some of these -programs contain constructs that haven't been covered yet. (The -description of the program will give you a good idea of what is going -on, but please read the rest of the Info file to become an `awk' -expert!) Most of the examples use a data file named `data'. This is -just a placeholder; if you use these programs yourself, substitute your -own file names for `data'. For future reference, note that there is -often more than one way to do things in `awk'. At some point, you may -want to look back at these examples and see if you can come up with -different ways to do the same things shown here: + Many practical `awk' programs are just a line or two long. +Following is a collection of useful, short programs to get you started. +Some of these programs contain constructs that haven't been covered +yet. (The description of the program will give you a good idea of what +is going on, but you'll need to read the rest of the Info file to +become an `awk' expert!) Most of the examples use a data file named +`data'. This is just a placeholder; if you use these programs +yourself, substitute your own file names for `data'. For future +reference, note that there is often more than one way to do things in +`awk'. At some point, you may want to look back at these examples and +see if you can come up with different ways to do the same things shown +here: * Print every line that is longer than 80 characters: awk 'length($0) > 80' data - The sole rule has a relational expression as its pattern and it - has no action--so it uses the default action, printing the record. + The sole rule has a relational expression as its pattern and has no + action--so it uses the default action, printing the record. * Print the length of the longest input line: @@ -1957,7 +2013,7 @@ different ways to do the same things shown here: expand data | awk '{ if (x < length($0)) x = length($0) } END { print "maximum line length is " x }' - This example differs slightly from the previous one: The input is + This example differs slightly from the previous one: the input is processed by the `expand' utility to change TABs into spaces, so the widths compared are actually the right-margin columns, as opposed to the number of input characters on each line. @@ -1997,8 +2053,8 @@ different ways to do the same things shown here: awk 'NR % 2 == 0' data - If you use the expression `NR % 2 == 1' instead, the program would - print the odd-numbered lines. + If you used the expression `NR % 2 == 1' instead, the program + would print the odd-numbered lines. File: gawk.info, Node: Two Rules, Next: More Complex, Prev: Very Simple, Up: Getting Started @@ -2151,10 +2207,10 @@ or a string. the C shell._ It works for `awk' programs in files and for one-shot programs, _provided_ you are using a POSIX-compliant shell, such as the Unix Bourne shell or Bash. But the C shell - behaves differently! There, you must use two backslashes in a - row, followed by a newline. Note also that when using the C - shell, _every_ newline in your `awk' program must be escaped with - a backslash. To illustrate: + behaves differently! There you must use two backslashes in a row, + followed by a newline. Note also that when using the C shell, + _every_ newline in your `awk' program must be escaped with a + backslash. To illustrate: % awk 'BEGIN { \ ? print \\ @@ -2231,9 +2287,10 @@ built-in functions for working with timestamps, performing bit manipulation, for runtime string translation (internationalization), determining the type of a variable, and array sorting. - As we develop our presentation of the `awk' language, we introduce -most of the variables and many of the functions. They are described -systematically in *note Built-in Variables::, and in *note Built-in::. + As we develop our presentation of the `awk' language, we will +introduce most of the variables and many of the functions. They are +described systematically in *note Built-in Variables::, and in *note +Built-in::. File: gawk.info, Node: When, Next: Intro Summary, Prev: Other Features, Up: Getting Started @@ -2259,8 +2316,8 @@ edit-compile-test-debug cycle of software development. Complex programs have been written in `awk', including a complete retargetable assembler for eight-bit microprocessors (*note Glossary::, for more information), and a microcode assembler for a special-purpose -Prolog computer. While the original `awk''s capabilities were strained -by tasks of such complexity, modern versions are more capable. +Prolog computer. The original `awk''s capabilities were strained by +tasks of such complexity, but modern versions are more capable. If you find yourself writing `awk' scripts of more than, say, a few hundred lines, you might consider using a different programming @@ -2298,7 +2355,7 @@ File: gawk.info, Node: Intro Summary, Prev: When, Up: Getting Started * You may use backslash continuation to continue a source line. Lines are automatically continued after a comma, open brace, - question mark, colon, `||', `&&', `do' and `else'. + question mark, colon, `||', `&&', `do', and `else'. File: gawk.info, Node: Invoking Gawk, Next: Regexp, Prev: Getting Started, Up: Top @@ -2308,10 +2365,10 @@ File: gawk.info, Node: Invoking Gawk, Next: Regexp, Prev: Getting Started, U This major node covers how to run `awk', both POSIX-standard and `gawk'-specific command-line options, and what `awk' and `gawk' do with -non-option arguments. It then proceeds to cover how `gawk' searches -for source files, reading standard input along with other files, -`gawk''s environment variables, `gawk''s exit status, using include -files, and obsolete and undocumented options and/or features. +nonoption arguments. It then proceeds to cover how `gawk' searches for +source files, reading standard input along with other files, `gawk''s +environment variables, `gawk''s exit status, using include files, and +obsolete and undocumented options and/or features. Many of the options and features described here are discussed in more detail later in the Info file; feel free to skip over things in @@ -2345,8 +2402,8 @@ enclosed in [...] in these templates are optional: `awk' [OPTIONS] `-f' PROGFILE [`--'] FILE ... `awk' [OPTIONS] [`--'] `'PROGRAM'' FILE ... - Besides traditional one-letter POSIX-style options, `gawk' also -supports GNU long options. + In addition to traditional one-letter POSIX-style options, `gawk' +also supports GNU long options. It is possible to invoke `awk' with an empty program: @@ -2365,8 +2422,8 @@ File: gawk.info, Node: Options, Next: Other Arguments, Prev: Command Line, U Options begin with a dash and consist of a single character. GNU-style long options consist of two dashes and a keyword. The keyword can be abbreviated, as long as the abbreviation allows the option to be -uniquely identified. If the option takes an argument, then the keyword -is either immediately followed by an equals sign (`=') and the +uniquely identified. If the option takes an argument, either the +keyword is immediately followed by an equals sign (`=') and the argument's value, or the keyword and the argument's value are separated by whitespace. If a particular option with a value is given more than once, it is the last value that counts. @@ -2381,10 +2438,10 @@ The following list describes options mandated by the POSIX standard: `-f SOURCE-FILE' `--file SOURCE-FILE' - Read `awk' program source from SOURCE-FILE instead of in the first - non-option argument. This option may be given multiple times; the - `awk' program consists of the concatenation of the contents of - each specified SOURCE-FILE. + Read the `awk' program source from SOURCE-FILE instead of in the + first nonoption argument. This option may be given multiple + times; the `awk' program consists of the concatenation of the + contents of each specified SOURCE-FILE. `-v VAR=VAL' `--assign VAR=VAL' @@ -2425,7 +2482,7 @@ The following list describes options mandated by the POSIX standard: `-b' `--characters-as-bytes' Cause `gawk' to treat all input data as single-byte characters. - In addition, all output written with `print' or `printf' are + In addition, all output written with `print' or `printf' is treated as single-byte characters. Normally, `gawk' follows the POSIX standard and attempts to process @@ -2433,7 +2490,7 @@ The following list describes options mandated by the POSIX standard: This can often involve converting multibyte characters into wide characters (internally), and can lead to problems or confusion if the input data does not contain valid multibyte characters. This - option is an easy way to tell `gawk': "hands off my data!". + option is an easy way to tell `gawk', "Hands off my data!" `-c' `--traditional' @@ -2466,10 +2523,10 @@ The following list describes options mandated by the POSIX standard: `--debug'[`='FILE] Enable debugging of `awk' programs (*note Debugging::). By default, the debugger reads commands interactively from the - keyboard. The optional FILE argument allows you to specify a file - with a list of commands for the debugger to execute - non-interactively. No space is allowed between the `-D' and FILE, - if FILE is supplied. + keyboard (standard input). The optional FILE argument allows you + to specify a file with a list of commands for the debugger to + execute noninteractively. No space is allowed between the `-D' + and FILE, if FILE is supplied. `-e' PROGRAM-TEXT `--source' PROGRAM-TEXT @@ -2494,8 +2551,8 @@ The following list describes options mandated by the POSIX standard: applications that pass arguments through the URL; using this option prevents a malicious (or other) user from passing in options, assignments, or `awk' source code (via `-e') to the CGI - application. This option should be used with `#!' scripts (*note - Executable Scripts::), like so: + application.(1) This option should be used with `#!' scripts + (*note Executable Scripts::), like so: #! /usr/local/bin/gawk -E @@ -2503,23 +2560,23 @@ The following list describes options mandated by the POSIX standard: `-g' `--gen-pot' - Analyze the source program and generate a GNU `gettext' Portable - Object Template file on standard output for all string constants + Analyze the source program and generate a GNU `gettext' portable + object template file on standard output for all string constants that have been marked for translation. *Note Internationalization::, for information about this option. `-h' `--help' - Print a "usage" message summarizing the short and long style + Print a "usage" message summarizing the short- and long-style options that `gawk' accepts and then exit. `-i' SOURCE-FILE `--include' SOURCE-FILE Read an `awk' source library from SOURCE-FILE. This option is completely equivalent to using the `@include' directive inside - your program. This option is very similar to the `-f' option, but - there are two important differences. First, when `-i' is used, - the program source is not loaded if it has been previously loaded, + your program. It is very similar to the `-f' option, but there + are two important differences. First, when `-i' is used, the + program source is not loaded if it has been previously loaded, whereas with `-f', `gawk' always loads the file. Second, because this option is intended to be used with code libraries, `gawk' does not recognize such files as constituting main program input. @@ -2561,7 +2618,7 @@ The following list describes options mandated by the POSIX standard: `-M' `--bignum' - Force arbitrary precision arithmetic on numbers. This option has + Force arbitrary-precision arithmetic on numbers. This option has no effect if `gawk' is not compiled to use the GNU MPFR and MP libraries (*note Arbitrary Precision Arithmetic::). @@ -2571,9 +2628,8 @@ The following list describes options mandated by the POSIX standard: input data (*note Nondecimal Data::). CAUTION: This option can severely break old programs. Use - with care. - - This option may disappear in a future version of `gawk'. + with care. Also note that this option may disappear in a + future version of `gawk'. `-N' `--use-lc-numeric' @@ -2582,7 +2638,7 @@ The following list describes options mandated by the POSIX standard: `-o'[FILE] `--pretty-print'[`='FILE] - Enable pretty-printing of `awk' programs. By default, output + Enable pretty-printing of `awk' programs. By default, the output program is created in a file named `awkprof.out' (*note Profiling::). The optional FILE argument allows you to specify a different file name for the output. No space is allowed between @@ -2594,7 +2650,8 @@ The following list describes options mandated by the POSIX standard: `-O' `--optimize' Enable some optimizations on the internal representation of the - program. At the moment this includes just simple constant folding. + program. At the moment, this includes just simple constant + folding. `-p'[FILE] `--profile'[`='FILE] @@ -2637,8 +2694,8 @@ The following list describes options mandated by the POSIX standard: `--re-interval' Allow interval expressions (*note Regexp Operators::) in regexps. This is now `gawk''s default behavior. Nevertheless, this option - remains both for backward compatibility, and for use in - combination with `--traditional'. + remains (both for backward compatibility and for use in + combination with `--traditional'). `-S' `--sandbox' @@ -2674,7 +2731,7 @@ it is, `awk' reads its program source from all of the named files, as if they had been concatenated together into one big file. This is useful for creating libraries of `awk' functions. These functions can be written once and then retrieved from a standard place, instead of -having to be included into each individual program. The `-i' option is +having to be included in each individual program. The `-i' option is similar in this regard. (As mentioned in *note Definition Syntax::, function names must be unique.) @@ -2682,18 +2739,18 @@ function names must be unique.) the program is entered at the keyboard, by specifying `-f /dev/tty'. After typing your program, type `Ctrl-d' (the end-of-file character) to terminate it. (You may also use `-f -' to read program source from the -standard input but then you will not be able to also use the standard +standard input, but then you will not be able to also use the standard input as a source of data.) Because it is clumsy using the standard `awk' mechanisms to mix source file and command-line `awk' programs, `gawk' provides the `-e' -option. This does not require you to pre-empt the standard input for +option. This does not require you to preempt the standard input for your source code; it allows you to easily mix command-line and library source code (*note AWKPATH Variable::). As with `-f', the `-e' and `-i' options may also be used multiple times on the command line. If no `-f' or `-e' option is specified, then `gawk' uses the first -non-option command-line argument as the text of the program source code. +nonoption command-line argument as the text of the program source code. If the environment variable `POSIXLY_CORRECT' exists, then `gawk' behaves in strict POSIX mode, exactly as if you had supplied `--posix'. @@ -2710,7 +2767,7 @@ you would add these lines to the `.profile' file in your home directory: POSIXLY_CORRECT=true export POSIXLY_CORRECT - For a C shell-compatible shell,(1) you would add this line to the + For a C shell-compatible shell,(2) you would add this line to the `.login' file in your home directory: setenv POSIXLY_CORRECT true @@ -2721,7 +2778,12 @@ environments. ---------- Footnotes ---------- - (1) Not recommended. + (1) For more detail, please see Section 4.4 of RFC 3875 +(http://www.ietf.org/rfc/rfc3875). Also see the explanatory note sent +to the `gawk' bug mailing list +(http://lists.gnu.org/archive/html/bug-gawk/2014-11/msg00022.html). + + (2) Not recommended. File: gawk.info, Node: Other Arguments, Next: Naming Standard Input, Prev: Options, Up: Invoking Gawk @@ -2742,8 +2804,8 @@ not a file name: program in the `ARGV' array (*note Built-in Variables::). Command-line options and the program text (if present) are omitted from `ARGV'. All other arguments, including variable assignments, are included. As -each element of `ARGV' is processed, `gawk' sets the variable `ARGIND' -to the index in `ARGV' of the current element. +each element of `ARGV' is processed, `gawk' sets `ARGIND' to the index +in `ARGV' of the current element. Changing `ARGC' and `ARGV' in your `awk' program lets you control how `awk' processes the input files; this is described in more detail @@ -2837,17 +2899,17 @@ File: gawk.info, Node: AWKPATH Variable, Next: AWKLIBPATH Variable, Up: Envir The previous minor node described how `awk' program files can be named on the command line with the `-f' option. In most `awk' -implementations, you must supply a precise path name for each program -file, unless the file is in the current directory. But in `gawk', if +implementations, you must supply a precise pathname for each program +file, unless the file is in the current directory. But with `gawk', if the file name supplied to the `-f' or `-i' options does not contain a directory separator `/', then `gawk' searches a list of directories -(called the "search path"), one by one, looking for a file with the +(called the "search path") one by one, looking for a file with the specified name. The search path is a string consisting of directory names separated by -colons(1). `gawk' gets its search path from the `AWKPATH' environment -variable. If that variable does not exist, `gawk' uses a default path, -`.:/usr/local/share/awk'.(2) +colons.(1) `gawk' gets its search path from the `AWKPATH' environment +variable. If that variable does not exist, or if it has an empty value, +`gawk' uses a default path (described shortly). The search path feature is particularly helpful for building libraries of useful `awk' functions. The library files can be placed @@ -2855,15 +2917,13 @@ in a standard directory in the default path and then specified on the command line with a short file name. Otherwise, you would have to type the full file name for each file. - By using the `-i' option, or the `-e' and `-f' options, your -command-line `awk' programs can use facilities in `awk' library files -(*note Library Functions::). Path searching is not done if `gawk' is -in compatibility mode. This is true for both `--traditional' and -`--posix'. *Note Options::. + By using the `-i' or `-f' options, your command-line `awk' programs +can use facilities in `awk' library files (*note Library Functions::). +Path searching is not done if `gawk' is in compatibility mode. This is +true for both `--traditional' and `--posix'. *Note Options::. If the source code file is not found after the initial search, the -path is searched again after adding the default `.awk' suffix to the -file name. +path is searched again after adding the suffix `.awk' to the file name. `gawk''s path search mechanism is similar to the shell's. (See `The Bourne-Again SHell manual' (http://www.gnu.org/software/bash/manual/).) @@ -2871,21 +2931,32 @@ It treats a null entry in the path as indicating the current directory. (A null entry is indicated by starting or ending the path with a colon or by placing two colons next to each other [`::'].) - NOTE: `gawk' always looks in the current directory _before_ - searching `AWKPATH'. Thus, while you can include the current - directory in the search path, either explicitly or with a null - entry, there is no real reason to do so. + NOTE: To include the current directory in the path, either place + `.' as an entry in the path or write a null entry in the path. - If `AWKPATH' is not defined in the environment, `gawk' places its -default search path into `ENVIRON["AWKPATH"]'. This makes it easy to -determine the actual search path that `gawk' used from within an `awk' -program. + Different past versions of `gawk' would also look explicitly in + the current directory, either before or after the path search. As + of version 4.1.2, this no longer happens; if you wish to look in + the current directory, you must include `.' either as a separate + entry or as a null entry in the search path. + + The default value for `AWKPATH' is `.:/usr/local/share/awk'.(2) +Since `.' is included at the beginning, `gawk' searches first in the +current directory and then in `/usr/local/share/awk'. In practice, +this means that you will rarely need to change the value of `AWKPATH'. - While you can change `ENVIRON["AWKPATH"]' within your `awk' program, -this has no effect on the running program's behavior. This makes -sense: the `AWKPATH' environment variable is used to find the program -source files. Once your program is running, all the files have been -found, and `gawk' no longer needs to use `AWKPATH'. + *Note Shell Startup Files::, for information on functions that help +to manipulate the `AWKPATH' variable. + + `gawk' places the value of the search path that it used into +`ENVIRON["AWKPATH"]'. This provides access to the actual search path +value from within an `awk' program. + + Although you can change `ENVIRON["AWKPATH"]' within your `awk' +program, this has no effect on the running program's behavior. This +makes sense: the `AWKPATH' environment variable is used to find the +program source files. Once your program is running, all the files have +been found, and `gawk' no longer needs to use `AWKPATH'. ---------- Footnotes ---------- @@ -2911,6 +2982,18 @@ platform. For example, on GNU/Linux systems, the suffix `.so' is used. The search path specified is also used for extensions loaded via the `@load' keyword (*note Loading Shared Libraries::). + If `AWKLIBPATH' does not exist in the environment, or if it has an +empty value, `gawk' uses a default path; this is typically +`/usr/local/lib/gawk', although it can vary depending upon how `gawk' +was built. + + *Note Shell Startup Files::, for information on functions that help +to manipulate the `AWKLIBPATH' variable. + + `gawk' places the value of the search path that it used into +`ENVIRON["AWKLIBPATH"]'. This provides access to the actual search path +value from within an `awk' program. + File: gawk.info, Node: Other Environment Variables, Prev: AWKLIBPATH Variable, Up: Environment Variables @@ -2919,7 +3002,7 @@ File: gawk.info, Node: Other Environment Variables, Prev: AWKLIBPATH Variable, A number of other environment variables affect `gawk''s behavior, but they are more specialized. Those in the following list are meant to be -used by regular users. +used by regular users: `GAWK_MSEC_SLEEP' Specifies the interval between connection retries, in @@ -2933,10 +3016,11 @@ used by regular users. `GAWK_SOCK_RETRIES' Controls the number of times `gawk' attempts to retry a two-way TCP/IP (socket) connection before giving up. *Note TCP/IP - Networking::. + Networking::. Note that when nonfatal I/O is enabled (*note + Nonfatal::), `gawk' only tries to open a TCP/IP socket once. `POSIXLY_CORRECT' - Causes `gawk' to switch to POSIX compatibility mode, disabling all + Causes `gawk' to switch to POSIX-compatibility mode, disabling all traditional and GNU extensions. *Note Options::. The environment variables in the following list are meant for use by @@ -2967,8 +3051,8 @@ change. The variables are: If this variable exists, `gawk' includes the file name and line number within the `gawk' source code from which warning and/or fatal messages are generated. Its purpose is to help isolate the - source of a message, since there are multiple places which produce - the same warning or error message. + source of a message, as there are multiple places that produce the + same warning or error message. `GAWK_NO_DFA' If this variable exists, `gawk' does not use the DFA regexp matcher @@ -2983,16 +3067,16 @@ change. The variables are: evaluation stack, when needed. `INT_CHAIN_MAX' - The intended maximum number of items `gawk' will maintain on a - hash chain for managing arrays indexed by integers. + This specifies intended maximum number of items `gawk' will + maintain on a hash chain for managing arrays indexed by integers. `STR_CHAIN_MAX' - The intended maximum number of items `gawk' will maintain on a - hash chain for managing arrays indexed by strings. + This specifies intended maximum number of items `gawk' will + maintain on a hash chain for managing arrays indexed by strings. `TIDYMEM' If this variable exists, `gawk' uses the `mtrace()' library calls - from GNU LIBC to help track down possible memory leaks. + from the GNU C library to help track down possible memory leaks. File: gawk.info, Node: Exit Status, Next: Include Files, Prev: Environment Variables, Up: Invoking Gawk @@ -3009,13 +3093,13 @@ with the value of the C constant `EXIT_SUCCESS'. This is usually zero. If an error occurs, `gawk' exits with the value of the C constant `EXIT_FAILURE'. This is usually one. - If `gawk' exits because of a fatal error, the exit status is 2. On -non-POSIX systems, this value may be mapped to `EXIT_FAILURE'. + If `gawk' exits because of a fatal error, the exit status is two. +On non-POSIX systems, this value may be mapped to `EXIT_FAILURE'. File: gawk.info, Node: Include Files, Next: Loading Shared Libraries, Prev: Exit Status, Up: Invoking Gawk -2.7 Including Other Files Into Your Program +2.7 Including Other Files into Your Program =========================================== This minor node describes a feature that is specific to `gawk'. @@ -3024,11 +3108,11 @@ This minor node describes a feature that is specific to `gawk'. files. This gives you the ability to split large `awk' source files into smaller, more manageable pieces, and also lets you reuse common `awk' code from various `awk' scripts. In other words, you can group -together `awk' functions, used to carry out specific tasks, into -external files. These files can be used just like function libraries, -using the `@include' keyword in conjunction with the `AWKPATH' -environment variable. Note that source files may also be included -using the `-i' option. +together `awk' functions used to carry out specific tasks into external +files. These files can be used just like function libraries, using the +`@include' keyword in conjunction with the `AWKPATH' environment +variable. Note that source files may also be included using the `-i' +option. Let's see an example. We'll start with two (trivial) `awk' scripts, namely `test1' and `test2'. Here is the `test1' script: @@ -3050,9 +3134,9 @@ and here is `test2': -| This is script test1. -| This is script test2. - `gawk' runs the `test2' script which includes `test1' using the -`@include' keyword. So, to include external `awk' source files you just -use `@include' followed by the name of the file to be included, + `gawk' runs the `test2' script, which includes `test1' using the +`@include' keyword. So, to include external `awk' source files, you +just use `@include' followed by the name of the file to be included, enclosed in double quotes. NOTE: Keep in mind that this is a language construct and the file @@ -3078,23 +3162,23 @@ Running `gawk' with the `test3' script produces the following results: @include "../io_funcs" -or: +and: @include "/usr/awklib/network" -are valid. The `AWKPATH' environment variable can be of great value -when using `@include'. The same rules for the use of the `AWKPATH' -variable in command-line file searches (*note AWKPATH Variable::) apply -to `@include' also. +are both valid. The `AWKPATH' environment variable can be of great +value when using `@include'. The same rules for the use of the +`AWKPATH' variable in command-line file searches (*note AWKPATH +Variable::) apply to `@include' also. This is very helpful in constructing `gawk' function libraries. If -you have a large script with useful, general purpose `awk' functions, +you have a large script with useful, general-purpose `awk' functions, you can break it down into library files and put those files in a -special directory. You can then include those "libraries," using -either the full pathnames of the files, or by setting the `AWKPATH' +special directory. You can then include those "libraries," either by +using the full pathnames of the files, or by setting the `AWKPATH' environment variable accordingly and then using `@include' with just -the file part of the full pathname. Of course you can have more than -one directory to keep library files; the more complex the working +the file part of the full pathname. Of course, you can keep library +files in more than one directory; the more complex the working environment is, the more directories you may need to organize the files to be included. @@ -3106,13 +3190,13 @@ particular, `@include' is very useful for writing CGI scripts to be run from web pages. As mentioned in *note AWKPATH Variable::, the current directory is -always searched first for source files, before searching in `AWKPATH', -and this also applies to files named with `@include'. +always searched first for source files, before searching in `AWKPATH'; +this also applies to files named with `@include'. File: gawk.info, Node: Loading Shared Libraries, Next: Obsolete, Prev: Include Files, Up: Invoking Gawk -2.8 Loading Dynamic Extensions Into Your Program +2.8 Loading Dynamic Extensions into Your Program ================================================ This minor node describes a feature that is specific to `gawk'. @@ -3127,7 +3211,7 @@ The `AWKLIBPATH' variable is used to search for the extension. Using If the extension is not initially found in `AWKLIBPATH', another search is conducted after appending the platform's default shared library suffix to the file name. For example, on GNU/Linux systems, -the suffix `.so' is used. +the suffix `.so' is used: $ gawk '@load "ordchr"; BEGIN {print chr(65)}' -| A @@ -3152,8 +3236,8 @@ File: gawk.info, Node: Obsolete, Next: Undocumented, Prev: Loading Shared Lib ==================================== This minor node describes features and/or command-line options from -previous releases of `gawk' that are either not available in the -current version or that are still supported but deprecated (meaning that +previous releases of `gawk' that either are not available in the +current version or are still supported but deprecated (meaning that they will _not_ be in the next release). The process-related special files `/dev/pid', `/dev/ppid', @@ -3181,15 +3265,15 @@ File: gawk.info, Node: Invoking Summary, Prev: Undocumented, Up: Invoking Gaw run `awk'. * The three standard options for all versions of `awk' are `-f', - `-F' and `-v'. `gawk' supplies these and many others, as well as + `-F', and `-v'. `gawk' supplies these and many others, as well as corresponding GNU-style long options. - * Non-option command-line arguments are usually treated as file - names, unless they have the form `VAR=VALUE', in which case they - are taken as variable assignments to be performed at that point in + * Nonoption command-line arguments are usually treated as file names, + unless they have the form `VAR=VALUE', in which case they are + taken as variable assignments to be performed at that point in processing the input. - * All non-option command-line arguments, excluding the program text, + * All nonoption command-line arguments, excluding the program text, are placed in the `ARGV' array. Adjusting `ARGC' and `ARGV' affects how `awk' processes input. @@ -3211,7 +3295,7 @@ File: gawk.info, Node: Invoking Summary, Prev: Undocumented, Up: Invoking Gaw * `gawk' allows you to load additional functions written in C or C++ using the `@load' statement and/or the `-l' option. (This - advanced feature is described later on in *note Dynamic + advanced feature is described later, in *note Dynamic Extensions::.) @@ -3228,7 +3312,7 @@ strings. Because regular expressions are such a fundamental part of that matches every input record whose text belongs to that set. The simplest regular expression is a sequence of letters, numbers, or both. Such a regexp matches any string that contains that sequence. Thus, -the regexp `foo' matches any string containing `foo'. Therefore, the +the regexp `foo' matches any string containing `foo'. Thus, the pattern `/foo/' matches any input record containing the three adjacent characters `foo' _anywhere_ in the record. Other kinds of regexps let you specify more complicated classes of strings. @@ -3269,13 +3353,13 @@ expressions allow you to specify the string to match against; it need not be the entire current input record. The two operators `~' and `!~' perform regular expression comparisons. Expressions using these operators can be used as patterns, or in `if', `while', `for', and `do' -statements. (*Note Statements::.) For example: +statements. (*Note Statements::.) For example, the following is true +if the expression EXP (taken as a string) matches REGEXP: EXP ~ /REGEXP/ -is true if the expression EXP (taken as a string) matches REGEXP. The -following example matches, or selects, all input records with the -uppercase letter `J' somewhere in the first field: +This example matches, or selects, all input records with the uppercase +letter `J' somewhere in the first field: $ awk '$1 ~ /J/' inventory-shipped -| Jan 13 25 15 115 @@ -3329,9 +3413,9 @@ string or regexp. Thus, the string whose contents are the two characters `"' and `\' must be written `"\"\\"'. Other escape sequences represent unprintable characters such as TAB -or newline. While there is nothing to stop you from entering most +or newline. There is nothing to stop you from entering most unprintable characters directly in a string constant or regexp constant, -they may look ugly. +but they may look ugly. The following list presents all the escape sequences used in `awk' and what they represent. Unless noted otherwise, all these escape @@ -3360,7 +3444,7 @@ sequences apply to both string constants and regexp constants: Horizontal TAB, `Ctrl-i', ASCII code 9 (HT). `\v' - Vertical tab, `Ctrl-k', ASCII code 11 (VT). + Vertical TAB, `Ctrl-k', ASCII code 11 (VT). `\NNN' The octal value NNN, where NNN stands for 1 to 3 digits between @@ -3372,20 +3456,21 @@ sequences apply to both string constants and regexp constants: hexadecimal digits (`0'-`9', and either `A'-`F' or `a'-`f'). A maximum of two digts are allowed after the `\x'. Any further hexadecimal digits are treated as simple letters or numbers. - (c.e.) + (c.e.) (The `\x' escape sequence is not allowed in POSIX awk.) CAUTION: In ISO C, the escape sequence continues until the first nonhexadecimal digit is seen. For many years, `gawk' would continue incorporating hexadecimal digits into the value until a non-hexadecimal digit or the end of the string was encountered. However, using more than two hexadecimal - digits produces + digits produced undefined results. As of version 4.2, only + two digits are processed. `\/' A literal slash (necessary for regexp constants only). This sequence is used when you want to write a regexp constant that contains a slash (such as `/.*:\/home\/[[:alnum:]]+:.*/'; the - `[[:alnum:]]' notation is discussed shortly, in *note Bracket + `[[:alnum:]]' notation is discussed in *note Bracket Expressions::). Because the regexp is delimited by slashes, you need to escape any slash that is part of the pattern, in order to tell `awk' to keep processing the rest of the regexp. @@ -3409,20 +3494,7 @@ normally be a regexp operator. For example, `/a\+b/' matches the three characters `a+b'. For complete portability, do not use a backslash before any -character not shown in the previous list and that is not an operator. - - To summarize: - - * The escape sequences in the list above are always processed first, - for both string constants and regexp constants. This happens very - early, as soon as `awk' reads your program. - - * `gawk' processes both regexp constants and dynamic regexps (*note - Computed Regexps::), for the special operators listed in *note GNU - Regexp Operators::. - - * A backslash before any other character means to treat that - character literally. +character not shown in the previous list or that is not an operator. Backslash Before Regular Characters @@ -3442,6 +3514,19 @@ Leave the backslash alone Some other `awk' implementations do this. In such implementations, typing `"a\qc"' is the same as typing `"a\\qc"'. + To summarize: + + * The escape sequences in the preceding list are always processed + first, for both string constants and regexp constants. This + happens very early, as soon as `awk' reads your program. + + * `gawk' processes both regexp constants and dynamic regexps (*note + Computed Regexps::), for the special operators listed in *note GNU + Regexp Operators::. + + * A backslash before any other character means to treat that + character literally. + Escape Sequences for Metacharacters Suppose you use an octal or hexadecimal escape to represent a regexp @@ -3471,18 +3556,18 @@ and converted into corresponding real characters as the very first step in processing regexps. Here is a list of metacharacters. All characters that are not escape -sequences and that are not listed in the following stand for themselves: +sequences and that are not listed here stand for themselves: `\' - This is used to suppress the special meaning of a character when - matching. For example, `\$' matches the character `$'. + This suppresses the special meaning of a character when matching. + For example, `\$' matches the character `$'. `^' - This matches the beginning of a string. For example, `^@chapter' - matches `@chapter' at the beginning of a string and can be used to - identify chapter beginnings in Texinfo source files. The `^' is - known as an "anchor", because it anchors the pattern to match only - at the beginning of the string. + This matches the beginning of a string. `^@chapter' matches + `@chapter' at the beginning of a string, for example, and can be + used to identify chapter beginnings in Texinfo source files. The + `^' is known as an "anchor", because it anchors the pattern to + match only at the beginning of the string. It is important to realize that `^' does not match the beginning of a line (the point right after a `\n' newline character) embedded @@ -3554,8 +3639,8 @@ sequences and that are not listed in the following stand for themselves: There are two subtle points to understand about how `*' works. First, the `*' applies only to the single preceding regular expression component (e.g., in `ph*', it applies just to the `h'). - To cause `*' to apply to a larger sub-expression, use parentheses: - `(ph)*' matches `ph', `phph', `phphph' and so on. + To cause `*' to apply to a larger subexpression, use parentheses: + `(ph)*' matches `ph', `phph', `phphph', and so on. Second, `*' finds as many repetitions as possible. If the text to be matched is `phhhhhhhhhhhhhhooey', `ph*' matches all of the `h's. @@ -3585,10 +3670,10 @@ sequences and that are not listed in the following stand for themselves: Matches `whhhy', but not `why' or `whhhhy'. `wh{3,5}y' - Matches `whhhy', `whhhhy', or `whhhhhy', only. + Matches `whhhy', `whhhhy', or `whhhhhy' only. `wh{2,}y' - Matches `whhy' or `whhhy', and so on. + Matches `whhy', `whhhy', and so on. Interval expressions were not traditionally available in `awk'. They were added as part of the POSIX standard to make `awk' and @@ -3639,7 +3724,7 @@ File: gawk.info, Node: Bracket Expressions, Next: Leftmost Longest, Prev: Reg 3.4 Using Bracket Expressions ============================= -As mentioned earlier, a bracket expression matches any character amongst +As mentioned earlier, a bracket expression matches any character among those listed between the opening and closing square brackets. Within a bracket expression, a "range expression" consists of two @@ -3679,24 +3764,24 @@ character classes defined by the POSIX standard. Class Meaning -------------------------------------------------------------------------- -`[:alnum:]' Alphanumeric characters. -`[:alpha:]' Alphabetic characters. -`[:blank:]' Space and TAB characters. -`[:cntrl:]' Control characters. -`[:digit:]' Numeric characters. -`[:graph:]' Characters that are both printable and visible. (A space is - printable but not visible, whereas an `a' is both.) -`[:lower:]' Lowercase alphabetic characters. +`[:alnum:]' Alphanumeric characters +`[:alpha:]' Alphabetic characters +`[:blank:]' Space and TAB characters +`[:cntrl:]' Control characters +`[:digit:]' Numeric characters +`[:graph:]' Characters that are both printable and visible (a space is + printable but not visible, whereas an `a' is both) +`[:lower:]' Lowercase alphabetic characters `[:print:]' Printable characters (characters that are not control - characters). + characters) `[:punct:]' Punctuation characters (characters that are not letters, - digits, control characters, or space characters). + digits, control characters, or space characters) `[:space:]' Space characters (such as space, TAB, and formfeed, to name - a few). -`[:upper:]' Uppercase alphabetic characters. -`[:xdigit:]'Characters that are hexadecimal digits. + a few) +`[:upper:]' Uppercase alphabetic characters +`[:xdigit:]'Characters that are hexadecimal digits -Table 3.1: POSIX Character Classes +Table 3.1: POSIX character classes For example, before the POSIX standard, you had to write `/[A-Za-z0-9]/' to match alphanumeric characters. If your character @@ -3704,7 +3789,7 @@ set had other alphabetic characters in it, this would not match them. With the POSIX character classes, you can write `/[[:alnum:]]/' to match the alphabetic and numeric characters in your character set. - Some utilities that match regular expressions provide a non-standard + Some utilities that match regular expressions provide a nonstandard `[:ascii:]' character class; `awk' does not. However, you can simulate such a construct using `[\x00-\x7F]'. This matches all values numerically between zero and 127, which is the defined range of the @@ -3728,8 +3813,9 @@ Collating symbols Equivalence classes Locale-specific names for a list of characters that are equal. The name is enclosed between `[=' and `=]'. For example, the name `e' - might be used to represent all of "e," "e`," and "e'." In this - case, `[[=e=]]' is a regexp that matches any of `e', `e'', or `e`'. + might be used to represent all of "e," "e^," "e`," and "e'." In + this case, `[[=e=]]' is a regexp that matches any of `e', `e^', + `e'', or `e`'. These features are very valuable in non-English-speaking locales. @@ -3751,7 +3837,7 @@ Consider the following: This example uses the `sub()' function to make a change to the input record. (`sub()' replaces the first instance of any text matched by the first argument with the string provided as the second argument; -*note String Functions::). Here, the regexp `/a+/' indicates "one or +*note String Functions::.) Here, the regexp `/a+/' indicates "one or more `a' characters," and the replacement text is `<A>'. The input contains four `a' characters. `awk' (and POSIX) regular @@ -3788,15 +3874,16 @@ regexp": This sets `digits_regexp' to a regexp that describes one or more digits, and tests whether the input record matches this regexp. - NOTE: When using the `~' and `!~' operators, there is a difference - between a regexp constant enclosed in slashes and a string - constant enclosed in double quotes. If you are going to use a - string constant, you have to understand that the string is, in - essence, scanned _twice_: the first time when `awk' reads your + NOTE: When using the `~' and `!~' operators, be aware that there + is a difference between a regexp constant enclosed in slashes and + a string constant enclosed in double quotes. If you are going to + use a string constant, you have to understand that the string is, + in essence, scanned _twice_: the first time when `awk' reads your program, and the second time when it goes to match the string on the lefthand side of the operator with the pattern on the right. This is true of any string-valued expression (such as - `digits_regexp', shown previously), not just string constants. + `digits_regexp', shown in the previous example), not just string + constants. What difference does it make if the string is scanned twice? The answer has to do with escape sequences, and particularly with @@ -3893,19 +3980,19 @@ letters, digits, or underscores (`_'): `\B' Matches the empty string that occurs between two word-constituent - characters. For example, `/\Brat\B/' matches `crate' but it does + characters. For example, `/\Brat\B/' matches `crate', but it does not match `dirty rat'. `\B' is essentially the opposite of `\y'. There are two other operators that work on buffers. In Emacs, a -"buffer" is, naturally, an Emacs buffer. For other programs, `gawk''s -regexp library routines consider the entire string to match as the -buffer. The operators are: +"buffer" is, naturally, an Emacs buffer. Other GNU programs, including +`gawk', consider the entire string to match as the buffer. The +operators are: `\`' - Matches the empty string at the beginning of a buffer (string). + Matches the empty string at the beginning of a buffer (string) `\'' - Matches the empty string at the end of a buffer (string). + Matches the empty string at the end of a buffer (string) Because `^' and `$' always work in terms of the beginning and end of strings, these operators don't add any new capabilities for `awk'. @@ -3927,15 +4014,14 @@ No options Operators::. `--posix' - Only POSIX regexps are supported; the GNU operators are not special - (e.g., `\w' matches a literal `w'). Interval expressions are - allowed. + Match only POSIX regexps; the GNU operators are not special (e.g., + `\w' matches a literal `w'). Interval expressions are allowed. `--traditional' - Traditional Unix `awk' regexps are matched. The GNU operators are - not special, and interval expressions are not available. The - POSIX character classes (`[[:alnum:]]', etc.) are supported, as - BWK `awk' supports them. Characters described by octal and + Match traditional Unix `awk' regexps. The GNU operators are not + special, and interval expressions are not available. Because BWK + `awk' supports them, the POSIX character classes (`[[:alnum:]]', + etc.) are available. Characters described by octal and hexadecimal escape sequences are treated literally, even if they represent regexp metacharacters. @@ -3975,10 +4061,9 @@ works in any POSIX-compliant `awk'. `IGNORECASE' is not zero, _all_ regexp and string operations ignore case. - Changing the value of `IGNORECASE' dynamically controls the -case-sensitivity of the program as it runs. Case is significant by -default because `IGNORECASE' (like most variables) is initialized to -zero: + Changing the value of `IGNORECASE' dynamically controls the case +sensitivity of the program as it runs. Case is significant by default +because `IGNORECASE' (like most variables) is initialized to zero: x = "aB" if (x ~ /ab/) ... # this test will fail @@ -3986,17 +4071,17 @@ zero: IGNORECASE = 1 if (x ~ /ab/) ... # now it will succeed - In general, you cannot use `IGNORECASE' to make certain rules -case-insensitive and other rules case-sensitive, because there is no + In general, you cannot use `IGNORECASE' to make certain rules case +insensitive and other rules case sensitive, as there is no straightforward way to set `IGNORECASE' just for the pattern of a particular rule.(1) To do this, use either bracket expressions or `tolower()'. However, one thing you can do with `IGNORECASE' only is -dynamically turn case-sensitivity on or off for all the rules at once. +dynamically turn case sensitivity on or off for all the rules at once. `IGNORECASE' can be set on the command line or in a `BEGIN' rule (*note Other Arguments::; also *note Using BEGIN/END::). Setting -`IGNORECASE' from the command line is a way to make a program -case-insensitive without having to edit it. +`IGNORECASE' from the command line is a way to make a program case +insensitive without having to edit it. In multibyte locales, the equivalences between upper- and lowercase characters are tested based on the wide-character values of the @@ -4033,11 +4118,11 @@ File: gawk.info, Node: Regexp Summary, Prev: Case-sensitivity, Up: Regexp conditional expressions, or as part of matching expressions using the `~' and `!~' operators. - * Escape sequences let you represent non-printable characters and + * Escape sequences let you represent nonprintable characters and also let you represent regexp metacharacters as literal characters to be matched. - * Regexp operators provide grouping, alternation and repetition. + * Regexp operators provide grouping, alternation, and repetition. * Bracket expressions give you a shorthand for specifying sets of characters that can match at a particular point in a regexp. @@ -4049,8 +4134,8 @@ File: gawk.info, Node: Regexp Summary, Prev: Case-sensitivity, Up: Regexp extent of the match, such as for text substitution and when the record separator is a regexp. - * Matching expressions may use dynamic regexps, that is, string - values treated as regular expressions. + * Matching expressions may use dynamic regexps (i.e., string values + treated as regular expressions). * `gawk''s `IGNORECASE' variable lets you control the case sensitivity of regexp matching. In other `awk' versions, use @@ -4078,7 +4163,7 @@ one line. Each record is automatically split into chunks called parts of a record. On rare occasions, you may need to use the `getline' command. The -`getline' command is valuable, both because it can do explicit input +`getline' command is valuable both because it can do explicit input from any number of files, and because the files used with it do not have to be named on the `awk' command line (*note Getline::). @@ -4095,6 +4180,7 @@ have to be named on the `awk' command line (*note Getline::). * Getline:: Reading files under explicit program control using the `getline' function. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -4109,7 +4195,7 @@ File: gawk.info, Node: Records, Next: Fields, Up: Reading Files `awk' divides the input for your program into records and fields. It keeps track of the number of records that have been read so far from the current input file. This value is stored in a predefined variable -called `FNR' which is reset to zero every time a new file is started. +called `FNR', which is reset to zero every time a new file is started. Another predefined variable, `NR', records the total number of input records read so far from all data files. It starts at zero, but is never automatically reset to zero. @@ -4122,19 +4208,19 @@ never automatically reset to zero. File: gawk.info, Node: awk split records, Next: gawk split records, Up: Records -4.1.1 Record Splitting With Standard `awk' +4.1.1 Record Splitting with Standard `awk' ------------------------------------------ Records are separated by a character called the "record separator". By default, the record separator is the newline character. This is why -records are, by default, single lines. A different character can be -used for the record separator by assigning the character to the +records are, by default, single lines. To use a different character +for the record separator, simply assign that character to the predefined variable `RS'. Like any other variable, the value of `RS' can be changed in the `awk' program with the assignment operator, `=' (*note Assignment Ops::). The new record-separator character should be enclosed in -quotation marks, which indicate a string constant. Often the right +quotation marks, which indicate a string constant. Often, the right time to do this is at the beginning of execution, before any input is processed, so that the very first record is read with the proper separator. To do this, use the special `BEGIN' pattern (*note @@ -4143,14 +4229,14 @@ BEGIN/END::). For example: awk 'BEGIN { RS = "u" } { print $0 }' mail-list -changes the value of `RS' to `u', before reading any input. This is a -string whose first character is the letter "u;" as a result, records -are separated by the letter "u." Then the input file is read, and the -second rule in the `awk' program (the action with no pattern) prints -each record. Because each `print' statement adds a newline at the end -of its output, this `awk' program copies the input with each `u' -changed to a newline. Here are the results of running the program on -`mail-list': +changes the value of `RS' to `u', before reading any input. The new +value is a string whose first character is the letter "u"; as a result, +records are separated by the letter "u". Then the input file is read, +and the second rule in the `awk' program (the action with no pattern) +prints each record. Because each `print' statement adds a newline at +the end of its output, this `awk' program copies the input with each +`u' changed to a newline. Here are the results of running the program +on `mail-list': $ awk 'BEGIN { RS = "u" } > { print $0 }' mail-list @@ -4198,11 +4284,11 @@ data file (*note Sample Data Files::), the line looks like this: Bill 555-1675 bill.drowning@hotmail.com A -It contains no `u' so there is no reason to split the record, unlike -the others which have one or more occurrences of the `u'. In fact, -this record is treated as part of the previous record; the newline -separating them in the output is the original newline in the data file, -not the one added by `awk' when it printed the record! +It contains no `u', so there is no reason to split the record, unlike +the others, which each have one or more occurrences of the `u'. In +fact, this record is treated as part of the previous record; the +newline separating them in the output is the original newline in the +data file, not the one added by `awk' when it printed the record! Another way to change the record separator is on the command line, using the variable-assignment feature (*note Other Arguments::): @@ -4250,7 +4336,7 @@ variable `RT' to the text in the input that matched `RS'. File: gawk.info, Node: gawk split records, Prev: awk split records, Up: Records -4.1.2 Record Splitting With `gawk' +4.1.2 Record Splitting with `gawk' ---------------------------------- When using `gawk', the value of `RS' is not limited to a one-character @@ -4268,8 +4354,8 @@ part of either record. character. However, when `RS' is a regular expression, `RT' contains the actual input text that matched the regular expression. - If the input file ended without any text that matches `RS', `gawk' -sets `RT' to the null string. + If the input file ends without any text matching `RS', `gawk' sets +`RT' to the null string. The following example illustrates both of these features. It sets `RS' equal to a regular expression that matches either a newline or a @@ -4290,7 +4376,7 @@ leading and trailing whitespace. The final value of `RT' is a newline. `RT'. If you set `RS' to a regular expression that allows optional -trailing text, such as `RS = "abc(XYZ)?"' it is possible, due to +trailing text, such as `RS = "abc(XYZ)?"', it is possible, due to implementation constraints, that `gawk' may match the leading part of the regular expression, but not the trailing part, particularly if the input text that could match the trailing part is fairly long. `gawk' @@ -4303,7 +4389,7 @@ that this will never happen. `RS = "^[[:upper:]]"' can only match at the beginning of a file. This is because `gawk' views the input file as one long string that happens to contain newline characters. It is thus best to - avoid anchor characters in the value of `RS'. + avoid anchor metacharacters in the value of `RS'. The use of `RS' as a regular expression and the `RT' variable are `gawk' extensions; they are not available in compatibility mode (*note @@ -4358,8 +4444,8 @@ When `awk' reads an input record, the record is automatically "parsed" or separated by the `awk' utility into chunks called "fields". By default, fields are separated by "whitespace", like words in a line. Whitespace in `awk' means any string of one or more spaces, TABs, or -newlines;(1) other characters, such as formfeed, vertical tab, etc., -that are considered whitespace by other languages, are _not_ considered +newlines;(1) other characters that are considered whitespace by other +languages (such as formfeed, vertical tab, etc.) are _not_ considered whitespace by `awk'. The purpose of fields is to make it more convenient for you to refer @@ -4367,12 +4453,12 @@ to these pieces of the record. You don't have to use them--you can operate on the whole record if you want--but fields are what make simple `awk' programs so powerful. - You use a dollar-sign (`$') to refer to a field in an `awk' program, + You use a dollar sign (`$') to refer to a field in an `awk' program, followed by the number of the field you want. Thus, `$1' refers to the -first field, `$2' to the second, and so on. (Unlike the Unix shells, -the field numbers are not limited to single digits. `$127' is the one -hundred twenty-seventh field in the record.) For example, suppose the -following is a line of input: +first field, `$2' to the second, and so on. (Unlike in the Unix +shells, the field numbers are not limited to single digits. `$127' is +the 127th field in the record.) For example, suppose the following is +a line of input: This seems like a pretty nice example. @@ -4389,10 +4475,9 @@ as `$7', which is `example.'. If you try to reference a field beyond the last one (such as `$8' when the record has only seven fields), you get the empty string. (If used in a numeric operation, you get zero.) - The use of `$0', which looks like a reference to the "zero-th" -field, is a special case: it represents the whole input record. Use it -when you are not interested in specific fields. Here are some more -examples: + The use of `$0', which looks like a reference to the "zeroth" field, +is a special case: it represents the whole input record. Use it when +you are not interested in specific fields. Here are some more examples: $ awk '$1 ~ /li/ { print $0 }' mail-list -| Amelia 555-5553 amelia.zodiacusque@gmail.com F @@ -4431,17 +4516,17 @@ example: awk '{ print $NR }' Recall that `NR' is the number of records read so far: one in the first -record, two in the second, etc. So this example prints the first field -of the first record, the second field of the second record, and so on. -For the twentieth record, field number 20 is printed; most likely, the -record has fewer than 20 fields, so this prints a blank line. Here is -another example of using expressions as field numbers: +record, two in the second, and so on. So this example prints the first +field of the first record, the second field of the second record, and so +on. For the twentieth record, field number 20 is printed; most likely, +the record has fewer than 20 fields, so this prints a blank line. Here +is another example of using expressions as field numbers: awk '{ print $(2*2) }' mail-list `awk' evaluates the expression `(2*2)' and uses its value as the -number of the field to print. The `*' sign represents multiplication, -so the expression `2*2' evaluates to four. The parentheses are used so +number of the field to print. The `*' represents multiplication, so +the expression `2*2' evaluates to four. The parentheses are used so that the multiplication is done before the `$' operation; they are necessary whenever there is a binary operator(1) in the field-number expression. This example, then, prints the type of relationship (the @@ -4465,7 +4550,7 @@ field number. ---------- Footnotes ---------- (1) A "binary operator", such as `*' for multiplication, is one that -takes two operands. The distinction is required, since `awk' also has +takes two operands. The distinction is required because `awk' also has unary (one-operand) and ternary (three-operand) operators. @@ -4511,8 +4596,8 @@ subtracted from the second field of each line: -| Mar 5 24 34 228 ... - It is also possible to also assign contents to fields that are out -of range. For example: + It is also possible to assign contents to fields that are out of +range. For example: $ awk '{ $6 = ($5 + $4 + $3 + $2) > print $6 }' inventory-shipped @@ -4587,7 +4672,7 @@ value of `NF' and recomputes `$0'. (d.c.) Here is an example: decremented. Finally, there are times when it is convenient to force `awk' to -rebuild the entire record, using the current value of the fields and +rebuild the entire record, using the current values of the fields and `OFS'. To do this, use the seemingly innocuous assignment: $1 = $1 # force record to be reconstituted @@ -4607,13 +4692,13 @@ built-in function that updates `$0', such as `sub()' and `gsub()' It is important to remember that `$0' is the _full_ record, exactly as it was read from the input. This includes any leading or trailing whitespace, and the exact whitespace (or other characters) that -separate the fields. +separates the fields. It is a common error to try to change the field separators in a record simply by setting `FS' and `OFS', and then expecting a plain `print' or `print $0' to print the modified record. - But this does not work, since nothing was done to change the record + But this does not work, because nothing was done to change the record itself. Instead, you must force the record to be rebuilt, typically with a statement such as `$1 = $1', as described earlier. @@ -4653,7 +4738,7 @@ is used by the POSIX-compliant shells (such as the Unix Bourne shell, `sh', or Bash). The value of `FS' can be changed in the `awk' program with the -assignment operator, `=' (*note Assignment Ops::). Often the right +assignment operator, `=' (*note Assignment Ops::). Often, the right time to do this is at the beginning of execution before any input has been processed, so that the very first record is read with the proper separator. To do this, use the special `BEGIN' pattern (*note @@ -4675,7 +4760,7 @@ attached, such as: John Q. Smith, LXIX, 29 Oak St., Walamazoo, MI 42139 -The same program would extract `*LXIX', instead of `*29*Oak*St.'. If +The same program would extract `*LXIX' instead of `*29*Oak*St.'. If you were expecting the program to print the address, you would be surprised. The moral is to choose your data layout and separator characters carefully to prevent such problems. (If the data is not in @@ -4765,7 +4850,7 @@ was ignored when finding `$1', it is not part of the new `$0'. Finally, the last `print' statement prints the new `$0'. There is an additional subtlety to be aware of when using regular -expressions for field splitting. It is not well-specified in the POSIX +expressions for field splitting. It is not well specified in the POSIX standard, or anywhere else, what `^' means when splitting fields. Does the `^' match only at the beginning of the entire record? Or is each field separator a new string? It turns out that different `awk' @@ -4874,11 +4959,11 @@ your field and record separators. Perhaps the most common use of a single character as the field separator occurs when processing the Unix system password file. On many Unix systems, each user has a separate entry in the system -password file, one line per user. The information in these lines is -separated by colons. The first field is the user's login name and the -second is the user's encrypted or shadow password. (A shadow password -is indicated by the presence of a single `x' in the second field.) A -password file entry might look like this: +password file, with one line per user. The information in these lines +is separated by colons. The first field is the user's login name and +the second is the user's encrypted or shadow password. (A shadow +password is indicated by the presence of a single `x' in the second +field.) A password file entry might look like this: arnold:x:2076:10:Arnold Robbins:/home/arnold:/bin/bash @@ -4890,21 +4975,51 @@ the entries for users whose full name is not indicated: File: gawk.info, Node: Full Line Fields, Next: Field Splitting Summary, Prev: Command Line Field Separator, Up: Field Separators -4.5.5 Making The Full Line Be A Single Field +4.5.5 Making the Full Line Be a Single Field -------------------------------------------- Occasionally, it's useful to treat the whole input line as a single field. This can be done easily and portably simply by setting `FS' to -`"\n"' (a newline).(1) +`"\n"' (a newline):(1) awk -F'\n' 'PROGRAM' FILES ... When you do this, `$1' is the same as `$0'. + Changing `FS' Does Not Affect the Fields + + According to the POSIX standard, `awk' is supposed to behave as if +each record is split into fields at the time it is read. In +particular, this means that if you change the value of `FS' after a +record is read, the values of the fields (i.e., how they were split) +should reflect the old value of `FS', not the new one. + + However, many older implementations of `awk' do not work this way. +Instead, they defer splitting the fields until a field is actually +referenced. The fields are split using the _current_ value of `FS'! +(d.c.) This behavior can be difficult to diagnose. The following +example illustrates the difference between the two methods: + + sed 1q /etc/passwd | awk '{ FS = ":" ; print $1 }' + +which usually prints: + + root + +on an incorrect implementation of `awk', while `gawk' prints the full +first line of the file, something like: + + root:x:0:0:Root:/: + + (The `sed'(2) command prints just the first line of `/etc/passwd'.) + ---------- Footnotes ---------- (1) Thanks to Andrew Schorr for this tip. + (2) The `sed' utility is a "stream editor." Its behavior is also +defined by the POSIX standard. + File: gawk.info, Node: Field Splitting Summary, Prev: Full Line Fields, Up: Field Separators @@ -4943,32 +5058,6 @@ value of `FS' (`==' means "is equal to"): (This is a common extension; it is not specified by the POSIX standard.) - Changing `FS' Does Not Affect the Fields - - According to the POSIX standard, `awk' is supposed to behave as if -each record is split into fields at the time it is read. In -particular, this means that if you change the value of `FS' after a -record is read, the value of the fields (i.e., how they were split) -should reflect the old value of `FS', not the new one. - - However, many older implementations of `awk' do not work this way. -Instead, they defer splitting the fields until a field is actually -referenced. The fields are split using the _current_ value of `FS'! -(d.c.) This behavior can be difficult to diagnose. The following -example illustrates the difference between the two methods. (The -`sed'(1) command prints just the first line of `/etc/passwd'.) - - sed 1q /etc/passwd | awk '{ FS = ":" ; print $1 }' - -which usually prints: - - root - -on an incorrect implementation of `awk', while `gawk' prints the full -first line of the file, something like: - - root:nSijPlPhZZwgE:0:0:Root:/: - `FS' and `IGNORECASE' The `IGNORECASE' variable (*note User-modified::) affects field @@ -4983,23 +5072,17 @@ Thus, in the following code: The output is `aCa'. If you really want to split fields on an alphabetic character while ignoring case, use a regexp that will do it -for you. E.g., `FS = "[c]"'. In this case, `IGNORECASE' will take +for you (e.g., `FS = "[c]"'). In this case, `IGNORECASE' will take effect. - ---------- Footnotes ---------- - - (1) The `sed' utility is a "stream editor." Its behavior is also -defined by the POSIX standard. - File: gawk.info, Node: Constant Size, Next: Splitting By Content, Prev: Field Separators, Up: Reading Files 4.6 Reading Fixed-Width Data ============================ - NOTE: This minor node discusses an advanced feature of `gawk'. If - you are a novice `awk' user, you might want to skip it on the - first reading. +This minor node discusses an advanced feature of `gawk'. If you are a +novice `awk' user, you might want to skip it on the first reading. `gawk' provides a facility for dealing with fixed-width fields with no distinctive field separator. For example, data of this nature @@ -5020,9 +5103,9 @@ the built-in variable `FIELDWIDTHS'. Each number specifies the width of the field, _including_ columns between fields. If you want to ignore the columns between fields, you can specify the width as a separate field that is subsequently ignored. It is a fatal error to -supply a field width that is not a positive number. The following data -is the output of the Unix `w' utility. It is useful to illustrate the -use of `FIELDWIDTHS': +supply a field width that has a negative value. The following data is +the output of the Unix `w' utility. It is useful to illustrate the use +of `FIELDWIDTHS': 10:06pm up 21 days, 14:04, 23 users User tty login idle JCPU PCPU what @@ -5035,13 +5118,10 @@ use of `FIELDWIDTHS': brent ttyp0 26Jun91 4:46 26:46 4:41 bash dave ttyq4 26Jun9115days 46 46 wnewmail - The following program takes the above input, converts the idle time -to number of seconds, and prints out the first two fields and the + The following program takes this input, converts the idle time to +number of seconds, and prints out the first two fields and the calculated idle time: - NOTE: This program uses a number of `awk' features that haven't - been introduced yet. - BEGIN { FIELDWIDTHS = "9 6 10 6 7 7 35" } NR > 2 { idle = $4 @@ -5058,6 +5138,9 @@ calculated idle time: print $1, $2, idle } + NOTE: The preceding program uses a number of `awk' features that + haven't been introduced yet. + Running the program on the data produces the following results: hzuo ttyV0 0 @@ -5083,7 +5166,7 @@ run on a system with card readers is another story!) splitting again. Use `FS = FS' to make this happen, without having to know the current value of `FS'. In order to tell which kind of field splitting is in effect, use `PROCINFO["FS"]' (*note Auto-set::). The -value is `"FS"' if regular field splitting is being used, or it is +value is `"FS"' if regular field splitting is being used, or `"FIELDWIDTHS"' if fixed-width field splitting is being used: if (PROCINFO["FS"] == "FS") @@ -5101,12 +5184,11 @@ of such a function). File: gawk.info, Node: Splitting By Content, Next: Multiple Line, Prev: Constant Size, Up: Reading Files -4.7 Defining Fields By Content +4.7 Defining Fields by Content ============================== - NOTE: This minor node discusses an advanced feature of `gawk'. If - you are a novice `awk' user, you might want to skip it on the - first reading. +This minor node discusses an advanced feature of `gawk'. If you are a +novice `awk' user, you might want to skip it on the first reading. Normally, when using `FS', `gawk' defines the fields as the parts of the record that occur in between each field separator. In other words, @@ -5114,14 +5196,13 @@ the record that occur in between each field separator. In other words, However, there are times when you really want to define the fields by what they are, and not by what they are not. - The most notorious such case is so-called "comma separated value" + The most notorious such case is so-called "comma-separated values" (CSV) data. Many spreadsheet programs, for example, can export their data into text files, where each record is terminated with a newline, -and fields are separated by commas. If only commas separated the data, +and fields are separated by commas. If commas only separated the data, there wouldn't be an issue. The problem comes when one of the fields -contains an _embedded_ comma. While there is no formal standard -specification for CSV data(1), in such cases, most programs embed the -field in double quotes. So we might have data like this: +contains an _embedded_ comma. In such cases, most programs embed the +field in double quotes.(1) So, we might have data like this: Robbins,Arnold,"1234 A Pretty Street, NE",MyTown,MyState,12345-6789,USA @@ -5129,7 +5210,7 @@ field in double quotes. So we might have data like this: value of `FPAT' should be a string that provides a regular expression. This regular expression describes the contents of each field. - In the case of CSV data as presented above, each field is either + In the case of CSV data as presented here, each field is either "anything that is not a comma," or "a double quote, anything that is not a double quote, and a closing double quote." If written as a regular expression constant (*note Regexp::), we would have @@ -5183,14 +5264,14 @@ being used. NOTE: Some programs export CSV data that contains embedded newlines between the double quotes. `gawk' provides no way to - deal with this. Since there is no formal specification for CSV - data, there isn't much more to be done; the `FPAT' mechanism + deal with this. Even though a formal specification for CSV data + exists, there isn't much more to be done; the `FPAT' mechanism provides an elegant solution for the majority of cases, and the `gawk' developers are satisfied with that. - As written, the regexp used for `FPAT' requires that each field have -a least one character. A straightforward modification (changing -changed the first `+' to `*') allows fields to be empty: + As written, the regexp used for `FPAT' requires that each field +contain at least one character. A straightforward modification +(changing the first `+' to `*') allows fields to be empty: FPAT = "([^,]*)|(\"[^\"]+\")" @@ -5198,13 +5279,14 @@ changed the first `+' to `*') allows fields to be empty: available for splitting regular strings (*note String Functions::). To recap, `gawk' provides three independent methods to split input -records into fields. `gawk' uses whichever mechanism was last chosen -based on which of the three variables--`FS', `FIELDWIDTHS', and -`FPAT'--was last assigned to. +records into fields. The mechanism used is based on which of the three +variables--`FS', `FIELDWIDTHS', or `FPAT'--was last assigned to. ---------- Footnotes ---------- - (1) At least, we don't know of one. + (1) The CSV format lacked a formal standard definition for many +years. RFC 4180 (http://www.ietf.org/rfc/rfc4180.txt) standardizes the +most common practices. File: gawk.info, Node: Multiple Line, Next: Getline, Prev: Splitting By Content, Up: Reading Files @@ -5236,7 +5318,7 @@ empty; lines that contain only whitespace do not count.) `"\n\n+"' to `RS'. This regexp matches the newline at the end of the record and one or more blank lines after the record. In addition, a regular expression always matches the longest possible sequence when -there is a choice (*note Leftmost Longest::). So the next record +there is a choice (*note Leftmost Longest::). So, the next record doesn't start until the first nonblank line that follows--no matter how many blank lines appear in a row, they are considered one record separator. @@ -5248,12 +5330,12 @@ last record, the final newline is removed from the record. In the second case, this special processing is not done. (d.c.) Now that the input is separated into records, the second step is to -separate the fields in the record. One way to do this is to divide each -of the lines into fields in the normal manner. This happens by default -as the result of a special feature. When `RS' is set to the empty -string, _and_ `FS' is set to a single character, the newline character -_always_ acts as a field separator. This is in addition to whatever -field separations result from `FS'.(1) +separate the fields in the records. One way to do this is to divide +each of the lines into fields in the normal manner. This happens by +default as the result of a special feature. When `RS' is set to the +empty string _and_ `FS' is set to a single character, the newline +character _always_ acts as a field separator. This is in addition to +whatever field separations result from `FS'.(1) The original motivation for this special exception was probably to provide useful behavior in the default case (i.e., `FS' is equal to @@ -5261,17 +5343,17 @@ provide useful behavior in the default case (i.e., `FS' is equal to newline character to separate fields, because there is no way to prevent it. However, you can work around this by using the `split()' function to break up the record manually (*note String Functions::). -If you have a single character field separator, you can work around the +If you have a single-character field separator, you can work around the special feature in a different way, by making `FS' into a regexp for that single character. For example, if the field separator is a percent character, instead of `FS = "%"', use `FS = "[%]"'. Another way to separate fields is to put each field on a separate line: to do this, just set the variable `FS' to the string `"\n"'. -(This single character separator matches a single newline.) A +(This single-character separator matches a single newline.) A practical example of a data file organized this way might be a mailing -list, where each entry is separated by blank lines. Consider a mailing -list in a file named `addresses', which looks like this: +list, where blank lines separate the entries. Consider a mailing list +in a file named `addresses', which looks like this: Jane Doe 123 Main Street @@ -5310,7 +5392,7 @@ A simple program to process this file is as follows: -| ... - *Note Labels Program::, for a more realistic program that deals with + *Note Labels Program::, for a more realistic program dealing with address lists. The following list summarizes how records are split, based on the value of `RS'. (`==' means "is equal to.") @@ -5354,7 +5436,7 @@ File: gawk.info, Node: Getline, Next: Read Timeout, Prev: Multiple Line, Up: So far we have been getting our input data from `awk''s main input stream--either the standard input (usually your keyboard, sometimes the -output from another program) or from the files specified on the command +output from another program) or the files specified on the command line. The `awk' language has a special built-in command called `getline' that can be used to read input under your explicit control. @@ -5371,11 +5453,16 @@ record, such as a file that cannot be opened, then `getline' returns -1. In this case, `gawk' sets the variable `ERRNO' to a string describing the error that occurred. + If `ERRNO' indicates that the I/O operation may be retried, and +`PROCINFO["INPUT", "RETRY"]' is set, then `getline' returns -2 instead +of -1, and further calls to `getline' may be attemped. *Note Retrying +Input::, for further information about this feature. + In the following examples, COMMAND stands for a string value that represents a shell command. NOTE: When `--sandbox' is specified (*note Options::), reading - lines from files, pipes and coprocesses is disabled. + lines from files, pipes, and coprocesses is disabled. * Menu: @@ -5492,7 +5579,7 @@ and produces these results: free The `getline' command used in this way sets only the variables `NR', -`FNR' and `RT' (and of course, VAR). The record is not split into +`FNR', and `RT' (and, of course, VAR). The record is not split into fields, so the values of the fields (including `$0') and the value of `NF' do not change. @@ -5502,8 +5589,8 @@ File: gawk.info, Node: Getline/File, Next: Getline/Variable/File, Prev: Getli 4.9.3 Using `getline' from a File --------------------------------- -Use `getline < FILE' to read the next record from FILE. Here FILE is a -string-valued expression that specifies the file name. `< FILE' is +Use `getline < FILE' to read the next record from FILE. Here, FILE is +a string-valued expression that specifies the file name. `< FILE' is called a "redirection" because it directs input to come from a different place. For example, the following program reads its input record from the file `secondary.input' when it encounters a first field @@ -5536,8 +5623,8 @@ File: gawk.info, Node: Getline/Variable/File, Next: Getline/Pipe, Prev: Getli ------------------------------------------------- Use `getline VAR < FILE' to read input from the file FILE, and put it -in the variable VAR. As above, FILE is a string-valued expression that -specifies the file from which to read. +in the variable VAR. As earlier, FILE is a string-valued expression +that specifies the file from which to read. In this version of `getline', none of the predefined variables are changed and the record is not split into fields. The only variable @@ -5639,8 +5726,8 @@ all `awk' implementations. treatment of a construct like `"echo " "date" | getline'. Most versions, including the current version, treat it at as `("echo " "date") | getline'. (This is also how BWK `awk' behaves.) Some - versions changed and treated it as `"echo " ("date" | getline)'. - (This is how `mawk' behaves.) In short, _always_ use explicit + versions instead treat it as `"echo " ("date" | getline)'. (This + is how `mawk' behaves.) In short, _always_ use explicit parentheses, and then you won't have to worry. @@ -5676,15 +5763,16 @@ File: gawk.info, Node: Getline/Coprocess, Next: Getline/Variable/Coprocess, P 4.9.7 Using `getline' from a Coprocess -------------------------------------- -Input into `getline' from a pipe is a one-way operation. The command -that is started with `COMMAND | getline' only sends data _to_ your -`awk' program. +Reading input into `getline' from a pipe is a one-way operation. The +command that is started with `COMMAND | getline' only sends data _to_ +your `awk' program. On occasion, you might want to send data to another program for processing and then read the results back. `gawk' allows you to start a "coprocess", with which two-way communications are possible. This is done with the `|&' operator. Typically, you write data to the -coprocess first and then read results back, as shown in the following: +coprocess first and then read the results back, as shown in the +following: print "SOME QUERY" |& "db_server" "db_server" |& getline @@ -5746,23 +5834,23 @@ in mind: files. (d.c.) (See *note BEGIN/END::; also *note Auto-set::.) * Using `FILENAME' with `getline' (`getline < FILENAME') is likely - to be a source for confusion. `awk' opens a separate input stream + to be a source of confusion. `awk' opens a separate input stream from the current input file. However, by not using a variable, - `$0' and `NR' are still updated. If you're doing this, it's + `$0' and `NF' are still updated. If you're doing this, it's probably by accident, and you should reconsider what it is you're trying to accomplish. * *note Getline Summary::, presents a table summarizing the `getline' variants and which variables they can affect. It is - worth noting that those variants which do not use redirection can + worth noting that those variants that do not use redirection can cause `FILENAME' to be updated if they cause `awk' to start reading a new input file. * If the variable being assigned is an expression with side effects, different versions of `awk' behave differently upon encountering end-of-file. Some versions don't evaluate the expression; many - versions (including `gawk') do. Here is an example, due to Duncan - Moore: + versions (including `gawk') do. Here is an example, courtesy of + Duncan Moore: BEGIN { system("echo 1 > f") @@ -5770,8 +5858,8 @@ in mind: print c } - Here, the side effect is the `++c'. Is `c' incremented if end of - file is encountered, before the element in `a' is assigned? + Here, the side effect is the `++c'. Is `c' incremented if + end-of-file is encountered before the element in `a' is assigned? `gawk' treats `getline' like a function call, and evaluates the expression `a[++c]' before attempting to read from `f'. However, @@ -5803,20 +5891,20 @@ COMMAND `|& getline' Sets `$0', `NF', and `RT' `gawk' COMMAND `|& getline' Sets VAR and `RT' `gawk' VAR -Table 4.1: `getline' Variants and What They Set +Table 4.1: `getline' variants and what they set -File: gawk.info, Node: Read Timeout, Next: Command-line directories, Prev: Getline, Up: Reading Files +File: gawk.info, Node: Read Timeout, Next: Retrying Input, Prev: Getline, Up: Reading Files -4.10 Reading Input With A Timeout +4.10 Reading Input with a Timeout ================================= This minor node describes a feature that is specific to `gawk'. You may specify a timeout in milliseconds for reading input from the keyboard, a pipe, or two-way communication, including TCP/IP sockets. -This can be done on a per input, command or connection basis, by -setting a special element in the `PROCINFO' array (*note Auto-set::): +This can be done on a per-input, per-command, or per-connection basis, +by setting a special element in the `PROCINFO' array (*note Auto-set::): PROCINFO["input_name", "READ_TIMEOUT"] = TIMEOUT IN MILLISECONDS @@ -5840,7 +5928,7 @@ for more than five seconds: print $0 `gawk' terminates the read operation if input does not arrive after -waiting for the timeout period, returns failure and sets `ERRNO' to an +waiting for the timeout period, returns failure, and sets `ERRNO' to an appropriate string value. A negative or zero value for the timeout is the same as specifying no timeout at all. @@ -5848,7 +5936,7 @@ the same as specifying no timeout at all. implicit loop that reads input records and matches them against patterns, like so: - $ gawk 'BEGIN { PROCINFO["-", "READ_TIMEOUT"] = 5000 } + $ gawk 'BEGIN { PROCINFO["-", "READ_TIMEOUT"] = 5000 } > { print "You entered: " $0 }' gawk -| You entered: gawk @@ -5867,25 +5955,26 @@ input to arrive: PROCINFO[Service, "READ_TIMEOUT"] = 1000 while ((Service |& getline) > 0) { print $0 - PROCINFO[S, "READ_TIMEOUT"] -= 100 + PROCINFO[Service, "READ_TIMEOUT"] -= 100 } NOTE: You should not assume that the read operation will block exactly after the tenth record has been printed. It is possible that `gawk' will read and buffer more than one record's worth of data the first time. Because of this, changing the value of - timeout like in the above example is not very useful. + timeout like in the preceding example is not very useful. - If the `PROCINFO' element is not present and the environment -variable `GAWK_READ_TIMEOUT' exists, `gawk' uses its value to -initialize the timeout value. The exclusive use of the environment -variable to specify timeout has the disadvantage of not being able to -control it on a per command or connection basis. + If the `PROCINFO' element is not present and the `GAWK_READ_TIMEOUT' +environment variable exists, `gawk' uses its value to initialize the +timeout value. The exclusive use of the environment variable to +specify timeout has the disadvantage of not being able to control it on +a per-command or per-connection basis. `gawk' considers a timeout event to be an error even though the attempt to read from the underlying device may succeed in a later attempt. This is a limitation, and it also means that you cannot use -this to multiplex input from two or more sources. +this to multiplex input from two or more sources. *Note Retrying +Input::, for a way to enable later I/O attempts to succeed. Assigning a timeout value prevents read operations from blocking indefinitely. But bear in mind that there are other ways `gawk' can @@ -5900,9 +5989,36 @@ writing. (1) This assumes that standard input is the keyboard. -File: gawk.info, Node: Command-line directories, Next: Input Summary, Prev: Read Timeout, Up: Reading Files +File: gawk.info, Node: Retrying Input, Next: Command-line directories, Prev: Read Timeout, Up: Reading Files + +4.11 Retrying Reads After Certain Input Errors +============================================== + +This minor node describes a feature that is specific to `gawk'. + + When `gawk' encounters an error while reading input, by default +`getline' returns -1, and subsequent attempts to read from that file +result in an end-of-file indication. However, you may optionally +instruct `gawk' to allow I/O to be retried when certain errors are +encountered by setting a special element in the `PROCINFO' array (*note +Auto-set::): + + PROCINFO["INPUT_NAME", "RETRY"] = 1 + + When this element exists, `gawk' checks the value of the system (C +language) `errno' variable when an I/O error occurs. If `errno' +indicates a subsequent I/O attempt may succeed, `getline' instead +returns -2 and further calls to `getline' may succeed. This applies to +the `errno' values `EAGAIN', `EWOULDBLOCK', `EINTR', or `ETIMEDOUT'. + + This feature is useful in conjunction with `PROCINFO["INPUT_NAME", +"READ_TIMEOUT"]' or situations where a file descriptor has been +configured to behave in a non-blocking fashion. -4.11 Directories On The Command Line + +File: gawk.info, Node: Command-line directories, Next: Input Summary, Prev: Retrying Input, Up: Reading Files + +4.12 Directories on the Command Line ==================================== According to the POSIX standard, files named on the `awk' command line @@ -5913,7 +6029,7 @@ of `awk' treat a directory on the command line as a fatal error. line, but otherwise ignores it. This makes it easier to use shell wildcards with your `awk' program: - $ gawk -f whizprog.awk * Directories could kill this progam + $ gawk -f whizprog.awk * Directories could kill this program If either of the `--posix' or `--traditional' options is given, then `gawk' reverts to treating a directory on the command line as a fatal @@ -5925,13 +6041,14 @@ usable data from an `awk' program. File: gawk.info, Node: Input Summary, Next: Input Exercises, Prev: Command-line directories, Up: Reading Files -4.12 Summary +4.13 Summary ============ * Input is split into records based on the value of `RS'. The possibilities are as follows: Value of `RS' Records are split on `awk' / `gawk' + ... ---------------------------------------------------------------------- Any single That character `awk' character @@ -5947,8 +6064,8 @@ File: gawk.info, Node: Input Summary, Next: Input Exercises, Prev: Command-li * `gawk' sets `RT' to the text matched by `RS'. * After splitting the input into records, `awk' further splits the - record into individual fields, named `$1', `$2' and so on. `$0' is - the whole record, and `NF' indicates how many fields there are. + records into individual fields, named `$1', `$2', and so on. `$0' + is the whole record, and `NF' indicates how many fields there are. The default way to split fields is between whitespace characters. * Fields may be referenced using a variable, as in `$NF'. Fields @@ -5959,21 +6076,23 @@ File: gawk.info, Node: Input Summary, Next: Input Exercises, Prev: Command-li does the same thing. Decrementing `NF' throws away fields and rebuilds the record. - * Field splitting is more complicated than record splitting. + * Field splitting is more complicated than record splitting: - Field separator value Fields are split ... `awk' / - `gawk' + Field separator value Fields are split ... `awk' / + `gawk' ---------------------------------------------------------------------- - `FS == " "' On runs of whitespace `awk' - `FS == ANY SINGLE On that character `awk' - CHARACTER' - `FS == REGEXP' On text matching the regexp `awk' - `FS == ""' Each individual character is `gawk' - a separate field - `FIELDWIDTHS == LIST OF Based on character position `gawk' - COLUMNS' - `FPAT == REGEXP' On the text surrounding text `gawk' - matching the regexp + `FS == " "' On runs of whitespace `awk' + `FS == ANY SINGLE On that character `awk' + CHARACTER' + `FS == REGEXP' On text matching the `awk' + regexp + `FS == ""' Such that each individual `gawk' + character is a separate + field + `FIELDWIDTHS == LIST OF Based on character `gawk' + COLUMNS' position + `FPAT == REGEXP' On the text surrounding `gawk' + text matching the regexp * Using `FS = "\n"' causes the entire record to be a single field (assuming that newlines separate records). @@ -5983,12 +6102,11 @@ File: gawk.info, Node: Input Summary, Next: Input Exercises, Prev: Command-li * Use `PROCINFO["FS"]' to see how fields are being split. - * Use `getline' in its various forms to read additional records, - from the default input stream, from a file, or from a pipe or - coprocess. + * Use `getline' in its various forms to read additional records from + the default input stream, from a file, or from a pipe or coprocess. - * Use `PROCINFO[FILE, "READ_TIMEOUT"]' to cause reads to timeout for - FILE. + * Use `PROCINFO[FILE, "READ_TIMEOUT"]' to cause reads to time out + for FILE. * Directories on the command line are fatal for standard `awk'; `gawk' ignores them if not in POSIX mode. @@ -5997,7 +6115,7 @@ File: gawk.info, Node: Input Summary, Next: Input Exercises, Prev: Command-li File: gawk.info, Node: Input Exercises, Prev: Input Summary, Up: Reading Files -4.13 Exercises +4.14 Exercises ============== 1. Using the `FIELDWIDTHS' variable (*note Constant Size::), write a @@ -6048,6 +6166,7 @@ function. `gawk' allows access to inherited file descriptors. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. @@ -6082,7 +6201,7 @@ you will probably get an error. Keep in mind that a space is printed between any two items. Note that the `print' statement is a statement and not an -expression--you can't use it in the pattern part of a PATTERN-ACTION +expression--you can't use it in the pattern part of a pattern-action statement, for example. @@ -6176,14 +6295,15 @@ separated by commas. In the output, the items are normally separated by single spaces. However, this doesn't need to be the case; a single space is simply the default. Any string of characters may be used as the "output field separator" by setting the predefined variable `OFS'. -The initial value of this variable is the string `" "'--that is, a -single space. +The initial value of this variable is the string `" "' (i.e., a single +space). The output from an entire `print' statement is called an "output record". Each `print' statement outputs one output record, and then outputs a string called the "output record separator" (or `ORS'). The -initial value of `ORS' is the string `"\n"'; i.e., a newline character. -Thus, each `print' statement normally makes a separate line. +initial value of `ORS' is the string `"\n"' (i.e., a newline +character). Thus, each `print' statement normally makes a separate +line. In order to change how output fields and records are separated, assign new values to the variables `OFS' and `ORS'. The usual place to @@ -6229,7 +6349,7 @@ File: gawk.info, Node: OFMT, Next: Printf, Prev: Output Separators, Up: Prin =========================================== When printing numeric values with the `print' statement, `awk' -internally converts the number to a string of characters and prints +internally converts each number to a string of characters and prints that string. `awk' uses the `sprintf()' function to do this conversion (*note String Functions::). For now, it suffices to say that the `sprintf()' function accepts a "format specification" that tells it how @@ -6282,10 +6402,10 @@ A simple `printf' statement looks like this: printf FORMAT, ITEM1, ITEM2, ... -As print `print', the entire list of arguments may optionally be -enclosed in parentheses. Here too, the parentheses are necessary if any -of the item expressions use the `>' relational operator; otherwise, it -can be confused with an output redirection (*note Redirection::). +As for `print', the entire list of arguments may optionally be enclosed +in parentheses. Here too, the parentheses are necessary if any of the +item expressions uses the `>' relational operator; otherwise, it can be +confused with an output redirection (*note Redirection::). The difference between `printf' and `print' is the FORMAT argument. This is an expression whose value is taken as a string; it specifies @@ -6311,7 +6431,7 @@ statements. For example: > }' -| Don't Panic! -Here, neither the `+' nor the `OUCH' appear in the output message. +Here, neither the `+' nor the `OUCH!' appears in the output message. File: gawk.info, Node: Control Letters, Next: Format Modifiers, Prev: Basic Printf, Up: Printf @@ -6350,7 +6470,7 @@ width. Here is a list of the format-control letters: (The `%i' specification is for compatibility with ISO C.) `%e', `%E' - Print a number in scientific (exponential) notation; for example: + Print a number in scientific (exponential) notation. For example: printf "%4.3e\n", 1950 @@ -6368,14 +6488,14 @@ width. Here is a list of the format-control letters: of which follow the decimal point. (The `4.3' represents two modifiers, discussed in the next node.) - On systems supporting IEEE 754 floating point format, values + On systems supporting IEEE 754 floating-point format, values representing negative infinity are formatted as `-inf' or - `-infinity', and positive infinity as `inf' and `infinity'. The + `-infinity', and positive infinity as `inf' or `infinity'. The special "not a number" value formats as `-nan' or `nan' (*note Math Definitions::). `%F' - Like `%f' but the infinity and "not a number" values are spelled + Like `%f', but the infinity and "not a number" values are spelled using uppercase letters. The `%F' format is a POSIX extension to ISO C; not all systems @@ -6394,7 +6514,7 @@ width. Here is a list of the format-control letters: `%u' Print an unsigned decimal integer. (This format is of marginal - use, because all numbers in `awk' are floating-point; it is + use, because all numbers in `awk' are floating point; it is provided primarily for compatibility with C.) `%x', `%X' @@ -6444,7 +6564,7 @@ which they may appear: messages at runtime. *Note Printf Ordering::, which describes how and why to use positional specifiers. For now, we ignore them. -`-' +`-' (Minus) The minus sign, used before the width modifier (see later on in this list), says to left-justify the argument within its specified width. Normally, the argument is printed right-justified in the @@ -6454,7 +6574,7 @@ which they may appear: prints `foo*'. -`SPACE' +SPACE For numeric conversions, prefix positive values with a space and negative values with a minus sign. @@ -6465,7 +6585,7 @@ which they may appear: space modifier. `#' - Use an "alternate form" for certain control letters. For `%o', + Use an "alternative form" for certain control letters. For `%o', supply a leading zero. For `%x' and `%X', supply a leading `0x' or `0X' for a nonzero result. For `%e', `%E', `%f', and `%F', the result always contains a decimal point. For `%g' and `%G', @@ -6479,7 +6599,7 @@ which they may appear: `'' A single quote or apostrophe character is a POSIX extension to ISO - C. It indicates that the integer part of a floating point value, + C. It indicates that the integer part of a floating-point value, or the entire part of an integer decimal value, should have a thousands-separator character in it. This only works in locales that support such characters. For example: @@ -6499,7 +6619,7 @@ which they may appear: programs. For information on appropriate quoting tricks, see *note Quoting::. -`WIDTH' +WIDTH This is a number specifying the desired minimum width of a field. Inserting any number between the `%' sign and the format-control character forces the field to expand to this width. The default @@ -6544,10 +6664,10 @@ which they may appear: prints `foob'. - The C library `printf''s dynamic WIDTH and PREC capability (for -example, `"%*.*s"') is supported. Instead of supplying explicit WIDTH -and/or PREC values in the format string, they are passed in the -argument list. For example: + The C library `printf''s dynamic WIDTH and PREC capability (e.g., +`"%*.*s"') is supported. Instead of supplying explicit WIDTH and/or +PREC values in the format string, they are passed in the argument list. +For example: w = 5 p = 3 @@ -6569,7 +6689,7 @@ string, like so: s = "abcdefg" printf "%" w "." p "s\n", s -This is not particularly easy to read but it does work. +This is not particularly easy to read, but it does work. C programmers may be used to supplying additional modifiers (`h', `j', `l', `L', `t', and `z') in `printf' format strings. These are not @@ -6608,7 +6728,7 @@ an aligned two-column table of names and phone numbers, as shown here: -| Jean-Paul 555-2127 In this case, the phone numbers had to be printed as strings because -the numbers are separated by a dash. Printing the phone numbers as +the numbers are separated by dashes. Printing the phone numbers as numbers would have produced just the first three digits: `555'. This would have been pretty confusing. @@ -6625,8 +6745,9 @@ beginning of the `awk' program: print "---- ------" } { printf "%-10s %s\n", $1, $2 }' mail-list - The above example mixes `print' and `printf' statements in the same -program. Using just `printf' statements can produce the same results: + The preceding example mixes `print' and `printf' statements in the +same program. Using just `printf' statements can produce the same +results: awk 'BEGIN { printf "%-10s %s\n", "Name", "Number" printf "%-10s %s\n", "----", "------" } @@ -6655,7 +6776,7 @@ output, usually the screen. Both `print' and `printf' can also send their output to other places. This is called "redirection". NOTE: When `--sandbox' is specified (*note Options::), redirecting - output to files, pipes and coprocesses is disabled. + output to files, pipes, and coprocesses is disabled. A redirection appears after the `print' or `printf' statement. Redirections in `awk' are written just like redirections in shell @@ -6695,7 +6816,7 @@ work identically for `printf': Each output file contains one name or number per line. `print ITEMS >> OUTPUT-FILE' - This redirection prints the items into the pre-existing output file + This redirection prints the items into the preexisting output file named OUTPUT-FILE. The difference between this and the single-`>' redirection is that the old contents (if any) of OUTPUT-FILE are not erased. Instead, the `awk' output is appended to the file. @@ -6743,8 +6864,8 @@ work identically for `printf': `print ITEMS |& COMMAND' This redirection prints the items to the input of COMMAND. The difference between this and the single-`|' redirection is that the - output from COMMAND can be read with `getline'. Thus COMMAND is a - "coprocess", which works together with, but subsidiary to, the + output from COMMAND can be read with `getline'. Thus, COMMAND is + a "coprocess", which works together with but is subsidiary to the `awk' program. This feature is a `gawk' extension, and is not available in POSIX @@ -6767,8 +6888,8 @@ a file, and then to use `>>' for subsequent output: This is indeed how redirections must be used from the shell. But in `awk', it isn't necessary. In this kind of case, a program should use -`>' for all the `print' statements, since the output file is only -opened once. (It happens that if you mix `>' and `>>' that output is +`>' for all the `print' statements, because the output file is only +opened once. (It happens that if you mix `>' and `>>' output is produced in the expected order. However, mixing the operators for the same file is definitely poor style, and is confusing to readers of your program.) @@ -6801,14 +6922,14 @@ command lines to be fed to the shell. File: gawk.info, Node: Special FD, Next: Special Files, Prev: Redirection, Up: Printing -5.7 Special Files for Standard Pre-Opened Data Streams -====================================================== +5.7 Special Files for Standard Preopened Data Streams +===================================================== Running programs conventionally have three input and output streams already available to them for reading and writing. These are known as the "standard input", "standard output", and "standard error output". -These open streams (and any other open file or pipe) are often referred -to by the technical term "file descriptors". +These open streams (and any other open files or pipes) are often +referred to by the technical term "file descriptors". These streams are, by default, connected to your keyboard and screen, but they are often redirected with the shell, via the `<', `<<', @@ -6833,13 +6954,13 @@ error messages to the screen, like this: (`/dev/tty' is a special file supplied by the operating system that is connected to your keyboard and screen. It represents the "terminal,"(1) which on modern systems is a keyboard and screen, not a serial console.) -This generally has the same effect but not always: although the +This generally has the same effect, but not always: although the standard error stream is usually the screen, it can be redirected; when that happens, writing to the screen is not correct. In fact, if `awk' is run from a background job, it may not have a terminal at all. Then opening `/dev/tty' fails. - `gawk', BWK `awk' and `mawk' provide special file names for + `gawk', BWK `awk', and `mawk' provide special file names for accessing the three standard streams. If the file name matches one of these special names when `gawk' (or one of the others) redirects input or output, then it directly uses the descriptor that the file name @@ -6860,14 +6981,14 @@ becomes: print "Serious error detected!" > "/dev/stderr" - Note the use of quotes around the file name. Like any other + Note the use of quotes around the file name. Like with any other redirection, the value must be a string. It is a common error to omit the quotes, which leads to confusing results. - `gawk' does not treat these file names as special when in POSIX -compatibility mode. However, since BWK `awk' supports them, `gawk' does -support them even when invoked with the `--traditional' option (*note -Options::). + `gawk' does not treat these file names as special when in +POSIX-compatibility mode. However, because BWK `awk' supports them, +`gawk' does support them even when invoked with the `--traditional' +option (*note Options::). ---------- Footnotes ---------- @@ -6876,10 +6997,10 @@ Options::). File: gawk.info, Node: Special Files, Next: Close Files And Pipes, Prev: Special FD, Up: Printing -5.8 Special File Names in `gawk' +5.8 Special File names in `gawk' ================================ -Besides access to standard input, stanard output, and standard error, +Besides access to standard input, standard output, and standard error, `gawk' provides access to any open file descriptor. Additionally, there are special file names reserved for TCP/IP networking. @@ -6893,7 +7014,7 @@ there are special file names reserved for TCP/IP networking. File: gawk.info, Node: Other Inherited Files, Next: Special Network, Up: Special Files -5.8.1 Accessing Other Open Files With `gawk' +5.8.1 Accessing Other Open Files with `gawk' -------------------------------------------- Besides the `/dev/stdin', `/dev/stdout', and `/dev/stderr' special file @@ -6926,7 +7047,7 @@ form: `/NET-TYPE/PROTOCOL/LOCAL-PORT/REMOTE-HOST/REMOTE-PORT' - The NET-TYPE is one of `inet', `inet4' or `inet6'. The PROTOCOL is + The NET-TYPE is one of `inet', `inet4', or `inet6'. The PROTOCOL is one of `tcp' or `udp', and the other fields represent the other essential pieces of information for making a networking connection. These file names are used with the `|&' operator for communicating with @@ -6937,13 +7058,13 @@ mentioned here only for completeness. Full discussion is delayed until File: gawk.info, Node: Special Caveats, Prev: Special Network, Up: Special Files -5.8.3 Special File Name Caveats +5.8.3 Special File name Caveats ------------------------------- Here are some things to bear in mind when using the special file names that `gawk' provides: - * Recognition of the file names for the three standard pre-opened + * Recognition of the file names for the three standard preopened files is disabled only in POSIX mode. * Recognition of the other special file names is disabled if `gawk' @@ -6952,14 +7073,14 @@ that `gawk' provides: * `gawk' _always_ interprets these special file names. For example, using `/dev/fd/4' for output actually writes on file descriptor 4, - and not on a new file descriptor that is `dup()''ed from file + and not on a new file descriptor that is `dup()'ed from file descriptor 4. Most of the time this does not matter; however, it is important to _not_ close any of the files related to file descriptors 0, 1, and 2. Doing so results in unpredictable behavior. -File: gawk.info, Node: Close Files And Pipes, Next: Output Summary, Prev: Special Files, Up: Printing +File: gawk.info, Node: Close Files And Pipes, Next: Nonfatal, Prev: Special Files, Up: Printing 5.9 Closing Input and Output Redirections ========================================= @@ -7068,9 +7189,10 @@ terminated;(1) more importantly, the file descriptor for the pipe is not closed and released until `close()' is called or `awk' exits. `close()' silently does nothing if given an argument that does not -represent a file, pipe or coprocess that was opened with a redirection. -In such a case, it returns a negative value, indicating an error. In -addition, `gawk' sets `ERRNO' to a string indicating the error. +represent a file, pipe, or coprocess that was opened with a +redirection. In such a case, it returns a negative value, indicating +an error. In addition, `gawk' sets `ERRNO' to a string indicating the +error. Note also that `close(FILENAME)' has no "magic" effects on the implicit loop that reads through the files named on the command line. @@ -7091,8 +7213,8 @@ describes it in more detail and gives an example. Using `close()''s Return Value In many older versions of Unix `awk', the `close()' function is -actually a statement. It is a syntax error to try and use the return -value from `close()': (d.c.) +actually a statement. (d.c.) It is a syntax error to try and use the +return value from `close()': command = "..." command | getline info @@ -7111,8 +7233,8 @@ closing input or output files, respectively. This value is zero if the close succeeds, or -1 if it fails. The POSIX standard is very vague; it says that `close()' returns -zero on success and nonzero otherwise. In general, different -implementations vary in what they report when closing pipes; thus the +zero on success and a nonzero value otherwise. In general, different +implementations vary in what they report when closing pipes; thus, the return value cannot be used portably. (d.c.) In POSIX mode (*note Options::), `gawk' just returns zero when closing a pipe. @@ -7127,9 +7249,68 @@ call. See the system manual pages for information on how to decode this value. -File: gawk.info, Node: Output Summary, Next: Output Exercises, Prev: Close Files And Pipes, Up: Printing +File: gawk.info, Node: Nonfatal, Next: Output Summary, Prev: Close Files And Pipes, Up: Printing + +5.10 Enabling Nonfatal Output +============================= + +This minor node describes a `gawk'-specific feature. + + In standard `awk', output with `print' or `printf' to a nonexistent +file, or some other I/O error (such as filling up the disk) is a fatal +error. + + $ gawk 'BEGIN { print "hi" > "/no/such/file" }' + error--> gawk: cmd. line:1: fatal: can't redirect to `/no/such/file' (No such file or directory) + + `gawk' makes it possible to detect that an error has occurred, +allowing you to possibly recover from the error, or at least print an +error message of your choosing before exiting. You can do this in one +of two ways: + + * For all output files, by assigning any value to + `PROCINFO["NONFATAL"]'. + + * On a per-file basis, by assigning any value to `PROCINFO[FILENAME, + "NONFATAL"]'. Here, FILENAME is the name of the file to which you + wish output to be nonfatal. + + Once you have enabled nonfatal output, you must check `ERRNO' after +every relevant `print' or `printf' statement to see if something went +wrong. It is also a good idea to initialize `ERRNO' to zero before +attempting the output. For example: + + $ gawk ' + > BEGIN { + > PROCINFO["NONFATAL"] = 1 + > ERRNO = 0 + > print "hi" > "/no/such/file" + > if (ERRNO) { + > print("Output failed:", ERRNO) > "/dev/stderr" + > exit 1 + > } + > }' + error--> Output failed: No such file or directory + + Here, `gawk' did not produce a fatal error; instead it let the `awk' +program code detect the problem and handle it. + + This mechanism works also for standard output and standard error. +For standard output, you may use `PROCINFO["-", "NONFATAL"]' or +`PROCINFO["/dev/stdout", "NONFATAL"]'. For standard error, use +`PROCINFO["/dev/stderr", "NONFATAL"]'. + + When attempting to open a TCP/IP socket (*note TCP/IP Networking::), +`gawk' tries multiple times. The `GAWK_SOCK_RETRIES' environment +variable (*note Other Environment Variables::) allows you to override +`gawk''s builtin default number of attempts. However, once nonfatal +I/O is enabled for a given socket, `gawk' only retries once, relying on +`awk'-level code to notice that there was a problem. -5.10 Summary + +File: gawk.info, Node: Output Summary, Next: Output Exercises, Prev: Nonfatal, Up: Printing + +5.11 Summary ============ * The `print' statement prints comma-separated expressions. Each @@ -7138,24 +7319,29 @@ File: gawk.info, Node: Output Summary, Next: Output Exercises, Prev: Close Fi numeric values for the `print' statement. * The `printf' statement provides finer-grained control over output, - with format control letters for different data types and various - flags that modify the behavior of the format control letters. + with format-control letters for different data types and various + flags that modify the behavior of the format-control letters. * Output from both `print' and `printf' may be redirected to files, pipes, and coprocesses. * `gawk' provides special file names for access to standard input, - output and error, and for network communications. + output, and error, and for network communications. - * Use `close()' to close open file, pipe and coprocess redirections. + * Use `close()' to close open file, pipe, and coprocess redirections. For coprocesses, it is possible to close only one direction of the communications. + * Normally errors with `print' or `printf' are fatal. `gawk' lets + you make output errors be nonfatal either for all files or on a + per-file basis. You must then check for errors after every + relevant output statement. + File: gawk.info, Node: Output Exercises, Prev: Output Summary, Up: Printing -5.11 Exercises +5.12 Exercises ============== 1. Rewrite the program: @@ -7190,9 +7376,9 @@ value to a variable or a field by using an assignment operator. An expression can serve as a pattern or action statement on its own. Most other kinds of statements contain one or more expressions that specify the data on which to operate. As in other languages, -expressions in `awk' include variables, array references, constants, -and function calls, as well as combinations of these with various -operators. +expressions in `awk' can include variables, array references, +constants, and function calls, as well as combinations of these with +various operators. * Menu: @@ -7207,12 +7393,12 @@ operators. File: gawk.info, Node: Values, Next: All Operators, Up: Expressions -6.1 Constants, Variables and Conversions -======================================== +6.1 Constants, Variables, and Conversions +========================================= Expressions are built up from values and the operations performed upon -them. This minor node describes the elementary objects which provide -the values used in expressions. +them. This minor node describes the elementary objects that provide the +values used in expressions. * Menu: @@ -7234,7 +7420,7 @@ regular expression. Each is used in the appropriate context when you need a data value that isn't going to change. Numeric constants can have different -forms, but are stored identically internally. +forms, but are internally stored in an identical manner. * Menu: @@ -7257,21 +7443,21 @@ the same value: 1.05e+2 1050e-1 - A string constant consists of a sequence of characters enclosed in -double-quotation marks. For example: + A "string constant" consists of a sequence of characters enclosed in +double quotation marks. For example: "parrot" represents the string whose contents are `parrot'. Strings in `gawk' can be of any length, and they can contain any of the possible -eight-bit ASCII characters including ASCII NUL (character code zero). +eight-bit ASCII characters, including ASCII NUL (character code zero). Other `awk' implementations may have difficulty with some character codes. ---------- Footnotes ---------- (1) The internal representation of all numbers, including integers, -uses double precision floating-point numbers. On most modern systems, +uses double-precision floating-point numbers. On most modern systems, these are in IEEE 754 standard format. *Note Arbitrary Precision Arithmetic::, for much more information. @@ -7281,16 +7467,16 @@ File: gawk.info, Node: Nondecimal-numbers, Next: Regexp Constants, Prev: Scal 6.1.1.2 Octal and Hexadecimal Numbers ..................................... -In `awk', all numbers are in decimal; i.e., base 10. Many other +In `awk', all numbers are in decimal (i.e., base 10). Many other programming languages allow you to specify numbers in other bases, often octal (base 8) and hexadecimal (base 16). In octal, the numbers go 0, -1, 2, 3, 4, 5, 6, 7, 10, 11, 12, etc. Just as `11', in decimal, is 1 -times 10 plus 1, so `11', in octal, is 1 times 8, plus 1. This equals 9 -in decimal. In hexadecimal, there are 16 digits. Since the everyday +1, 2, 3, 4, 5, 6, 7, 10, 11, 12, and so on. Just as `11' in decimal is +1 times 10 plus 1, so `11' in octal is 1 times 8 plus 1. This equals 9 +in decimal. In hexadecimal, there are 16 digits. Because the everyday decimal number system only has ten digits (`0'-`9'), the letters `a' through `f' are used to represent the rest. (Case in the letters is usually irrelevant; hexadecimal `a' and `A' have the same value.) -Thus, `11', in hexadecimal, is 1 times 16 plus 1, which equals 17 in +Thus, `11' in hexadecimal is 1 times 16 plus 1, which equals 17 in decimal. Just by looking at plain `11', you can't tell what base it's in. @@ -7299,13 +7485,13 @@ notation to signify the base. Octal numbers start with a leading `0', and hexadecimal numbers start with a leading `0x' or `0X': `11' - Decimal value 11. + Decimal value 11 `011' - Octal 11, decimal value 9. + Octal 11, decimal value 9 `0x11' - Hexadecimal 11, decimal value 17. + Hexadecimal 11, decimal value 17 This example shows the difference: @@ -7324,12 +7510,12 @@ really need to do this, use the `--non-decimal-data' command-line option; *note Nondecimal Data::.) If you have octal or hexadecimal data, you can use the `strtonum()' function (*note String Functions::) to convert the data into a number. Most of the time, you will want to -use octal or hexadecimal constants when working with the built-in bit -manipulation functions; see *note Bitwise Functions::, for more +use octal or hexadecimal constants when working with the built-in +bit-manipulation functions; see *note Bitwise Functions::, for more information. - Unlike some early C implementations, `8' and `9' are not valid in -octal constants; e.g., `gawk' treats `018' as decimal 18: + Unlike in some early C implementations, `8' and `9' are not valid in +octal constants. For example, `gawk' treats `018' as decimal 18: $ gawk 'BEGIN { print "021 is", 021 ; print 018 }' -| 021 is 17 @@ -7355,12 +7541,12 @@ File: gawk.info, Node: Regexp Constants, Prev: Nondecimal-numbers, Up: Consta 6.1.1.3 Regular Expression Constants .................................... -A regexp constant is a regular expression description enclosed in +A "regexp constant" is a regular expression description enclosed in slashes, such as `/^beginning and end$/'. Most regexps used in `awk' programs are constant, but the `~' and `!~' matching operators can also match computed or dynamic regexps (which are typically just ordinary -strings or variables that contain a regexp, but could be a more complex -expression). +strings or variables that contain a regexp, but could be more complex +expressions). File: gawk.info, Node: Using Constant Regexps, Next: Variables, Prev: Constants, Up: Values @@ -7372,8 +7558,8 @@ When used on the righthand side of the `~' or `!~' operators, a regexp constant merely stands for the regexp that is to be matched. However, regexp constants (such as `/foo/') may be used like simple expressions. When a regexp constant appears by itself, it has the same meaning as if -it appeared in a pattern, i.e., `($0 ~ /foo/)' (d.c.) *Note Expression -Patterns::. This means that the following two code segments: +it appeared in a pattern (i.e., `($0 ~ /foo/)'). (d.c.) *Note +Expression Patterns::. This means that the following two code segments: if ($0 ~ /barfly/ || $0 ~ /camelot/) print "found" @@ -7412,7 +7598,7 @@ and `patsplit()' functions (*note String Functions::). Modern implementations of `awk', including `gawk', allow the third argument of `split()' to be a regexp constant, but some older implementations do not. (d.c.) Because some built-in functions accept regexp constants -as arguments, it can be confusing when attempting to use regexp +as arguments, confusion can arise when attempting to use regexp constants as arguments to user-defined functions (*note User-defined::). For example: @@ -7435,10 +7621,11 @@ User-defined::). For example: In this example, the programmer wants to pass a regexp constant to the user-defined function `mysub()', which in turn passes it on to either `sub()' or `gsub()'. However, what really happens is that the -`pat' parameter is either one or zero, depending upon whether or not -`$0' matches `/hi/'. `gawk' issues a warning when it sees a regexp -constant used as a parameter to a user-defined function, since passing -a truth value in this way is probably not what was intended. +`pat' parameter is assigned a value of either one or zero, depending +upon whether or not `$0' matches `/hi/'. `gawk' issues a warning when +it sees a regexp constant used as a parameter to a user-defined +function, because passing a truth value in this way is probably not +what was intended. File: gawk.info, Node: Variables, Next: Conversion, Prev: Using Constant Regexps, Up: Values @@ -7446,7 +7633,7 @@ File: gawk.info, Node: Variables, Next: Conversion, Prev: Using Constant Rege 6.1.3 Variables --------------- -Variables are ways of storing values at one point in your program for +"Variables" are ways of storing values at one point in your program for use later in another part of your program. They can be manipulated entirely within the program text, and they can also be assigned values on the `awk' command line. @@ -7475,14 +7662,14 @@ variables. A variable name is a valid expression by itself; it represents the variable's current value. Variables are given new values with -"assignment operators", "increment operators", and "decrement -operators". *Note Assignment Ops::. In addition, the `sub()' and -`gsub()' functions can change a variable's value, and the `match()', -`split()' and `patsplit()' functions can change the contents of their -array parameters. *Note String Functions::. +"assignment operators", "increment operators", and "decrement operators" +(*note Assignment Ops::). In addition, the `sub()' and `gsub()' +functions can change a variable's value, and the `match()', `split()', +and `patsplit()' functions can change the contents of their array +parameters (*note String Functions::). A few variables have special built-in meanings, such as `FS' (the -field separator), and `NF' (the number of fields in the current input +field separator) and `NF' (the number of fields in the current input record). *Note Built-in Variables::, for a list of the predefined variables. These predefined variables can be used and assigned just like all other variables, but their values are also used or changed @@ -7550,7 +7737,7 @@ File: gawk.info, Node: Conversion, Prev: Variables, Up: Values 6.1.4 Conversion of Strings and Numbers --------------------------------------- -Number to string and string to number conversion are generally +Number-to-string and string-to-number conversion are generally straightforward. There can be subtleties to be aware of; this minor node discusses this important facet of `awk'. @@ -7563,7 +7750,7 @@ node discusses this important facet of `awk'. File: gawk.info, Node: Strings And Numbers, Next: Locale influences conversions, Up: Conversion -6.1.4.1 How `awk' Converts Between Strings And Numbers +6.1.4.1 How `awk' Converts Between Strings and Numbers ...................................................... Strings are converted to numbers and numbers are converted to strings, @@ -7586,7 +7773,7 @@ string, concatenate that number with the empty string, `""'. To force a string to be converted to a number, add zero to that string. A string is converted to a number by interpreting any numeric prefix of the string as numerals: `"2.5"' converts to 2.5, `"1e3"' converts to -1000, and `"25fix"' has a numeric value of 25. Strings that can't be +1,000, and `"25fix"' has a numeric value of 25. Strings that can't be interpreted as valid numbers convert to zero. The exact manner in which numbers are converted into strings is @@ -7615,7 +7802,7 @@ value of `CONVFMT' may be. Given the following code fragment: `b' has the value `"12"', not `"12.00"'. (d.c.) - Pre-POSIX `awk' Used `OFMT' For String Conversion + Pre-POSIX `awk' Used `OFMT' for String Conversion Prior to the POSIX standard, `awk' used the value of `OFMT' for converting numbers to strings. `OFMT' specifies the output format to @@ -7652,7 +7839,7 @@ separator, if they have one. decimal point when reading the `awk' program source code, and for command-line variable assignments (*note Other Arguments::). However, when interpreting input data, for `print' and `printf' output, and for -number to string conversion, the local decimal point character is used. +number-to-string conversion, the local decimal point character is used. (d.c.) In all cases, numbers in source code and in input data cannot have a thousands separator. Here are some examples indicating the difference in behavior, on a GNU/Linux system: @@ -7674,12 +7861,12 @@ full number including the fractional part, 4.321. Some earlier versions of `gawk' fully complied with this aspect of the standard. However, many users in non-English locales complained -about this behavior, since their data used a period as the decimal +about this behavior, because their data used a period as the decimal point, so the default behavior was restored to use a period as the decimal point character. You can use the `--use-lc-numeric' option (*note Options::) to force `gawk' to use the locale's decimal point character. (`gawk' also uses the locale's decimal point character when -in POSIX mode, either via `--posix', or the `POSIXLY_CORRECT' +in POSIX mode, either via `--posix' or the `POSIXLY_CORRECT' environment variable, as shown previously.) *note table-locale-affects:: describes the cases in which the @@ -7693,20 +7880,20 @@ Feature Default `--posix' or `--use-lc-numeric' Input Use period Use locale `strtonum()'Use period Use locale -Table 6.1: Locale Decimal Point versus A Period +Table 6.1: Locale decimal point versus a period - Finally, modern day formal standards and IEEE standard floating point -representation can have an unusual but important effect on the way -`gawk' converts some special string values to numbers. The details are -presented in *note POSIX Floating Point Problems::. + Finally, modern-day formal standards and the IEEE standard +floating-point representation can have an unusual but important effect +on the way `gawk' converts some special string values to numbers. The +details are presented in *note POSIX Floating Point Problems::. File: gawk.info, Node: All Operators, Next: Truth Values and Conditions, Prev: Values, Up: Expressions -6.2 Operators: Doing Something With Values +6.2 Operators: Doing Something with Values ========================================== -This minor node introduces the "operators" which make use of the values +This minor node introduces the "operators" that make use of the values provided by constants and variables. * Menu: @@ -7765,9 +7952,9 @@ order from the highest precedence to the lowest: Division; because all numbers in `awk' are floating-point numbers, the result is _not_ rounded to an integer--`3 / 4' has the value 0.75. (It is a common mistake, especially for C - programmers, to forget that _all_ numbers in `awk' are - floating-point, and that division of integer-looking constants - produces a real number, not an integer.) + programmers, to forget that _all_ numbers in `awk' are floating + point, and that division of integer-looking constants produces a + real number, not an integer.) `X % Y' Remainder; further discussion is provided in the text, just after @@ -7829,7 +8016,7 @@ runs together. For example: ... Because string concatenation does not have an explicit operator, it -is often necessary to insure that it happens at the right time by using +is often necessary to ensure that it happens at the right time by using parentheses to enclose the items to concatenate. For example, you might expect that the following code fragment concatenates `file' and `name': @@ -7887,7 +8074,7 @@ you'll get. ---------- Footnotes ---------- - (1) It happens that BWK `awk', `gawk' and `mawk' all "get it right," + (1) It happens that BWK `awk', `gawk', and `mawk' all "get it right," but you should not rely on this. @@ -8004,7 +8191,7 @@ righthand expression. For example: The indices of `bar' are practically guaranteed to be different, because `rand()' returns different values each time it is called. (Arrays and the `rand()' function haven't been covered yet. *Note Arrays::, and -see *note Numeric Functions::, for more information). This example +*note Numeric Functions::, for more information.) This example illustrates an important fact about assignment operators: the lefthand expression is only evaluated _once_. @@ -8028,10 +8215,10 @@ LVALUE `*=' Multiply the value of LVALUE by COEFFICIENT. COEFFICIENT LVALUE `/=' DIVISOR Divide the value of LVALUE by DIVISOR. LVALUE `%=' MODULUS Set LVALUE to its remainder by MODULUS. -LVALUE `^=' POWER +LVALUE `^=' POWER Raise LVALUE to the power POWER. LVALUE `**=' POWER Raise LVALUE to the power POWER. (c.e.) -Table 6.2: Arithmetic Assignment Operators +Table 6.2: Arithmetic assignment operators NOTE: Only the `^=' operator is specified by POSIX. For maximum portability, do not use the `**=' operator. @@ -8080,7 +8267,7 @@ effect of incrementing it. The post-increment `foo++' is nearly the same as writing `(foo += 1) - 1'. It is not perfectly equivalent because all numbers in `awk' are -floating-point--in floating-point, `foo + 1 - 1' does not necessarily +floating point--in floating point, `foo + 1 - 1' does not necessarily equal `foo'. But the difference is minute as long as you stick to numbers that are fairly small (less than 10e12). @@ -8114,8 +8301,8 @@ is a summary of increment and decrement expressions: Operator Evaluation Order - Doctor, doctor! It hurts when I do this! - So don't do that! -- Groucho Marx + Doctor, it hurts when I do this! + Then don't do that! -- Groucho Marx What happens for something like the following? @@ -8130,7 +8317,7 @@ Or something even stranger? In other words, when do the various side effects prescribed by the postfix operators (`b++') take effect? When side effects happen is -"implementation defined". In other words, it is up to the particular +"implementation-defined". In other words, it is up to the particular version of `awk'. The result for the first example may be 12 or 13, and for the second, it may be 22 or 23. @@ -8144,7 +8331,7 @@ File: gawk.info, Node: Truth Values and Conditions, Next: Function Calls, Pre 6.3 Truth Values and Conditions =============================== -In certain contexts, expression values also serve as "truth values;" +In certain contexts, expression values also serve as "truth values"; i.e., they determine what should happen next as the program runs. This minor node describes how `awk' defines "true" and "false" and how values are compared. @@ -8196,13 +8383,13 @@ File: gawk.info, Node: Typing and Comparison, Next: Boolean Ops, Prev: Truth 6.3.2 Variable Typing and Comparison Expressions ------------------------------------------------ - The Guide is definitive. Reality is frequently inaccurate. -- The - Hitchhiker's Guide to the Galaxy + The Guide is definitive. Reality is frequently inaccurate. -- + Douglas Adams, `The Hitchhiker's Guide to the Galaxy' - Unlike other programming languages, `awk' variables do not have a -fixed type. Instead, they can be either a number or a string, depending -upon the value that is assigned to them. We look now at how variables -are typed, and how `awk' compares variables. + Unlike in other programming languages, in `awk' variables do not +have a fixed type. Instead, they can be either a number or a string, +depending upon the value that is assigned to them. We look now at how +variables are typed, and how `awk' compares variables. * Menu: @@ -8213,7 +8400,7 @@ are typed, and how `awk' compares variables. File: gawk.info, Node: Variable Typing, Next: Comparison Operators, Up: Typing and Comparison -6.3.2.1 String Type Versus Numeric Type +6.3.2.1 String Type versus Numeric Type ....................................... The POSIX standard introduced the concept of a "numeric string", which @@ -8223,16 +8410,16 @@ of the variable is important because the types of two variables determine how they are compared. Variable typing follows these rules: * A numeric constant or the result of a numeric operation has the - NUMERIC attribute. + "numeric" attribute. * A string constant or the result of a string operation has the - STRING attribute. + "string" attribute. * Fields, `getline' input, `FILENAME', `ARGV' elements, `ENVIRON' elements, and the elements of an array created by `match()', - `split()' and `patsplit()' that are numeric strings have the - STRNUM attribute. Otherwise, they have the STRING attribute. - Uninitialized variables also have the STRNUM attribute. + `split()', and `patsplit()' that are numeric strings have the + "strnum" attribute. Otherwise, they have the "string" attribute. + Uninitialized variables also have the "strnum" attribute. * Attributes propagate across assignments but are not changed by any use. @@ -8274,12 +8461,13 @@ constant, then a string comparison is performed. Otherwise, a numeric comparison is performed. This point bears additional emphasis: All user input is made of -characters, and so is first and foremost of STRING type; input strings -that look numeric are additionally given the STRNUM attribute. Thus, -the six-character input string ` +3.14' receives the STRNUM attribute. +characters, and so is first and foremost of string type; input strings +that look numeric are additionally given the strnum attribute. Thus, +the six-character input string ` +3.14' receives the strnum attribute. In contrast, the eight characters `" +3.14"' appearing in program text comprise a string constant. The following examples print `1' when the -comparison between the two different constants is true, `0' otherwise: +comparison between the two different constants is true, and `0' +otherwise: $ echo ' +3.14' | awk '{ print($0 == " +3.14") }' True -| 1 @@ -8311,19 +8499,19 @@ them. Expression Result -------------------------------------------------------------------------- -X `<' Y True if X is less than Y. -X `<=' Y True if X is less than or equal to Y. -X `>' Y True if X is greater than Y. -X `>=' Y True if X is greater than or equal to Y. -X `==' Y True if X is equal to Y. -X `!=' Y True if X is not equal to Y. -X `~' Y True if the string X matches the regexp denoted by Y. +X `<' Y True if X is less than Y +X `<=' Y True if X is less than or equal to Y +X `>' Y True if X is greater than Y +X `>=' Y True if X is greater than or equal to Y +X `==' Y True if X is equal to Y +X `!=' Y True if X is not equal to Y +X `~' Y True if the string X matches the regexp denoted by Y X `!~' Y True if the string X does not match the regexp - denoted by Y. + denoted by Y SUBSCRIPT `in' True if the array ARRAY has an element with the -ARRAY subscript SUBSCRIPT. +ARRAY subscript SUBSCRIPT -Table 6.3: Relational Operators +Table 6.3: Relational operators Comparison expressions have the value one if true and zero if false. When comparing operands of mixed types, numeric operands are converted @@ -8353,24 +8541,24 @@ comparisons `awk' performs, as well as what the result of each comparison is: `1.5 <= 2.0' - numeric comparison (true) + Numeric comparison (true) `"abc" >= "xyz"' - string comparison (false) + String comparison (false) `1.5 != " +2"' - string comparison (true) + String comparison (true) `"1e2" < "3"' - string comparison (true) + String comparison (true) `a = 2; b = "2"' `a == b' - string comparison (true) + String comparison (true) `a = 2; b = " +2"' `a == b' - string comparison (false) + String comparison (false) In this example: @@ -8378,7 +8566,7 @@ comparison is: -| false the result is `false' because both `$1' and `$2' are user input. They -are numeric strings--therefore both have the STRNUM attribute, +are numeric strings--therefore both have the strnum attribute, dictating a numeric comparison. The purpose of the comparison rules and the use of numeric strings is to attempt to produce the behavior that is "least surprising," while still "doing the right thing." @@ -8402,8 +8590,8 @@ case, the value of the expression as a string is used as a dynamic regexp (*note Regexp Usage::; also *note Computed Regexps::). A constant regular expression in slashes by itself is also an -expression. The regexp `/REGEXP/' is an abbreviation for the following -comparison expression: +expression. `/REGEXP/' is an abbreviation for the following comparison +expression: $0 ~ /REGEXP/ @@ -8414,7 +8602,7 @@ Constant Regexps::, where this is discussed in more detail. File: gawk.info, Node: POSIX String Comparison, Prev: Comparison Operators, Up: Typing and Comparison -6.3.2.3 String Comparison With POSIX Rules +6.3.2.3 String Comparison with POSIX Rules .......................................... The POSIX standard says that string comparison is performed based on @@ -8437,7 +8625,7 @@ is an example to illustrate the difference, in an `en_US.UTF-8' locale: ---------- Footnotes ---------- (1) Technically, string comparison is supposed to behave the same -way as if the strings are compared with the C `strcoll()' function. +way as if the strings were compared with the C `strcoll()' function. File: gawk.info, Node: Boolean Ops, Next: Conditional Exp, Prev: Typing and Comparison, Up: Truth Values and Conditions @@ -8500,7 +8688,7 @@ Boolean operators are: The `&&' and `||' operators are called "short-circuit" operators because of the way they work. Evaluation of the full expression is -"short-circuited" if the result can be determined part way through its +"short-circuited" if the result can be determined partway through its evaluation. Statements that end with `&&' or `||' can be continued simply by @@ -8553,15 +8741,15 @@ File: gawk.info, Node: Conditional Exp, Prev: Boolean Ops, Up: Truth Values a A "conditional expression" is a special kind of expression that has three operands. It allows you to use one expression's value to select -one of two other expressions. The conditional expression is the same -as in the C language, as shown here: +one of two other expressions. The conditional expression in `awk' is +the same as in the C language, as shown here: SELECTOR ? IF-TRUE-EXP : IF-FALSE-EXP There are three subexpressions. The first, SELECTOR, is always computed first. If it is "true" (not zero or not null), then -IF-TRUE-EXP is computed next and its value becomes the value of the -whole expression. Otherwise, IF-FALSE-EXP is computed next and its +IF-TRUE-EXP is computed next, and its value becomes the value of the +whole expression. Otherwise, IF-FALSE-EXP is computed next, and its value becomes the value of the whole expression. For example, the following expression produces the absolute value of `x': @@ -8595,13 +8783,13 @@ A "function" is a name for a particular calculation. This enables you to ask for it by name at any point in the program. For example, the function `sqrt()' computes the square root of a number. - A fixed set of functions are "built-in", which means they are + A fixed set of functions are "built in", which means they are available in every `awk' program. The `sqrt()' function is one of these. *Note Built-in::, for a list of built-in functions and their descriptions. In addition, you can define functions for use in your program. *Note User-defined::, for instructions on how to do this. Finally, `gawk' lets you write functions in C or C++ that may be called -from your program: see *note Dynamic Extensions::. +from your program (*note Dynamic Extensions::). The way to use a function is with a "function call" expression, which consists of the function name followed immediately by a list of @@ -8616,7 +8804,7 @@ examples show function calls with and without arguments: rand() no arguments CAUTION: Do not put any space between the function name and the - open-parenthesis! A user-defined function name looks just like + opening parenthesis! A user-defined function name looks just like the name of a variable--a space would make the expression look like concatenation of a variable with an expression inside parentheses. With built-in functions, space before the @@ -8730,7 +8918,7 @@ precedence: Increment, decrement. `^ **' - Exponentiation. These operators group right-to-left. + Exponentiation. These operators group right to left. `+ - !' Unary plus, minus, logical "not." @@ -8741,7 +8929,7 @@ precedence: `+ -' Addition, subtraction. -String Concatenation +String concatenation There is no special symbol for concatenation. The operands are simply written side by side (*note Concatenation::). @@ -8756,9 +8944,9 @@ String Concatenation redirection does not produce an expression that could be the operand of another operator. As a result, it does not make sense to use a redirection operator near another operator of lower - precedence without parentheses. Such combinations (for example, - `print foo > a ? b : c'), result in syntax errors. The correct - way to write this statement is `print foo > (a ? b : c)'. + precedence without parentheses. Such combinations (e.g., `print + foo > a ? b : c') result in syntax errors. The correct way to + write this statement is `print foo > (a ? b : c)'. `~ !~' Matching, nonmatching. @@ -8767,16 +8955,16 @@ String Concatenation Array membership. `&&' - Logical "and". + Logical "and." `||' - Logical "or". + Logical "or." `?:' - Conditional. This operator groups right-to-left. + Conditional. This operator groups right to left. `= += -= *= /= %= ^= **=' - Assignment. These operators group right-to-left. + Assignment. These operators group right to left. NOTE: The `|&', `**', and `**=' operators are not specified by POSIX. For maximum portability, do not use them. @@ -8784,7 +8972,7 @@ String Concatenation File: gawk.info, Node: Locales, Next: Expressions Summary, Prev: Precedence, Up: Expressions -6.6 Where You Are Makes A Difference +6.6 Where You Are Makes a Difference ==================================== Modern systems support the notion of "locales": a way to tell the @@ -8804,7 +8992,7 @@ terminator. Locales can affect how dates and times are formatted (*note Time Functions::). For example, a common way to abbreviate the date -September 4, 2015 in the United States is "9/4/15." In many countries +September 4, 2015, in the United States is "9/4/15." In many countries in Europe, however, it is abbreviated "4.9.15." Thus, the `%x' specification in a `"US"' locale might produce `9/4/15', while in a `"EUROPE"' locale, it might produce `4.9.15'. @@ -8824,12 +9012,12 @@ File: gawk.info, Node: Expressions Summary, Prev: Locales, Up: Expressions =========== * Expressions are the basic elements of computation in programs. - They are built from constants, variables, function calls and + They are built from constants, variables, function calls, and combinations of the various kinds of values with operators. * `awk' supplies three kinds of constants: numeric, string, and regexp. `gawk' lets you specify numeric constants in octal and - hexadecimal (bases 8 and 16) in addition to decimal (base 10). In + hexadecimal (bases 8 and 16) as well as decimal (base 10). In certain contexts, a standalone regexp constant such as `/foo/' has the same meaning as `$0 ~ /foo/'. @@ -8844,28 +9032,27 @@ File: gawk.info, Node: Expressions Summary, Prev: Locales, Up: Expressions * `awk' provides the usual arithmetic operators (addition, subtraction, multiplication, division, modulus), and unary plus - and minus. It also provides comparison operators, boolean - operators, array membership testing, and regexp matching - operators. String concatenation is accomplished by placing two - expressions next to each other; there is no explicit operator. - The three-operand `?:' operator provides an "if-else" test within - expressions. + and minus. It also provides comparison operators, Boolean + operators, an array membership testing operator, and regexp + matching operators. String concatenation is accomplished by + placing two expressions next to each other; there is no explicit + operator. The three-operand `?:' operator provides an "if-else" + test within expressions. * Assignment operators provide convenient shorthands for common arithmetic operations. - * In `awk', a value is considered to be true if it is non-zero _or_ + * In `awk', a value is considered to be true if it is nonzero _or_ non-null. Otherwise, the value is false. * A variable's type is set upon each assignment and may change over its lifetime. The type determines how it behaves in comparisons (string or numeric). - * Function calls return a value which may be used as part of a larger + * Function calls return a value that may be used as part of a larger expression. Expressions used to pass parameter values are fully evaluated before the function is called. `awk' provides built-in - and user-defined functions; this is described later on in this - Info file. + and user-defined functions; this is described in *note Functions::. * Operator precedence specifies the order in which operations are performed, unless explicitly overridden by parentheses. `awk''s @@ -9034,10 +9221,10 @@ _not_ contain the string `li': constant regular expressions, comparisons, or any other `awk' expressions. Range patterns are not expressions, so they cannot appear inside Boolean patterns. Likewise, the special patterns `BEGIN', `END', -`BEGINFILE' and `ENDFILE', which never match any input record, are not +`BEGINFILE', and `ENDFILE', which never match any input record, are not expressions and cannot appear inside Boolean patterns. - The precedence of the different operators which can appear in + The precedence of the different operators that can appear in patterns is described in *note Precedence::. @@ -9057,8 +9244,8 @@ following: prints every record in `myfile' between `on'/`off' pairs, inclusive. A range pattern starts out by matching BEGPAT against every input -record. When a record matches BEGPAT, the range pattern is "turned on" -and the range pattern matches this record as well. As long as the +record. When a record matches BEGPAT, the range pattern is "turned +on", and the range pattern matches this record as well. As long as the range pattern stays turned on, it automatically matches every input record read. The range pattern also matches ENDPAT against every input record; when this succeeds, the range pattern is "turned off" again for @@ -9120,8 +9307,7 @@ All the patterns described so far are for matching input records. The and cleanup actions for `awk' programs. `BEGIN' and `END' rules must have actions; there is no default action for these rules because there is no current record when they run. `BEGIN' and `END' rules are often -referred to as "`BEGIN' and `END' blocks" by long-time `awk' -programmers. +referred to as "`BEGIN' and `END' blocks" by longtime `awk' programmers. * Menu: @@ -9148,7 +9334,7 @@ input is read. For example: This program finds the number of records in the input file `mail-list' that contain the string `li'. The `BEGIN' rule prints a title for the report. There is no need to use the `BEGIN' rule to -initialize the counter `n' to zero, since `awk' does this automatically +initialize the counter `n' to zero, as `awk' does this automatically (*note Variables::). The second rule increments the variable `n' every time a record containing the pattern `li' is read. The `END' rule prints the value of `n' at the end of the run. @@ -9177,7 +9363,7 @@ for more information on using library functions. *Note Library Functions::, for a number of useful library functions. If an `awk' program has only `BEGIN' rules and no other rules, then -the program exits after the `BEGIN' rule is run.(1) However, if an +the program exits after the `BEGIN' rules are run.(1) However, if an `END' rule exists, then the input is read, even if there are no other rules in the program. This is necessary in case the `END' rule checks the `FNR' and `NR' variables. @@ -9203,7 +9389,7 @@ give `$0' a real value is to execute a `getline' command without a variable (*note Getline::). Another way is simply to assign a value to `$0'. - The second point is similar to the first but from the other + The second point is similar to the first, but from the other direction. Traditionally, due largely to implementation issues, `$0' and `NF' were _undefined_ inside an `END' rule. The POSIX standard specifies that `NF' is available in an `END' rule. It contains the @@ -9216,20 +9402,20 @@ many older versions of Unix `awk' do not. The third point follows from the first two. The meaning of `print' inside a `BEGIN' or `END' rule is the same as always: `print $0'. If -`$0' is the null string, then this prints an empty record. Many long -time `awk' programmers use an unadorned `print' in `BEGIN' and `END' -rules, to mean `print ""', relying on `$0' being null. Although one -might generally get away with this in `BEGIN' rules, it is a very bad -idea in `END' rules, at least in `gawk'. It is also poor style, since -if an empty line is needed in the output, the program should print one -explicitly. +`$0' is the null string, then this prints an empty record. Many +longtime `awk' programmers use an unadorned `print' in `BEGIN' and +`END' rules, to mean `print ""', relying on `$0' being null. Although +one might generally get away with this in `BEGIN' rules, it is a very +bad idea in `END' rules, at least in `gawk'. It is also poor style, +because if an empty line is needed in the output, the program should +print one explicitly. Finally, the `next' and `nextfile' statements are not allowed in a `BEGIN' rule, because the implicit read-a-record-and-match-against-the-rules loop has not started yet. -Similarly, those statements are not valid in an `END' rule, since all -the input has been read. (*Note Next Statement::, and see *note -Nextfile Statement::.) +Similarly, those statements are not valid in an `END' rule, because all +the input has been read. (*Note Next Statement::, and *note Nextfile +Statement::,.) File: gawk.info, Node: BEGINFILE/ENDFILE, Next: Empty, Prev: BEGIN/END, Up: Pattern Overview @@ -9264,7 +9450,7 @@ tasks that would otherwise be difficult or impossible to perform: entirely. Otherwise, `gawk' exits with the usual fatal error. * If you have written extensions that modify the record handling (by - inserting an "input parser," *note Input Parsers::), you can invoke + inserting an "input parser"; *note Input Parsers::), you can invoke them at this point, before `gawk' has started processing the file. (This is a _very_ advanced feature, currently used only by the `gawkextlib' project (http://gawkextlib.sourceforge.net).) @@ -9274,16 +9460,15 @@ last record in an input file. For the last input file, it will be called before any `END' rules. The `ENDFILE' rule is executed even for empty input files. - Normally, when an error occurs when reading input in the normal input -processing loop, the error is fatal. However, if an `ENDFILE' rule is -present, the error becomes non-fatal, and instead `ERRNO' is set. This -makes it possible to catch and process I/O errors at the level of the -`awk' program. + Normally, when an error occurs when reading input in the normal +input-processing loop, the error is fatal. However, if an `ENDFILE' +rule is present, the error becomes non-fatal, and instead `ERRNO' is +set. This makes it possible to catch and process I/O errors at the +level of the `awk' program. The `next' statement (*note Next Statement::) is not allowed inside either a `BEGINFILE' or an `ENDFILE' rule. The `nextfile' statement is -allowed only inside a `BEGINFILE' rule, but not inside an `ENDFILE' -rule. +allowed only inside a `BEGINFILE' rule, not inside an `ENDFILE' rule. The `getline' statement (*note Getline::) is restricted inside both `BEGINFILE' and `ENDFILE': only redirected forms of `getline' are @@ -9332,7 +9517,7 @@ concatenated together to form the program. The first part is double-quoted, which allows substitution of the `pattern' shell variable inside the quotes. The second part is single-quoted. - Variable substitution via quoting works, but can be potentially + Variable substitution via quoting works, but can potentially be messy. It requires a good understanding of the shell's quoting rules (*note Quoting::), and it's often difficult to correctly match up the quotes when reading the program. @@ -9352,10 +9537,10 @@ Now, the `awk' program is just one single-quoted string. The assignment `-v pat="$pattern"' still requires double quotes, in case there is whitespace in the value of `$pattern'. The `awk' variable `pat' could be named `pattern' too, but that would be more confusing. -Using a variable also provides more flexibility, since the variable can -be used anywhere inside the program--for printing, as an array -subscript, or for any other use--without requiring the quoting tricks -at every point in the program. +Using a variable also provides more flexibility, as the variable can be +used anywhere inside the program--for printing, as an array subscript, +or for any other use--without requiring the quoting tricks at every +point in the program. File: gawk.info, Node: Action Overview, Next: Statements, Prev: Using Shell Variables, Up: Patterns and Actions @@ -9407,7 +9592,7 @@ Compound statements Input statements Use the `getline' command (*note Getline::). Also supplied in - `awk' are the `next' statement (*note Next Statement::), and the + `awk' are the `next' statement (*note Next Statement::) and the `nextfile' statement (*note Nextfile Statement::). Output statements @@ -9475,7 +9660,7 @@ following: else print "x is odd" - In this example, if the expression `x % 2 == 0' is true (that is, if + In this example, if the expression `x % 2 == 0' is true (i.e., if the value of `x' is evenly divisible by two), then the first `print' statement is executed; otherwise, the second `print' statement is executed. If the `else' keyword appears on the same line as THEN-BODY @@ -9529,15 +9714,15 @@ The body of this loop is a compound statement enclosed in braces, containing two statements. The loop works in the following manner: first, the value of `i' is set to one. Then, the `while' statement tests whether `i' is less than or equal to three. This is true when -`i' equals one, so the `i'-th field is printed. Then the `i++' +`i' equals one, so the `i'th field is printed. Then the `i++' increments the value of `i' and the loop repeats. The loop terminates when `i' reaches four. A newline is not required between the condition and the body; -however using one makes the program clearer unless the body is a -compound statement or else is very simple. The newline after the -open-brace that begins the compound statement is not required either, -but the program is harder to read without it. +however, using one makes the program clearer unless the body is a +compound statement or else is very simple. The newline after the open +brace that begins the compound statement is not required either, but the +program is harder to read without it. File: gawk.info, Node: Do Statement, Next: For Statement, Prev: While Statement, Up: Statements @@ -9560,7 +9745,7 @@ Contrast this with the corresponding `while' statement: while (CONDITION) BODY -This statement does not execute BODY even once if the CONDITION is +This statement does not execute the BODY even once if the CONDITION is false to begin with. The following is an example of a `do' statement: { @@ -9572,8 +9757,8 @@ false to begin with. The following is an example of a `do' statement: } This program prints each input record 10 times. However, it isn't a -very realistic example, since in this case an ordinary `while' would do -just as well. This situation reflects actual experience; only +very realistic example, because in this case an ordinary `while' would +do just as well. This situation reflects actual experience; only occasionally is there a real use for a `do' statement. @@ -9616,7 +9801,7 @@ loop.) The same is true of the INCREMENT part. Incrementing additional variables requires separate statements at the end of the loop. The C compound expression, using C's comma operator, is useful in this -context but it is not supported in `awk'. +context, but it is not supported in `awk'. Most often, INCREMENT is an increment expression, as in the previous example. But this is not required; it can be any expression @@ -9652,7 +9837,7 @@ natural to think of. Counting the number of iterations is very common in loops. It can be easier to think of this counting as part of looping rather than as something to do inside the loop. - There is an alternate version of the `for' loop, for iterating over + There is an alternative version of the `for' loop, for iterating over all the indices of an array: for (i in array) @@ -9692,7 +9877,7 @@ statement looks like this: Control flow in the `switch' statement works as it does in C. Once a match to a given case is made, the case statement bodies execute until -a `break', `continue', `next', `nextfile' or `exit' is encountered, or +a `break', `continue', `next', `nextfile', or `exit' is encountered, or the end of the `switch' statement itself. For example: while ((c = getopt(ARGC, ARGV, "aksx")) != -1) { @@ -9719,12 +9904,12 @@ the end of the `switch' statement itself. For example: } } - Note that if none of the statements specified above halt execution -of a matched `case' statement, execution falls through to the next -`case' until execution halts. In the above example, the `case' for -`"?"' falls through to the `default' case, which is to call a function -named `usage()'. (The `getopt()' function being called here is -described in *note Getopt Function::.) + Note that if none of the statements specified here halt execution of +a matched `case' statement, execution falls through to the next `case' +until execution halts. In this example, the `case' for `"?"' falls +through to the `default' case, which is to call a function named +`usage()'. (The `getopt()' function being called here is described in +*note Getopt Function::.) File: gawk.info, Node: Break Statement, Next: Continue Statement, Prev: Switch Statement, Up: Statements @@ -9739,12 +9924,12 @@ divisor of any integer, and also identifies prime numbers: # find smallest divisor of num { num = $1 - for (div = 2; div * div <= num; div++) { - if (num % div == 0) + for (divisor = 2; divisor * divisor <= num; divisor++) { + if (num % divisor == 0) break } - if (num % div == 0) - printf "Smallest divisor of %d is %d\n", num, div + if (num % divisor == 0) + printf "Smallest divisor of %d is %d\n", num, divisor else printf "%d is prime\n", num } @@ -9762,12 +9947,12 @@ Statement::.) # find smallest divisor of num { num = $1 - for (div = 2; ; div++) { - if (num % div == 0) { - printf "Smallest divisor of %d is %d\n", num, div + for (divisor = 2; ; divisor++) { + if (num % divisor == 0) { + printf "Smallest divisor of %d is %d\n", num, divisor break } - if (div * div > num) { + if (divisor * divisor > num) { printf "%d is prime\n", num break } @@ -9825,7 +10010,7 @@ the previous example with the following `while' loop: print "" } -This program loops forever once `x' reaches 5, since the increment +This program loops forever once `x' reaches 5, because the increment (`x++') is never reached. The `continue' statement has no special meaning with respect to the @@ -9871,7 +10056,7 @@ beginning, in the following manner: Because of the `next' statement, the program's subsequent rules won't see the bad record. The error message is redirected to the standard -error output stream, as error messages should be. For more detail see +error output stream, as error messages should be. For more detail, see *note Special Files::. If the `next' statement causes the end of the input to be reached, @@ -9924,23 +10109,22 @@ over a file that would otherwise cause `gawk' to exit with a fatal error. In this case, `ENDFILE' rules are not executed. *Note BEGINFILE/ENDFILE::. - While one might think that `close(FILENAME)' would accomplish the + Although it might seem that `close(FILENAME)' would accomplish the same as `nextfile', this isn't true. `close()' is reserved for closing files, pipes, and coprocesses that are opened with redirections. It is not related to the main processing that `awk' does with the files listed in `ARGV'. NOTE: For many years, `nextfile' was a common extension. In - September, 2012, it was accepted for inclusion into the POSIX + September 2012, it was accepted for inclusion into the POSIX standard. See the Austin Group website (http://austingroupbugs.net/view.php?id=607). - The current version of BWK `awk', and `mawk' also support -`nextfile'. However, they don't allow the `nextfile' statement inside -function bodies (*note User-defined::). `gawk' does; a `nextfile' -inside a function body reads the next record and starts processing it -with the first rule in the program, just as any other `nextfile' -statement. + The current version of BWK `awk' and `mawk' also support `nextfile'. +However, they don't allow the `nextfile' statement inside function +bodies (*note User-defined::). `gawk' does; a `nextfile' inside a +function body reads the next record and starts processing it with the +first rule in the program, just as any other `nextfile' statement. File: gawk.info, Node: Exit Statement, Prev: Nextfile Statement, Up: Statements @@ -9968,9 +10152,9 @@ record, skips reading any remaining input records, and executes the they do not execute. In such a case, if you don't want the `END' rule to do its job, set -a variable to nonzero before the `exit' statement and check that -variable in the `END' rule. *Note Assert Function::, for an example -that does this. +a variable to a nonzero value before the `exit' statement and check +that variable in the `END' rule. *Note Assert Function::, for an +example that does this. If an argument is supplied to `exit', its value is used as the exit status code for the `awk' process. If no argument is supplied, `exit' @@ -10052,11 +10236,11 @@ description of each variable.) use binary I/O. Any other string value is treated the same as `"rw"', but causes `gawk' to generate a warning message. `BINMODE' is described in more detail in *note PC Using::. `mawk' - (*note Other Versions::), also supports this variable, but only + (*note Other Versions::) also supports this variable, but only using numeric values. ``CONVFMT'' - This string controls conversion of numbers to strings (*note + A string that controls the conversion of numbers to strings (*note Conversion::). It works by being passed, in effect, as the first argument to the `sprintf()' function (*note String Functions::). Its default value is `"%.6g"'. `CONVFMT' was introduced by the @@ -10103,15 +10287,14 @@ description of each variable.) `IGNORECASE #' If `IGNORECASE' is nonzero or non-null, then all string comparisons - and all regular expression matching are case independent. Thus, - regexp matching with `~' and `!~', as well as the `gensub()', + and all regular expression matching are case-independent. This + applies to regexp matching with `~' and `!~', the `gensub()', `gsub()', `index()', `match()', `patsplit()', `split()', and `sub()' functions, record termination with `RS', and field - splitting with `FS' and `FPAT', all ignore case when doing their - particular regexp operations. However, the value of `IGNORECASE' - does _not_ affect array subscripting and it does not affect field - splitting when using a single-character field separator. *Note - Case-sensitivity::. + splitting with `FS' and `FPAT'. However, the value of + `IGNORECASE' does _not_ affect array subscripting and it does not + affect field splitting when using a single-character field + separator. *Note Case-sensitivity::. `LINT #' When this variable is true (nonzero or non-null), `gawk' behaves @@ -10123,7 +10306,7 @@ description of each variable.) Assigning a false value to `LINT' turns off the lint warnings. This variable is a `gawk' extension. It is not special in other - `awk' implementations. Unlike the other special variables, + `awk' implementations. Unlike with the other special variables, changing `LINT' does affect the production of lint warnings, even if `gawk' is in compatibility mode. Much as the `--lint' and `--traditional' options independently control different aspects of @@ -10131,17 +10314,18 @@ description of each variable.) execution is independent of the flavor of `awk' being executed. `OFMT' - Controls conversion of numbers to strings (*note Conversion::) for - printing with the `print' statement. It works by being passed as - the first argument to the `sprintf()' function (*note String - Functions::). Its default value is `"%.6g"'. Earlier versions of - `awk' used `OFMT' to specify the format for converting numbers to - strings in general expressions; this is now done by `CONVFMT'. + A string that controls conversion of numbers to strings (*note + Conversion::) for printing with the `print' statement. It works + by being passed as the first argument to the `sprintf()' function + (*note String Functions::). Its default value is `"%.6g"'. + Earlier versions of `awk' used `OFMT' to specify the format for + converting numbers to strings in general expressions; this is now + done by `CONVFMT'. `OFS' - This is the output field separator (*note Output Separators::). - It is output between the fields printed by a `print' statement. - Its default value is `" "', a string consisting of a single space. + The output field separator (*note Output Separators::). It is + output between the fields printed by a `print' statement. Its + default value is `" "', a string consisting of a single space. `ORS' The output record separator. It is output at the end of every @@ -10149,11 +10333,11 @@ description of each variable.) character. (*Note Output Separators::.) `PREC #' - The working precision of arbitrary precision floating-point + The working precision of arbitrary-precision floating-point numbers, 53 bits by default (*note Setting precision::). `ROUNDMODE #' - The rounding mode to use for arbitrary precision arithmetic on + The rounding mode to use for arbitrary-precision arithmetic on numbers, by default `"N"' (`roundTiesToEven' in the IEEE 754 standard; *note Setting the rounding mode::). @@ -10180,7 +10364,7 @@ description of each variable.) Used for internationalization of programs at the `awk' level. It sets the default text domain for specially marked string constants in the source text, as well as for the `dcgettext()', - `dcngettext()' and `bindtextdomain()' functions (*note + `dcngettext()', and `bindtextdomain()' functions (*note Internationalization::). The default value of `TEXTDOMAIN' is `"messages"'. @@ -10201,7 +10385,7 @@ your program. The variables that are specific to `gawk' are marked with a pound sign (`#'). These variables are `gawk' extensions. In other `awk' implementations or if `gawk' is in compatibility mode (*note -Options::), they are not special. +Options::), they are not special: `ARGC', `ARGV' The command-line arguments available to `awk' programs are stored @@ -10265,6 +10449,12 @@ Options::), they are not special. `ENVIRON["PATH"]"', which is the search path for finding executable programs. + This can also affect the running `gawk' program, since some of the + built-in functions may pay attention to certain environment + variables. The most notable instance of this is `mktime()' (*note + Time Functions::), which pays attention the value of the `TZ' + environment variable on many systems. + Some operating systems may not have environment variables. On such systems, the `ENVIRON' array is empty (except for `ENVIRON["AWKPATH"]' and `ENVIRON["AWKLIBPATH"]'; *note AWKPATH @@ -10286,14 +10476,20 @@ Options::), they are not special. `getline' returning -1. You are, of course, free to clear it yourself before doing an I/O operation. + If the value of `ERRNO' corresponds to a system error in the C + `errno' variable, then `PROCINFO["errno"]' will be set to the value + of `errno'. For non-system errors, `PROCINFO["errno"]' will be + zero. + `FILENAME' The name of the current input file. When no data files are listed on the command line, `awk' reads from the standard input and `FILENAME' is set to `"-"'. `FILENAME' changes each time a new file is read (*note Reading Files::). Inside a `BEGIN' rule, the - value of `FILENAME' is `""', since there are no input files being - processed yet.(1) (d.c.) Note, though, that using `getline' (*note - Getline::) inside a `BEGIN' rule can give `FILENAME' a value. + value of `FILENAME' is `""', because there are no input files + being processed yet.(1) (d.c.) Note, though, that using `getline' + (*note Getline::) inside a `BEGIN' rule can give `FILENAME' a + value. `FNR' The current record number in the current file. `awk' increments @@ -10302,18 +10498,18 @@ Options::), they are not special. `NF' The number of fields in the current input record. `NF' is set - each time a new record is read, when a new field is created or + each time a new record is read, when a new field is created, or when `$0' changes (*note Fields::). Unlike most of the variables described in this node, assigning a value to `NF' has the potential to affect `awk''s internal workings. In particular, assignments to `NF' can be used to - create or remove fields from the current record. *Note Changing - Fields::. + create fields in or remove fields from the current record. *Note + Changing Fields::. `FUNCTAB #' An array whose indices and corresponding values are the names of - all the built-in, user-defined and extension functions in the + all the built-in, user-defined, and extension functions in the program. NOTE: Attempting to use the `delete' statement with the @@ -10333,6 +10529,10 @@ Options::), they are not special. `PROCINFO["egid"]' The value of the `getegid()' system call. + `PROCINFO["errno"]' + The value of the C `errno' variable when `ERRNO' is set to + the associated error message. + `PROCINFO["euid"]' The value of the `geteuid()' system call. @@ -10344,7 +10544,7 @@ Options::), they are not special. `PROCINFO["identifiers"]' A subarray, indexed by the names of all identifiers used in - the text of the AWK program. An "identifier" is simply the + the text of the `awk' program. An "identifier" is simply the name of a variable (be it scalar or array), built-in function, user-defined function, or extension function. For each identifier, the value of the element is one of the @@ -10365,7 +10565,7 @@ Options::), they are not special. `"untyped"' The identifier is untyped (could be used as a scalar or - array, `gawk' doesn't know yet). + an array; `gawk' doesn't know yet). `"user"' The identifier is a user-defined function. @@ -10389,8 +10589,8 @@ Options::), they are not special. `PROCINFO["sorted_in"]' If this element exists in `PROCINFO', its value controls the order in which array indices will be processed by `for (INDX - in ARRAY)' loops. Since this is an advanced feature, we - defer the full description until later; see *note Scanning an + in ARRAY)' loops. This is an advanced feature, so we defer + the full description until later; see *note Scanning an Array::. `PROCINFO["strftime"]' @@ -10406,7 +10606,7 @@ Options::), they are not special. The following additional elements in the array are available to provide information about the MPFR and GMP libraries if your - version of `gawk' supports arbitrary precision arithmetic (*note + version of `gawk' supports arbitrary-precision arithmetic (*note Arbitrary Precision Arithmetic::): `PROCINFO["mpfr_version"]' @@ -10443,6 +10643,10 @@ Options::), they are not special. open input file, pipe, or coprocess. *Note Read Timeout::, for more information. + * It may be used to indicate that input may be retried when it + fails due to certain errors. *Note Retrying Input::, for + more information. + * It may be used to cause coprocesses to communicate over pseudo-ttys instead of through two-way pipes; this is discussed further in *note Two-way I/O::. @@ -10454,7 +10658,7 @@ Options::), they are not special. string, or -1 if no match is found. `RSTART' - The start-index in characters of the substring that is matched by + The start index in characters of the substring that is matched by the `match()' function (*note String Functions::). `RSTART' is set by invoking the `match()' function. Its value is the position of the string where the matched substring starts, or zero if no @@ -10504,7 +10708,7 @@ Options::), they are not special. } NOTE: In order to avoid severe time-travel paradoxes,(2) - neither `FUNCTAB' nor `SYMTAB' are available as elements + neither `FUNCTAB' nor `SYMTAB' is available as an element within the `SYMTAB' array. Changing `NR' and `FNR' @@ -10598,7 +10802,7 @@ string. Another option is to use the `delete' statement to remove elements from `ARGV' (*note Delete::). All of these actions are typically done in the `BEGIN' rule, before -actual processing of the input begins. *Note Split Program::, and see +actual processing of the input begins. *Note Split Program::, and *note Tee Program::, for examples of each way of removing elements from `ARGV'. @@ -10609,7 +10813,7 @@ manner: awk -f myprog.awk -- -v -q file1 file2 ... The following fragment processes `ARGV' in order to examine, and -then remove, the above command-line options: +then remove, the previously mentioned command-line options: BEGIN { for (i = 1; i < ARGC; i++) { @@ -10641,10 +10845,10 @@ are passed on to the `awk' program. (*Note Getopt Function::, for an `awk' library function that parses command-line options.) When designing your program, you should choose options that don't -conflict with `gawk''s, since it will process any options that it +conflict with `gawk''s, because it will process any options that it accepts before passing the rest of the command line on to your program. Using `#!' with the `-E' option may help (*note Executable Scripts::, -and *note Options::). +and *note Options::,). File: gawk.info, Node: Pattern Action Summary, Prev: Built-in Variables, Up: Patterns and Actions @@ -10654,14 +10858,14 @@ File: gawk.info, Node: Pattern Action Summary, Prev: Built-in Variables, Up: * Pattern-action pairs make up the basic elements of an `awk' program. Patterns are either normal expressions, range - expressions, regexp constants, one of the special keywords - `BEGIN', `END', `BEGINFILE', `ENDFILE', or empty. The action + expressions, or regexp constants; one of the special keywords + `BEGIN', `END', `BEGINFILE', or `ENDFILE'; or empty. The action executes if the current record matches the pattern. Empty (missing) patterns match all records. - * I/O from `BEGIN' and `END' rules have certain constraints. This - is also true, only more so, for `BEGINFILE' and `ENDFILE' rules. - The latter two give you "hooks" into `gawk''s file processing, + * I/O from `BEGIN' and `END' rules has certain constraints. This is + also true, only more so, for `BEGINFILE' and `ENDFILE' rules. The + latter two give you "hooks" into `gawk''s file processing, allowing you to recover from a file that otherwise would cause a fatal error (such as a file that cannot be opened). @@ -10682,11 +10886,11 @@ File: gawk.info, Node: Pattern Action Summary, Prev: Built-in Variables, Up: iteration of a loop (or get out of a `switch'). * `next' and `nextfile' let you read the next record and start over - at the top of your program, or skip to the next input file and + at the top of your program or skip to the next input file and start over, respectively. * The `exit' statement terminates your program. When executed from - an action (or function body) it transfers control to the `END' + an action (or function body), it transfers control to the `END' statements. From an `END' statement body, it exits immediately. You may pass an optional numeric value to be used as `awk''s exit status. @@ -10778,7 +10982,7 @@ be used as an array index. including a specification of how many elements or components they contain. In such languages, the declaration causes a contiguous block of memory to be allocated for that many elements. Usually, an index in -the array must be a positive integer. For example, the index zero +the array must be a nonnegative integer. For example, the index zero specifies the first element in the array, which is actually stored at the beginning of the block of memory. Index one specifies the second element, which is stored in memory right after the first element, and @@ -10788,28 +10992,30 @@ languages allow arbitrary starting and ending indices--e.g., `15 .. 27'--but the size of the array is still fixed when the array is declared.) - A contiguous array of four elements might look like the following -example, conceptually, if the element values are 8, `"foo"', `""', and -30 as shown in *note figure-array-elements::: + A contiguous array of four elements might look like *note +figure-array-elements::, conceptually, if the element values are eight, +`"foo"', `""', and 30. +---------+---------+--------+---------+ | 8 | "foo" | "" | 30 | @r{Value} +---------+---------+--------+---------+ 0 1 2 3 @r{Index} -Figure 8.1: A Contiguous Array +Figure 8.1: A contiguous array Only the values are stored; the indices are implicit from the order of -the values. Here, 8 is the value at index zero, because 8 appears in the -position with zero elements before it. +the values. Here, eight is the value at index zero, because eight +appears in the position with zero elements before it. Arrays in `awk' are different--they are "associative". This means -that each array is a collection of pairs: an index and its corresponding +that each array is a collection of pairs--an index and its corresponding array element value: - Index 3 Value 30 - Index 1 Value "foo" - Index 0 Value 8 - Index 2 Value "" + Index Value +------------------------ + `3' `30' + `1' `"foo"' + `0' `8' + `2' `""' The pairs are shown in jumbled order because their order is irrelevant.(1) @@ -10818,32 +11024,36 @@ irrelevant.(1) at any time. For example, suppose a tenth element is added to the array whose value is `"number ten"'. The result is: - Index 10 Value "number ten" - Index 3 Value 30 - Index 1 Value "foo" - Index 0 Value 8 - Index 2 Value "" + Index Value +------------------------------- + `10' `"number ten"' + `3' `30' + `1' `"foo"' + `0' `8' + `2' `""' Now the array is "sparse", which just means some indices are missing. It has elements 0-3 and 10, but doesn't have elements 4, 5, 6, 7, 8, or 9. Another consequence of associative arrays is that the indices don't -have to be positive integers. Any number, or even a string, can be an -index. For example, the following is an array that translates words +have to be nonnegative integers. Any number, or even a string, can be +an index. For example, the following is an array that translates words from English to French: - Index "dog" Value "chien" - Index "cat" Value "chat" - Index "one" Value "un" - Index 1 Value "un" + Index Value +------------------------ + `"dog"' `"chien"' + `"cat"' `"chat"' + `"one"' `"un"' + `1' `"un"' Here we decided to translate the number one in both spelled-out and numeric form--thus illustrating that a single array can have both numbers and strings as indices. (In fact, array subscripts are always strings. There are some subtleties to how numbers work when used as array subscripts; this is discussed in more detail in *note Numeric -Array Subscripts::.) Here, the number `1' isn't double-quoted, since +Array Subscripts::.) Here, the number `1' isn't double-quoted, because `awk' automatically converts it to a string. The value of `IGNORECASE' has no effect upon array subscripting. @@ -10867,7 +11077,7 @@ File: gawk.info, Node: Reference to Elements, Next: Assigning Elements, Prev: ----------------------------------- The principal way to use an array is to refer to one of its elements. -An array reference is an expression as follows: +An "array reference" is an expression as follows: ARRAY[INDEX-EXPRESSION] @@ -10875,8 +11085,8 @@ Here, ARRAY is the name of an array. The expression INDEX-EXPRESSION is the index of the desired element of the array. The value of the array reference is the current value of that array -element. For example, `foo[4.3]' is an expression for the element of -array `foo' at index `4.3'. +element. For example, `foo[4.3]' is an expression referencing the +element of array `foo' at index `4.3'. A reference to an array element that has no recorded value yields a value of `""', the null string. This includes elements that have not @@ -10906,8 +11116,8 @@ index, use the following expression: This expression tests whether the particular index INDX exists, without the side effect of creating that element if it is not present. The expression has the value one (true) if `ARRAY[INDX]' exists and zero -(false) if it does not exist. (We use INDX here, since `index' is the -name of a built-in function.) For example, this statement tests +(false) if it does not exist. (We use INDX here, because `index' is +the name of a built-in function.) For example, this statement tests whether the array `frequencies' contains the index `2': if (2 in frequencies) @@ -10943,7 +11153,7 @@ File: gawk.info, Node: Array Example, Next: Scanning an Array, Prev: Assignin The following program takes a list of lines, each beginning with a line number, and prints them out in order of line number. The line numbers -are not in order when they are first read--instead they are scrambled. +are not in order when they are first read--instead, they are scrambled. This program sorts the lines by making an array using the line numbers as subscripts. The program then prints out the lines in sorted order of their numbers. It is a very simple program and gets confused upon @@ -10999,7 +11209,7 @@ File: gawk.info, Node: Scanning an Array, Next: Controlling Scanning, Prev: A In programs that use arrays, it is often necessary to use a loop that executes once for each element of an array. In other languages, where -arrays are contiguous and indices are limited to positive integers, +arrays are contiguous and indices are limited to nonnegative integers, this is easy: all the valid indices can be found by counting from the lowest index up to the highest. This technique won't do the job in `awk', because any number or string can be an array index. So `awk' @@ -11014,7 +11224,7 @@ has previously used, with the variable VAR set to that index. The following program uses this form of the `for' statement. The first rule scans the input records and notes which words appear (at least once) in the input, by storing a one into the array `used' with -the word as index. The second rule scans the elements of `used' to +the word as the index. The second rule scans the elements of `used' to find all the distinct words that appear in the input. It prints each word that is more than 10 characters long and also prints the number of such words. *Note String Functions::, for more information on the @@ -11083,7 +11293,7 @@ all `awk' versions do so. Consider this program, named `loopcheck.awk': File: gawk.info, Node: Controlling Scanning, Prev: Scanning an Array, Up: Array Basics -8.1.6 Using Predefined Array Scanning Orders With `gawk' +8.1.6 Using Predefined Array Scanning Orders with `gawk' -------------------------------------------------------- This node describes a feature that is specific to `gawk'. @@ -11097,14 +11307,14 @@ internal implementation of arrays and will vary from one version of Often, though, you may wish to do something simple, such as "traverse the array by comparing the indices in ascending order," or "traverse the array by comparing the values in descending order." -`gawk' provides two mechanisms which give you this control. +`gawk' provides two mechanisms that give you this control: * Set `PROCINFO["sorted_in"]' to one of a set of predefined values. We describe this now. * Set `PROCINFO["sorted_in"]' to the name of a user-defined function to use for comparison of array elements. This advanced feature is - described later, in *note Array Sorting::. + described later in *note Array Sorting::. The following special values for `PROCINFO["sorted_in"]' are available: @@ -11145,22 +11355,26 @@ available: which `gawk' uses internally to perform the sorting. `"@ind_str_desc"' - String indices ordered from high to low. + Like `"@ind_str_asc"', but the string indices are ordered from + high to low. `"@ind_num_desc"' - Numeric indices ordered from high to low. + Like `"@ind_num_asc"', but the numeric indices are ordered from + high to low. `"@val_type_desc"' - Element values, based on type, ordered from high to low. - Subarrays, if present, come out first. + Like `"@val_type_asc"', but the element values, based on type, are + ordered from high to low. Subarrays, if present, come out first. `"@val_str_desc"' - Element values, treated as strings, ordered from high to low. - Subarrays, if present, come out first. + Like `"@val_str_asc"', but the element values, treated as strings, + are ordered from high to low. Subarrays, if present, come out + first. `"@val_num_desc"' - Element values, treated as numbers, ordered from high to low. - Subarrays, if present, come out first. + Like `"@val_num_asc"', but the element values, treated as numbers, + are ordered from high to low. Subarrays, if present, come out + first. The array traversal order is determined before the `for' loop starts to run. Changing `PROCINFO["sorted_in"]' in the loop body does not @@ -11193,7 +11407,7 @@ subarrays are treated as being equal to each other. Their order relative to each other is determined by their index strings. Here are some additional things to bear in mind about sorted array -traversal. +traversal: * The value of `PROCINFO["sorted_in"]' is global. That is, it affects all array traversal `for' loops. If you need to change it within @@ -11209,12 +11423,12 @@ traversal. if (save_sorted) PROCINFO["sorted_in"] = save_sorted - * As mentioned, the default array traversal order is represented by - `"@unsorted"'. You can also get the default behavior by assigning - the null string to `PROCINFO["sorted_in"]' or by just deleting the - `"sorted_in"' element from the `PROCINFO' array with the `delete' - statement. (The `delete' statement hasn't been described yet; - *note Delete::.) + * As already mentioned, the default array traversal order is + represented by `"@unsorted"'. You can also get the default + behavior by assigning the null string to `PROCINFO["sorted_in"]' + or by just deleting the `"sorted_in"' element from the `PROCINFO' + array with the `delete' statement. (The `delete' statement hasn't + been described yet; *note Delete::.) In addition, `gawk' provides built-in functions for sorting arrays; see *note Array Sorting Functions::. @@ -11255,8 +11469,8 @@ string value `"12.153"' (using the default conversion value of assigned the value one. The program then changes the value of `CONVFMT'. The test `(xyz in data)' generates a new string value from `xyz'--this time `"12.15"'--because the value of `CONVFMT' only allows -two significant digits. This test fails, since `"12.15"' is different -from `"12.153"'. +two significant digits. This test fails, because `"12.15"' is +different from `"12.153"'. According to the rules for conversions (*note Conversion::), integer values always convert to strings as integers, no matter what the value @@ -11275,7 +11489,7 @@ the same element! As with many things in `awk', the majority of the time things work as you would expect them to. But it is useful to have a precise -knowledge of the actual rules since they can sometimes have a subtle +knowledge of the actual rules, as they can sometimes have a subtle effect on your programs. @@ -11346,8 +11560,8 @@ deleting elements in an array: This example removes all the elements from the array `frequencies'. Once an element is deleted, a subsequent `for' statement to scan the -array does not report that element and the `in' operator to check for -the presence of that element returns zero (i.e., false): +array does not report that element and using the `in' operator to check +for the presence of that element returns zero (i.e., false): delete foo[4] if (4 in foo) @@ -11378,7 +11592,7 @@ at a time. and `mawk', as well as by a number of other implementations. NOTE: For many years, using `delete' without a subscript was a - common extension. In September, 2012, it was accepted for + common extension. In September 2012, it was accepted for inclusion into the POSIX standard. See the Austin Group website (http://austingroupbugs.net/view.php?id=544). @@ -11415,7 +11629,7 @@ File: gawk.info, Node: Multidimensional, Next: Arrays of Arrays, Prev: Delete * Multiscanning:: Scanning multidimensional arrays. - A multidimensional array is an array in which an element is + A "multidimensional array" is an array in which an element is identified by a sequence of indices instead of a single index. For example, a two-dimensional array requires two indices. The usual way (in many languages, including `awk') to refer to an element of a @@ -11450,7 +11664,7 @@ stored as `foo["a@b@c"]'. To test whether a particular index sequence exists in a multidimensional array, use the same operator (`in') that is used for -single dimensional arrays. Write the whole sequence of indices in +single-dimensional arrays. Write the whole sequence of indices in parentheses, separated by commas, as the left operand: if ((SUBSCRIPT1, SUBSCRIPT2, ...) in ARRAY) @@ -11550,22 +11764,22 @@ two-element subarray at index `1' of the main array `a': This simulates a true two-dimensional array. Each subarray element can contain another subarray as a value, which in turn can hold other arrays as well. In this way, you can create arrays of three or more -dimensions. The indices can be any `awk' expression, including scalars -separated by commas (that is, a regular `awk' simulated -multidimensional subscript). So the following is valid in `gawk': +dimensions. The indices can be any `awk' expressions, including scalars +separated by commas (i.e., a regular `awk' simulated multidimensional +subscript). So the following is valid in `gawk': a[1][3][1, "name"] = "barney" Each subarray and the main array can be of different length. In fact, the elements of an array or its subarray do not all have to have the same type. This means that the main array and any of its subarrays -can be non-rectangular, or jagged in structure. You can assign a scalar +can be nonrectangular, or jagged in structure. You can assign a scalar value to the index `4' of the main array `a', even though `a[1]' is itself an array and not a scalar: a[4] = "An element in a jagged array" - The terms "dimension", "row" and "column" are meaningless when + The terms "dimension", "row", and "column" are meaningless when applied to such an array, but we will use "dimension" henceforth to imply the maximum number of indices needed to refer to an existing element. The type of any element that has already been assigned cannot @@ -11577,8 +11791,8 @@ the element at that index: a[4][5][6][7] = "An element in a four-dimensional array" This removes the scalar value from index `4' and then inserts a -subarray of subarray of subarray containing a scalar. You can also -delete an entire subarray or subarray of subarrays: +three-level nested subarray containing a scalar. You can also delete an +entire subarray or subarray of subarrays: delete a[4][5] a[4][5] = "An element in subarray a[4]" @@ -11586,7 +11800,7 @@ delete an entire subarray or subarray of subarrays: But recall that you can not delete the main array `a' and then use it as a scalar. - The built-in functions which take array arguments can also be used + The built-in functions that take array arguments can also be used with subarrays. For example, the following code fragment uses `length()' (*note String Functions::) to determine the number of elements in the main array `a' and its subarrays: @@ -11607,7 +11821,7 @@ be nested to scan all the elements of an array of arrays if it is rectangular in structure. In order to print the contents (scalar values) of a two-dimensional array of arrays (i.e., in which each first-level element is itself an array, not necessarily of the same -length) you could use the following code: +length), you could use the following code: for (i in array) for (j in array[i]) @@ -11641,7 +11855,7 @@ the following code prints the elements of our main array `a': } *Note Walking Arrays::, for a user-defined function that "walks" an -arbitrarily-dimensioned array of arrays. +arbitrarily dimensioned array of arrays. Recall that a reference to an uninitialized array element yields a value of `""', the null string. This has one important implication when @@ -11682,16 +11896,16 @@ File: gawk.info, Node: Arrays Summary, Prev: Arrays of Arrays, Up: Arrays `gawk' lets you control the order by assigning special predefined values to `PROCINFO["sorted_in"]'. - * Use `delete ARRAY[INDX]' to delete an individual element. You may - also use `delete ARRAY' to delete all of the elements in the - array. This latter feature has been a common extension for many - years and is now standard, but may not be supported by all - commercial versions of `awk'. + * Use `delete ARRAY[INDX]' to delete an individual element. To + delete all of the elements in an array, use `delete ARRAY'. This + latter feature has been a common extension for many years and is + now standard, but may not be supported by all commercial versions + of `awk'. * Standard `awk' simulates multidimensional arrays by separating - subscript values with a comma. The values are concatenated into a + subscript values with commas. The values are concatenated into a single string, separated by the value of `SUBSEP'. The fact that - such a subscript was created in this way is not retained; thus + such a subscript was created in this way is not retained; thus, changing `SUBSEP' may have unexpected consequences. You can use `(SUB1, SUB2, ...) in ARRAY' to see if such a multidimensional subscript exists in ARRAY. @@ -11699,7 +11913,7 @@ File: gawk.info, Node: Arrays Summary, Prev: Arrays of Arrays, Up: Arrays * `gawk' provides true arrays of arrays. You use a separate set of square brackets for each dimension in such an array: `data[row][col]', for example. Array elements may thus be either - scalar values (number or string) or another array. + scalar values (number or string) or other arrays. * Use the `isarray()' built-in function to determine if an array element is itself a subarray. @@ -11714,11 +11928,14 @@ File: gawk.info, Node: Functions, Next: Library Functions, Prev: Arrays, Up: This major node describes `awk''s built-in functions, which fall into three categories: numeric, string, and I/O. `gawk' provides additional groups of functions to work with values that represent time, do bit -manipulation, sort arrays, and internationalize and localize programs. +manipulation, sort arrays, provide type information, and +internationalize and localize programs. Besides the built-in functions, `awk' has provisions for writing new functions that the rest of a program can use. The second half of this -major node describes these "user-defined" functions. +major node describes these "user-defined" functions. Finally, we +explore indirect function calls, a `gawk'-specific extension that lets +you determine at runtime what function is to be called. * Menu: @@ -11763,7 +11980,7 @@ function followed by arguments in parentheses. For example, `atan2(y + z, 1)' is a call to the function `atan2()' and has two arguments. Whitespace is ignored between the built-in function name and the -open parenthesis, but nonetheless it is good practice to avoid using +opening parenthesis, but nonetheless it is good practice to avoid using whitespace there. User-defined functions do not permit whitespace in this way, and it is easier to avoid mistakes by following a simple convention that always works--no whitespace after a function name. @@ -11789,12 +12006,13 @@ undefined. Thus, avoid writing programs that assume that parameters are evaluated from left to right or from right to left. For example: i = 5 - j = atan2(i++, i *= 2) + j = atan2(++i, i *= 2) If the order of evaluation is left to right, then `i' first becomes -6, and then 12, and `atan2()' is called with the two arguments 6 and -12. But if the order of evaluation is right to left, `i' first becomes -10, then 11, and `atan2()' is called with the two arguments 11 and 10. +six, and then 12, and `atan2()' is called with the two arguments six +and 12. But if the order of evaluation is right to left, `i' first +becomes 10, then 11, and `atan2()' is called with the two arguments 11 +and 10. File: gawk.info, Node: Numeric Functions, Next: String Functions, Prev: Calling Built-in, Up: Built-in @@ -11813,7 +12031,17 @@ brackets ([ ]): `cos(X)' Return the cosine of X, with X in radians. -`div(NUMERATOR, DENOMINATOR, RESULT)' +`exp(X)' + Return the exponential of X (`e ^ X') or report an error if X is + out of range. The range of values X can have depends on your + machine's floating-point representation. + +`int(X)' + Return the nearest integer to X, located between X and zero and + truncated toward zero. For example, `int(3)' is 3, `int(3.9)' is + 3, `int(-3.9)' is -3, and `int(-3)' is -3 as well. + +`intdiv(NUMERATOR, DENOMINATOR, RESULT)' Perform integer division, similar to the standard C function of the same name. First, truncate `numerator' and `denominator' towards zero, creating integer values. Clear the `result' array, and then @@ -11828,18 +12056,6 @@ brackets ([ ]): This function is a `gawk' extension. It is not available in compatibility mode (*note Options::). -`exp(X)' - Return the exponential of X (`e ^ X') or report an error if X is - out of range. The range of values X can have depends on your - machine's floating-point representation. - -`int(X)' - Return the nearest integer to X, located between X and zero and - truncated toward zero. - - For example, `int(3)' is 3, `int(3.9)' is 3, `int(-3.9)' is -3, - and `int(-3)' is -3 as well. - `log(X)' Return the natural logarithm of X, if X is positive; otherwise, return `NaN' ("not a number") on IEEE 754 systems. Additionally, @@ -11852,16 +12068,16 @@ brackets ([ ]): Often random integers are needed instead. Following is a user-defined function that can be used to obtain a random - non-negative integer less than N: + nonnegative integer less than N: function randint(n) { return int(n * rand()) } - The multiplication produces a random number greater than zero and - less than `n'. Using `int()', this result is made into an integer - between zero and `n' - 1, inclusive. + The multiplication produces a random number greater than or equal + to zero and less than `n'. Using `int()', this result is made into + an integer between zero and `n' - 1, inclusive. The following example uses a similar function to produce random integers between one and N. This program prints a new random @@ -11929,7 +12145,7 @@ numbers. (2) `mawk' uses a different seed each time. (3) Computer-generated random numbers really are not truly random. -They are technically known as "pseudorandom." This means that while +They are technically known as "pseudorandom". This means that although the numbers in a sequence appear to be random, you can in fact generate the same sequence of random numbers over and over again. @@ -11942,7 +12158,7 @@ File: gawk.info, Node: String Functions, Next: I/O Functions, Prev: Numeric F The functions in this minor node look at or change the text of one or more strings. - `gawk' understands locales (*note Locales::), and does all string + `gawk' understands locales (*note Locales::) and does all string processing in terms of _characters_, not _bytes_. This distinction is particularly important to understand for locales where one character may be represented by multiple bytes. Thus, for example, `length()' @@ -11960,7 +12176,8 @@ with character indices, and not byte indices. In the following list, optional parameters are enclosed in square brackets ([ ]). Several functions perform string substitution; the full discussion is provided in the description of the `sub()' function, -which comes towards the end since the list is presented alphabetically. +which comes toward the end, because the list is presented +alphabetically. Those functions that are specific to `gawk' are marked with a pound sign (`#'). They are not available in compatibility mode (*note @@ -11978,10 +12195,10 @@ Options::): together. NOTE: The following description ignores the third argument, - HOW, since it requires understanding features that we have - not discussed yet. Thus, the discussion here is a deliberate - simplification. (We do provide all the details later on: - *Note Array Sorting Functions::, for the full story.) + HOW, as it requires understanding features that we have not + discussed yet. Thus, the discussion here is a deliberate + simplification. (We do provide all the details later on; see + *note Array Sorting Functions::, for the full story.) Both functions return the number of elements in the array SOURCE. For `asort()', `gawk' sorts the values of SOURCE and replaces the @@ -12012,7 +12229,7 @@ Options::): a[2] = "de" a[3] = "sac" - The `asorti()' function works similarly to `asort()', however, the + The `asorti()' function works similarly to `asort()'; however, the _indices_ are sorted, instead of the values. Thus, in the previous example, starting with the same initial set of indices and values in `a', calling `asorti(a)' would yield: @@ -12100,7 +12317,7 @@ Options::): With BWK `awk' and `gawk', it is a fatal error to use a regexp constant for FIND. Other implementations allow it, simply treating the regexp constant as an expression meaning `$0 ~ - /regexp/'. (d.c.). + /regexp/'. (d.c.) `length('[STRING]`)' Return the number of characters in STRING. If STRING is a number, @@ -12144,9 +12361,9 @@ Options::): `match(STRING, REGEXP' [`, ARRAY']`)' Search STRING for the longest, leftmost substring matched by the - regular expression, REGEXP and return the character position - (index) at which that substring begins (one, if it starts at the - beginning of STRING). If no match is found, return zero. + regular expression REGEXP and return the character position (index) + at which that substring begins (one, if it starts at the beginning + of STRING). If no match is found, return zero. The REGEXP argument may be either a regexp constant (`/'...`/') or a string constant (`"'...`"'). In the latter case, the string is @@ -12154,7 +12371,7 @@ Options::): discussion of the difference between the two forms, and the implications for writing your program correctly. - The order of the first two arguments is backwards from most other + The order of the first two arguments is the opposite of most other string functions that work with regular expressions, such as `sub()' and `gsub()'. It might help to remember that for `match()', the order is the same as for the `~' operator: `STRING @@ -12220,9 +12437,9 @@ Options::): -| 9 7 There may not be subscripts for the start and index for every - parenthesized subexpression, since they may not all have matched - text; thus they should be tested for with the `in' operator (*note - Reference to Elements::). + parenthesized subexpression, because they may not all have matched + text; thus, they should be tested for with the `in' operator + (*note Reference to Elements::). The ARRAY argument to `match()' is a `gawk' extension. In compatibility mode (*note Options::), using a third argument is a @@ -12255,11 +12472,11 @@ Options::): FIELDSEP, is a regexp describing where to split STRING (much as `FS' can be a regexp describing where to split input records). If FIELDSEP is omitted, the value of `FS' is used. `split()' returns - the number of elements created. SEPS is a `gawk' extension with + the number of elements created. SEPS is a `gawk' extension, with `SEPS[I]' being the separator string between `ARRAY[I]' and - `ARRAY[I+1]'. If FIELDSEP is a single space then any leading + `ARRAY[I+1]'. If FIELDSEP is a single space, then any leading whitespace goes into `SEPS[0]' and any trailing whitespace goes - into `SEPS[N]' where N is the return value of `split()' (that is, + into `SEPS[N]', where N is the return value of `split()' (i.e., the number of elements in ARRAY). The `split()' function splits strings into pieces in a manner @@ -12267,7 +12484,7 @@ Options::): split("cul-de-sac", a, "-", seps) - splits the string `cul-de-sac' into three fields using `-' as the + splits the string `"cul-de-sac"' into three fields using `-' as the separator. It sets the contents of the array `a' as follows: a[1] = "cul" @@ -12284,17 +12501,18 @@ Options::): As with input field-splitting, when the value of FIELDSEP is `" "', leading and trailing whitespace is ignored in values assigned to the elements of ARRAY but not in SEPS, and the elements - are separated by runs of whitespace. Also as with input - field-splitting, if FIELDSEP is the null string, each individual + are separated by runs of whitespace. Also, as with input field + splitting, if FIELDSEP is the null string, each individual character in the string is split into its own array element. (c.e.) Note, however, that `RS' has no effect on the way `split()' works. - Even though `RS = ""' causes newline to also be an input field - separator, this does not affect how `split()' splits strings. + Even though `RS = ""' causes the newline character to also be an + input field separator, this does not affect how `split()' splits + strings. Modern implementations of `awk', including `gawk', allow the third - argument to be a regexp constant (`/abc/') as well as a string. + argument to be a regexp constant (`/'...`/') as well as a string. (d.c.) The POSIX standard allows this as well. *Note Computed Regexps::, for a discussion of the difference between using a string constant or a regexp constant, and the implications for @@ -12395,7 +12613,7 @@ Options::): { sub(/\|/, "\\&"); print } As mentioned, the third argument to `sub()' must be a variable, - field or array element. Some versions of `awk' allow the third + field, or array element. Some versions of `awk' allow the third argument to be an expression that is not an lvalue. In such a case, `sub()' still searches for the pattern and returns zero or one, but the result of the substitution (if any) is thrown away @@ -12493,7 +12711,7 @@ is number zero. File: gawk.info, Node: Gory Details, Up: String Functions -9.1.3.1 More About `\' and `&' with `sub()', `gsub()', and `gensub()' +9.1.3.1 More about `\' and `&' with `sub()', `gsub()', and `gensub()' ..................................................................... CAUTION: This subsubsection has been reported to cause headaches. @@ -12520,11 +12738,11 @@ example, `"a\qb"' is treated as `"aqb"'. At the runtime level, the various functions handle sequences of `\' and `&' differently. The situation is (sadly) somewhat complex. -Historically, the `sub()' and `gsub()' functions treated the two -character sequence `\&' specially; this sequence was replaced in the -generated text with a single `&'. Any other `\' within the REPLACEMENT -string that did not precede an `&' was passed through unchanged. This -is illustrated in *note table-sub-escapes::. +Historically, the `sub()' and `gsub()' functions treated the +two-character sequence `\&' specially; this sequence was replaced in +the generated text with a single `&'. Any other `\' within the +REPLACEMENT string that did not precede an `&' was passed through +unchanged. This is illustrated in *note table-sub-escapes::. You type `sub()' sees `sub()' generates ------- --------- -------------- @@ -12536,13 +12754,13 @@ is illustrated in *note table-sub-escapes::. `\\\\\\&' `\\\&' A literal `\\&' `\\q' `\q' A literal `\q' -Table 9.1: Historical Escape Sequence Processing for `sub()' and +Table 9.1: Historical escape sequence processing for `sub()' and `gsub()' -This table shows both the lexical-level processing, where an odd number -of backslashes becomes an even number at the runtime level, as well as -the runtime processing done by `sub()'. (For the sake of simplicity, -the rest of the following tables only show the case of even numbers of +This table shows the lexical-level processing, where an odd number of +backslashes becomes an even number at the runtime level, as well as the +runtime processing done by `sub()'. (For the sake of simplicity, the +rest of the following tables only show the case of even numbers of backslashes entered at the lexical level.) The problem with the historical approach is that there is no way to @@ -12566,10 +12784,10 @@ This is shown in *note table-sub-proposed::. `\\q' `\q' A literal `\q' `\\\\' `\\' `\\' -Table 9.2: GNU `awk' Rules For `sub()' And Backslash +Table 9.2: `gawk' rules for `sub()' and backslash In a nutshell, at the runtime level, there are now three special -sequences of characters (`\\\&', `\\&' and `\&') whereas historically +sequences of characters (`\\\&', `\\&', and `\&') whereas historically there was only one. However, as in the historical case, any `\' that is not part of one of these three sequences is not special and appears in the output literally. @@ -12593,20 +12811,19 @@ rules are presented in *note table-posix-sub::. `\\q' `\q' A literal `\q' `\\\\' `\\' `\' -Table 9.3: POSIX Rules For `sub()' And `gsub()' +Table 9.3: POSIX rules for `sub()' and `gsub()' The only case where the difference is noticeable is the last one: `\\\\' is seen as `\\' and produces `\' instead of `\\'. Starting with version 3.1.4, `gawk' followed the POSIX rules when -`--posix' is specified (*note Options::). Otherwise, it continued to -follow the proposed rules, since that had been its behavior for many -years. +`--posix' was specified (*note Options::). Otherwise, it continued to +follow the proposed rules, as that had been its behavior for many years. When version 4.0.0 was released, the `gawk' maintainer made the POSIX rules the default, breaking well over a decade's worth of -backwards compatibility.(1) Needless to say, this was a bad idea, and -as of version 4.0.1, `gawk' resumed its historical behavior, and only +backward compatibility.(1) Needless to say, this was a bad idea, and as +of version 4.0.1, `gawk' resumed its historical behavior, and only follows the POSIX rules when `--posix' is given. The rules for `gensub()' are considerably simpler. At the runtime @@ -12625,11 +12842,11 @@ the `\' does not, as shown in *note table-gensub-escapes::. `\\\\\\&' `\\\&' A literal `\&' `\\q' `\q' A literal `q' -Table 9.4: Escape Sequence Processing For `gensub()' +Table 9.4: Escape sequence processing for `gensub()' - Because of the complexity of the lexical and runtime level processing -and the special cases for `sub()' and `gsub()', we recommend the use of -`gawk' and `gensub()' when you have to do substitutions. + Because of the complexity of the lexical- and runtime-level +processing and the special cases for `sub()' and `gsub()', we recommend +the use of `gawk' and `gensub()' when you have to do substitutions. ---------- Footnotes ---------- @@ -12656,10 +12873,10 @@ parameters are enclosed in square brackets ([ ]): When closing a coprocess, it is occasionally useful to first close one end of the two-way pipe and then to close the other. This is done by providing a second argument to `close()'. This second - argument should be one of the two string values `"to"' or `"from"', - indicating which end of the pipe to close. Case in the string does - not matter. *Note Two-way I/O::, which discusses this feature in - more detail and gives an example. + argument (HOW) should be one of the two string values `"to"' or + `"from"', indicating which end of the pipe to close. Case in the + string does not matter. *Note Two-way I/O::, which discusses this + feature in more detail and gives an example. Note that the second argument to `close()' is a `gawk' extension; it is not available in compatibility mode (*note Options::). @@ -12669,23 +12886,23 @@ parameters are enclosed in square brackets ([ ]): either a file opened for writing or a shell command for redirecting output to a pipe or coprocess. - Many utility programs "buffer" their output; i.e., they save + Many utility programs "buffer" their output (i.e., they save information to write to a disk file or the screen in memory until there is enough for it to be worthwhile to send the data to the - output device. This is often more efficient than writing every + output device). This is often more efficient than writing every little bit of information as soon as it is ready. However, sometimes it is necessary to force a program to "flush" its - buffers; that is, write the information to its destination, even - if a buffer is not full. This is the purpose of the `fflush()' - function--`gawk' also buffers its output and the `fflush()' + buffers (i.e., write the information to its destination, even if a + buffer is not full). This is the purpose of the `fflush()' + function--`gawk' also buffers its output, and the `fflush()' function forces `gawk' to flush its buffers. - Brian Kernighan added `fflush()' to his `awk' in April of 1992. - For two decades, it was a common extension. In December, 2012, it - was accepted for inclusion into the POSIX standard. See the - Austin Group website (http://austingroupbugs.net/view.php?id=634). + Brian Kernighan added `fflush()' to his `awk' in April 1992. For + two decades, it was a common extension. In December 2012, it was + accepted for inclusion into the POSIX standard. See the Austin + Group website (http://austingroupbugs.net/view.php?id=634). - POSIX standardizes `fflush()' as follows: If there is no argument, + POSIX standardizes `fflush()' as follows: if there is no argument, or if the argument is the null string (`""'), then `awk' flushes the buffers for _all_ open output files and pipes. @@ -12694,24 +12911,57 @@ parameters are enclosed in square brackets ([ ]): output files and pipes if the argument was the null string. This was changed in order to be compatible with Brian Kernighan's `awk', in the hope that standardizing this - feature in POSIX would then be easier (which indeed helped). + feature in POSIX would then be easier (which indeed proved to + be the case). With `gawk', you can use `fflush("/dev/stdout")' if you wish to flush only the standard output. `fflush()' returns zero if the buffer is successfully flushed; - otherwise, it returns non-zero. (`gawk' returns -1.) In the case - where all buffers are flushed, the return value is zero only if - all buffers were flushed successfully. Otherwise, it is -1, and - `gawk' warns about the problem FILENAME. + otherwise, it returns a nonzero value. (`gawk' returns -1.) In + the case where all buffers are flushed, the return value is zero + only if all buffers were flushed successfully. Otherwise, it is + -1, and `gawk' warns about the problem FILENAME. `gawk' also issues a warning message if you attempt to flush a file or pipe that was opened for reading (such as with `getline'), or if FILENAME is not an open file, pipe, or coprocess. In such a case, `fflush()' returns -1, as well. + Interactive Versus Noninteractive Buffering + + As a side point, buffering issues can be even more confusing if + your program is "interactive" (i.e., communicating with a user + sitting at a keyboard).(1) + + Interactive programs generally "line buffer" their output (i.e., + they write out every line). Noninteractive programs wait until + they have a full buffer, which may be many lines of output. Here + is an example of the difference: + + $ awk '{ print $1 + $2 }' + 1 1 + -| 2 + 2 3 + -| 5 + Ctrl-d + + Each line of output is printed immediately. Compare that behavior + with this example: + + $ awk '{ print $1 + $2 }' | cat + 1 1 + 2 3 + Ctrl-d + -| 2 + -| 5 + + Here, no output is printed until after the `Ctrl-d' is typed, + because it is all buffered and sent down the pipe to `cat' in one + shot. + `system(COMMAND)' - Execute the operating-system command COMMAND and then return to + Execute the operating system command COMMAND and then return to the `awk' program. Return COMMAND's exit status. For example, if the following fragment of code is put in your `awk' @@ -12743,37 +12993,6 @@ parameters are enclosed in square brackets ([ ]): is disabled (*note Options::). - Interactive Versus Noninteractive Buffering - - As a side point, buffering issues can be even more confusing, -depending upon whether your program is "interactive", i.e., -communicating with a user sitting at a keyboard.(1) - - Interactive programs generally "line buffer" their output; i.e., they -write out every line. Noninteractive programs wait until they have a -full buffer, which may be many lines of output. Here is an example of -the difference: - - $ awk '{ print $1 + $2 }' - 1 1 - -| 2 - 2 3 - -| 5 - Ctrl-d - -Each line of output is printed immediately. Compare that behavior with -this example: - - $ awk '{ print $1 + $2 }' | cat - 1 1 - 2 3 - Ctrl-d - -| 2 - -| 5 - -Here, no output is printed until after the `Ctrl-d' is typed, because -it is all buffered and sent down the pipe to `cat' in one shot. - Controlling Output Buffering with `system()' The `fflush()' function provides explicit control over output @@ -12787,8 +13006,8 @@ argument: `gawk' treats this use of the `system()' function as a special case and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with `gawk', this idiom is not only -useful, it is also efficient. While this method should work with other -`awk' implementations, it does not necessarily avoid starting an +useful, it is also efficient. Although this method should work with +other `awk' implementations, it does not necessarily avoid starting an unnecessary shell. (Other implementations may only flush the buffer associated with the standard output and not necessarily all buffered output.) @@ -12831,14 +13050,14 @@ File: gawk.info, Node: Time Functions, Next: Bitwise Functions, Prev: I/O Fun `awk' programs are commonly used to process log files containing timestamp information, indicating when a particular log record was -written. Many programs log their timestamp in the form returned by the -`time()' system call, which is the number of seconds since a particular -epoch. On POSIX-compliant systems, it is the number of seconds since -1970-01-01 00:00:00 UTC, not counting leap seconds.(1) All known -POSIX-compliant systems support timestamps from 0 through 2^31 - 1, -which is sufficient to represent times through 2038-01-19 03:14:07 UTC. -Many systems support a wider range of timestamps, including negative -timestamps that represent times before the epoch. +written. Many programs log their timestamps in the form returned by +the `time()' system call, which is the number of seconds since a +particular epoch. On POSIX-compliant systems, it is the number of +seconds since 1970-01-01 00:00:00 UTC, not counting leap seconds.(1) +All known POSIX-compliant systems support timestamps from 0 through +2^31 - 1, which is sufficient to represent times through 2038-01-19 +03:14:07 UTC. Many systems support a wider range of timestamps, +including negative timestamps that represent times before the epoch. In order to make it easier to process such log files and to produce useful reports, `gawk' provides the following functions for working @@ -12861,9 +13080,9 @@ enclosed in square brackets ([ ]): specified; for example, an hour of -1 means 1 hour before midnight. The origin-zero Gregorian calendar is assumed, with year 0 preceding year 1 and year -1 preceding year 0. The time is - assumed to be in the local timezone. If the daylight-savings flag - is positive, the time is assumed to be daylight savings time; if - zero, the time is assumed to be standard time; and if negative + assumed to be in the local time zone. If the daylight-savings + flag is positive, the time is assumed to be daylight savings time; + if zero, the time is assumed to be standard time; and if negative (the default), `mktime()' attempts to determine whether daylight savings time is in effect for the specified time. @@ -12879,14 +13098,14 @@ enclosed in square brackets ([ ]): Otherwise, the value is formatted for the local time zone. The TIMESTAMP is in the same format as the value returned by the `systime()' function. If no TIMESTAMP argument is supplied, - `gawk' uses the current time of day as the timestamp. If no - FORMAT argument is supplied, `strftime()' uses the value of + `gawk' uses the current time of day as the timestamp. Without a + FORMAT argument, `strftime()' uses the value of `PROCINFO["strftime"]' as the format string (*note Built-in Variables::). The default string value is `"%a %b %e %H:%M:%S %Z %Y"'. This format string produces output that is equivalent to that of the `date' utility. You can assign a new value to `PROCINFO["strftime"]' to change the default - format; see below for the various format directives. + format; see the following list for the various format directives. `systime()' Return the current time as the number of seconds since the system @@ -12950,9 +13169,9 @@ the following date format specifications: `%g' The year modulo 100 of the ISO 8601 week number, as a decimal - number (00-99). For example, January 1, 2012 is in week 53 of + number (00-99). For example, January 1, 2012, is in week 53 of 2011. Thus, the year of its ISO 8601 week number is 2011, even - though its year is 2012. Similarly, December 31, 2012 is in week + though its year is 2012. Similarly, December 31, 2012, is in week 1 of 2013. Thus, the year of its ISO week number is 2013, even though its year is 2012. @@ -13004,23 +13223,23 @@ the following date format specifications: The weekday as a decimal number (1-7). Monday is day one. `%U' - The week number of the year (the first Sunday as the first day of - week one) as a decimal number (00-53). + The week number of the year (with the first Sunday as the first + day of week one) as a decimal number (00-53). `%V' - The week number of the year (the first Monday as the first day of - week one) as a decimal number (01-53). The method for determining - the week number is as specified by ISO 8601. (To wit: if the week - containing January 1 has four or more days in the new year, then - it is week one; otherwise it is week 53 of the previous year and - the next week is week one.) + The week number of the year (with the first Monday as the first + day of week one) as a decimal number (01-53). The method for + determining the week number is as specified by ISO 8601. (To wit: + if the week containing January 1 has four or more days in the new + year, then it is week one; otherwise it is week 53 of the previous + year and the next week is week one.) `%w' The weekday as a decimal number (0-6). Sunday is day zero. `%W' - The week number of the year (the first Monday as the first day of - week one) as a decimal number (00-53). + The week number of the year (with the first Monday as the first + day of week one) as a decimal number (00-53). `%x' The locale's "appropriate" date representation. (This is `%A %B @@ -13037,8 +13256,8 @@ the following date format specifications: The full year as a decimal number (e.g., 2015). `%z' - The timezone offset in a +HHMM format (e.g., the format necessary - to produce RFC 822/RFC 1036 date headers). + The time zone offset in a `+HHMM' format (e.g., the format + necessary to produce RFC 822/RFC 1036 date headers). `%Z' The time zone name or abbreviation; no characters if no time zone @@ -13046,15 +13265,15 @@ the following date format specifications: `%Ec %EC %Ex %EX %Ey %EY %Od %Oe %OH' `%OI %Om %OM %OS %Ou %OU %OV %Ow %OW %Oy' - "Alternate representations" for the specifications that use only + "Alternative representations" for the specifications that use only the second letter (`%c', `%C', and so on).(5) (These facilitate compliance with the POSIX `date' utility.) `%%' A literal `%'. - If a conversion specifier is not one of the above, the behavior is -undefined.(6) + If a conversion specifier is not one of those just listed, the +behavior is undefined.(6) For systems that are not yet fully standards-compliant, `gawk' supplies a copy of `strftime()' from the GNU C Library. It supports @@ -13074,8 +13293,8 @@ format specifications are available: The time as a decimal timestamp in seconds since the epoch. - Additionally, the alternate representations are recognized but their -normal representations are used. + Additionally, the alternative representations are recognized but +their normal representations are used. The following example is an `awk' implementation of the POSIX `date' utility. Normally, the `date' utility prints the current date and time @@ -13155,7 +13374,7 @@ each successive pair of bits in the operands. Three common operations are bitwise AND, OR, and XOR. The operations are described in *note table-bitwise-ops::. - Bit Operator + Bit operator | AND | OR | XOR |--+--+--+--+--+-- Operands | 0 | 1 | 0 | 1 | 0 | 1 @@ -13163,7 +13382,7 @@ table-bitwise-ops::. 0 | 0 0 | 0 1 | 0 1 1 | 0 1 | 1 1 | 1 0 -Table 9.5: Bitwise Operations +Table 9.5: Bitwise operations As you can see, the result of an AND operation is 1 only when _both_ bits are 1. The result of an OR operation is 1 if _either_ bit is 1. @@ -13200,7 +13419,7 @@ are enclosed in square brackets ([ ]): Return the bitwise XOR of the arguments. There must be at least two. - For all of these functions, first the double precision + For all of these functions, first the double-precision floating-point value is converted to the widest C unsigned integer type, then the bitwise operation is performed. If the result cannot be represented exactly as a C `double', leading nonzero bits are removed @@ -13211,7 +13430,7 @@ paragraph, don't worry about it.) Here is a user-defined function (*note User-defined::) that illustrates the use of these functions: - # bits2str --- turn a byte into readable 1's and 0's + # bits2str --- turn a byte into readable ones and zeros function bits2str(bits, data, mask) { @@ -13250,15 +13469,16 @@ This program produces the following output when run: -| lshift(0x99, 2) = 0x264 = 0000001001100100 -| rshift(0x99, 2) = 0x26 = 00100110 - The `bits2str()' function turns a binary number into a string. The -number `1' represents a binary value where the rightmost bit is set to -1. Using this mask, the function repeatedly checks the rightmost bit. -ANDing the mask with the value indicates whether the rightmost bit is 1 -or not. If so, a `"1"' is concatenated onto the front of the string. -Otherwise, a `"0"' is added. The value is then shifted right by one -bit and the loop continues until there are no more 1 bits. + The `bits2str()' function turns a binary number into a string. +Initializing `mask' to one creates a binary value where the rightmost +bit is set to one. Using this mask, the function repeatedly checks the +rightmost bit. ANDing the mask with the value indicates whether the +rightmost bit is one or not. If so, a `"1"' is concatenated onto the +front of the string. Otherwise, a `"0"' is added. The value is then +shifted right by one bit and the loop continues until there are no more +one bits. - If the initial value is zero it returns a simple `"0"'. Otherwise, + If the initial value is zero, it returns a simple `"0"'. Otherwise, at the end, it pads the value with zeros to represent multiples of 8-bit quantities. This is typical in modern computers. @@ -13269,9 +13489,9 @@ Nondecimal-numbers::), and then demonstrates the results of the ---------- Footnotes ---------- - (1) This example shows that 0's come in on the left side. For + (1) This example shows that zeros come in on the left side. For `gawk', this is always true, but in some languages, it's possible to -have the left side fill with 1's. +have the left side fill with ones. File: gawk.info, Node: Type Functions, Next: I18N Functions, Prev: Bitwise Functions, Up: Built-in @@ -13285,7 +13505,7 @@ traverses every element of an array of arrays (*note Arrays of Arrays::). `isarray(X)' - Return a true value if X is an array. Otherwise return false. + Return a true value if X is an array. Otherwise, return false. `isarray()' is meant for use in two circumstances. The first is when traversing a multidimensional array: you can test if an element is @@ -13294,11 +13514,11 @@ user-defined function (not discussed yet; *note User-defined::), to test if a parameter is an array or not. NOTE: Using `isarray()' at the global level to test variables - makes no sense. Since you are the one writing the program, you are - supposed to know if your variables are arrays or not. And in fact, - due to the way `gawk' works, if you pass the name of a variable - that has not been previously used to `isarray()', `gawk' ends up - turning it into a scalar. + makes no sense. Because you are the one writing the program, you + are supposed to know if your variables are arrays or not. And in + fact, due to the way `gawk' works, if you pass the name of a + variable that has not been previously used to `isarray()', `gawk' + ends up turning it into a scalar. File: gawk.info, Node: I18N Functions, Prev: Type Functions, Up: Built-in @@ -13332,8 +13552,8 @@ brackets ([ ]): Return the plural form used for NUMBER of the translation of STRING1 and STRING2 in text domain DOMAIN for locale category CATEGORY. STRING1 is the English singular variant of a message, - and STRING2 the English plural variant of the same message. The - default value for DOMAIN is the current value of `TEXTDOMAIN'. + and STRING2 is the English plural variant of the same message. + The default value for DOMAIN is the current value of `TEXTDOMAIN'. The default value for CATEGORY is `"LC_MESSAGES"'. @@ -13344,8 +13564,8 @@ File: gawk.info, Node: User-defined, Next: Indirect Calls, Prev: Built-in, U Complicated `awk' programs can often be simplified by defining your own functions. User-defined functions can be called just like built-in -ones (*note Function Calls::), but it is up to you to define them, -i.e., to tell `awk' what they should do. +ones (*note Function Calls::), but it is up to you to define them +(i.e., to tell `awk' what they should do). * Menu: @@ -13362,7 +13582,7 @@ File: gawk.info, Node: Definition Syntax, Next: Function Example, Up: User-de 9.2.1 Function Definition Syntax -------------------------------- - It's entirely fair to say that the `awk' syntax for local variable + It's entirely fair to say that the awk syntax for local variable definitions is appallingly awful. -- Brian Kernighan Definitions of functions can appear anywhere between the rules of an @@ -13392,17 +13612,22 @@ the argument names are used to hold the argument values given in the call. A function cannot have two parameters with the same name, nor may it -have a parameter with the same name as the function itself. In -addition, according to the POSIX standard, function parameters cannot -have the same name as one of the special predefined variables (*note -Built-in Variables::). Not all versions of `awk' enforce this -restriction. +have a parameter with the same name as the function itself. + + CAUTION: According to the POSIX standard, function parameters + cannot have the same name as one of the special predefined + variables (*note Built-in Variables::), nor may a function + parameter have the same name as another function. + + Not all versions of `awk' enforce these restrictions. `gawk' + always enforces the first restriction. With `--posix' (*note + Options::), it also enforces the second restriction. Local variables act like the empty string if referenced where a string value is required, and like zero if referenced where a numeric -value is required. This is the same as regular variables that have -never been assigned a value. (There is more to understand about local -variables; *note Dynamic Typing::.) +value is required. This is the same as the behavior of regular +variables that have never been assigned a value. (There is more to +understand about local variables; *note Dynamic Typing::.) The BODY-OF-FUNCTION consists of `awk' statements. It is the most important part of the definition, because it says what the function @@ -13431,9 +13656,9 @@ function is supposed to be used. variable values hide, or "shadow", any variables of the same names used in the rest of the program. The shadowed variables are not accessible in the function definition, because there is no way to name them while -their names have been taken away for the local variables. All other -variables used in the `awk' program can be referenced or set normally -in the function's body. +their names have been taken away for the arguments and local variables. +All other variables used in the `awk' program can be referenced or set +normally in the function's body. The arguments and local variables last only as long as the function body is executing. Once the body finishes, you can once again access @@ -13458,7 +13683,7 @@ function: func foo() { a = sqrt($1) ; print a } -Instead it defines a rule that, for each record, concatenates the value +Instead, it defines a rule that, for each record, concatenates the value of the variable `func' with the return value of the function `foo'. If the resulting string is non-null, the action is executed. This is probably not what is desired. (`awk' accepts this input as @@ -13470,7 +13695,7 @@ keyword `function' when defining a function. ---------- Footnotes ---------- - (1) This program won't actually run, since `foo()' is undefined. + (1) This program won't actually run, because `foo()' is undefined. File: gawk.info, Node: Function Example, Next: Function Caveats, Prev: Definition Syntax, Up: User-defined @@ -13486,7 +13711,7 @@ takes a number and prints it in a specific format: printf "%6.3g\n", num } -To illustrate, here is an `awk' rule that uses our `myprint' function: +To illustrate, here is an `awk' rule that uses our `myprint()' function: $3 > 0 { myprint($3) } @@ -13515,16 +13740,16 @@ extra whitespace signifies the start of the local variable list): When working with arrays, it is often necessary to delete all the elements in an array and start over with a new list of elements (*note Delete::). Instead of having to repeat this loop everywhere that you -need to clear out an array, your program can just call `delarray'. +need to clear out an array, your program can just call `delarray()'. (This guarantees portability. The use of `delete ARRAY' to delete the contents of an entire array is a relatively recent(1) addition to the POSIX standard.) The following is an example of a recursive function. It takes a -string as an input parameter and returns the string in backwards order. +string as an input parameter and returns the string in reverse order. Recursive functions must always have a test that stops the recursion. In this case, the recursion terminates when the input string is already -empty. +empty: function rev(str) { @@ -13560,9 +13785,9 @@ an `awk' version of `ctime()': } You might think that `ctime()' could use `PROCINFO["strftime"]' for -its format string. That would be a mistake, since `ctime()' is supposed -to return the time formatted in a standard fashion, and user-level code -could have changed `PROCINFO["strftime"]'. +its format string. That would be a mistake, because `ctime()' is +supposed to return the time formatted in a standard fashion, and +user-level code could have changed `PROCINFO["strftime"]'. ---------- Footnotes ---------- @@ -13587,7 +13812,7 @@ the function. File: gawk.info, Node: Calling A Function, Next: Variable Scope, Up: Function Caveats -9.2.3.1 Writing A Function Call +9.2.3.1 Writing a Function Call ............................... A function call consists of the function name followed by the arguments @@ -13600,10 +13825,10 @@ string concatenation): foo(x y, "lose", 4 * z) CAUTION: Whitespace characters (spaces and TABs) are not allowed - between the function name and the open-parenthesis of the argument - list. If you write whitespace by mistake, `awk' might think that - you mean to concatenate a variable with an expression in - parentheses. However, it notices that you used a function name + between the function name and the opening parenthesis of the + argument list. If you write whitespace by mistake, `awk' might + think that you mean to concatenate a variable with an expression + in parentheses. However, it notices that you used a function name and not a variable name, and reports an error. @@ -13612,14 +13837,14 @@ File: gawk.info, Node: Variable Scope, Next: Pass By Value/Reference, Prev: C 9.2.3.2 Controlling Variable Scope .................................. -Unlike many languages, there is no way to make a variable local to a +Unlike in many languages, there is no way to make a variable local to a `{' ... `}' block in `awk', but you can make a variable local to a function. It is good practice to do so whenever a variable is needed only in that function. To make a variable local to a function, simply declare the variable as an argument after the actual function arguments (*note Definition -Syntax::). Look at the following example where variable `i' is a +Syntax::). Look at the following example, where variable `i' is a global variable used by both functions `foo()' and `bar()': function bar() @@ -13655,8 +13880,8 @@ variable instance: foo's i=3 top's i=3 - If you want `i' to be local to both `foo()' and `bar()' do as -follows (the extra-space before `i' is a coding convention to indicate + If you want `i' to be local to both `foo()' and `bar()', do as +follows (the extra space before `i' is a coding convention to indicate that `i' is a local variable, not an argument): function bar( i) @@ -13729,20 +13954,17 @@ create new arrays. Consider this example: File: gawk.info, Node: Pass By Value/Reference, Prev: Variable Scope, Up: Function Caveats -9.2.3.3 Passing Function Arguments By Value Or By Reference +9.2.3.3 Passing Function Arguments by Value Or by Reference ........................................................... In `awk', when you declare a function, there is no way to declare explicitly whether the arguments are passed "by value" or "by reference". - Instead the passing convention is determined at runtime when the -function is called according to the following rule: - - * If the argument is an array variable, then it is passed by - reference, - - * Otherwise the argument is passed by value. + Instead, the passing convention is determined at runtime when the +function is called, according to the following rule: if the argument is +an array variable, then it is passed by reference. Otherwise, the +argument is passed by value. Passing an argument by value means that when a function is called, it is given a _copy_ of the value of this argument. The caller may use a @@ -13798,7 +14020,7 @@ function _are_ visible outside that function. stores `"two"' in the second element of `a'. Some `awk' implementations allow you to call a function that has not -been defined. They only report a problem at runtime when the program +been defined. They only report a problem at runtime, when the program actually tries to call the function. For example: BEGIN { @@ -13819,7 +14041,7 @@ undefined functions. Some `awk' implementations generate a runtime error if you use either the `next' statement or the `nextfile' statement (*note Next -Statement::, also *note Nextfile Statement::) inside a user-defined +Statement::, and *note Nextfile Statement::) inside a user-defined function. `gawk' does not have this limitation. @@ -13843,15 +14065,15 @@ undefined, and therefore, unpredictable. In practice, though, all versions of `awk' simply return the null string, which acts like zero if used in a numeric context. - A `return' statement with no value expression is assumed at the end -of every function definition. So if control reaches the end of the -function body, then technically, the function returns an unpredictable + A `return' statement without an EXPRESSION is assumed at the end of +every function definition. So, if control reaches the end of the +function body, then technically the function returns an unpredictable value. In practice, it returns the empty string. `awk' does _not_ warn you if you use the return value of such a function. Sometimes, you want to write a function for what it does, not for what it returns. Such a function corresponds to a `void' function in -C, C++ or Java, or to a `procedure' in Ada. Thus, it may be +C, C++, or Java, or to a `procedure' in Ada. Thus, it may be appropriate to not return any value; simply bear in mind that you should not be using the return value of such a function. @@ -13868,11 +14090,12 @@ a value for the largest number among the elements of an array: } You call `maxelt()' with one argument, which is an array name. The -local variables `i' and `ret' are not intended to be arguments; while -there is nothing to stop you from passing more than one argument to -`maxelt()', the results would be strange. The extra space before `i' -in the function parameter list indicates that `i' and `ret' are local -variables. You should follow this convention when defining functions. +local variables `i' and `ret' are not intended to be arguments; there +is nothing to stop you from passing more than one argument to +`maxelt()' but the results would be strange. The extra space before +`i' in the function parameter list indicates that `i' and `ret' are +local variables. You should follow this convention when defining +functions. The following program uses the `maxelt()' function. It loads an array, calls `maxelt()', and then reports the maximum number in that @@ -13956,13 +14179,13 @@ you can specify the name of the function to call as a string variable, and then call the function. Let's look at an example. Suppose you have a file with your test scores for the classes you -are taking. The first field is the class name. The following fields -are the functions to call to process the data, up to a "marker" field +are taking, and you wish to get the sum and the average of your test +scores. The first field is the class name. The following fields are +the functions to call to process the data, up to a "marker" field `data:'. Following the marker, to the end of the record, are the various numeric test scores. - Here is the initial file; you wish to get the sum and the average of -your test scores: + Here is the initial file: Biology_101 sum average data: 87.0 92.4 78.5 94.9 Chemistry_305 sum average data: 75.2 98.3 94.7 88.2 @@ -13986,15 +14209,15 @@ function calls, you tell `gawk' to use the _value_ of a variable as the _name_ of the function to call. The syntax is similar to that of a regular function call: an -identifier immediately followed by a left parenthesis, any arguments, -and then a closing right parenthesis, with the addition of a leading `@' -character: +identifier immediately followed by an opening parenthesis, any +arguments, and then a closing parenthesis, with the addition of a +leading `@' character: the_func = "sum" result = @the_func() # calls the sum() function Here is a full program that processes the previously shown data, -using indirect function calls. +using indirect function calls: # indirectcall.awk --- Demonstrate indirect function calls @@ -14020,9 +14243,9 @@ using indirect function calls. return ret } - These two functions expect to work on fields; thus the parameters + These two functions expect to work on fields; thus, the parameters `first' and `last' indicate where in the fields to start and end. -Otherwise they perform the expected computations and are not unusual. +Otherwise, they perform the expected computations and are not unusual: # For each record, print the class name and the requested statistics { @@ -14075,18 +14298,19 @@ to force it to be a string value.) may think at first. The C and C++ languages provide "function pointers," which are a mechanism for calling a function chosen at runtime. One of the most well-known uses of this ability is the C -`qsort()' function, which sorts an array using the famous "quick sort" +`qsort()' function, which sorts an array using the famous "quicksort" algorithm (see the Wikipedia article -(http://en.wikipedia.org/wiki/Quick_sort) for more information). To -use this function, you supply a pointer to a comparison function. This +(http://en.wikipedia.org/wiki/Quicksort) for more information). To use +this function, you supply a pointer to a comparison function. This mechanism allows you to sort arbitrary data in an arbitrary fashion. We can do something similar using `gawk', like this: # quicksort.awk --- Quicksort algorithm, with user-supplied # comparison function - # quicksort --- C.A.R. Hoare's quick sort algorithm. See Wikipedia - # or almost any algorithms or computer science text + + # quicksort --- C.A.R. Hoare's quicksort algorithm. See Wikipedia + # or almost any algorithms or computer science text. function quicksort(data, left, right, less_than, i, last) { @@ -14115,7 +14339,7 @@ mechanism allows you to sort arbitrary data in an arbitrary fashion. The `quicksort()' function receives the `data' array, the starting and ending indices to sort (`left' and `right'), and the name of a function that performs a "less than" comparison. It then implements -the quick sort algorithm. +the quicksort algorithm. To make use of the sorting function, we return to our previous example. The first thing to do is write some comparison functions: @@ -14209,70 +14433,24 @@ names of the two comparison functions: -| rsort: <100.0 95.6 93.4 87.1> Another example where indirect functions calls are useful can be -found in processing arrays. *note Walking Arrays::, presented a simple -function for "walking" an array of arrays. That function simply -printed the name and value of each scalar array element. However, it is -easy to generalize that function, by passing in the name of a function -to call when walking an array. The modified function looks like this: - - function process_array(arr, name, process, do_arrays, i, new_name) - { - for (i in arr) { - new_name = (name "[" i "]") - if (isarray(arr[i])) { - if (do_arrays) - @process(new_name, arr[i]) - process_array(arr[i], new_name, process, do_arrays) - } else - @process(new_name, arr[i]) - } - } - - The arguments are as follows: - -`arr' - The array. - -`name' - The name of the array (a string). - -`process' - The name of the function to call. - -`do_arrays' - If this is true, the function can handle elements that are - subarrays. - - If subarrays are to be processed, that is done before walking them -further. - - When run with the following scaffolding, the function produces the -same results as does the earlier `walk_array()' function: - - BEGIN { - a[1] = 1 - a[2][1] = 21 - a[2][2] = 22 - a[3] = 3 - a[4][1][1] = 411 - a[4][2] = 42 - - process_array(a, "a", "do_print", 0) - } - - function do_print(name, element) - { - printf "%s = %s\n", name, element - } +found in processing arrays. This is described in *note Walking Arrays::. Remember that you must supply a leading `@' in front of an indirect function call. Starting with version 4.1.2 of `gawk', indirect function calls may also be used with built-in functions and with extension functions -(*note Dynamic Extensions::). The only thing you cannot do is pass a -regular expression constant to a built-in function through an indirect -function call.(1) +(*note Dynamic Extensions::). There are some limitations when calling +built-in functions indirectly, as follows. + + * You cannot pass a regular expression constant to a built-in + function through an indirect function call.(1) This applies to the + `sub()', `gsub()', `gensub()', `match()', `split()' and + `patsplit()' functions. + + * If calling `sub()' or `gsub()', you may only pass two arguments, + since those functions are unusual in that they update their third + argument. This means that `$0' will be updated. `gawk' does its best to make indirect function calls efficient. For example, in the following case: @@ -14299,7 +14477,7 @@ File: gawk.info, Node: Functions Summary, Prev: Indirect Calls, Up: Functions * POSIX `awk' provides three kinds of built-in functions: numeric, string, and I/O. `gawk' provides functions that sort arrays, work with values representing time, do bit manipulation, determine - variable type (array vs. scalar), and internationalize and + variable type (array versus scalar), and internationalize and localize programs. `gawk' also provides several extensions to some of standard functions, typically in the form of additional arguments. @@ -14355,7 +14533,7 @@ File: gawk.info, Node: Library Functions, Next: Sample Programs, Prev: Functi *note User-defined::, describes how to write your own `awk' functions. Writing functions is important, because it allows you to encapsulate algorithms and program tasks in a single place. It simplifies -programming, making program development more manageable, and making +programming, making program development more manageable and making programs more readable. In their seminal 1976 book, `Software Tools',(1) Brian Kernighan and @@ -14389,8 +14567,8 @@ functions and would like to contribute them to the `awk' user community, see *note How To Contribute::, for more information. The programs in this major node and in *note Sample Programs::, -freely use features that are `gawk'-specific. Rewriting these programs -for different implementations of `awk' is pretty straightforward. +freely use `gawk'-specific features. Rewriting these programs for +different implementations of `awk' is pretty straightforward: * Diagnostic error messages are sent to `/dev/stderr'. Use `| "cat 1>&2"' instead of `> "/dev/stderr"' if your system does not have a @@ -14448,8 +14626,8 @@ specific function). There is no intermediate state analogous to Library functions often need to have global variables that they can use to preserve state information between calls to the function--for example, `getopt()''s variable `_opti' (*note Getopt Function::). Such -variables are called "private", since the only functions that need to -use them are the ones in the library. +variables are called "private", as the only functions that need to use +them are the ones in the library. When writing a library function, you should try to choose names for your private variables that will not conflict with any variables used by @@ -14460,20 +14638,20 @@ often use variable names like these for their own purposes. The example programs shown in this major node all start the names of their private variables with an underscore (`_'). Users generally don't use leading underscores in their variable names, so this -convention immediately decreases the chances that the variable name +convention immediately decreases the chances that the variable names will be accidentally shared with the user's program. In addition, several of the library functions use a prefix that helps indicate what function or set of functions use the variables--for example, `_pw_byname()' in the user database routines (*note Passwd -Functions::). This convention is recommended, since it even further +Functions::). This convention is recommended, as it even further decreases the chance of inadvertent conflict among variable names. Note that this convention is used equally well for variable names and for private function names.(1) As a final note on variable naming, if a function makes global variables available for use by a main program, it is a good convention -to start that variable's name with a capital letter--for example, +to start those variables' names with a capital letter--for example, `getopt()''s `Opterr' and `Optind' variables (*note Getopt Function::). The leading capital letter indicates that it is global, while the fact that the variable name is not all capital letters indicates that the @@ -14481,7 +14659,7 @@ variable is not one of `awk''s predefined variables, such as `FS'. It is also important that _all_ variables in library functions that do not need to save state are, in fact, declared local.(2) If this is -not done, the variable could accidentally be used in the user's +not done, the variables could accidentally be used in the user's program, leading to bugs that are very difficult to track down: function lib_func(x, y, l1, l2) @@ -14507,9 +14685,9 @@ merely recommend that you do so. ---------- Footnotes ---------- - (1) While all the library routines could have been rewritten to use -this convention, this was not done, in order to show how our own `awk' -programming style has evolved and to provide some basis for this + (1) Although all the library routines could have been rewritten to +use this convention, this was not done, in order to show how our own +`awk' programming style has evolved and to provide some basis for this discussion. (2) `gawk''s `--dump-variables' command-line option is useful for @@ -14543,7 +14721,7 @@ programming use. File: gawk.info, Node: Strtonum Function, Next: Assert Function, Up: General Functions -10.2.1 Converting Strings To Numbers +10.2.1 Converting Strings to Numbers ------------------------------------ The `strtonum()' function (*note String Functions::) is a `gawk' @@ -14610,8 +14788,8 @@ then `mystrtonum()' loops through each character in the string. It sets `k' to the index in `"1234567"' of the current octal digit. The return value will either be the same number as the digit, or zero if the character is not there, which will be true for a `0'. This is -safe, since the regexp test in the `if' ensures that only octal values -are converted. +safe, because the regexp test in the `if' ensures that only octal +values are converted. Similar logic applies to the code that checks for and converts a hexadecimal value, which starts with `0x' or `0X'. The use of @@ -14659,7 +14837,7 @@ for use in printing the diagnostic message. This is not possible in `awk', so this `assert()' function also requires a string version of the condition that is being tested. Following is the function: - # assert --- assert that a condition is true. Otherwise exit. + # assert --- assert that a condition is true. Otherwise, exit. function assert(condition, string) { @@ -14680,7 +14858,7 @@ the condition that is being tested. Following is the function: false, it prints a message to standard error, using the `string' parameter to describe the failed condition. It then sets the variable `_assert_exit' to one and executes the `exit' statement. The `exit' -statement jumps to the `END' rule. If the `END' rules finds +statement jumps to the `END' rule. If the `END' rule finds `_assert_exit' to be true, it exits immediately. The purpose of the test in the `END' rule is to keep any other `END' @@ -14840,8 +15018,8 @@ distant past, at least one minicomputer manufacturer used ASCII, but with mark parity, meaning that the leftmost bit in the byte is always 1. This means that on those systems, characters have numeric values from 128 to 255. Finally, large mainframe systems use the EBCDIC -character set, which uses all 256 values. While there are other -character sets in use on some older systems, they are not really worth +character set, which uses all 256 values. There are other character +sets in use on some older systems, but they are not really worth worrying about: function ord(str, c) @@ -14895,11 +15073,11 @@ the strings in an array into one long string. The following function, `join()', accomplishes this task. It is used later in several of the application programs (*note Sample Programs::). - Good function design is important; this function needs to be general -but it should also have a reasonable default behavior. It is called -with an array as well as the beginning and ending indices of the + Good function design is important; this function needs to be +general, but it should also have a reasonable default behavior. It is +called with an array as well as the beginning and ending indices of the elements in the array to be merged. This assumes that the array -indices are numeric--a reasonable assumption since the array was likely +indices are numeric--a reasonable assumption, as the array was likely created with `split()' (*note String Functions::): # join.awk --- join an array into a string @@ -14939,7 +15117,7 @@ File: gawk.info, Node: Getlocaltime Function, Next: Readfile Function, Prev: The `systime()' and `strftime()' functions described in *note Time Functions::, provide the minimum functionality necessary for dealing -with the time of day in human readable form. While `strftime()' is +with the time of day in human-readable form. Although `strftime()' is extensive, the control formats are not necessarily easy to remember or intuitively obvious when reading a program. @@ -15016,7 +15194,7 @@ optional timestamp value to use instead of the current time. File: gawk.info, Node: Readfile Function, Next: Shell Quoting, Prev: Getlocaltime Function, Up: General Functions -10.2.8 Reading A Whole File At Once +10.2.8 Reading a Whole File at Once ----------------------------------- Often, it is convenient to have the entire contents of a file available @@ -15058,13 +15236,13 @@ reads the entire contents of the named file in one shot: It works by setting `RS' to `^$', a regular expression that will never match if the file has contents. `gawk' reads data from the file -into `tmp' attempting to match `RS'. The match fails after each read, +into `tmp', attempting to match `RS'. The match fails after each read, but fails quickly, such that `gawk' fills `tmp' with the entire contents of the file. (*Note Records::, for information on `RT' and `RS'.) In the case that `file' is empty, the return value is the null -string. Thus calling code may use something like: +string. Thus, calling code may use something like: contents = readfile("/some/path") if (length(contents) == 0) @@ -15079,7 +15257,7 @@ also reads an entire file into memory. File: gawk.info, Node: Shell Quoting, Prev: Readfile Function, Up: General Functions -10.2.9 Quoting Strings to Pass to The Shell +10.2.9 Quoting Strings to Pass to the Shell ------------------------------------------- Michael Brennan offers the following programming pattern, which he uses @@ -15107,7 +15285,7 @@ frequently: Note the need for shell quoting. The function `shell_quote()' does it. `SINGLE' is the one-character string `"'"' and `QSINGLE' is the -three-character string `"\"'\""'. +three-character string `"\"'\""': # shell_quote --- quote an argument for passing to the shell @@ -15131,7 +15309,7 @@ three-character string `"\"'\""'. File: gawk.info, Node: Data File Management, Next: Getopt Function, Prev: General Functions, Up: Library Functions -10.3 Data File Management +10.3 Data file Management ========================= This minor node presents functions that are useful for managing @@ -15148,14 +15326,15 @@ command-line data files. File: gawk.info, Node: Filetrans Function, Next: Rewind Function, Up: Data File Management -10.3.1 Noting Data File Boundaries +10.3.1 Noting Data file Boundaries ---------------------------------- -The `BEGIN' and `END' rules are each executed exactly once at the +The `BEGIN' and `END' rules are each executed exactly once, at the beginning and end of your `awk' program, respectively (*note BEGIN/END::). We (the `gawk' authors) once had a user who mistakenly -thought that the `BEGIN' rule is executed at the beginning of each data -file and the `END' rule is executed at the end of each data file. +thought that the `BEGIN' rules were executed at the beginning of each +data file and the `END' rules were executed at the end of each data +file. When informed that this was not the case, the user requested that we add new special patterns to `gawk', named `BEGIN_FILE' and `END_FILE', @@ -15189,7 +15368,7 @@ does so _portably_; this works with any implementation of `awk': This file must be loaded before the user's "main" program, so that the rule it supplies is executed first. - This rule relies on `awk''s `FILENAME' variable that automatically + This rule relies on `awk''s `FILENAME' variable, which automatically changes for each new data file. The current file name is saved in a private variable, `_oldfilename'. If `FILENAME' does not equal `_oldfilename', then a new data file is being processed and it is @@ -15204,14 +15383,14 @@ correctly even for the first data file. The program also supplies an `END' rule to do the final processing for the last file. Because this `END' rule comes before any `END' rules supplied in the "main" program, `endfile()' is called first. Once -again the value of multiple `BEGIN' and `END' rules should be clear. +again, the value of multiple `BEGIN' and `END' rules should be clear. If the same data file occurs twice in a row on the command line, then `endfile()' and `beginfile()' are not executed at the end of the first pass and at the beginning of the second pass. The following version solves the problem: - # ftrans.awk --- handle data file transitions + # ftrans.awk --- handle datafile transitions # # user supplies beginfile() and endfile() functions @@ -15227,19 +15406,20 @@ solves the problem: *note Wc Program::, shows how this library function can be used and how it simplifies writing the main program. - So Why Does `gawk' have `BEGINFILE' and `ENDFILE'? + So Why Does `gawk' Have `BEGINFILE' and `ENDFILE'? You are probably wondering, if `beginfile()' and `endfile()' functions can do the job, why does `gawk' have `BEGINFILE' and -`ENDFILE' patterns (*note BEGINFILE/ENDFILE::)? +`ENDFILE' patterns? Good question. Normally, if `awk' cannot open a file, this causes an immediate fatal error. In this case, there is no way for a -user-defined function to deal with the problem, since the mechanism for +user-defined function to deal with the problem, as the mechanism for calling it relies on the file being open and at the first record. Thus, the main reason for `BEGINFILE' is to give you a "hook" to catch files that cannot be processed. `ENDFILE' exists for symmetry, and because -it provides an easy way to do per-file cleanup processing. +it provides an easy way to do per-file cleanup processing. For more +information, refer to *note BEGINFILE/ENDFILE::. File: gawk.info, Node: Rewind Function, Next: File Checking, Prev: Filetrans Function, Up: Data File Management @@ -15247,15 +15427,14 @@ File: gawk.info, Node: Rewind Function, Next: File Checking, Prev: Filetrans 10.3.2 Rereading the Current File --------------------------------- -Another request for a new built-in function was for a `rewind()' -function that would make it possible to reread the current file. The -requesting user didn't want to have to use `getline' (*note Getline::) -inside a loop. +Another request for a new built-in function was for a function that +would make it possible to reread the current file. The requesting user +didn't want to have to use `getline' (*note Getline::) inside a loop. However, as long as you are not in the `END' rule, it is quite easy to arrange to immediately close the current input file and then start -over with it from the top. For lack of a better name, we'll call it -`rewind()': +over with it from the top. For lack of a better name, we'll call the +function `rewind()': # rewind.awk --- rewind the current file and start over @@ -15279,13 +15458,13 @@ over with it from the top. For lack of a better name, we'll call it Auto-set::), which is specific to `gawk'. It also relies on the `nextfile' keyword (*note Nextfile Statement::). Because of this, you should not call it from an `ENDFILE' rule. (This isn't necessary -anyway, since as soon as an `ENDFILE' rule finishes `gawk' goes to the -next file!) +anyway, because `gawk' goes to the next file as soon as an `ENDFILE' +rule finishes!) File: gawk.info, Node: File Checking, Next: Empty Files, Prev: Rewind Function, Up: Data File Management -10.3.3 Checking for Readable Data Files +10.3.3 Checking for Readable Data files --------------------------------------- Normally, if you give `awk' a data file that isn't readable, it stops @@ -15308,12 +15487,12 @@ following program to your `awk' program: } This works, because the `getline' won't be fatal. Removing the -element from `ARGV' with `delete' skips the file (since it's no longer -in the list). See also *note ARGC and ARGV::. +element from `ARGV' with `delete' skips the file (because it's no +longer in the list). See also *note ARGC and ARGV::. - The regular expression check purposely does not use character classes -such as `[:alpha:]' and `[:alnum:]' (*note Bracket Expressions::) since -`awk' variable names only allow the English letters. + Because `awk' variable names only allow the English letters, the +regular expression check purposely does not use character classes such +as `[:alpha:]' and `[:alnum:]' (*note Bracket Expressions::). ---------- Footnotes ---------- @@ -15324,14 +15503,14 @@ opened. However, the code here provides a portable solution. File: gawk.info, Node: Empty Files, Next: Ignoring Assigns, Prev: File Checking, Up: Data File Management -10.3.4 Checking for Zero-length Files +10.3.4 Checking for Zero-Length Files ------------------------------------- All known `awk' implementations silently skip over zero-length files. This is a by-product of `awk''s implicit read-a-record-and-match-against-the-rules loop: when `awk' tries to -read a record from an empty file, it immediately receives an end of -file indication, closes the file, and proceeds on to the next +read a record from an empty file, it immediately receives an +end-of-file indication, closes the file, and proceeds on to the next command-line data file, _without_ executing any user-level `awk' program code. @@ -15375,13 +15554,13 @@ of the `for' loop uses the `<=' operator, not `<'. File: gawk.info, Node: Ignoring Assigns, Prev: Empty Files, Up: Data File Management -10.3.5 Treating Assignments as File Names +10.3.5 Treating Assignments as File names ----------------------------------------- Occasionally, you might not want `awk' to process command-line variable assignments (*note Assignment Options::). In particular, if you have a file name that contains an `=' character, `awk' treats the file name as -an assignment, and does not process it. +an assignment and does not process it. Some users have suggested an additional command-line option for `gawk' to disable command-line assignments. However, some simple @@ -15421,11 +15600,11 @@ File: gawk.info, Node: Getopt Function, Next: Passwd Functions, Prev: Data Fi 10.4 Processing Command-Line Options ==================================== -Most utilities on POSIX compatible systems take options on the command +Most utilities on POSIX-compatible systems take options on the command line that can be used to change the way a program behaves. `awk' is an example of such a program (*note Options::). Often, options take -"arguments"; i.e., data that the program needs to correctly obey the -command-line option. For example, `awk''s `-F' option requires a +"arguments" (i.e., data that the program needs to correctly obey the +command-line option). For example, `awk''s `-F' option requires a string to use as the field separator. The first occurrence on the command line of either `--' or a string that does not begin with `-' ends the options. @@ -15650,10 +15829,10 @@ next element in `argv'. If neither condition is true, then only on the next call to `getopt()'. The `BEGIN' rule initializes both `Opterr' and `Optind' to one. -`Opterr' is set to one, since the default behavior is for `getopt()' to -print a diagnostic message upon seeing an invalid option. `Optind' is -set to one, since there's no reason to look at the program name, which -is in `ARGV[0]': +`Opterr' is set to one, because the default behavior is for `getopt()' +to print a diagnostic message upon seeing an invalid option. `Optind' +is set to one, because there's no reason to look at the program name, +which is in `ARGV[0]': BEGIN { Opterr = 1 # default is to diagnose @@ -15671,8 +15850,8 @@ is in `ARGV[0]': } } - The rest of the `BEGIN' rule is a simple test program. Here is the -result of two sample runs of the test program: + The rest of the `BEGIN' rule is a simple test program. Here are the +results of two sample runs of the test program: $ awk -f getopt.awk -v _getopt_test=1 -- -a -cbARG bax -x -| c = <a>, Optarg = <> @@ -15693,14 +15872,14 @@ result of two sample runs of the test program: In both runs, the first `--' terminates the arguments to `awk', so that it does not try to interpret the `-a', etc., as its own options. - NOTE: After `getopt()' is through, user level code must clear out + NOTE: After `getopt()' is through, user-level code must clear out all the elements of `ARGV' from 1 to `Optind', so that `awk' does not try to process the command-line options as file names. Using `#!' with the `-E' option may help avoid conflicts between -your program's options and `gawk''s options, since `-E' causes `gawk' -to abandon processing of further options (*note Executable Scripts::, -and *note Options::). +your program's options and `gawk''s options, as `-E' causes `gawk' to +abandon processing of further options (*note Executable Scripts::, and +*note Options::). Several of the sample programs presented in *note Sample Programs::, use `getopt()' to process their arguments. @@ -15709,7 +15888,7 @@ use `getopt()' to process their arguments. (1) This function was written before `gawk' acquired the ability to split strings into single characters using `""' as the separator. We -have left it alone, since using `substr()' is more portable. +have left it alone, as using `substr()' is more portable. File: gawk.info, Node: Passwd Functions, Next: Group Functions, Prev: Getopt Function, Up: Library Functions @@ -15718,10 +15897,10 @@ File: gawk.info, Node: Passwd Functions, Next: Group Functions, Prev: Getopt ============================== The `PROCINFO' array (*note Built-in Variables::) provides access to -the current user's real and effective user and group ID numbers, and if -available, the user's supplementary group set. However, because these -are numbers, they do not provide very useful information to the average -user. There needs to be some way to find the user information +the current user's real and effective user and group ID numbers, and, +if available, the user's supplementary group set. However, because +these are numbers, they do not provide very useful information to the +average user. There needs to be some way to find the user information associated with the user and group ID numbers. This minor node presents a suite of functions for retrieving information from the user database. *Note Group Functions::, for a similar suite that retrieves @@ -15732,9 +15911,9 @@ kept. Instead, it provides the `<pwd.h>' header file and several C language subroutines for obtaining user information. The primary function is `getpwent()', for "get password entry." The "password" comes from the original user database file, `/etc/passwd', which stores -user information, along with the encrypted passwords (hence the name). +user information along with the encrypted passwords (hence the name). - While an `awk' program could simply read `/etc/passwd' directly, + Although an `awk' program could simply read `/etc/passwd' directly, this file may not contain complete information about the system's set of users.(1) To be sure you are able to produce a readable and complete version of the user database, it is necessary to write a small C @@ -15779,13 +15958,13 @@ Encrypted password systems. User-ID - The user's numeric user ID number. (On some systems it's a C - `long', and not an `int'. Thus we cast it to `long' for all + The user's numeric user ID number. (On some systems, it's a C + `long', and not an `int'. Thus, we cast it to `long' for all cases.) Group-ID The user's numeric group ID number. (Similar comments about - `long' vs. `int' apply here.) + `long' versus `int' apply here.) Full name The user's full name, and perhaps other information associated @@ -15802,7 +15981,7 @@ Login shell A few lines representative of `pwcat''s output are as follows: $ pwcat - -| root:3Ov02d5VaUPB6:0:1:Operator:/:/bin/sh + -| root:x:0:1:Operator:/:/bin/sh -| nobody:*:65534:65534::/: -| daemon:*:1:1::/: -| sys:*:2:2::/:/bin/csh @@ -15863,14 +16042,14 @@ you might want it to be in a different directory on your system. into three associative arrays. The arrays are indexed by username (`_pw_byname'), by user ID number (`_pw_byuid'), and by order of occurrence (`_pw_bycount'). The variable `_pw_inited' is used for -efficiency, since `_pw_init()' needs to be called only once. +efficiency, as `_pw_init()' needs to be called only once. Because this function uses `getline' to read information from `pwcat', it first saves the values of `FS', `RS', and `$0'. It notes in the variable `using_fw' whether field splitting with `FIELDWIDTHS' -is in effect or not. Doing so is necessary, since these functions -could be called from anywhere within a user's program, and the user may -have his or her own way of splitting records and fields. This makes it +is in effect or not. Doing so is necessary, as these functions could +be called from anywhere within a user's program, and the user may have +his or her own way of splitting records and fields. This makes it possible to restore the correct field-splitting mechanism later. The test can only be true for `gawk'. It is false if using `FS' or `FPAT', or on some other `awk' implementation. @@ -15879,8 +16058,8 @@ or on some other `awk' implementation. `PROCINFO["FS"]', is similar. The main part of the function uses a loop to read database lines, -split the line into fields, and then store the line into each array as -necessary. When the loop is done, `_pw_init()' cleans up by closing +split the lines into fields, and then store the lines into each array +as necessary. When the loop is done, `_pw_init()' cleans up by closing the pipeline, setting `_pw_inited' to one, and restoring `FS' (and `FIELDWIDTHS' or `FPAT' if necessary), `RS', and `$0'. The use of `_pw_count' is explained shortly. @@ -15941,8 +16120,8 @@ simplifies the code but runs an extra process that may never be needed.) once. If you are worried about squeezing every last cycle out of your `awk' program, the check of `_pw_inited' could be moved out of `_pw_init()' and duplicated in all the other functions. In practice, -this is not necessary, since most `awk' programs are I/O-bound, and -such a change would clutter up the code. +this is not necessary, as most `awk' programs are I/O-bound, and such a +change would clutter up the code. The `id' program in *note Id Program::, uses these functions. @@ -16008,11 +16187,11 @@ Group Password Group ID Number The group's numeric group ID number; the association of name to number must be unique within the file. (On some systems it's a C - `long', and not an `int'. Thus we cast it to `long' for all + `long', and not an `int'. Thus, we cast it to `long' for all cases.) Group Member List - A comma-separated list of user names. These users are members of + A comma-separated list of usernames. These users are members of the group. Modern Unix systems allow users to be members of several groups simultaneously. If your system does, then there are elements `"group1"' through `"groupN"' in `PROCINFO' for those @@ -16098,29 +16277,30 @@ to ensure that the database is scanned no more than once. The `_gr_init()' function first saves `FS', `RS', and `$0', and then sets `FS' and `RS' to the correct values for scanning the group information. It also takes care to note whether `FIELDWIDTHS' or `FPAT' is being -used, and to restore the appropriate field splitting mechanism. +used, and to restore the appropriate field-splitting mechanism. - The group information is stored is several associative arrays. The + The group information is stored in several associative arrays. The arrays are indexed by group name (`_gr_byname'), by group ID number (`_gr_bygid'), and by position in the database (`_gr_bycount'). There -is an additional array indexed by user name (`_gr_groupsbyuser'), which +is an additional array indexed by username (`_gr_groupsbyuser'), which is a space-separated list of groups to which each user belongs. - Unlike the user database, it is possible to have multiple records in -the database for the same group. This is common when a group has a + Unlike in the user database, it is possible to have multiple records +in the database for the same group. This is common when a group has a large number of members. A pair of such entries might look like the following: - tvpeople:*:101:johny,jay,arsenio + tvpeople:*:101:johnny,jay,arsenio tvpeople:*:101:david,conan,tom,joan For this reason, `_gr_init()' looks to see if a group name or group -ID number is already seen. If it is, then the user names are simply +ID number is already seen. If so, the usernames are simply concatenated onto the previous list of users.(1) Finally, `_gr_init()' closes the pipeline to `grcat', restores `FS' -(and `FIELDWIDTHS' or `FPAT' if necessary), `RS', and `$0', initializes -`_gr_count' to zero (it is used later), and makes `_gr_inited' nonzero. +(and `FIELDWIDTHS' or `FPAT', if necessary), `RS', and `$0', +initializes `_gr_count' to zero (it is used later), and makes +`_gr_inited' nonzero. The `getgrnam()' function takes a group name as its argument, and if that group exists, it is returned. Otherwise, it relies on the array @@ -16143,7 +16323,7 @@ looks up the information associated with that group ID: } The `getgruser()' function does not have a C counterpart. It takes a -user name and returns the list of groups that have the user as a member: +username and returns the list of groups that have the user as a member: function getgruser(user) { @@ -16183,9 +16363,9 @@ very simple, relying on `awk''s associative arrays to do work. ---------- Footnotes ---------- - (1) There is actually a subtle problem with the code just presented. -Suppose that the first time there were no names. This code adds the -names with a leading comma. It also doesn't check that there is a `$4'. + (1) There is a subtle problem with the code just presented. Suppose +that the first time there were no names. This code adds the names with +a leading comma. It also doesn't check that there is a `$4'. File: gawk.info, Node: Walking Arrays, Next: Library Functions Summary, Prev: Group Functions, Up: Library Functions @@ -16194,11 +16374,11 @@ File: gawk.info, Node: Walking Arrays, Next: Library Functions Summary, Prev: ================================ *note Arrays of Arrays::, described how `gawk' provides arrays of -arrays. In particular, any element of an array may be either a scalar, +arrays. In particular, any element of an array may be either a scalar or another array. The `isarray()' function (*note Type Functions::) lets you distinguish an array from a scalar. The following function, -`walk_array()', recursively traverses an array, printing each element's -indices and value. You call it with the array and a string +`walk_array()', recursively traverses an array, printing the element +indices and values. You call it with the array and a string representing the name of the array: function walk_array(arr, name, i) @@ -16238,6 +16418,61 @@ value. Here is a main program to demonstrate: -| a[4][1][1] = 411 -| a[4][2] = 42 + The function just presented simply prints the name and value of each +scalar array element. However, it is easy to generalize it, by passing +in the name of a function to call when walking an array. The modified +function looks like this: + + function process_array(arr, name, process, do_arrays, i, new_name) + { + for (i in arr) { + new_name = (name "[" i "]") + if (isarray(arr[i])) { + if (do_arrays) + @process(new_name, arr[i]) + process_array(arr[i], new_name, process, do_arrays) + } else + @process(new_name, arr[i]) + } + } + + The arguments are as follows: + +`arr' + The array. + +`name' + The name of the array (a string). + +`process' + The name of the function to call. + +`do_arrays' + If this is true, the function can handle elements that are + subarrays. + + If subarrays are to be processed, that is done before walking them +further. + + When run with the following scaffolding, the function produces the +same results as does the earlier version of `walk_array()': + + BEGIN { + a[1] = 1 + a[2][1] = 21 + a[2][2] = 22 + a[3] = 3 + a[4][1][1] = 411 + a[4][2] = 42 + + process_array(a, "a", "do_print", 0) + } + + function do_print(name, element) + { + printf "%s = %s\n", name, element + } + File: gawk.info, Node: Library Functions Summary, Next: Library Exercises, Prev: Walking Arrays, Up: Library Functions @@ -16255,24 +16490,24 @@ File: gawk.info, Node: Library Functions Summary, Next: Library Exercises, Pr * The functions presented here fit into the following categories: General problems - Number to string conversion, assertions, rounding, random - number generation, converting characters to numbers, joining - strings, getting easily usable time-of-day information, and - reading a whole file in one shot. + Number-to-string conversion, testing assertions, rounding, + random number generation, converting characters to numbers, + joining strings, getting easily usable time-of-day + information, and reading a whole file in one shot Managing data files Noting data file boundaries, rereading the current file, checking for readable files, checking for zero-length files, - and treating assignments as file names. + and treating assignments as file names Processing command-line options - An `awk' version of the standard C `getopt()' function. + An `awk' version of the standard C `getopt()' function Reading the user and group databases - Two sets of routines that parallel the C library versions. + Two sets of routines that parallel the C library versions Traversing arrays of arrays - A simple function to traverse an array of arrays to any depth. + Two functions that traverse an array of arrays to any depth @@ -16367,7 +16602,7 @@ you. to replace the installed versions on your system. Nor may all of these programs be fully compliant with the most recent POSIX standard. This is not a problem; their purpose is to illustrate `awk' language -programming for "real world" tasks. +programming for "real-world" tasks. The programs are presented in alphabetical order. @@ -16384,7 +16619,7 @@ programming for "real world" tasks. File: gawk.info, Node: Cut Program, Next: Egrep Program, Up: Clones -11.2.1 Cutting out Fields and Columns +11.2.1 Cutting Out Fields and Columns ------------------------------------- The `cut' utility selects, or "cuts," characters or fields from its @@ -16393,7 +16628,7 @@ separated by TABs by default, but you may supply a command-line option to change the field "delimiter" (i.e., the field-separator character). `cut''s definition of fields is less general than `awk''s. - A common use of `cut' might be to pull out just the login name of + A common use of `cut' might be to pull out just the login names of logged-on users from the output of `who'. For example, the following pipeline generates a sorted, unique list of the logged-on users: @@ -16601,10 +16836,10 @@ filler fields: nfields = j - 1 } - Next is the rule that actually processes the data. If the `-s' -option is given, then `suppress' is true. The first `if' statement -makes sure that the input record does have the field separator. If -`cut' is processing fields, `suppress' is true, and the field separator + Next is the rule that processes the data. If the `-s' option is +given, then `suppress' is true. The first `if' statement makes sure +that the input record does have the field separator. If `cut' is +processing fields, `suppress' is true, and the field separator character is not in the record, then the record is skipped. If the record is valid, then `gawk' has split the data into fields, @@ -16629,8 +16864,8 @@ out between the fields: } This version of `cut' relies on `gawk''s `FIELDWIDTHS' variable to -do the character-based cutting. While it is possible in other `awk' -implementations to use `substr()' (*note String Functions::), it is +do the character-based cutting. It is possible in other `awk' +implementations to use `substr()' (*note String Functions::), but it is also extremely painful. The `FIELDWIDTHS' variable supplies an elegant solution to the problem of picking the input line apart by characters. @@ -16741,14 +16976,14 @@ the matched lines in the output: # pattern = tolower(pattern) } - The last two lines are commented out, since they are not needed in + The last two lines are commented out, as they are not needed in `gawk'. They should be uncommented if you have to use another version of `awk'. The next set of lines should be uncommented if you are not using `gawk'. This rule translates all the characters in the input line into lowercase if the `-i' option is specified.(1) The rule is commented out -since it is not necessary with `gawk': +as it is not necessary with `gawk': #{ # if (IGNORECASE) @@ -16802,7 +17037,7 @@ unsuccessful match. If the line does not match, the `next' statement just moves on to the next record. A number of additional tests are made, but they are only done if we -are not counting lines. First, if the user only wants exit status +are not counting lines. First, if the user only wants the exit status (`no_print' is true), then it is enough to know that _one_ line in this file matched, and we can skip on to the next file with `nextfile'. Similarly, if we are only printing file names, we can print the file @@ -16836,7 +17071,7 @@ line is printed, with a leading file name and colon if necessary: } The `END' rule takes care of producing the correct exit status. If -there are no matches, the exit status is one; otherwise it is zero: +there are no matches, the exit status is one; otherwise, it is zero: END { exit (total == 0) @@ -16860,7 +17095,7 @@ the translated line, not the original. File: gawk.info, Node: Id Program, Next: Split Program, Prev: Egrep Program, Up: Clones -11.2.3 Printing out User Information +11.2.3 Printing Out User Information ------------------------------------ The `id' utility lists a user's real and effective user ID numbers, @@ -16878,7 +17113,8 @@ a more palatable output than just individual numbers. Here is a simple version of `id' written in `awk'. It uses the user database library functions (*note Passwd Functions::) and the group -database library functions (*note Group Functions::): +database library functions (*note Group Functions::) from *note Library +Functions::. The program is fairly straightforward. All the work is done in the `BEGIN' rule. The user and group ID numbers are obtained from @@ -16944,7 +17180,7 @@ and the group numbers: The test in the `for' loop is worth noting. Any supplementary groups in the `PROCINFO' array have the indices `"group1"' through -`"groupN"' for some N, i.e., the total number of supplementary groups. +`"groupN"' for some N (i.e., the total number of supplementary groups). However, we don't know in advance how many of these groups there are. This loop works by starting at one, concatenating the value with @@ -16973,11 +17209,11 @@ is as follows:(1) `split' [`-COUNT'] [FILE] [PREFIX] By default, the output files are named `xaa', `xab', and so on. Each -file has 1000 lines in it, with the likely exception of the last file. +file has 1,000 lines in it, with the likely exception of the last file. To change the number of lines in each file, supply a number on the -command line preceded with a minus; e.g., `-500' for files with 500 -lines in them instead of 1000. To change the name of the output files -to something like `myfileaa', `myfileab', and so on, supply an +command line preceded with a minus sign (e.g., `-500' for files with +500 lines in them instead of 1,000). To change the names of the output +files to something like `myfileaa', `myfileab', and so on, supply an additional argument that specifies the file name prefix. Here is a version of `split' in `awk'. It uses the `ord()' and @@ -17010,7 +17246,7 @@ output file names: } # test argv in case reading from stdin instead of file if (i in ARGV) - i++ # skip data file name + i++ # skip datafile name if (i in ARGV) { outfile = ARGV[i] ARGV[i] = "" @@ -17082,8 +17318,8 @@ files named on the command line. Its usage is as follows: truncating them and starting over. The `BEGIN' rule first makes a copy of all the command-line arguments -into an array named `copy'. `ARGV[0]' is not copied, since it is not -needed. `tee' cannot use `ARGV' directly, since `awk' attempts to +into an array named `copy'. `ARGV[0]' is not needed, so it is not +copied. `tee' cannot use `ARGV' directly, because `awk' attempts to process each file name in `ARGV' as input data. If the first argument is `-a', then the flag variable `append' is @@ -17115,7 +17351,7 @@ input by setting `ARGV[1]' to `"-"' and `ARGC' to two: ARGC = 2 } - The following single rule does all the work. Since there is no + The following single rule does all the work. Because there is no pattern, it is executed for each line of input. The body of the rule simply prints the line into each file on the command line, and then to the standard output: @@ -17139,11 +17375,12 @@ It is also possible to write the loop this way: else print > copy[i] -This is more concise but it is also less efficient. The `if' is tested -for each record and for each output file. By duplicating the loop -body, the `if' is only tested once for each input record. If there are -N input records and M output files, the first method only executes N -`if' statements, while the second executes N`*'M `if' statements. +This is more concise, but it is also less efficient. The `if' is +tested for each record and for each output file. By duplicating the +loop body, the `if' is only tested once for each input record. If +there are N input records and M output files, the first method only +executes N `if' statements, while the second executes N`*'M `if' +statements. Finally, the `END' rule cleans up by closing all the output files: @@ -17321,13 +17558,13 @@ to. depending upon the results of `are_equal()''s comparison. If `uniq' is counting repeated lines, and the lines are equal, then it increments the `count' variable. Otherwise, it prints the line and resets `count', -since the two lines are not equal. +because the two lines are not equal. If `uniq' is not counting, and if the lines are equal, `count' is -incremented. Nothing is printed, since the point is to remove -duplicates. Otherwise, if `uniq' is counting repeated lines and more -than one line is seen, or if `uniq' is counting nonrepeated lines and -only one line is seen, then the line is printed, and `count' is reset. +incremented. Nothing is printed, as the point is to remove duplicates. +Otherwise, if `uniq' is counting repeated lines and more than one line +is seen, or if `uniq' is counting nonrepeated lines and only one line +is seen, then the line is printed, and `count' is reset. Finally, similar logic is used in the `END' rule to print the final line of input data: @@ -17399,10 +17636,10 @@ follows: `-c' Count only characters. - Implementing `wc' in `awk' is particularly elegant, since `awk' does -a lot of the work for us; it splits lines into words (i.e., fields) and -counts them, it counts lines (i.e., records), and it can easily tell us -how long a line is. + Implementing `wc' in `awk' is particularly elegant, because `awk' +does a lot of the work for us; it splits lines into words (i.e., +fields) and counts them, it counts lines (i.e., records), and it can +easily tell us how long a line is. This program uses the `getopt()' library function (*note Getopt Function::) and the file-transition functions (*note Filetrans @@ -17509,7 +17746,7 @@ in its length. Next, `lines' is incremented for each line read, and ---------- Footnotes ---------- - (1) Since `gawk' understands multibyte locales, this code counts + (1) Because `gawk' understands multibyte locales, this code counts characters, not bytes. @@ -17612,7 +17849,7 @@ checking and setting of defaults: the delay, the count, and the message to print. If the user supplied a message without the ASCII BEL character (known as the "alert" character, `"\a"'), then it is added to the message. (On many systems, printing the ASCII BEL generates an -audible alert. Thus when the alarm goes off, the system calls attention +audible alert. Thus, when the alarm goes off, the system calls attention to itself in case the user is not looking at the computer.) Just for a change, this program uses a `switch' statement (*note Switch Statement::), but the processing could be done with a series of @@ -17744,7 +17981,7 @@ the "from" list. Once upon a time, a user proposed adding a transliteration function to `gawk'. The following program was written to prove that character transliteration could be done with a user-level function. This program -is not as complete as the system `tr' utility but it does most of the +is not as complete as the system `tr' utility, but it does most of the job. The `translate' program was written long before `gawk' acquired the @@ -17754,13 +17991,13 @@ and `gsub()' built-in functions (*note String Functions::). There are two functions. The first, `stranslate()', takes three arguments: `from' - A list of characters from which to translate. + A list of characters from which to translate `to' - A list of characters to which to translate. + A list of characters to which to translate `target' - The string on which to do the translation. + The string on which to do the translation Associative arrays make the translation part fairly easy. `t_ar' holds the "to" characters, indexed by the "from" characters. Then a @@ -17768,7 +18005,7 @@ simple loop goes through `from', one character at a time. For each character in `from', if the character appears in `target', it is replaced with the corresponding `to' character. - The `translate()' function calls `stranslate()' using `$0' as the + The `translate()' function calls `stranslate()', using `$0' as the target. The main program sets two global variables, `FROM' and `TO', from the command line, and then changes `ARGV' so that `awk' reads from the standard input. @@ -17777,7 +18014,7 @@ the standard input. record: # translate.awk --- do tr-like stuff - # Bugs: does not handle things like: tr A-Z a-z, it has + # Bugs: does not handle things like tr A-Z a-z; it has # to be spelled out. However, if `to' is shorter than `from', # the last character in `to' is used for the rest of `from'. @@ -17823,14 +18060,15 @@ record: print } - While it is possible to do character transliteration in a user-level -function, it is not necessarily efficient, and we (the `gawk' authors) -started to consider adding a built-in function. However, shortly after -writing this program, we learned that Brian Kernighan had added the -`toupper()' and `tolower()' functions to his `awk' (*note String -Functions::). These functions handle the vast majority of the cases -where character transliteration is necessary, and so we chose to simply -add those functions to `gawk' as well and then leave well enough alone. + It is possible to do character transliteration in a user-level +function, but it is not necessarily efficient, and we (the `gawk' +developers) started to consider adding a built-in function. However, +shortly after writing this program, we learned that Brian Kernighan had +added the `toupper()' and `tolower()' functions to his `awk' (*note +String Functions::). These functions handle the vast majority of the +cases where character transliteration is necessary, and so we chose to +simply add those functions to `gawk' as well and then leave well enough +alone. An obvious improvement to this program would be to set up the `t_ar' array only once, in a `BEGIN' rule. However, this assumes that the @@ -17854,31 +18092,31 @@ File: gawk.info, Node: Labels Program, Next: Word Sorting, Prev: Translate Pr 11.3.4 Printing Mailing Labels ------------------------------ -Here is a "real world"(1) program. This script reads lists of names and +Here is a "real-world"(1) program. This script reads lists of names and addresses and generates mailing labels. Each page of labels has 20 labels on it, two across and 10 down. The addresses are guaranteed to be no more than five lines of data. Each address is separated from the next by a blank line. - The basic idea is to read 20 labels worth of data. Each line of + The basic idea is to read 20 labels' worth of data. Each line of each label is stored in the `line' array. The single rule takes care of filling the `line' array and printing the page when 20 labels have been read. The `BEGIN' rule simply sets `RS' to the empty string, so that `awk' splits records at blank lines (*note Records::). It sets `MAXLINES' to -100, since 100 is the maximum number of lines on the page (20 * 5 = +100, because 100 is the maximum number of lines on the page (20 * 5 = 100). Most of the work is done in the `printpage()' function. The label lines are stored sequentially in the `line' array. But they have to -print horizontally; `line[1]' next to `line[6]', `line[2]' next to +print horizontally: `line[1]' next to `line[6]', `line[2]' next to `line[7]', and so on. Two loops accomplish this. The outer loop, controlled by `i', steps through every 10 lines of data; this is each row of labels. The inner loop, controlled by `j', goes through the -lines within the row. As `j' goes from 0 to 4, `i+j' is the `j'-th -line in the row, and `i+j+5' is the entry next to it. The output ends -up looking something like this: +lines within the row. As `j' goes from 0 to 4, `i+j' is the `j'th line +in the row, and `i+j+5' is the entry next to it. The output ends up +looking something like this: line 1 line 6 line 2 line 7 @@ -17981,8 +18219,8 @@ a useful format. printf "%s\t%d\n", word, freq[word] } - The program relies on `awk''s default field splitting mechanism to -break each line up into "words," and uses an associative array named + The program relies on `awk''s default field-splitting mechanism to +break each line up into "words" and uses an associative array named `freq', indexed by each word, to count the number of times the word occurs. In the `END' rule, it prints the counts. @@ -17991,9 +18229,9 @@ on real text files: * The `awk' language considers upper- and lowercase characters to be distinct. Therefore, "bartender" and "Bartender" are not treated - as the same word. This is undesirable, since in normal text, words - are capitalized if they begin sentences, and a frequency analyzer - should not be sensitive to capitalization. + as the same word. This is undesirable, because words are + capitalized if they begin sentences in normal text, and a + frequency analyzer should not be sensitive to capitalization. * Words are detected using the `awk' convention that fields are separated just by whitespace. Other characters in the input @@ -18068,7 +18306,7 @@ File: gawk.info, Node: History Sorting, Next: Extract Program, Prev: Word Sor 11.3.6 Removing Duplicates from Unsorted Text --------------------------------------------- -The `uniq' program (*note Uniq Program::), removes duplicate lines from +The `uniq' program (*note Uniq Program::) removes duplicate lines from _sorted_ data. Suppose, however, you need to remove duplicate lines from a data @@ -18116,12 +18354,12 @@ File: gawk.info, Node: Extract Program, Next: Simple Sed, Prev: History Sorti The nodes *note Library Functions::, and *note Sample Programs::, are the top level nodes for a large number of `awk' programs. If you want -to experiment with these programs, it is tedious to have to type them -in by hand. Here we present a program that can extract parts of a -Texinfo input file into separate files. +to experiment with these programs, it is tedious to type them in by +hand. Here we present a program that can extract parts of a Texinfo +input file into separate files. This Info file is written in Texinfo -(http://www.gnu.org/software/texinfo/), the GNU project's document +(http://www.gnu.org/software/texinfo/), the GNU Project's document formatting language. A single Texinfo source file can be used to produce both printed documentation, with TeX, and online documentation. (The Texinfo language is described fully, starting with *note @@ -18162,7 +18400,7 @@ them in a standard directory where `gawk' can find them. The Texinfo file looks something like this: ... - This program has a @code{BEGIN} rule, + This program has a @code{BEGIN} rule that prints a nice message: @example @@ -18175,7 +18413,7 @@ file looks something like this: @example @c file examples/messages.awk - END @{ print "Always avoid bored archeologists!" @} + END @{ print "Always avoid bored archaeologists!" @} @c end file @end example ... @@ -18187,7 +18425,7 @@ upper- and lowercase letters in the directives won't matter. given (`NF' is at least three) and also checking that the command exits with a zero exit status, signifying OK: - # extract.awk --- extract files and run programs from texinfo files + # extract.awk --- extract files and run programs from Texinfo files BEGIN { IGNORECASE = 1 } @@ -18214,11 +18452,11 @@ The variable `e' is used so that the rule fits nicely on the screen. file name is given in the directive. If the file named is not the current file, then the current file is closed. Keeping the current file open until a new file is encountered allows the use of the `>' -redirection for printing the contents, keeping open file management +redirection for printing the contents, keeping open-file management simple. The `for' loop does the work. It reads lines using `getline' (*note -Getline::). For an unexpected end of file, it calls the +Getline::). For an unexpected end-of-file, it calls the `unexpected_eof()' function. If the line is an "endfile" line, then it breaks out of the loop. If the line is an `@group' or `@end group' line, then it ignores it and goes on to the next line. Similarly, @@ -18308,18 +18546,18 @@ File: gawk.info, Node: Simple Sed, Next: Igawk Program, Prev: Extract Program 11.3.8 A Simple Stream Editor ----------------------------- -The `sed' utility is a stream editor, a program that reads a stream of -data, makes changes to it, and passes it on. It is often used to make -global changes to a large file or to a stream of data generated by a -pipeline of commands. While `sed' is a complicated program in its own -right, its most common use is to perform global substitutions in the -middle of a pipeline: +The `sed' utility is a "stream editor", a program that reads a stream +of data, makes changes to it, and passes it on. It is often used to +make global changes to a large file or to a stream of data generated by +a pipeline of commands. Although `sed' is a complicated program in its +own right, its most common use is to perform global substitutions in +the middle of a pipeline: COMMAND1 < orig.data | sed 's/old/new/g' | COMMAND2 > result Here, `s/old/new/g' tells `sed' to look for the regexp `old' on each -input line and globally replace it with the text `new', i.e., all the -occurrences on a line. This is similar to `awk''s `gsub()' function +input line and globally replace it with the text `new' (i.e., all the +occurrences on a line). This is similar to `awk''s `gsub()' function (*note String Functions::). The following program, `awksed.awk', accepts at least two @@ -18380,7 +18618,7 @@ arguments and calling `usage()' if there is a problem. Then it sets (*note ARGC and ARGV::). The `usage()' function prints an error message and exits. Finally, -the single rule handles the printing scheme outlined above, using +the single rule handles the printing scheme outlined earlier, using `print' or `printf' as appropriate, depending upon the value of `RT'. @@ -18418,14 +18656,14 @@ to be able to write programs in the following manner: The following program, `igawk.sh', provides this service. It simulates `gawk''s searching of the `AWKPATH' variable and also allows -"nested" includes; i.e., a file that is included with `@include' can -contain further `@include' statements. `igawk' makes an effort to only -include files once, so that nested includes don't accidentally include -a library function twice. +"nested" includes (i.e., a file that is included with `@include' can +contain further `@include' statements). `igawk' makes an effort to +only include files once, so that nested includes don't accidentally +include a library function twice. `igawk' should behave just like `gawk' externally. This means it should accept all of `gawk''s command-line arguments, including the -ability to have multiple source files specified via `-f', and the +ability to have multiple source files specified via `-f' and the ability to mix command-line and library source files. The program is written using the POSIX Shell (`sh') command @@ -18442,8 +18680,8 @@ language.(1) It works as follows: b. Source file names, provided with `-f'. We use a neat trick and append `@include FILENAME' to the shell variable's - contents. Since the file-inclusion program works the way - `gawk' does, this gets the text of the file included into the + contents. Because the file-inclusion program works the way + `gawk' does, this gets the text of the file included in the program at the correct point. 3. Run an `awk' program (naturally) over the shell variable's @@ -18455,8 +18693,8 @@ language.(1) It works as follows: file names). This program uses shell variables extensively: for storing -command-line arguments, the text of the `awk' program that will expand -the user's program, for the user's original program, and for the +command-line arguments and the text of the `awk' program that will +expand the user's program, for the user's original program, and for the expanded program. Doing so removes some potential problems that might arise were we to use temporary files instead, at the cost of making the script somewhat more complicated. @@ -18703,18 +18941,18 @@ is saved as a single string, even if the results contain whitespace. It's done in these steps: 1. Run `gawk' with the `@include'-processing program (the value of - the `expand_prog' shell variable) on standard input. + the `expand_prog' shell variable) reading standard input. 2. Standard input is the contents of the user's program, from the - shell variable `program'. Its contents are fed to `gawk' via a - here document. + shell variable `program'. Feed its contents to `gawk' via a here + document. - 3. The results of this processing are saved in the shell variable + 3. Save the results of this processing in the shell variable `processed_program' by using command substitution. The last step is to call `gawk' with the expanded program, along with the original options and command-line arguments that the user -supplied. +supplied: eval gawk $opts -- '"$processed_program"' '"$@"' @@ -18766,26 +19004,26 @@ use of `awk' programs as Web CGI scripts. File: gawk.info, Node: Anagram Program, Next: Signature Program, Prev: Igawk Program, Up: Miscellaneous Programs -11.3.10 Finding Anagrams From A Dictionary +11.3.10 Finding Anagrams from a Dictionary ------------------------------------------ An interesting programming challenge is to search for "anagrams" in a word list (such as `/usr/share/dict/words' on many GNU/Linux systems). One word is an anagram of another if both words contain the same letters -(for example, "babbling" and "blabbing"). +(e.g., "babbling" and "blabbing"). - Column 2, Problem C of Jon Bentley's `Programming Pearls', second -edition, presents an elegant algorithm. The idea is to give words that + Column 2, Problem C, of Jon Bentley's `Programming Pearls', Second +Edition, presents an elegant algorithm. The idea is to give words that are anagrams a common signature, sort all the words together by their -signature, and then print them. Dr. Bentley observes that taking the -letters in each word and sorting them produces that common signature. +signatures, and then print them. Dr. Bentley observes that taking the +letters in each word and sorting them produces those common signatures. The following program uses arrays of arrays to bring together words with the same signature and array sorting to print the words in sorted -order. +order: - # anagram.awk --- An implementation of the anagram finding algorithm - # from Jon Bentley's "Programming Pearls", 2nd edition. + # anagram.awk --- An implementation of the anagram-finding algorithm + # from Jon Bentley's "Programming Pearls," 2nd edition. # Addison Wesley, 2000, ISBN 0-201-65788-0. # Column 2, Problem C, section 2.8, pp 18-20. @@ -18805,7 +19043,7 @@ signature; the second dimension is the word itself: apart into individual letters, sorts the letters, and then joins them back together: - # word2key --- split word apart into letters, sort, joining back together + # word2key --- split word apart into letters, sort, and join back together function word2key(word, a, i, n, result) { @@ -18819,8 +19057,8 @@ back together: } Finally, the `END' rule traverses the array and prints out the -anagram lists. It sends the output to the system `sort' command, since -otherwise the anagrams would appear in arbitrary order: +anagram lists. It sends the output to the system `sort' command +because otherwise the anagrams would appear in arbitrary order: END { sort = "sort" @@ -18855,7 +19093,7 @@ otherwise the anagrams would appear in arbitrary order: File: gawk.info, Node: Signature Program, Prev: Anagram Program, Up: Miscellaneous Programs -11.3.11 And Now For Something Completely Different +11.3.11 And Now for Something Completely Different -------------------------------------------------- The following program was written by Davide Brini and is published on @@ -18903,12 +19141,13 @@ File: gawk.info, Node: Programs Summary, Next: Programs Exercises, Prev: Misc characters. The ability to use `split()' with the empty string as the separator can considerably simplify such tasks. - * The library functions from *note Library Functions::, proved their - usefulness for a number of real (if small) programs. + * The examples here demonstrate the usefulness of the library + functions from *note Library Functions::, for a number of real (if + small) programs. * Besides reinventing POSIX wheels, other programs solved a - selection of interesting problems, such as finding duplicates - words in text, printing mailing labels, and finding anagrams. + selection of interesting problems, such as finding duplicate words + in text, printing mailing labels, and finding anagrams. @@ -19025,16 +19264,16 @@ File: gawk.info, Node: Advanced Features, Next: Internationalization, Prev: S This major node discusses advanced features in `gawk'. It's a bit of a "grab bag" of items that are otherwise unrelated to each other. -First, a command-line option allows `gawk' to recognize nondecimal -numbers in input data, not just in `awk' programs. Then, `gawk''s -special features for sorting arrays are presented. Next, two-way I/O, -discussed briefly in earlier parts of this Info file, is described in -full detail, along with the basics of TCP/IP networking. Finally, -`gawk' can "profile" an `awk' program, making it possible to tune it -for performance. +First, we look at a command-line option that allows `gawk' to recognize +nondecimal numbers in input data, not just in `awk' programs. Then, +`gawk''s special features for sorting arrays are presented. Next, +two-way I/O, discussed briefly in earlier parts of this Info file, is +described in full detail, along with the basics of TCP/IP networking. +Finally, we see how `gawk' can "profile" an `awk' program, making it +possible to tune it for performance. - A number of advanced features require separate major nodes of their -own: + Additional advanced features are discussed in separate major nodes +of their own: * *note Internationalization::, discusses how to internationalize your `awk' programs, so that they can speak multiple national @@ -19108,7 +19347,7 @@ File: gawk.info, Node: Array Sorting, Next: Two-way I/O, Prev: Nondecimal Dat 12.2 Controlling Array Traversal and Array Sorting ================================================== -`gawk' lets you control the order in which a `for (i in array)' loop +`gawk' lets you control the order in which a `for (INDX in ARRAY)' loop traverses an array. In addition, two built-in functions, `asort()' and `asorti()', let @@ -19127,16 +19366,16 @@ File: gawk.info, Node: Controlling Array Traversal, Next: Array Sorting Functi 12.2.1 Controlling Array Traversal ---------------------------------- -By default, the order in which a `for (i in array)' loop scans an array -is not defined; it is generally based upon the internal implementation -of arrays inside `awk'. +By default, the order in which a `for (INDX in ARRAY)' loop scans an +array is not defined; it is generally based upon the internal +implementation of arrays inside `awk'. Often, though, it is desirable to be able to loop over the elements in a particular order that you, the programmer, choose. `gawk' lets you do this. *note Controlling Scanning::, describes how you can assign special, -pre-defined values to `PROCINFO["sorted_in"]' in order to control the +predefined values to `PROCINFO["sorted_in"]' in order to control the order in which `gawk' traverses an array during a `for' loop. In addition, the value of `PROCINFO["sorted_in"]' can be a function @@ -19151,21 +19390,22 @@ arguments: RETURN < 0; 0; OR > 0 } - Here, I1 and I2 are the indices, and V1 and V2 are the corresponding -values of the two elements being compared. Either V1 or V2, or both, -can be arrays if the array being traversed contains subarrays as values. -(*Note Arrays of Arrays::, for more information about subarrays.) The -three possible return values are interpreted as follows: + Here, `i1' and `i2' are the indices, and `v1' and `v2' are the +corresponding values of the two elements being compared. Either `v1' +or `v2', or both, can be arrays if the array being traversed contains +subarrays as values. (*Note Arrays of Arrays::, for more information +about subarrays.) The three possible return values are interpreted as +follows: `comp_func(i1, v1, i2, v2) < 0' - Index I1 comes before index I2 during loop traversal. + Index `i1' comes before index `i2' during loop traversal. `comp_func(i1, v1, i2, v2) == 0' - Indices I1 and I2 come together but the relative order with + Indices `i1' and `i2' come together, but the relative order with respect to each other is undefined. `comp_func(i1, v1, i2, v2) > 0' - Index I1 comes after index I2 during loop traversal. + Index `i1' comes after index `i2' during loop traversal. Our first comparison function can be used to scan an array in numerical order of the indices: @@ -19293,7 +19533,7 @@ Running the program produces the following output: The comparison should normally always return the same value when given a specific pair of array elements as its arguments. If -inconsistent results are returned then the order is undefined. This +inconsistent results are returned, then the order is undefined. This behavior can be exploited to introduce random order into otherwise seemingly ordered data: @@ -19303,12 +19543,12 @@ seemingly ordered data: return (2 - 4 * rand()) } - As mentioned above, the order of the indices is arbitrary if two + As already mentioned, the order of the indices is arbitrary if two elements compare equal. This is usually not a problem, but letting the tied elements come out in arbitrary order can be an issue, especially when comparing item values. The partial ordering of the equal elements may change the next time the array is traversed, if other elements are -added or removed from the array. One way to resolve ties when +added to or removed from the array. One way to resolve ties when comparing elements with otherwise equal values is to include the indices in the comparison rules. Note that doing this may make the loop traversal less efficient, so consider it only if necessary. The @@ -19337,19 +19577,19 @@ such a function. When string comparisons are made during a sort, either for element values where one or both aren't numbers, or for element indices handled as strings, the value of `IGNORECASE' (*note Built-in Variables::) -controls whether the comparisons treat corresponding uppercase and +controls whether the comparisons treat corresponding upper- and lowercase letters as equivalent or distinct. - Another point to keep in mind is that in the case of subarrays the + Another point to keep in mind is that in the case of subarrays, the element values can themselves be arrays; a production comparison -function should use the `isarray()' function (*note Type Functions::), +function should use the `isarray()' function (*note Type Functions::) to check for this, and choose a defined sorting order for subarrays. All sorting based on `PROCINFO["sorted_in"]' is disabled in POSIX -mode, since the `PROCINFO' array is not special in that case. +mode, because the `PROCINFO' array is not special in that case. As a side note, sorting the array indices before traversing the -array has been reported to add 15% to 20% overhead to the execution +array has been reported to add a 15% to 20% overhead to the execution time of `awk' programs. For this reason, sorted array traversal is not the default. @@ -19365,10 +19605,10 @@ File: gawk.info, Node: Array Sorting Functions, Prev: Controlling Array Traver --------------------------------------------------- In most `awk' implementations, sorting an array requires writing a -`sort()' function. While this can be educational for exploring -different sorting algorithms, usually that's not the point of the -program. `gawk' provides the built-in `asort()' and `asorti()' -functions (*note String Functions::) for sorting arrays. For example: +`sort()' function. This can be educational for exploring different +sorting algorithms, but usually that's not the point of the program. +`gawk' provides the built-in `asort()' and `asorti()' functions (*note +String Functions::) for sorting arrays. For example: POPULATE THE ARRAY data n = asort(data) @@ -19398,8 +19638,8 @@ array is not affected. Often, what's needed is to sort on the values of the _indices_ instead of the values of the elements. To do that, use the `asorti()' function. The interface and behavior are identical to that of -`asort()', except that the index values are used for sorting, and -become the values of the result array: +`asort()', except that the index values are used for sorting and become +the values of the result array: { source[$0] = some_func($0) } @@ -19431,8 +19671,8 @@ chooses_, taking into account just the indices, just the values, or both. This is extremely powerful. Once the array is sorted, `asort()' takes the _values_ in their -final order, and uses them to fill in the result array, whereas -`asorti()' takes the _indices_ in their final order, and uses them to +final order and uses them to fill in the result array, whereas +`asorti()' takes the _indices_ in their final order and uses them to fill in the result array. NOTE: Copying array indices and elements isn't expensive in terms @@ -19449,8 +19689,8 @@ comparisons are based on character values only.(1) ---------- Footnotes ---------- (1) This is true because locale-based comparison occurs only when in -POSIX compatibility mode, and since `asort()' and `asorti()' are `gawk' -extensions, they are not available in that case. +POSIX-compatibility mode, and because `asort()' and `asorti()' are +`gawk' extensions, they are not available in that case. File: gawk.info, Node: Two-way I/O, Next: TCP/IP Networking, Prev: Array Sorting, Up: Advanced Features @@ -19479,7 +19719,7 @@ the program be run in a directory that cannot be shared among users; for example, `/tmp' will not do, as another user might happen to be using a temporary file with the same name.(1) However, with `gawk', it is possible to open a _two-way_ pipe to another process. The second -process is termed a "coprocess", since it runs in parallel with `gawk'. +process is termed a "coprocess", as it runs in parallel with `gawk'. The two-way connection is created using the `|&' operator (borrowed from the Korn shell, `ksh'):(2) @@ -19558,7 +19798,7 @@ per-command basis, by setting a special element in the `PROCINFO' array command = "sort -nr" # command, save in convenience variable PROCINFO[command, "pty"] = 1 # update PROCINFO - print ... |& command # start two-way pipe + print ... |& command # start two-way pipe ... Using ptys usually avoids the buffer deadlock issues described earlier, @@ -19595,7 +19835,7 @@ connection. You can think of this as just a _very long_ two-way pipeline to a coprocess. The way `gawk' decides that you want to use TCP/IP networking is by recognizing special file names that begin with one of -`/inet/', `/inet4/' or `/inet6/'. +`/inet/', `/inet4/', or `/inet6/'. The full syntax of the special file name is `/NET-TYPE/PROTOCOL/LOCAL-PORT/REMOTE-HOST/REMOTE-PORT'. The @@ -19621,7 +19861,7 @@ LOCAL-PORT `getaddrinfo()' function. REMOTE-HOST - The IP address or fully-qualified domain name of the Internet host + The IP address or fully qualified domain name of the Internet host to which you want to connect. REMOTE-PORT @@ -19630,7 +19870,7 @@ REMOTE-PORT name. NOTE: Failure in opening a two-way socket will result in a - non-fatal error being returned to the calling code. The value of + nonfatal error being returned to the calling code. The value of `ERRNO' indicates the error (*note Auto-set::). Consider the following very simple example: @@ -19669,7 +19909,7 @@ used to change the name of the file where `gawk' will write the profile: gawk --profile=myprog.prof -f myprog.awk data1 data2 -In the above example, `gawk' places the profile in `myprog.prof' +In the preceding example, `gawk' places the profile in `myprog.prof' instead of in `awkprof.out'. Here is a sample session showing a simple `awk' program, its input @@ -19711,8 +19951,8 @@ First, the `awk' program: junk Here is the `awkprof.out' that results from running the `gawk' -profiler on this program and data. (This example also illustrates that -`awk' programmers sometimes get up very early in the morning to work.) +profiler on this program and data (this example also illustrates that +`awk' programmers sometimes get up very early in the morning to work): # gawk profile, created Mon Sep 29 05:16:21 2014 @@ -19765,7 +20005,7 @@ profiler on this program and data. (This example also illustrates that output. They are as follows: * The program is printed in the order `BEGIN' rules, `BEGINFILE' - rules, pattern/action rules, `ENDFILE' rules, `END' rules and + rules, pattern-action rules, `ENDFILE' rules, `END' rules, and functions, listed alphabetically. Multiple `BEGIN' and `END' rules retain their separate identities, as do multiple `BEGINFILE' and `ENDFILE' rules. @@ -19800,9 +20040,9 @@ output. They are as follows: * Parentheses are used only where needed, as indicated by the structure of the program and the precedence rules. For example, - `(3 + 5) * 4' means add three plus five, then multiply the total - by four. However, `3 + 5 * 4' has no parentheses, and means `3 + - (5 * 4)'. + `(3 + 5) * 4' means add three and five, then multiply the total by + four. However, `3 + 5 * 4' has no parentheses, and means `3 + (5 + * 4)'. * Parentheses are used around the arguments to `print' and `printf' only when the `print' or `printf' statement is followed by a @@ -19810,13 +20050,13 @@ output. They are as follows: scalar, it gets parenthesized. * `gawk' supplies leading comments in front of the `BEGIN' and `END' - rules, the `BEGINFILE' and `ENDFILE' rules, the pattern/action + rules, the `BEGINFILE' and `ENDFILE' rules, the pattern-action rules, and the functions. The profiled version of your program may not look exactly like what you typed when you wrote it. This is because `gawk' creates the -profiled version by "pretty printing" its internal representation of +profiled version by "pretty-printing" its internal representation of the program. The advantage to this is that `gawk' can produce a standard representation. Also, things such as: @@ -19866,15 +20106,15 @@ output profile file. produces the profile and the function call trace and then exits. When `gawk' runs on MS-Windows systems, it uses the `INT' and `QUIT' -signals for producing the profile and, in the case of the `INT' signal, +signals for producing the profile, and in the case of the `INT' signal, `gawk' exits. This is because these systems don't support the `kill' command, so the only signals you can deliver to a program are those generated by the keyboard. The `INT' signal is generated by the -`Ctrl-<C>' or `Ctrl-<BREAK>' key, while the `QUIT' signal is generated -by the `Ctrl-<\>' key. +`Ctrl-c' or `Ctrl-BREAK' key, while the `QUIT' signal is generated by +the `Ctrl-\' key. Finally, `gawk' also accepts another option, `--pretty-print'. When -called this way, `gawk' "pretty prints" the program into `awkprof.out', +called this way, `gawk' "pretty-prints" the program into `awkprof.out', without any execution counts. NOTE: Once upon a time, the `--pretty-print' option would also run @@ -19926,7 +20166,7 @@ File: gawk.info, Node: Advanced Features Summary, Prev: Profiling, Up: Advanc two-way communications. * By using special file names with the `|&' operator, you can open a - TCP/IP (or UDP/IP) connection to remote hosts in the Internet. + TCP/IP (or UDP/IP) connection to remote hosts on the Internet. `gawk' supports both IPv4 and IPv6. * You can generate statement count profiles of your program. This @@ -19935,7 +20175,7 @@ File: gawk.info, Node: Advanced Features Summary, Prev: Profiling, Up: Advanc `USR1' signal while profiling causes `gawk' to dump the profile and keep going, including a function call stack. - * You can also just "pretty print" the program. This currently also + * You can also just "pretty-print" the program. This currently also runs the program, but that will change in the next major release. @@ -19979,7 +20219,7 @@ File: gawk.info, Node: I18N and L10N, Next: Explaining gettext, Up: Internati "Internationalization" means writing (or modifying) a program once, in such a way that it can use multiple languages without requiring further -source-code changes. "Localization" means providing the data necessary +source code changes. "Localization" means providing the data necessary for an internationalized program to work in a particular language. Most typically, these terms refer to features such as the language used for printing error messages, the language used to read responses, and @@ -19993,7 +20233,7 @@ File: gawk.info, Node: Explaining gettext, Next: Programmer i18n, Prev: I18N ================== `gawk' uses GNU `gettext' to provide its internationalization features. -The facilities in GNU `gettext' focus on messages; strings printed by a +The facilities in GNU `gettext' focus on messages: strings printed by a program, either directly or via formatting with `printf' or `sprintf()'.(1) @@ -20007,12 +20247,12 @@ components--programs written in C or C++, as well as scripts written in named `guide'. Internationalization consists of the following steps, in this order: - 1. The programmer goes through the source for all of `guide''s - components and marks each string that is a candidate for - translation. For example, `"`-F': option required"' is a good - candidate for translation. A table with strings of option names - is not (e.g., `gawk''s `--profile' option should remain the same, - no matter what the local language). + 1. The programmer reviews the source for all of `guide''s components + and marks each string that is a candidate for translation. For + example, `"`-F': option required"' is a good candidate for + translation. A table with strings of option names is not (e.g., + `gawk''s `--profile' option should remain the same, no matter what + the local language). 2. The programmer indicates the application's text domain (`"guide"') to the `gettext' library, by calling the `textdomain()' function. @@ -20081,8 +20321,8 @@ are: a different category.) `LC_COLLATE' - Text-collation information; i.e., how different characters and/or - groups of characters sort in a given language. + Text-collation information (i.e., how different characters and/or + groups of characters sort in a given language). `LC_CTYPE' Character-type information (alphabetic, digit, upper- or @@ -20122,8 +20362,7 @@ File: gawk.info, Node: Programmer i18n, Next: Translator i18n, Prev: Explaini 13.3 Internationalizing `awk' Programs ====================================== -`gawk' provides the following variables and functions for -internationalization: +`gawk' provides the following variables for internationalization: `TEXTDOMAIN' This variable indicates the application's text domain. For @@ -20135,6 +20374,8 @@ internationalization: for translation at runtime. String constants without a leading underscore are not translated. + `gawk' provides the following functions for internationalization: + ``dcgettext(STRING' [`,' DOMAIN [`,' CATEGORY]]`)'' Return the translation of STRING in text domain DOMAIN for locale category CATEGORY. The default value for DOMAIN is the current @@ -20173,8 +20414,7 @@ internationalization: the null string (`""'), then `bindtextdomain()' returns the current binding for the given DOMAIN. - To use these facilities in your `awk' program, follow the steps -outlined in *note Explaining gettext::, like so: + To use these facilities in your `awk' program, follow these steps: 1. Set the variable `TEXTDOMAIN' to the text domain of your program. This is best done in a `BEGIN' rule (*note BEGIN/END::), or it can @@ -20387,7 +20627,7 @@ them to other versions of `awk'. Consider this program: As written, it won't work on other versions of `awk'. However, it is actually almost portable, requiring very little change: - * Assignments to `TEXTDOMAIN' won't have any effect, since + * Assignments to `TEXTDOMAIN' won't have any effect, because `TEXTDOMAIN' is not special in other `awk' implementations. * Non-GNU versions of `awk' treat marked strings as the @@ -20396,7 +20636,7 @@ actually almost portable, requiring very little change: its value, leaving the original string constant as the result. * By defining "dummy" functions to replace `dcgettext()', - `dcngettext()' and `bindtextdomain()', the `awk' program can be + `dcngettext()', and `bindtextdomain()', the `awk' program can be made to run, but all the messages are output in the original language. For example: @@ -20423,10 +20663,10 @@ actually almost portable, requiring very little change: and arguments unchanged to the underlying C library version of `sprintf()', but only one format and argument at a time. What happens if a positional specification is used is anybody's guess. - However, since the positional specifications are primarily for use - in _translated_ format strings, and since non-GNU `awk's never - retrieve the translated string, this should not be a problem in - practice. + However, because the positional specifications are primarily for + use in _translated_ format strings, and because non-GNU `awk's + never retrieve the translated string, this should not be a problem + in practice. ---------- Footnotes ---------- @@ -20489,7 +20729,7 @@ Following are the translations: The next step is to make the directory to hold the binary message object file and then to create the `guide.mo' file. We pretend that -our file is to be used in the `en_US.UTF-8' locale, since we have to +our file is to be used in the `en_US.UTF-8' locale, because we have to use a locale name known to the C `gettext' routines. The directory layout shown here is standard for GNU `gettext' on GNU/Linux systems. Other versions of `gettext' may use a different layout: @@ -20510,7 +20750,7 @@ proper directory (using the `-o' option) so that `gawk' can find it: -| Like, the scoop is 42 -| Pardon me, Zaphod who? - If the three replacement functions for `dcgettext()', `dcngettext()' + If the three replacement functions for `dcgettext()', `dcngettext()', and `bindtextdomain()' (*note I18N Portability::) are in a file named `libintl.awk', then we can run `guide.awk' unchanged as follows: @@ -20531,9 +20771,9 @@ File: gawk.info, Node: Gawk I18N, Next: I18N Summary, Prev: I18N Example, Up `gawk' itself has been internationalized using the GNU `gettext' package. (GNU `gettext' is described in complete detail in *note (GNU -`gettext' utilities)Top:: gettext, GNU gettext tools.) As of this -writing, the latest version of GNU `gettext' is version 0.19.2 -(ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.2.tar.gz). +`gettext' utilities)Top:: gettext, GNU `gettext' utilities.) As of +this writing, the latest version of GNU `gettext' is version 0.19.4 +(ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.4.tar.gz). If a translation of `gawk''s messages exists, then `gawk' produces usage messages, warnings, and fatal errors in the local language. @@ -20545,7 +20785,7 @@ File: gawk.info, Node: I18N Summary, Prev: Gawk I18N, Up: Internationalizatio ============ * Internationalization means writing a program such that it can use - multiple languages without requiring source-code changes. + multiple languages without requiring source code changes. Localization means providing the data necessary for an internationalized program to work in a particular language. @@ -20559,10 +20799,10 @@ File: gawk.info, Node: I18N Summary, Prev: Gawk I18N, Up: Internationalizatio file, and the `.po' files are compiled into `.gmo' files for use at runtime. - * You can use position specifications with `sprintf()' and `printf' - to rearrange the placement of argument values in formatted strings - and output. This is useful for the translations of format control - strings. + * You can use positional specifications with `sprintf()' and + `printf' to rearrange the placement of argument values in formatted + strings and output. This is useful for the translation of format + control strings. * The internationalization features have been designed so that they can be easily worked around in a standard `awk'. @@ -20599,7 +20839,7 @@ program is easy. File: gawk.info, Node: Debugging, Next: Sample Debugging Session, Up: Debugger -14.1 Introduction to The `gawk' Debugger +14.1 Introduction to the `gawk' Debugger ======================================== This minor node introduces debugging in general and begins the @@ -20618,12 +20858,11 @@ File: gawk.info, Node: Debugging Concepts, Next: Debugging Terms, Up: Debuggi --------------------------- (If you have used debuggers in other languages, you may want to skip -ahead to the next section on the specific features of the `gawk' -debugger.) +ahead to *note Awk Debugging::.) - Of course, a debugging program cannot remove bugs for you, since it -has no way of knowing what you or your users consider a "bug" and what -is a "feature." (Sometimes, we humans have a hard time with this + Of course, a debugging program cannot remove bugs for you, because +it has no way of knowing what you or your users consider a "bug" versus +a "feature." (Sometimes, we humans have a hard time with this ourselves.) In that case, what can you expect from such a tool? The answer to that depends on the language being debugged, but in general, you can expect at least the following: @@ -20639,7 +20878,7 @@ you can expect at least the following: * The chance to see the values of data in the program at any point in execution, and also to change that data on the fly, to see how that - affects what happens afterwards. (This often includes the ability + affects what happens afterward. (This often includes the ability to look at internal data structures besides the variables you actually defined in your code.) @@ -20659,9 +20898,9 @@ File: gawk.info, Node: Debugging Terms, Next: Awk Debugging, Prev: Debugging Before diving in to the details, we need to introduce several important concepts that apply to just about all debuggers. The following list -defines terms used throughout the rest of this major node. +defines terms used throughout the rest of this major node: -"Stack Frame" +"Stack frame" Programs generally call functions during the course of their execution. One function can call another, or a function can call itself (recursion). You can view the chain of called functions @@ -20697,39 +20936,39 @@ defines terms used throughout the rest of this major node. breakpoints are oriented around the code: stop when a certain point in the code is reached. A watchpoint, however, specifies that program execution should stop when a _data value_ is changed. - This is useful, since sometimes it happens that a variable - receives an erroneous value, and it's hard to track down where - this happens just by looking at the code. By using a watchpoint, - you can stop whenever a variable is assigned to, and usually find - the errant code quite quickly. + This is useful, as sometimes it happens that a variable receives + an erroneous value, and it's hard to track down where this happens + just by looking at the code. By using a watchpoint, you can stop + whenever a variable is assigned to, and usually find the errant + code quite quickly. File: gawk.info, Node: Awk Debugging, Prev: Debugging Terms, Up: Debugging -14.1.3 Awk Debugging --------------------- +14.1.3 `awk' Debugging +---------------------- Debugging an `awk' program has some specific aspects that are not -shared with other programming languages. - - First of all, the fact that `awk' programs usually take input -line-by-line from a file or files and operate on those lines using -specific rules makes it especially useful to organize viewing the -execution of the program in terms of these rules. As we will see, each -`awk' rule is treated almost like a function call, with its own -specific block of instructions. - - In addition, since `awk' is by design a very concise language, it is -easy to lose sight of everything that is going on "inside" each line of -`awk' code. The debugger provides the opportunity to look at the +shared with programs written in other languages. + + First of all, the fact that `awk' programs usually take input line +by line from a file or files and operate on those lines using specific +rules makes it especially useful to organize viewing the execution of +the program in terms of these rules. As we will see, each `awk' rule +is treated almost like a function call, with its own specific block of +instructions. + + In addition, because `awk' is by design a very concise language, it +is easy to lose sight of everything that is going on "inside" each line +of `awk' code. The debugger provides the opportunity to look at the individual primitive instructions carried out by the higher-level `awk' commands. File: gawk.info, Node: Sample Debugging Session, Next: List of Debugger Commands, Prev: Debugging, Up: Debugger -14.2 Sample Debugging Session -============================= +14.2 Sample `gawk' Debugging Session +==================================== In order to illustrate the use of `gawk' as a debugger, let's look at a sample debugging session. We will use the `awk' implementation of the @@ -20748,8 +20987,8 @@ File: gawk.info, Node: Debugger Invocation, Next: Finding The Bug, Up: Sample -------------------------------- Starting the debugger is almost exactly like running `gawk' normally, -except you have to pass an additional option `--debug', or the -corresponding short option `-D'. The file(s) containing the program +except you have to pass an additional option, `--debug', or the +corresponding short option, `-D'. The file(s) containing the program and any supporting code are given on the command line as arguments to one or more `-f' options. (`gawk' is not designed to debug command-line programs, only programs contained in files.) In our case, we invoke @@ -20759,7 +20998,7 @@ the debugger like this: where both `getopt.awk' and `uniq.awk' are in `$AWKPATH'. (Experienced users of GDB or similar debuggers should note that this syntax is -slightly different from what they are used to. With the `gawk' +slightly different from what you are used to. With the `gawk' debugger, you give the arguments for running the program in the command line to the debugger rather than as part of the `run' command at the debugger prompt.) The `-1' is an option to `uniq.awk'. @@ -20832,8 +21071,8 @@ the current stack frames: -| #1 in main() at `awklib/eg/prog/uniq.awk':88 This tells us that `are_equal()' was called by the main program at -line 88 of `uniq.awk'. (This is not a big surprise, since this is the -only call to `are_equal()' in the program, but in more complex +line 88 of `uniq.awk'. (This is not a big surprise, because this is +the only call to `are_equal()' in the program, but in more complex programs, knowing who called a function and with what parameters can be the key to finding the source of the problem.) @@ -20845,7 +21084,7 @@ Actually, the debugger gives us: gawk> p n -| n = untyped variable -In this case, `n' is an uninitialized local variable, since the +In this case, `n' is an uninitialized local variable, because the function was called without arguments (*note Function Calls::). A more useful variable to display might be the current record: @@ -20853,8 +21092,8 @@ function was called without arguments (*note Function Calls::). gawk> p $0 -| $0 = "gawk is a wonderful program!" -This might be a bit puzzling at first since this is the second line of -our test input above. Let's look at `NR': +This might be a bit puzzling at first, as this is the second line of +our test input. Let's look at `NR': gawk> p NR -| NR = 2 @@ -20883,10 +21122,10 @@ typing `n' (for "next"): -| 66 if (fcount > 0) { This tells us that `gawk' is now ready to execute line 66, which -decides whether to give the lines the special "field skipping" treatment +decides whether to give the lines the special "field-skipping" treatment indicated by the `-1' command-line option. (Notice that we skipped -from where we were before at line 63 to here, since the condition in -line 63 `if (fcount == 0 && charcount == 0)' was false.) +from where we were before, at line 63, to here, because the condition +in line 63, `if (fcount == 0 && charcount == 0)', was false.) Continuing to step, we now get to the splitting of the current and last records: @@ -20910,9 +21149,9 @@ split into, so we try to look: This is kind of disappointing, though. All we found out is that there are five elements in `alast'; `m' and `aline' don't have values -since we are at line 68 but haven't executed it yet. This information -is useful enough (we now know that none of the words were accidentally -left out), but what if we want to see inside the array? +because we are at line 68 but haven't executed it yet. This +information is useful enough (we now know that none of the words were +accidentally left out), but what if we want to see inside the array? The first choice would be to use subscripts: @@ -20944,15 +21183,15 @@ mentioned): Well, here we are at our error (sorry to spoil the suspense). What we had in mind was to join the fields starting from the second one to -make the virtual record to compare, and if the first field was numbered -zero, this would work. Let's look at what we've got: +make the virtual record to compare, and if the first field were +numbered zero, this would work. Let's look at what we've got: gawk> p cline clast -| cline = "gawk is a wonderful program!" -| clast = "awk is a wonderful program!" Hey, those look pretty familiar! They're just our original, -unaltered, input records. A little thinking (the human brain is still +unaltered input records. A little thinking (the human brain is still the best debugging tool), and we realize that we were off by one! We get out of the debugger: @@ -20989,12 +21228,12 @@ categories: * Miscellaneous Each of these are discussed in the following subsections. In the -following descriptions, commands which may be abbreviated show the +following descriptions, commands that may be abbreviated show the abbreviation on a second description line. A debugger command name may also be truncated if that partial name is unambiguous. The debugger has the built-in capability to automatically repeat the previous command -just by hitting <Enter>. This works for the commands `list', `next', -`nexti', `step', `stepi' and `continue' executed without any argument. +just by hitting `Enter'. This works for the commands `list', `next', +`nexti', `step', `stepi', and `continue' executed without any argument. * Menu: @@ -21012,10 +21251,10 @@ File: gawk.info, Node: Breakpoint Control, Next: Debugger Execution Control, 14.3.1 Control of Breakpoints ----------------------------- -As we saw above, the first thing you probably want to do in a debugging -session is to get your breakpoints set up, since otherwise your program -will just run as if it was not under the debugger. The commands for -controlling breakpoints are: +As we saw earlier, the first thing you probably want to do in a +debugging session is to get your breakpoints set up, because your +program will otherwise just run as if it was not under the debugger. +The commands for controlling breakpoints are: `break' [[FILENAME`:']N | FUNCTION] [`"EXPRESSION"'] `b' [[FILENAME`:']N | FUNCTION] [`"EXPRESSION"'] @@ -21033,8 +21272,8 @@ controlling breakpoints are: Set a breakpoint at entry to (the first instruction of) function FUNCTION. - Each breakpoint is assigned a number which can be used to delete - it from the breakpoint list using the `delete' command. + Each breakpoint is assigned a number that can be used to delete it + from the breakpoint list using the `delete' command. With a breakpoint, you may also supply a condition. This is an `awk' expression (enclosed in double quotes) that the debugger @@ -21067,31 +21306,31 @@ controlling breakpoints are: reached. If the condition is true, then the debugger stops execution and prompts for a command. Otherwise, the debugger continues executing the program. If the condition expression is - not specified, any existing condition is removed; i.e., the - breakpoint or watchpoint is made unconditional. + not specified, any existing condition is removed (i.e., the + breakpoint or watchpoint is made unconditional). `delete' [N1 N2 ...] [N-M] `d' [N1 N2 ...] [N-M] - Delete specified breakpoints or a range of breakpoints. Deletes - all defined breakpoints if no argument is supplied. + Delete specified breakpoints or a range of breakpoints. Delete all + defined breakpoints if no argument is supplied. `disable' [N1 N2 ... | N-M] Disable specified breakpoints or a range of breakpoints. Without - any argument, disables all breakpoints. + any argument, disable all breakpoints. `enable' [`del' | `once'] [N1 N2 ...] [N-M] `e' [`del' | `once'] [N1 N2 ...] [N-M] Enable specified breakpoints or a range of breakpoints. Without - any argument, enables all breakpoints. Optionally, you can - specify how to enable the breakpoint: + any argument, enable all breakpoints. Optionally, you can specify + how to enable the breakpoints: `del' - Enable the breakpoint(s) temporarily, then delete it when the - program stops at the breakpoint. + Enable the breakpoints temporarily, then delete each one when + the program stops at it. `once' - Enable the breakpoint(s) temporarily, then disable it when - the program stops at the breakpoint. + Enable the breakpoints temporarily, then disable each one when + the program stops at it. `ignore' N COUNT Ignore breakpoint number N the next COUNT times it is hit. @@ -21137,7 +21376,7 @@ execution of the program than we saw in our earlier example: `continue' [COUNT] `c' [COUNT] Resume program execution. If continued from a breakpoint and COUNT - is specified, ignores the breakpoint at that location the next + is specified, ignore the breakpoint at that location the next COUNT times before stopping. `finish' @@ -21159,7 +21398,7 @@ execution of the program than we saw in our earlier example: Cancel execution of a function call. If VALUE (either a string or a number) is specified, it is used as the function's return value. If used in a frame other than the innermost one (the currently - executing function, i.e., frame number 0), discard all inner + executing function; i.e., frame number 0), discard all inner frames in addition to the selected one, and the caller of that frame becomes the innermost frame. @@ -21172,10 +21411,10 @@ execution of the program than we saw in our earlier example: `step' [COUNT] `s' [COUNT] Continue execution until control reaches a different source line - in the current stack frame. `step' steps inside any function - called within the line. If the argument COUNT is supplied, steps - that many times before stopping, unless it encounters a breakpoint - or watchpoint. + in the current stack frame, stepping inside any function called + within the line. If the argument COUNT is supplied, steps that + many times before stopping, unless it encounters a breakpoint or + watchpoint. `stepi' [COUNT] `si' [COUNT] @@ -21207,7 +21446,7 @@ The commands for viewing and changing variables inside of `gawk' are: gawk> display x -| 10: x = 1 - displays the assigned item number, the variable name and its + This displays the assigned item number, the variable name, and its current value. If the display variable refers to a function parameter, it is silently deleted from the list as soon as the execution reaches a context where no such variable of the given @@ -21256,13 +21495,13 @@ AWK STATEMENTS (`"'...`"'). You can also set special `awk' variables, such as `FS', `NF', - `NR', etc. + `NR', and so on. `watch' VAR | `$'N [`"EXPRESSION"'] `w' VAR | `$'N [`"EXPRESSION"'] Add variable VAR (or field `$N') to the watch list. The debugger then stops whenever the value of the variable or field changes. - Each watched item is assigned a number which can be used to delete + Each watched item is assigned a number that can be used to delete it from the watch list using the `unwatch' command. With a watchpoint, you may also supply a condition. This is an @@ -21286,11 +21525,11 @@ File: gawk.info, Node: Execution Stack, Next: Debugger Info, Prev: Viewing An 14.3.4 Working with the Stack ----------------------------- -Whenever you run a program which contains any function calls, `gawk' +Whenever you run a program that contains any function calls, `gawk' maintains a stack of all of the function calls leading up to where the program is right now. You can see how you got to where you are, and also move around in the stack to see what the state of things was in the -functions which called the one you are in. The commands for doing this +functions that called the one you are in. The commands for doing this are: `backtrace' [COUNT] @@ -21300,8 +21539,8 @@ are: innermost COUNT frames if COUNT > 0. Print the outermost COUNT frames if COUNT < 0. The backtrace displays the name and arguments to each function, the source file name, and the line - number. The alias `where' for `backtrace' is provided for - long-time GDB users who may be used to that command. + number. The alias `where' for `backtrace' is provided for longtime + GDB users who may be used to that command. `down' [COUNT] Move COUNT (default 1) frames down the stack toward the innermost @@ -21310,8 +21549,8 @@ are: `frame' [N] `f' [N] Select and print stack frame N. Frame 0 is the currently - executing, or "innermost", frame (function call), frame 1 is the - frame that called the innermost one. The highest numbered frame is + executing, or "innermost", frame (function call); frame 1 is the + frame that called the innermost one. The highest-numbered frame is the one for the main program. The printed information consists of the frame number, function and argument names, source file, and the source line. @@ -21323,12 +21562,12 @@ are: File: gawk.info, Node: Debugger Info, Next: Miscellaneous Debugger Commands, Prev: Execution Stack, Up: List of Debugger Commands -14.3.5 Obtaining Information about the Program and the Debugger State +14.3.5 Obtaining Information About the Program and the Debugger State --------------------------------------------------------------------- Besides looking at the values of variables, there is often a need to get other sorts of information about the state of your program and of the -debugging environment itself. The `gawk' debugger has one command which +debugging environment itself. The `gawk' debugger has one command that provides this information, appropriately called `info'. `info' is used with one of a number of arguments that tell it exactly what you want to know: @@ -21385,11 +21624,12 @@ from a file. The commands are: option. The available options are: `history_size' - The maximum number of lines to keep in the history file + Set the maximum number of lines to keep in the history file `./.gawk_history'. The default is 100. `listsize' - The number of lines that `list' prints. The default is 15. + Specify the number of lines that `list' prints. The default + is 15. `outfile' Send `gawk' output to a file; debugger output still goes to @@ -21397,7 +21637,7 @@ from a file. The commands are: standard output. `prompt' - The debugger prompt. The default is `gawk> '. + Change the debugger prompt. The default is `gawk> '. `save_history' [`on' | `off'] Save command history to file `./.gawk_history'. The default @@ -21405,8 +21645,8 @@ from a file. The commands are: `save_options' [`on' | `off'] Save current options to file `./.gawkrc' upon exit. The - default is `on'. Options are read back in to the next - session upon startup. + default is `on'. Options are read back into the next session + upon startup. `trace' [`on' | `off'] Turn instruction tracing on or off. The default is `off'. @@ -21425,7 +21665,7 @@ from a file. The commands are: commands; however, the `gawk' debugger will not source the same file more than once in order to avoid infinite recursion. - In addition to, or instead of the `source' command, you can use + In addition to, or instead of, the `source' command, you can use the `-D FILE' or `--debug=FILE' command-line options to execute commands from a file non-interactively (*note Options::). @@ -21435,13 +21675,13 @@ File: gawk.info, Node: Miscellaneous Debugger Commands, Prev: Debugger Info, 14.3.6 Miscellaneous Commands ----------------------------- -There are a few more commands which do not fit into the previous +There are a few more commands that do not fit into the previous categories, as follows: `dump' [FILENAME] - Dump bytecode of the program to standard output or to the file + Dump byte code of the program to standard output or to the file named in FILENAME. This prints a representation of the internal - instructions which `gawk' executes to implement the `awk' commands + instructions that `gawk' executes to implement the `awk' commands in a program. This can be very enlightening, as the following partial dump of Davide Brini's obfuscated code (*note Signature Program::) demonstrates: @@ -21449,39 +21689,39 @@ categories, as follows: gawk> dump -| # BEGIN -| - -| [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] - -| [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] - -| [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] - -| [ 1:0xfcc280] Op_match : - -| [ 1:0xfcc1e0] Op_store_var : O - -| [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] - -| [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] - -| [ 1:0xfcc320] Op_equal : - -| [ 1:0xfcc200] Op_store_var : o - -| [ 1:0xfcc380] Op_push : o - -| [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] - -| [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] - -| [ 1:0xfcc300] Op_assign_plus : - -| [ :0xfcc2c0] Op_pop : - -| [ 1:0xfcc400] Op_push : O - -| [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] - -| [ :0xfcc4a0] Op_no_op : - -| [ 1:0xfcc480] Op_push : O - -| [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] - -| [ 1:0xfcc3c0] Op_store_var : x - -| [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] - -| [ 1:0xfcc3a0] Op_postincrement : - -| [ 1:0xfcc4e0] Op_push : x - -| [ 1:0xfcc540] Op_push : o - -| [ 1:0xfcc500] Op_plus : - -| [ 1:0xfcc580] Op_push : o - -| [ 1:0xfcc560] Op_plus : - -| [ 1:0xfcc460] Op_leq : - -| [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] - -| [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] - -| [ :0xfcc660] Op_no_op : - -| [ 1:0xfcc520] Op_assign_concat : c - -| [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] + -| [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] + -| [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] + -| [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] + -| [ 1:0xfcc280] Op_match : + -| [ 1:0xfcc1e0] Op_store_var : O + -| [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] + -| [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] + -| [ 1:0xfcc320] Op_equal : + -| [ 1:0xfcc200] Op_store_var : o + -| [ 1:0xfcc380] Op_push : o + -| [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] + -| [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] + -| [ 1:0xfcc300] Op_assign_plus : + -| [ :0xfcc2c0] Op_pop : + -| [ 1:0xfcc400] Op_push : O + -| [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] + -| [ :0xfcc4a0] Op_no_op : + -| [ 1:0xfcc480] Op_push : O + -| [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] + -| [ 1:0xfcc3c0] Op_store_var : x + -| [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] + -| [ 1:0xfcc3a0] Op_postincrement : + -| [ 1:0xfcc4e0] Op_push : x + -| [ 1:0xfcc540] Op_push : o + -| [ 1:0xfcc500] Op_plus : + -| [ 1:0xfcc580] Op_push : o + -| [ 1:0xfcc560] Op_plus : + -| [ 1:0xfcc460] Op_leq : + -| [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] + -| [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] + -| [ :0xfcc660] Op_no_op : + -| [ 1:0xfcc520] Op_assign_concat : c + -| [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] -| ... -| @@ -21525,22 +21765,21 @@ categories, as follows: FILENAME. This command may change the current source file. FUNCTION - Print lines centered around beginning of the function + Print lines centered around the beginning of the function FUNCTION. This command may change the current source file. `quit' `q' Exit the debugger. Debugging is great fun, but sometimes we all have to tend to other obligations in life, and sometimes we find - the bug, and are free to go on to the next one! As we saw above, - if you are running a program, the debugger warns you if you - accidentally type `q' or `quit', to make sure you really want to - quit. + the bug and are free to go on to the next one! As we saw earlier, + if you are running a program, the debugger warns you when you type + `q' or `quit', to make sure you really want to quit. `trace' [`on' | `off'] - Turn on or off a continuous printing of instructions which are - about to be executed, along with printing the `awk' line which they - implement. The default is `off'. + Turn on or off continuous printing of the instructions that are + about to be executed, along with the `awk' lines they implement. + The default is `off'. It is to be hoped that most of the "opcodes" in these instructions are fairly self-explanatory, and using `stepi' and `nexti' while @@ -21553,7 +21792,7 @@ File: gawk.info, Node: Readline Support, Next: Limitations, Prev: List of Deb 14.4 Readline Support ===================== -If `gawk' is compiled with the `readline' library +If `gawk' is compiled with the GNU Readline library (http://cnswww.cns.cwru.edu/php/chet/readline/readline.html), you can take advantage of that library's command completion and history expansion features. The following types of completion are available: @@ -21583,7 +21822,7 @@ File: gawk.info, Node: Limitations, Next: Debugging Summary, Prev: Readline S We hope you find the `gawk' debugger useful and enjoyable to work with, but as with any program, especially in its early releases, it still has -some limitations. A few which are worth being aware of are: +some limitations. A few that it's worth being aware of are: * At this point, the debugger does not give a detailed explanation of what you did wrong when you type in something it doesn't like. @@ -21594,29 +21833,30 @@ some limitations. A few which are worth being aware of are: Commands:: (or if you are already familiar with `gawk' internals), you will realize that much of the internal manipulation of data in `gawk', as in many interpreters, is done on a stack. `Op_push', - `Op_pop', etc., are the "bread and butter" of most `gawk' code. + `Op_pop', and the like are the "bread and butter" of most `gawk' + code. Unfortunately, as of now, the `gawk' debugger does not allow you to examine the stack's contents. That is, the intermediate results of expression evaluation are on the stack, but cannot be - printed. Rather, only variables which are defined in the program + printed. Rather, only variables that are defined in the program can be printed. Of course, a workaround for this is to use more explicit variables at the debugging stage and then change back to obscure, perhaps more optimal code later. * There is no way to look "inside" the process of compiling regular expressions to see if you got it right. As an `awk' programmer, - you are expected to know what `/[^[:alnum:][:blank:]]/' means. + you are expected to know the meaning of `/[^[:alnum:][:blank:]]/'. * The `gawk' debugger is designed to be used by running a program (with all its parameters) on the command line, as described in *note Debugger Invocation::. There is no way (as of now) to - attach or "break in" to a running program. This seems reasonable - for a language which is used mainly for quickly executing, short + attach or "break into" a running program. This seems reasonable + for a language that is used mainly for quickly executing, short programs. - * The `gawk' debugger only accepts source supplied with the `-f' - option. + * The `gawk' debugger only accepts source code supplied with the + `-f' option. File: gawk.info, Node: Debugging Summary, Prev: Limitations, Up: Debugger @@ -21625,8 +21865,8 @@ File: gawk.info, Node: Debugging Summary, Prev: Limitations, Up: Debugger ============ * Programs rarely work correctly the first time. Finding bugs is - "debugging" and a program that helps you find bugs is a - "debugger". `gawk' has a built-in debugger that works very + called debugging, and a program that helps you find bugs is a + debugger. `gawk' has a built-in debugger that works very similarly to the GNU Debugger, GDB. * Debuggers let you step through your program one statement at a @@ -21642,25 +21882,25 @@ File: gawk.info, Node: Debugging Summary, Prev: Limitations, Up: Debugger breakpoints, execution, viewing and changing data, working with the stack, getting information, and other tasks. - * If the `readline' library is available when `gawk' is compiled, it - is used by the debugger to provide command-line history and + * If the GNU Readline library is available when `gawk' is compiled, + it is used by the debugger to provide command-line history and editing. File: gawk.info, Node: Arbitrary Precision Arithmetic, Next: Dynamic Extensions, Prev: Debugger, Up: Top -15 Arithmetic and Arbitrary Precision Arithmetic with `gawk' +15 Arithmetic and Arbitrary-Precision Arithmetic with `gawk' ************************************************************ This major node introduces some basic concepts relating to how computers do arithmetic and defines some important terms. It then proceeds to describe floating-point arithmetic, which is what `awk' -uses for all its computations, including a discussion of arbitrary -precision floating point arithmetic, which is a feature available only -in `gawk'. It continues on to present arbitrary precision integers, and -concludes with a description of some points where `gawk' and the POSIX -standard are not quite in agreement. +uses for all its computations, including a discussion of +arbitrary-precision floating-point arithmetic, which is a feature +available only in `gawk'. It continues on to present +arbitrary-precision integers, and concludes with a description of some +points where `gawk' and the POSIX standard are not quite in agreement. NOTE: Most users of `gawk' can safely skip this chapter. But if you want to do scientific calculations with `gawk', this is the @@ -21704,7 +21944,7 @@ Decimal arithmetic sides) of the decimal point, and the results of a computation are always exact. - Some modern system can do decimal arithmetic in hardware, but + Some modern systems can do decimal arithmetic in hardware, but usually you need a special software library to provide access to these instructions. There are also libraries that do decimal arithmetic entirely in software. @@ -21720,37 +21960,36 @@ Integer arithmetic In computers, integer values come in two flavors: "signed" and "unsigned". Signed values may be negative or positive, whereas - unsigned values are always positive (that is, greater than or equal - to zero). + unsigned values are always greater than or equal to zero. In computer systems, integer arithmetic is exact, but the possible range of values is limited. Integer arithmetic is generally - faster than floating point arithmetic. + faster than floating-point arithmetic. -Floating point arithmetic +Floating-point arithmetic Floating-point numbers represent what were called in school "real" - numbers; i.e., those that have a fractional part, such as - 3.1415927. The advantage to floating-point numbers is that they + numbers (i.e., those that have a fractional part, such as + 3.1415927). The advantage to floating-point numbers is that they can represent a much larger range of values than can integers. The disadvantage is that there are numbers that they cannot represent exactly. - Modern systems support floating point arithmetic in hardware, with + Modern systems support floating-point arithmetic in hardware, with a limited range of values. There are software libraries that allow - the use of arbitrary precision floating point calculations. + the use of arbitrary-precision floating-point calculations. - POSIX `awk' uses "double precision" floating-point numbers, which - can hold more digits than "single precision" floating-point - numbers. `gawk' has facilities for performing arbitrary precision - floating point arithmetic, which we describe in more detail + POSIX `awk' uses "double-precision" floating-point numbers, which + can hold more digits than "single-precision" floating-point + numbers. `gawk' has facilities for performing arbitrary-precision + floating-point arithmetic, which we describe in more detail shortly. - Computers work with integer and floating point values of different -ranges. Integer values are usually either 32 or 64 bits in size. Single -precision floating point values occupy 32 bits, whereas double precision -floating point values occupy 64 bits. Floating point values are always -signed. The possible ranges of values are shown in *note -table-numeric-ranges::. + Computers work with integer and floating-point values of different +ranges. Integer values are usually either 32 or 64 bits in size. +Single-precision floating-point values occupy 32 bits, whereas +double-precision floating-point values occupy 64 bits. Floating-point +values are always signed. The possible ranges of values are shown in +*note table-numeric-ranges::. Numeric representation Minimum value Maximum value --------------------------------------------------------------------------- @@ -21758,14 +21997,8 @@ Numeric representation Minimum value Maximum value 32-bit unsigned integer 0 4,294,967,295 64-bit signed integer -9,223,372,036,854,775,8089,223,372,036,854,775,807 64-bit unsigned integer 0 18,446,744,073,709,551,615 -Single precision `1.175494e-38' `3.402823e+38' -floating point -(approximate) -Double precision `2.225074e-308' `1.797693e+308' -floating point -(approximate) -Table 15.1: Value Ranges for Different Numeric Representations +Table 15.1: Value ranges for different numeric representations ---------- Footnotes ---------- @@ -21774,12 +22007,12 @@ Table 15.1: Value Ranges for Different Numeric Representations File: gawk.info, Node: Math Definitions, Next: MPFR features, Prev: Computer Arithmetic, Up: Arbitrary Precision Arithmetic -15.2 Other Stuff To Know +15.2 Other Stuff to Know ======================== The rest of this major node uses a number of terms. Here are some informal definitions that should help you work your way through the -material here. +material here: "Accuracy" A floating-point calculation's accuracy is how close it comes to @@ -21799,7 +22032,7 @@ material here. number and infinity produce infinity. "NaN" - "Not A Number."(1) A special value that results from attempting a + "Not a number."(1) A special value that results from attempting a calculation that has no answer as a real number. In such a case, programs can either receive a floating-point exception, or get `NaN' back as the result. The IEEE 754 standard recommends that @@ -21825,15 +22058,15 @@ material here. PREC = 3.322 * DPS - Here, PREC denotes the binary precision (measured in bits) and DPS - (short for decimal places) is the decimal digits. + Here, _prec_ denotes the binary precision (measured in bits) and + _dps_ (short for decimal places) is the decimal digits. "Rounding mode" How numbers are rounded up or down when necessary. More details are provided later. "Significand" - A floating point value consists the significand multiplied by 10 + A floating-point value consists of the significand multiplied by 10 to the power of the exponent. For example, in `1.2345e67', the significand is `1.2345'. @@ -21849,13 +22082,13 @@ information on some of those terms. On modern systems, floating-point hardware uses the representation and operations defined by the IEEE 754 standard. Three of the standard -IEEE 754 types are 32-bit single precision, 64-bit double precision and -128-bit quadruple precision. The standard also specifies extended +IEEE 754 types are 32-bit single precision, 64-bit double precision, +and 128-bit quadruple precision. The standard also specifies extended precision formats to allow greater precisions and larger exponent -ranges. (`awk' uses only the 64-bit double precision format.) +ranges. (`awk' uses only the 64-bit double-precision format.) *note table-ieee-formats:: lists the precision and exponent field -values for the basic IEEE 754 binary formats: +values for the basic IEEE 754 binary formats. Name Total bits Precision Minimum Maximum exponent exponent @@ -21864,7 +22097,7 @@ Single 32 24 -126 +127 Double 64 53 -1022 +1023 Quadruple 128 113 -16382 +16383 -Table 15.2: Basic IEEE Format Values +Table 15.2: Basic IEEE format values NOTE: The precision numbers include the implied leading one that gives them one extra bit of significand. @@ -21877,19 +22110,19 @@ paraphrased, and for the examples. File: gawk.info, Node: MPFR features, Next: FP Math Caution, Prev: Math Definitions, Up: Arbitrary Precision Arithmetic -15.3 Arbitrary Precision Arithmetic Features In `gawk' +15.3 Arbitrary-Precision Arithmetic Features in `gawk' ====================================================== -By default, `gawk' uses the double precision floating-point values +By default, `gawk' uses the double-precision floating-point values supplied by the hardware of the system it runs on. However, if it was -compiled to do so, `gawk' uses the `http://www.mpfr.org GNU MPFR' and -GNU MP (http://gmplib.org) (GMP) libraries for arbitrary precision +compiled to do so, `gawk' uses the GNU MPFR (http://www.mpfr.org) and +GNU MP (http://gmplib.org) (GMP) libraries for arbitrary-precision arithmetic on numbers. You can see if MPFR support is available like so: $ gawk --version -| GNU Awk 4.1.2, API: 1.1 (GNU MPFR 3.1.0-p3, GNU MP 5.0.2) - -| Copyright (C) 1989, 1991-2014 Free Software Foundation. + -| Copyright (C) 1989, 1991-2015 Free Software Foundation. ... (You may see different version numbers than what's shown here. That's @@ -21915,12 +22148,12 @@ more information. File: gawk.info, Node: FP Math Caution, Next: Arbitrary Precision Integers, Prev: MPFR features, Up: Arbitrary Precision Arithmetic -15.4 Floating Point Arithmetic: Caveat Emptor! +15.4 Floating-Point Arithmetic: Caveat Emptor! ============================================== Math class is tough! -- Teen Talk Barbie, July 1992 - This minor node provides a high level overview of the issues + This minor node provides a high-level overview of the issues involved when doing lots of floating-point arithmetic.(1) The discussion applies to both hardware and arbitrary-precision floating-point arithmetic. @@ -21941,15 +22174,15 @@ floating-point arithmetic. (1) There is a very nice paper on floating-point arithmetic (http://www.validlab.com/goldberg/paper.pdf) by David Goldberg, "What -Every Computer Scientist Should Know About Floating-point Arithmetic," -`ACM Computing Surveys' *23*, 1 (1991-03), 5-48. This is worth reading +Every Computer Scientist Should Know About Floating-Point Arithmetic," +`ACM Computing Surveys' *23*, 1 (1991-03): 5-48. This is worth reading if you are interested in the details, but it does require a background in computer science. File: gawk.info, Node: Inexactness of computations, Next: Getting Accuracy, Up: FP Math Caution -15.4.1 Floating Point Arithmetic Is Not Exact +15.4.1 Floating-Point Arithmetic Is Not Exact --------------------------------------------- Binary floating-point representations and arithmetic are inexact. @@ -21957,9 +22190,10 @@ Simple values like 0.1 cannot be precisely represented using binary floating-point numbers, and the limited precision of floating-point numbers means that slight changes in the order of operations or the precision of intermediate storage can change the result. To make -matters worse, with arbitrary precision floating-point, you can set the -precision before starting a computation, but then you cannot be sure of -the number of significant decimal places in the final result. +matters worse, with arbitrary-precision floating-point arithmetic, you +can set the precision before starting a computation, but then you +cannot be sure of the number of significant decimal places in the final +result. * Menu: @@ -21981,8 +22215,8 @@ the following example: y = 0.425 Unlike the number in `y', the number stored in `x' is exactly -representable in binary since it can be written as a finite sum of one -or more fractions whose denominators are all powers of two. When +representable in binary because it can be written as a finite sum of +one or more fractions whose denominators are all powers of two. When `gawk' reads a floating-point number from program source, it automatically rounds that number to whatever precision your machine supports. If you try to print the numeric content of a variable using @@ -21995,7 +22229,7 @@ number as you assigned to it: Often the error is so small you do not even notice it, and if you do, you can always specify how much precision you would like in your output. -Usually this is a format string like `"%.15g"', which when used in the +Usually this is a format string like `"%.15g"', which, when used in the previous example, produces an output identical to the input. @@ -22015,7 +22249,7 @@ work like you would expect: The general wisdom when comparing floating-point values is to see if they are within some small range of each other (called a "delta", or "tolerance"). You have to decide how small a delta is important to -you. Code to do this looks something like this: +you. Code to do this looks something like the following: delta = 0.00001 # for example difference = abs(a) - abs(b) # subtract the two values @@ -22035,7 +22269,7 @@ File: gawk.info, Node: Errors accumulate, Prev: Comparing FP Values, Up: Inex The loss of accuracy during a single computation with floating-point numbers usually isn't enough to worry about. However, if you compute a -value which is the result of a sequence of floating point operations, +value that is the result of a sequence of floating-point operations, the error can accumulate and greatly affect the computation itself. Here is an attempt to compute the value of pi using one of its many series representations: @@ -22078,18 +22312,18 @@ representations yield an unexpected result: File: gawk.info, Node: Getting Accuracy, Next: Try To Round, Prev: Inexactness of computations, Up: FP Math Caution -15.4.2 Getting The Accuracy You Need +15.4.2 Getting the Accuracy You Need ------------------------------------ -Can arbitrary precision arithmetic give exact results? There are no +Can arbitrary-precision arithmetic give exact results? There are no easy answers. The standard rules of algebra often do not apply when using floating-point arithmetic. Among other things, the distributive and associative laws do not hold completely, and order of operation may be important for your computation. Rounding error, cumulative precision -loss and underflow are often troublesome. +loss, and underflow are often troublesome. When `gawk' tests the expressions `0.1 + 12.2' and `12.3' for -equality using the machine double precision arithmetic, it decides that +equality using the machine double-precision arithmetic, it decides that they are not equal! (*Note Comparing FP Values::.) You can get the result you want by increasing the precision; 56 bits in this case does the job: @@ -22108,21 +22342,22 @@ value of `PREC': forget that the finite number of bits used to store the value is often just an approximation after proper rounding. The test for equality succeeds if and only if _all_ bits in the two operands are exactly the -same. Since this is not necessarily true after floating-point +same. Because this is not necessarily true after floating-point computations with a particular precision and effective rounding mode, a straight test for equality may not work. Instead, compare the two numbers to see if they are within the desirable delta of each other. In applications where 15 or fewer decimal places suffice, hardware -double precision arithmetic can be adequate, and is usually much faster. +double-precision arithmetic can be adequate, and is usually much faster. But you need to keep in mind that every floating-point operation can -suffer a new rounding error with catastrophic consequences as +suffer a new rounding error with catastrophic consequences, as illustrated by our earlier attempt to compute the value of pi. Extra precision can greatly enhance the stability and the accuracy of your computation in such cases. - Repeated addition is not necessarily equivalent to multiplication in -floating-point arithmetic. In the example in *note Errors accumulate::: + Additionally, you should understand that repeated addition is not +necessarily equivalent to multiplication in floating-point arithmetic. +In the example in *note Errors accumulate::: $ gawk 'BEGIN { > for (d = 1.1; d <= 1.5; d += 0.1) # loop five times (?) @@ -22138,10 +22373,10 @@ hand is often the correct approach in such situations. File: gawk.info, Node: Try To Round, Next: Setting precision, Prev: Getting Accuracy, Up: FP Math Caution -15.4.3 Try A Few Extra Bits of Precision and Rounding +15.4.3 Try a Few Extra Bits of Precision and Rounding ----------------------------------------------------- -Instead of arbitrary precision floating-point arithmetic, often all you +Instead of arbitrary-precision floating-point arithmetic, often all you need is an adjustment of your logic or a different order for the operations in your calculation. The stability and the accuracy of the computation of pi in the earlier example can be enhanced by using the @@ -22149,7 +22384,7 @@ following simple algebraic transformation: (sqrt(x * x + 1) - 1) / x == x / (sqrt(x * x + 1) + 1) -After making this, change the program converges to pi in under 30 +After making this change, the program converges to pi in under 30 iterations: $ gawk -f pi2.awk @@ -22165,7 +22400,7 @@ iterations: File: gawk.info, Node: Setting precision, Next: Setting the rounding mode, Prev: Try To Round, Up: FP Math Caution -15.4.4 Setting The Precision +15.4.4 Setting the Precision ---------------------------- `gawk' uses a global working precision; it does not keep track of the @@ -22177,15 +22412,15 @@ set the value to one of the predefined case-insensitive strings shown in *note table-predefined-precision-strings::, to emulate an IEEE 754 binary format. -`PREC' IEEE 754 Binary Format +`PREC' IEEE 754 binary format --------------------------------------------------- -`"half"' 16-bit half-precision. -`"single"' Basic 32-bit single precision. -`"double"' Basic 64-bit double precision. -`"quad"' Basic 128-bit quadruple precision. -`"oct"' 256-bit octuple precision. +`"half"' 16-bit half-precision +`"single"' Basic 32-bit single precision +`"double"' Basic 64-bit double precision +`"quad"' Basic 128-bit quadruple precision +`"oct"' 256-bit octuple precision -Table 15.3: Predefined Precision Strings For `PREC' +Table 15.3: Predefined precision strings for `PREC' The following example illustrates the effects of changing precision on arithmetic operations: @@ -22210,41 +22445,41 @@ on arithmetic operations: example illustrates the differences among various ways to print a floating-point constant: - $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }' - -| 0.1000000000000000055511151 - $ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }' - -| 0.1000000000000000000000000 - $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }' - -| 0.1000000000000000000000000 - $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }' - -| 0.1000000000000000000000000 + $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 0.1) }' + -| 0.1000000000000000055511151 + $ gawk -M -v PREC=113 'BEGIN { printf("%0.25f\n", 0.1) }' + -| 0.1000000000000000000000000 + $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", "0.1") }' + -| 0.1000000000000000000000000 + $ gawk -M 'BEGIN { PREC = 113; printf("%0.25f\n", 1/10) }' + -| 0.1000000000000000000000000 File: gawk.info, Node: Setting the rounding mode, Prev: Setting precision, Up: FP Math Caution -15.4.5 Setting The Rounding Mode +15.4.5 Setting the Rounding Mode -------------------------------- -The `ROUNDMODE' variable provides program level control over the +The `ROUNDMODE' variable provides program-level control over the rounding mode. The correspondence between `ROUNDMODE' and the IEEE rounding modes is shown in *note table-gawk-rounding-modes::. -Rounding Mode IEEE Name `ROUNDMODE' +Rounding mode IEEE name `ROUNDMODE' --------------------------------------------------------------------------- Round to nearest, ties to even `roundTiesToEven' `"N"' or `"n"' -Round toward plus Infinity `roundTowardPositive' `"U"' or `"u"' -Round toward negative Infinity `roundTowardNegative' `"D"' or `"d"' +Round toward positive infinity `roundTowardPositive' `"U"' or `"u"' +Round toward negative infinity `roundTowardNegative' `"D"' or `"d"' Round toward zero `roundTowardZero' `"Z"' or `"z"' Round to nearest, ties away `roundTiesToAway' `"A"' or `"a"' from zero -Table 15.4: `gawk' Rounding Modes +Table 15.4: `gawk' rounding modes `ROUNDMODE' has the default value `"N"', which selects the IEEE 754 rounding mode `roundTiesToEven'. In *note Table 15.4: table-gawk-rounding-modes, the value `"A"' selects `roundTiesToAway'. This is only available if your version of the MPFR library supports it; -otherwise setting `ROUNDMODE' to `"A"' has no effect. +otherwise, setting `ROUNDMODE' to `"A"' has no effect. The default mode `roundTiesToEven' is the most preferred, but the least intuitive. This method does the obvious thing for most values, by @@ -22284,8 +22519,8 @@ distributes upward and downward rounds of exact halves, which might cause any accumulating round-off error to cancel itself out. This is the default rounding mode for IEEE 754 computing functions and operators. - The other rounding modes are rarely used. Round toward positive -infinity (`roundTowardPositive') and round toward negative infinity + The other rounding modes are rarely used. Rounding toward positive +infinity (`roundTowardPositive') and toward negative infinity (`roundTowardNegative') are often used to implement interval arithmetic, where you adjust the rounding mode to calculate upper and lower bounds for the range of output. The `roundTowardZero' mode can be @@ -22312,16 +22547,16 @@ to round halfway cases for `printf'. File: gawk.info, Node: Arbitrary Precision Integers, Next: POSIX Floating Point Problems, Prev: FP Math Caution, Up: Arbitrary Precision Arithmetic -15.5 Arbitrary Precision Integer Arithmetic with `gawk' +15.5 Arbitrary-Precision Integer Arithmetic with `gawk' ======================================================= When given the `-M' option, `gawk' performs all integer arithmetic -using GMP arbitrary precision integers. Any number that looks like an -integer in a source or data file is stored as an arbitrary precision +using GMP arbitrary-precision integers. Any number that looks like an +integer in a source or data file is stored as an arbitrary-precision integer. The size of the integer is limited only by the available memory. For example, the following computes 5^4^3^2, the result of -which is beyond the limits of ordinary hardware double precision -floating point values: +which is beyond the limits of ordinary hardware double-precision +floating-point values: $ gawk -M 'BEGIN { > x = 5^4^3^2 @@ -22331,10 +22566,10 @@ floating point values: -| number of digits = 183231 -| 62060698786608744707 ... 92256259918212890625 - If instead you were to compute the same value using arbitrary -precision floating-point values, the precision needed for correct -output (using the formula `prec = 3.322 * dps'), would be 3.322 x -183231, or 608693. + If instead you were to compute the same value using +arbitrary-precision floating-point values, the precision needed for +correct output (using the formula `prec = 3.322 * dps') would be 3.322 +x 183231, or 608693. The result from an arithmetic operation with an integer and a floating-point value is a floating-point value with a precision equal @@ -22357,14 +22592,14 @@ case), or replace the floating-point constant `2.0' with an integer, to perform all computations using integer arithmetic to get the correct output. - Sometimes `gawk' must implicitly convert an arbitrary precision -integer into an arbitrary precision floating-point value. This is + Sometimes `gawk' must implicitly convert an arbitrary-precision +integer into an arbitrary-precision floating-point value. This is primarily because the MPFR library does not always provide the relevant -interface to process arbitrary precision integers or mixed-mode numbers +interface to process arbitrary-precision integers or mixed-mode numbers as needed by an operation or function. In such a case, the precision is set to the minimum value necessary for exact conversion, and the working precision is not used for this purpose. If this is not what you need or -want, you can employ a subterfuge, and convert the integer to floating +want, you can employ a subterfuge and convert the integer to floating point first, like this: gawk -M 'BEGIN { n = 13; print (n + 0.0) % 2.0 }' @@ -22374,8 +22609,8 @@ floating-point value to begin with: gawk -M 'BEGIN { n = 13.0; print n % 2.0 }' - Note that for the particular example above, it is likely best to -just use the following: + Note that for this particular example, it is likely best to just use +the following: gawk -M 'BEGIN { n = 13; print n % 2 }' @@ -22383,14 +22618,15 @@ just use the following: `%', the result is typically an arbitrary precision floating point value (unless the denominator evenly divides into the numerator). In order to do integer division or remainder with arbitrary precision -integers, use the built-in `div()' function (*note Numeric Functions::). +integers, use the built-in `intdiv()' function (*note Numeric +Functions::). - You can simulate the `div()' function in standard `awk' using this -user-defined function: + You can simulate the `intdiv()' function in standard `awk' using +this user-defined function: - # div --- do integer division + # intdiv --- do integer division - function div(numerator, denominator, result) + function intdiv(numerator, denominator, result) { split("", result) @@ -22403,8 +22639,8 @@ user-defined function: } The following example program, contributed by Katie Wasserman, uses -`div()' to compute the digits of pi to as many places as you choose to -set: +`intdiv()' to compute the digits of pi to as many places as you choose +to set: # pi.awk --- compute the digits of pi @@ -22415,7 +22651,7 @@ set: for (m = digits * 4; m > 0; --m) { d = m * 2 + 1 x = pi * m - div(x, d, result) + intdiv(x, d, result) pi = result["quotient"] pi = pi + two } @@ -22427,7 +22663,7 @@ set: It's not that well known but it's not that obscure either. It's Euler's modification to Newton's method for calculating pi. Take a look at lines (23) - (25) here: - `http://mathworld.wolfram.com/PiFormulas.htm'. + `http://mathworld.wolfram.com/PiFormulas.html'. The algorithm I wrote simply expands the multiply by 2 and works from the innermost expression outwards. I used this to program HP @@ -22447,7 +22683,7 @@ File: gawk.info, Node: POSIX Floating Point Problems, Next: Floating point sum 15.6 Standards Versus Existing Practice ======================================= -Historically, `awk' has converted any non-numeric looking string to the +Historically, `awk' has converted any nonnumeric-looking string to the numeric value zero, when required. Furthermore, the original definition of the language and the original POSIX standards specified that `awk' only understands decimal numbers (base 10), and not octal @@ -22457,12 +22693,12 @@ that `awk' only understands decimal numbers (base 10), and not octal interpreted to imply that `awk' should support additional features. These features are: - * Interpretation of floating point data values specified in + * Interpretation of floating-point data values specified in hexadecimal notation (e.g., `0xDEADBEEF'). (Note: data values, _not_ source code constants.) - * Support for the special IEEE 754 floating point values "Not A - Number" (NaN), positive Infinity ("inf") and negative Infinity + * Support for the special IEEE 754 floating-point values "not a + number" (NaN), positive infinity ("inf"), and negative infinity ("-inf"). In particular, the format for these values is as specified by the ISO 1999 C standard, which ignores case and can allow implementation-dependent additional characters after the @@ -22471,29 +22707,29 @@ These features are: The first problem is that both of these are clear changes to historical practice: - * The `gawk' maintainer feels that supporting hexadecimal floating - point values, in particular, is ugly, and was never intended by the - original designers to be part of the language. + * The `gawk' maintainer feels that supporting hexadecimal + floating-point values, in particular, is ugly, and was never + intended by the original designers to be part of the language. * Allowing completely alphabetic strings to have valid numeric values is also a very severe departure from historical practice. The second problem is that the `gawk' maintainer feels that this -interpretation of the standard, which requires a certain amount of +interpretation of the standard, which required a certain amount of "language lawyering" to arrive at in the first place, was not even -intended by the standard developers. In other words, "we see how you +intended by the standard developers. In other words, "We see how you got where you are, but we don't think that that's where you want to be." - Recognizing the above issues, but attempting to provide compatibility + Recognizing these issues, but attempting to provide compatibility with the earlier versions of the standard, the 2008 POSIX standard added explicit wording to allow, but not require, that `awk' support -hexadecimal floating point values and special values for "Not A Number" +hexadecimal floating-point values and special values for "not a number" and infinity. Although the `gawk' maintainer continues to feel that providing those features is inadvisable, nevertheless, on systems that support IEEE floating point, it seems reasonable to provide _some_ way to -support NaN and Infinity values. The solution implemented in `gawk' is +support NaN and infinity values. The solution implemented in `gawk' is as follows: * With the `--posix' command-line option, `gawk' becomes "hands @@ -22508,7 +22744,7 @@ as follows: $ echo 0xDeadBeef | gawk --posix '{ print $1 + 0 }' -| 3735928559 - * Without `--posix', `gawk' interprets the four strings `+inf', + * Without `--posix', `gawk' interprets the four string values `+inf', `-inf', `+nan', and `-nan' specially, producing the corresponding special numeric values. The leading sign acts a signal to `gawk' (and the user) that the value is really numeric. Hexadecimal @@ -22522,7 +22758,7 @@ as follows: $ echo 0xDeadBeef | gawk '{ print $1 + 0 }' -| 0 - `gawk' ignores case in the four special values. Thus `+nan' and + `gawk' ignores case in the four special values. Thus, `+nan' and `+NaN' are the same. ---------- Footnotes ---------- @@ -22536,12 +22772,12 @@ File: gawk.info, Node: Floating point summary, Prev: POSIX Floating Point Prob ============ * Most computer arithmetic is done using either integers or - floating-point values. Standard `awk' uses double precision + floating-point values. Standard `awk' uses double-precision floating-point values. - * In the early 1990's, Barbie mistakenly said "Math class is tough!" - While math isn't tough, floating-point arithmetic isn't the same - as pencil and paper math, and care must be taken: + * In the early 1990s Barbie mistakenly said, "Math class is tough!" + Although math isn't tough, floating-point arithmetic isn't the same + as pencil-and-paper math, and care must be taken: - Not all numbers can be represented exactly. @@ -22561,19 +22797,19 @@ File: gawk.info, Node: Floating point summary, Prev: POSIX Floating Point Prob set the precision in bits, and `ROUNDMODE' to set the IEEE 754 rounding mode. - * With `-M', `gawk' performs arbitrary precision integer arithmetic - using the GMP library. This is faster and more space efficient + * With `-M', `gawk' performs arbitrary-precision integer arithmetic + using the GMP library. This is faster and more space-efficient than using MPFR for the same calculations. - * There are several "dark corners" with respect to floating-point - numbers where `gawk' disagrees with the POSIX standard. It pays - to be aware of them. + * There are several areas with respect to floating-point numbers + where `gawk' disagrees with the POSIX standard. It pays to be + aware of them. * Overall, there is no need to be unduly suspicious about the results from floating-point arithmetic. The lesson to remember is that floating-point arithmetic is always more complex than arithmetic using pencil and paper. In order to take advantage of - the power of computer floating-point, you need to know its + the power of floating-point arithmetic, you need to know its limitations and work within them. For most casual use of floating-point arithmetic, you will often get the expected result if you simply round the display of your final results to the @@ -22632,15 +22868,15 @@ the rest of this Info file. `gawk''s functionality. For example, they can provide access to system calls (such as `chdir()' to change directory) and to other C library routines that could be of use. As with most software, "the sky is the -limit;" if you can imagine something that you might want to do and can +limit"; if you can imagine something that you might want to do and can write in C or C++, you can write an extension to do it! - Extensions are written in C or C++, using the "Application -Programming Interface" (API) defined for this purpose by the `gawk' + Extensions are written in C or C++, using the "application +programming interface" (API) defined for this purpose by the `gawk' developers. The rest of this major node explains the facilities that the API provides and how to use them, and presents a small example extension. In addition, it documents the sample extensions included in -the `gawk' distribution, and describes the `gawkextlib' project. *Note +the `gawk' distribution and describes the `gawkextlib' project. *Note Extension Design::, for a discussion of the extension mechanism goals and design. @@ -22667,7 +22903,7 @@ the symbol exists in the global scope. Something like this is enough: File: gawk.info, Node: Extension Mechanism Outline, Next: Extension API Description, Prev: Plugin License, Up: Dynamic Extensions -16.3 At A High Level How It Works +16.3 How It Works at a High Level ================================= Communication between `gawk' and an extension is two-way. First, when @@ -22699,7 +22935,7 @@ figure-load-extension::. +-------+-+---+-+---+-+------------------+--------------------+ gawk Main Program Address Space Extension -Figure 16.1: Loading The Extension +Figure 16.1: Loading the extension The extension can call functions inside `gawk' through these function pointers, at runtime, without needing (link-time) access to @@ -22719,7 +22955,7 @@ figure-register-new-function::. +-------+-+---+-+---+-+------------------+--------------+-+---+ gawk Main Program Address Space Extension -Figure 16.2: Registering A New Function +Figure 16.2: Registering a new function In the other direction, the extension registers its new functions with `gawk' by passing function pointers to the functions that provide @@ -22740,7 +22976,7 @@ calling convention. This is shown in *note figure-call-new-function::. +-------+-+---+-+---+-+------------------+--------------+-+---+ gawk Main Program Address Space Extension -Figure 16.3: Calling The New Function +Figure 16.3: Calling the new function The `do_XXX()' function, in turn, then uses the function pointers in the API `struct' to do its work, such as updating variables or arrays, @@ -22758,7 +22994,7 @@ Example::) and also in the `testext.c' code for testing the APIs. Some other bits and pieces: * The API provides access to `gawk''s `do_XXX' values, reflecting - command-line options, like `do_lint', `do_profiling' and so on + command-line options, like `do_lint', `do_profiling', and so on (*note Extension API Variables::). These are informational: an extension cannot affect their values inside `gawk'. In addition, attempting to assign to them produces a compile-time error. @@ -22795,6 +23031,7 @@ describes the API in detail. * Symbol Table Access:: Functions for accessing global variables. * Array Manipulation:: Functions for working with arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension API Boilerplate:: Boilerplate code for using the API. @@ -22804,8 +23041,8 @@ File: gawk.info, Node: Extension API Functions Introduction, Next: General Dat 16.4.1 Introduction ------------------- -Access to facilities within `gawk' are made available by calling -through function pointers passed into your extension. +Access to facilities within `gawk' is achieved by calling through +function pointers passed into your extension. API function pointers are provided for the following kinds of operations: @@ -22813,18 +23050,20 @@ operations: * Allocating, reallocating, and releasing memory. * Registration functions. You may register: - - extension functions, - - exit callbacks, + - Extension functions + + - Exit callbacks - - a version string, + - A version string - - input parsers, + - Input parsers - - output wrappers, + - Output wrappers - - and two-way processors. - All of these are discussed in detail, later in this major node. + - Two-way processors + + All of these are discussed in detail later in this major node. * Printing fatal, warning, and "lint" warning messages. @@ -22850,16 +23089,19 @@ operations: - Clearing an array - - Flattening an array for easy C style looping over all its + - Flattening an array for easy C-style looping over all its indices and elements + * Accessing and manipulating redirections. + + Some points about using the API: - * The following types and/or macros and/or functions are referenced - in `gawkapi.h'. For correct use, you must therefore include the + * The following types, macros, and/or functions are referenced in + `gawkapi.h'. For correct use, you must therefore include the corresponding standard header file _before_ including `gawkapi.h': - C Entity Header File + C entity Header file ------------------------------------------- `EOF' `<stdio.h>' Values for `errno' `<errno.h>' @@ -22883,26 +23125,26 @@ operations: * Although the API only uses ISO C 90 features, there is an exception; the "constructor" functions use the `inline' keyword. If your compiler does not support this keyword, you should either - place `-Dinline=''' on your command line, or use the GNU Autotools + place `-Dinline=''' on your command line or use the GNU Autotools and include a `config.h' file in your extensions. * All pointers filled in by `gawk' point to memory managed by `gawk' and should be treated by the extension as read-only. Memory for _all_ strings passed into `gawk' from the extension _must_ come - from calling one of `gawk_malloc()', `gawk_calloc()' or + from calling one of `gawk_malloc()', `gawk_calloc()', or `gawk_realloc()', and is managed by `gawk' from then on. * The API defines several simple `struct's that map values as seen from `awk'. A value can be a `double', a string, or an array (as in multidimensional arrays, or when creating a new array). String - values maintain both pointer and length since embedded NUL + values maintain both pointer and length, because embedded NUL characters are allowed. NOTE: By intent, strings are maintained using the current multibyte encoding (as defined by `LC_XXX' environment variables) and not using wide characters. This matches how `gawk' stores strings internally and also how characters are - likely to be input and output from files. + likely to be input into and output from files. * When retrieving a value (such as a parameter or that of a global variable or array element), the extension requests a specific type @@ -22916,16 +23158,16 @@ operations: message (such as "scalar passed where array expected"). - While you may call the API functions by using the function pointers -directly, the interface is not so pretty. To make extension code look -more like regular code, the `gawkapi.h' header file defines several + You may call the API functions by using the function pointers +directly, but the interface is not so pretty. To make extension code +look more like regular code, the `gawkapi.h' header file defines several macros that you should use in your code. This minor node presents the macros as if they were functions. File: gawk.info, Node: General Data Types, Next: Memory Allocation Functions, Prev: Extension API Functions Introduction, Up: Extension API Description -16.4.2 General Purpose Data Types +16.4.2 General-Purpose Data Types --------------------------------- I have a true love/hate relationship with unions. -- Arnold @@ -22934,10 +23176,12 @@ File: gawk.info, Node: General Data Types, Next: Memory Allocation Functions, That's the thing about unions: the compiler will arrange things so they can accommodate both love and hate. -- Chet Ramey - The extension API defines a number of simple types and structures -for general purpose use. Additional, more specialized, data structures -are introduced in subsequent minor nodes, together with the functions -that use them. + The extension API defines a number of simple types and structures for +general-purpose use. Additional, more specialized, data structures are +introduced in subsequent minor nodes, together with the functions that +use them. + + The general-purpose types and structures are as follows: `typedef void *awk_ext_id_t;' A value of this type is received from `gawk' when an extension is @@ -22954,7 +23198,7 @@ that use them. ` awk_false = 0,' ` awk_true' `} awk_bool_t;' - A simple boolean type. + A simple Boolean type. `typedef struct awk_string {' ` char *str; /* data */' @@ -22962,8 +23206,8 @@ that use them. `} awk_string_t;' This represents a mutable string. `gawk' owns the memory pointed to if it supplied the value. Otherwise, it takes ownership of the - memory pointed to. *Such memory must come from calling one of the - `gawk_malloc()', `gawk_calloc()', or `gawk_realloc()' functions!* + memory pointed to. _Such memory must come from calling one of the + `gawk_malloc()', `gawk_calloc()', or `gawk_realloc()' functions!_ As mentioned earlier, strings are maintained using the current multibyte encoding. @@ -22998,19 +23242,19 @@ that use them. `#define array_cookie u.a' `#define scalar_cookie u.scl' `#define value_cookie u.vc' - These macros make accessing the fields of the `awk_value_t' more - readable. + Using these macros makes accessing the fields of the `awk_value_t' + more readable. `typedef void *awk_scalar_t;' Scalars can be represented as an opaque type. These values are obtained from `gawk' and then passed back into it. This is - discussed in a general fashion below, and in more detail in *note - Symbol table by cookie::. + discussed in a general fashion in the text following this list, + and in more detail in *note Symbol table by cookie::. `typedef void *awk_value_cookie_t;' A "value cookie" is an opaque type representing a cached value. - This is also discussed in a general fashion below, and in more - detail in *note Cached values::. + This is also discussed in a general fashion in the text following + this list, and in more detail in *note Cached values::. Scalar values in `awk' are either numbers or strings. The @@ -23018,9 +23262,9 @@ that use them. indicates what is in the `union'. Representing numbers is easy--the API uses a C `double'. Strings -require more work. Since `gawk' allows embedded NUL bytes in string -values, a string must be represented as a pair containing a -data-pointer and length. This is the `awk_string_t' type. +require more work. Because `gawk' allows embedded NUL bytes in string +values, a string must be represented as a pair containing a data +pointer and length. This is the `awk_string_t' type. Identifiers (i.e., the names of global variables) can be associated with either scalar values or with arrays. In addition, `gawk' provides @@ -23030,15 +23274,14 @@ Manipulation::. The various macros listed earlier make it easier to use the elements of the `union' as if they were fields in a `struct'; this is a common -coding practice in C. Such code is easier to write and to read, -however it remains _your_ responsibility to make sure that the -`val_type' member correctly reflects the type of the value in the -`awk_value_t'. +coding practice in C. Such code is easier to write and to read, but it +remains _your_ responsibility to make sure that the `val_type' member +correctly reflects the type of the value in the `awk_value_t' struct. Conceptually, the first three members of the `union' (number, string, and array) are all that is needed for working with `awk' values. -However, since the API provides routines for accessing and changing the -value of global scalar variables only by using the variable's name, +However, because the API provides routines for accessing and changing +the value of a global scalar variable only by using the variable's name, there is a performance penalty: `gawk' must find the variable each time it is accessed and changed. This turns out to be a real issue, not just a theoretical one. @@ -23047,17 +23290,19 @@ just a theoretical one. reading and/or changing the value of one or more scalar variables, you can obtain a "scalar cookie"(1) object for that variable, and then use the cookie for getting the variable's value or for changing the -variable's value. This is the `awk_scalar_t' type and `scalar_cookie' -macro. Given a scalar cookie, `gawk' can directly retrieve or modify -the value, as required, without having to find it first. +variable's value. The `awk_scalar_t' type holds a scalar cookie, and +the `scalar_cookie' macro provides access to the value of that type in +the `awk_value_t' struct. Given a scalar cookie, `gawk' can directly +retrieve or modify the value, as required, without having to find it +first. The `awk_value_cookie_t' type and `value_cookie' macro are similar. If you know that you wish to use the same numeric or string _value_ for one or more variables, you can create the value once, retaining a "value cookie" for it, and then pass in that value cookie whenever you -wish to set the value of a variable. This saves both storage space -within the running `gawk' process as well as the time needed to create -the value. +wish to set the value of a variable. This saves storage space within +the running `gawk' process and reduces the time needed to create the +value. ---------- Footnotes ---------- @@ -23076,7 +23321,7 @@ File: gawk.info, Node: Memory Allocation Functions, Next: Constructor Function The API provides a number of "memory allocation" functions for allocating memory that can be passed to `gawk', as well as a number of convenience macros. This node presents them all as function -prototypes, in the way that extension code would use them. +prototypes, in the way that extension code would use them: `void *gawk_malloc(size_t size);' Call the correct version of `malloc()' to allocate storage that may @@ -23092,7 +23337,7 @@ prototypes, in the way that extension code would use them. `void gawk_free(void *ptr);' Call the correct version of `free()' to release storage that was - allocated with `gawk_malloc()', `gawk_calloc()' or + allocated with `gawk_malloc()', `gawk_calloc()', or `gawk_realloc()'. The API has to provide these functions because it is possible for an @@ -23104,7 +23349,7 @@ version of `malloc()', unexpected behavior would likely result. Two convenience macros may be used for allocating storage from `gawk_malloc()' and `gawk_realloc()'. If the allocation fails, they cause `gawk' to exit with a fatal error message. They should be used -as if they were procedure calls that do not return a value. +as if they were procedure calls that do not return a value: `#define emalloc(pointer, type, size, message) ...' The arguments to this macro are as follows: @@ -23113,8 +23358,8 @@ as if they were procedure calls that do not return a value. The pointer variable to point at the allocated storage. `type' - The type of the pointer variable, used to create a cast for - the call to `gawk_malloc()'. + The type of the pointer variable. This is used to create a + cast for the call to `gawk_malloc()'. `size' The total number of bytes to be allocated. @@ -23134,13 +23379,13 @@ as if they were procedure calls that do not return a value. make_malloced_string(message, strlen(message), & result); `#define erealloc(pointer, type, size, message) ...' - This is like `emalloc()', but it calls `gawk_realloc()', instead - of `gawk_malloc()'. The arguments are the same as for the + This is like `emalloc()', but it calls `gawk_realloc()' instead of + `gawk_malloc()'. The arguments are the same as for the `emalloc()' macro. ---------- Footnotes ---------- - (1) This is more common on MS-Windows systems, but can happen on + (1) This is more common on MS-Windows systems, but it can happen on Unix-like systems as well. @@ -23152,32 +23397,32 @@ File: gawk.info, Node: Constructor Functions, Next: Registration Functions, P The API provides a number of "constructor" functions for creating string and numeric values, as well as a number of convenience macros. This node presents them all as function prototypes, in the way that -extension code would use them. +extension code would use them: `static inline awk_value_t *' -`make_const_string(const char *string, size_t length, awk_value_t *result)' +`make_const_string(const char *string, size_t length, awk_value_t *result);' This function creates a string value in the `awk_value_t' variable pointed to by `result'. It expects `string' to be a C string constant (or other string data), and automatically creates a _copy_ of the data for storage in `result'. It returns `result'. `static inline awk_value_t *' -`make_malloced_string(const char *string, size_t length, awk_value_t *result)' +`make_malloced_string(const char *string, size_t length, awk_value_t *result);' This function creates a string value in the `awk_value_t' variable pointed to by `result'. It expects `string' to be a `char *' value pointing to data previously obtained from `gawk_malloc()', - `gawk_calloc()' or `gawk_realloc()'. The idea here is that the + `gawk_calloc()', or `gawk_realloc()'. The idea here is that the data is passed directly to `gawk', which assumes responsibility for it. It returns `result'. `static inline awk_value_t *' -`make_null_string(awk_value_t *result)' +`make_null_string(awk_value_t *result);' This specialized function creates a null string (the "undefined" value) in the `awk_value_t' variable pointed to by `result'. It returns `result'. `static inline awk_value_t *' -`make_number(double num, awk_value_t *result)' +`make_number(double num, awk_value_t *result);' This function simply creates a numeric value in the `awk_value_t' variable pointed to by `result'. @@ -23216,7 +23461,7 @@ Extension functions are described by the following record: The fields are: `const char *name;' - The name of the new function. `awk' level code calls the function + The name of the new function. `awk'-level code calls the function by this name. This is a regular C string. Function names must obey the rules for `awk' identifiers. That is, @@ -23228,8 +23473,8 @@ Extension functions are described by the following record: This is a pointer to the C function that provides the extension's functionality. The function must fill in `*result' with either a number or a string. `gawk' takes ownership of any string memory. - As mentioned earlier, string memory *must* come from one of - `gawk_malloc()', `gawk_calloc()' or `gawk_realloc()'. + As mentioned earlier, string memory _must_ come from one of + `gawk_malloc()', `gawk_calloc()', or `gawk_realloc()'. The `num_actual_args' argument tells the C function how many actual parameters were passed from the calling `awk' code. @@ -23262,7 +23507,7 @@ An "exit callback" function is a function that `gawk' calls before it exits. Such functions are useful if you have general "cleanup" tasks that should be performed in your extension (such as closing database connections or other resource deallocations). You can register such a -function with `gawk' using the following function. +function with `gawk' using the following function: `void awk_atexit(void (*funcp)(void *data, int exit_status),' ` void *arg0);' @@ -23275,10 +23520,10 @@ function with `gawk' using the following function. `gawk' intends to pass to the `exit()' system call. `arg0' - A pointer to private data which `gawk' saves in order to pass + A pointer to private data that `gawk' saves in order to pass to the function pointed to by `funcp'. - Exit callback functions are called in Last-In-First-Out (LIFO) + Exit callback functions are called in last-in, first-out (LIFO) order--that is, in the reverse order in which they are registered with `gawk'. @@ -23288,12 +23533,13 @@ File: gawk.info, Node: Extension Version String, Next: Input Parsers, Prev: E 16.4.5.3 Registering An Extension Version String ................................................ -You can register a version string which indicates the name and version -of your extension, with `gawk', as follows: +You can register a version string that indicates the name and version +of your extension with `gawk', as follows: `void register_ext_version(const char *version);' - Register the string pointed to by `version' with `gawk'. `gawk' - does _not_ copy the `version' string, so it should not be changed. + Register the string pointed to by `version' with `gawk'. Note + that `gawk' does _not_ copy the `version' string, so it should not + be changed. `gawk' prints all registered extension version strings when it is invoked with the `--version' option. @@ -23311,27 +23557,27 @@ Files::). Additionally, it sets the value of `RT' (*note Built-in Variables::). If you want, you can provide your own custom input parser. An input -parser's job is to return a record to the `gawk' record processing +parser's job is to return a record to the `gawk' record-processing code, along with indicators for the value and length of the data to be used for `RT', if any. To provide an input parser, you must first provide two functions (where XXX is a prefix name for your extension): -`awk_bool_t XXX_can_take_file(const awk_input_buf_t *iobuf)' +`awk_bool_t XXX_can_take_file(const awk_input_buf_t *iobuf);' This function examines the information available in `iobuf' (which we discuss shortly). Based on the information there, it decides if the input parser should be used for this file. If so, it should return true. Otherwise, it should return false. It should not change any state (variable values, etc.) within `gawk'. -`awk_bool_t XXX_take_control_of(awk_input_buf_t *iobuf)' +`awk_bool_t XXX_take_control_of(awk_input_buf_t *iobuf);' When `gawk' decides to hand control of the file over to the input parser, it calls this function. This function in turn must fill - in certain fields in the `awk_input_buf_t' structure, and ensure + in certain fields in the `awk_input_buf_t' structure and ensure that certain conditions are true. It should then return true. If - an error of some kind occurs, it should not fill in any fields, - and should return false; then `gawk' will not use the input parser. + an error of some kind occurs, it should not fill in any fields and + should return false; then `gawk' will not use the input parser. The details are presented shortly. Your extension should package these functions inside an @@ -23366,7 +23612,7 @@ used for `RT', if any. 2. When your extension is loaded, register your input parser with `gawk' using the `register_input_parser()' API function (described - below). + next). An `awk_input_buf_t' looks like this: @@ -23404,14 +23650,14 @@ decide if the input parser should be used for the file. The decision can be made based upon `gawk' state (the value of a variable defined previously by the extension and set by `awk' code), the name of the file, whether or not the file descriptor is valid, the information in -the `struct stat', or any combination of the above. +the `struct stat', or any combination of these factors. Once `XXX_can_take_file()' has returned true, and `gawk' has decided to use your input parser, it calls `XXX_take_control_of()'. That -function then fills one of either the `get_record' field or the -`read_func' field in the `awk_input_buf_t'. It must also ensure that -`fd' is _not_ set to `INVALID_HANDLE'. All of the fields that may be -filled by `XXX_take_control_of()' are as follows: +function then fills either the `get_record' field or the `read_func' +field in the `awk_input_buf_t'. It must also ensure that `fd' is _not_ +set to `INVALID_HANDLE'. The following list describes the fields that +may be filled by `XXX_take_control_of()': `void *opaque;' This is used to hold any state information needed by the input @@ -23425,25 +23671,25 @@ filled by `XXX_take_control_of()' are as follows: ` size_t *rt_len);' This function pointer should point to a function that creates the input records. Said function is the core of the input parser. - Its behavior is described below. + Its behavior is described in the text following this list. `ssize_t (*read_func)();' - This function pointer should point to function that has the same + This function pointer should point to a function that has the same behavior as the standard POSIX `read()' system call. It is an alternative to the `get_record' pointer. Its behavior is also - described below. + described in the text following this list. `void (*close_func)(struct awk_input *iobuf);' This function pointer should point to a function that does the - "tear down." It should release any resources allocated by + "teardown." It should release any resources allocated by `XXX_take_control_of()'. It may also close the file. If it does so, it should set the `fd' field to `INVALID_HANDLE'. If `fd' is still not `INVALID_HANDLE' after the call to this function, `gawk' calls the regular `close()' system call. - Having a "tear down" function is optional. If your input parser - does not need it, do not set this field. Then, `gawk' calls the + Having a "teardown" function is optional. If your input parser does + not need it, do not set this field. Then, `gawk' calls the regular `close()' system call on the file descriptor, so it should be valid. @@ -23451,7 +23697,7 @@ filled by `XXX_take_control_of()' are as follows: records. The parameters are as follows: `char **out' - This is a pointer to a `char *' variable which is set to point to + This is a pointer to a `char *' variable that is set to point to the record. `gawk' makes its own copy of the data, so the extension must manage this storage. @@ -23500,16 +23746,16 @@ explicitly. NOTE: You must choose one method or the other: either a function that returns a record, or one that returns raw data. In particular, if you supply a function to get a record, `gawk' will - call it, and never call the raw read function. + call it, and will never call the raw read function. `gawk' ships with a sample extension that reads directories, -returning records for each entry in the directory (*note Extension -Sample Readdir::). You may wish to use that code as a guide for writing -your own input parser. +returning records for each entry in a directory (*note Extension Sample +Readdir::). You may wish to use that code as a guide for writing your +own input parser. When writing an input parser, you should think about (and document) how it is expected to interact with `awk' code. You may want it to -always be called, and take effect as appropriate (as the `readdir' +always be called, and to take effect as appropriate (as the `readdir' extension does). Or you may want it to take effect based upon the value of an `awk' variable, as the XML extension from the `gawkextlib' project does (*note gawkextlib::). In the latter case, code in a @@ -23556,8 +23802,8 @@ an extension to take over the output to a file opened with the `>' or The function pointed to by this field is called when `gawk' decides to let the output wrapper take control of the file. It should fill in appropriate members of the `awk_output_buf_t' - structure, as described below, and return true if successful, - false otherwise. + structure, as described next, and return true if successful, false + otherwise. `awk_const struct output_wrapper *awk_const next;' This is for use by `gawk'; therefore it is marked `awk_const' so @@ -23609,17 +23855,17 @@ in the `awk_output_buf_t'. The data members are as follows: These pointers should be set to point to functions that perform the equivalent function as the `<stdio.h>' functions do, if appropriate. `gawk' uses these function pointers for all output. - `gawk' initializes the pointers to point to internal, "pass - through" functions that just call the regular `<stdio.h>' - functions, so an extension only needs to redefine those functions - that are appropriate for what it does. + `gawk' initializes the pointers to point to internal "pass-through" + functions that just call the regular `<stdio.h>' functions, so an + extension only needs to redefine those functions that are + appropriate for what it does. The `XXX_can_take_file()' function should make a decision based upon the `name' and `mode' fields, and any additional state (such as `awk' variable values) that is appropriate. When `gawk' calls `XXX_take_control_of()', that function should fill -in the other fields, as appropriate, except for `fp', which it should +in the other fields as appropriate, except for `fp', which it should just use normally. You register your output wrapper with the following function: @@ -23656,16 +23902,17 @@ structures as described earlier. The name of the two-way processor. `awk_bool_t (*can_take_two_way)(const char *name);' - This function returns true if it wants to take over two-way I/O - for this file name. It should not change any state (variable - values, etc.) within `gawk'. + The function pointed to by this field should return true if it + wants to take over two-way I/O for this file name. It should not + change any state (variable values, etc.) within `gawk'. `awk_bool_t (*take_control_of)(const char *name,' ` awk_input_buf_t *inbuf,' ` awk_output_buf_t *outbuf);' - This function should fill in the `awk_input_buf_t' and - `awk_outut_buf_t' structures pointed to by `inbuf' and `outbuf', - respectively. These structures were described earlier. + The function pointed to by this field should fill in the + `awk_input_buf_t' and `awk_outut_buf_t' structures pointed to by + `inbuf' and `outbuf', respectively. These structures were + described earlier. `awk_const struct two_way_processor *awk_const next;' This is for use by `gawk'; therefore it is marked `awk_const' so @@ -23688,8 +23935,8 @@ File: gawk.info, Node: Printing Messages, Next: Updating `ERRNO', Prev: Regis ------------------------ You can print different kinds of warning messages from your extension, -as described below. Note that for these functions, you must pass in -the extension id received from `gawk' when the extension was loaded.(1) +as described here. Note that for these functions, you must pass in the +extension ID received from `gawk' when the extension was loaded:(1) `void fatal(awk_ext_id_t id, const char *format, ...);' Print a message and then cause `gawk' to exit immediately. @@ -23745,26 +23992,26 @@ value you expect. If the actual value matches what you requested, the function returns true and fills in the `awk_value_t' result. Otherwise, the function returns false, and the `val_type' member indicates the type of the actual value. You may then print an error -message, or reissue the request for the actual value type, as +message or reissue the request for the actual value type, as appropriate. This behavior is summarized in *note table-value-types-returned::. - Type of Actual Value: + Type of Actual Value -------------------------------------------------------------------------- String Number Array Undefined ------------------------------------------------------------------------------ - String String String false false - Number Number if can Number false false + String String String False False + Number Number if can Number False False be converted, else false -Type Array false false Array false -Requested: Scalar Scalar Scalar false false +Type Array False False Array False +Requested Scalar Scalar Scalar False False Undefined String Number Array Undefined - Value false false false false - Cookie + Value False False False False + cookie -Table 16.1: API Value Types Returned +Table 16.1: API value types returned File: gawk.info, Node: Accessing Parameters, Next: Symbol Table Access, Prev: Requesting Values, Up: Extension API Description @@ -23779,16 +24026,16 @@ your extension function. They are: ` awk_valtype_t wanted,' ` awk_value_t *result);' Fill in the `awk_value_t' structure pointed to by `result' with - the `count''th argument. Return true if the actual type matches - `wanted', false otherwise. In the latter case, `result->val_type' - indicates the actual type (*note Table 16.1: - table-value-types-returned.). Counts are zero based--the first + the `count'th argument. Return true if the actual type matches + `wanted', and false otherwise. In the latter case, + `result->val_type' indicates the actual type (*note Table 16.1: + table-value-types-returned.). Counts are zero-based--the first argument is numbered zero, the second one, and so on. `wanted' indicates the type of value expected. `awk_bool_t set_argument(size_t count, awk_array_t array);' Convert a parameter that was undefined into an array; this provides - call-by-reference for arrays. Return false if `count' is too big, + call by reference for arrays. Return false if `count' is too big, or if the argument's type is not undefined. *Note Array Manipulation::, for more information on creating arrays. @@ -23816,8 +24063,8 @@ File: gawk.info, Node: Symbol table by name, Next: Symbol table by cookie, Up The following routines provide the ability to access and update global `awk'-level variables by name. In compiler terminology, identifiers of different kinds are termed "symbols", thus the "sym" in the routines' -names. The data structure which stores information about symbols is -termed a "symbol table". +names. The data structure that stores information about symbols is +termed a "symbol table". The functions are as follows: `awk_bool_t sym_lookup(const char *name,' ` awk_valtype_t wanted,' @@ -23825,14 +24072,14 @@ termed a "symbol table". Fill in the `awk_value_t' structure pointed to by `result' with the value of the variable named by the string `name', which is a regular C string. `wanted' indicates the type of value expected. - Return true if the actual type matches `wanted', false otherwise. - In the latter case, `result->val_type' indicates the actual type - (*note Table 16.1: table-value-types-returned.). + Return true if the actual type matches `wanted', and false + otherwise. In the latter case, `result->val_type' indicates the + actual type (*note Table 16.1: table-value-types-returned.). `awk_bool_t sym_update(const char *name, awk_value_t *value);' Update the variable named by the string `name', which is a regular C string. The variable is added to `gawk''s symbol table if it is - not there. Return true if everything worked, false otherwise. + not there. Return true if everything worked, and false otherwise. Changing types (scalar to array or vice versa) of an existing variable is _not_ allowed, nor may this routine be used to update @@ -23845,8 +24092,8 @@ cannot change any of those variables. CAUTION: It is possible for the lookup of `PROCINFO' to fail. This happens if the `awk' program being run does not reference - `PROCINFO'; in this case `gawk' doesn't bother to create the array - and populate it. + `PROCINFO'; in this case, `gawk' doesn't bother to create the + array and populate it. File: gawk.info, Node: Symbol table by cookie, Next: Cached values, Prev: Symbol table by name, Up: Symbol Table Access @@ -23859,7 +24106,7 @@ variable or array. It is an optimization that avoids looking up variables in `gawk''s symbol table every time access is needed. This was discussed earlier, in *note General Data Types::. - The following functions let you work with scalar cookies. + The following functions let you work with scalar cookies: `awk_bool_t sym_lookup_scalar(awk_scalar_t cookie,' ` awk_valtype_t wanted,' @@ -23900,7 +24147,7 @@ variable based on the result of that evaluation, like so: This code looks (and is) simple and straightforward. So what's the problem? - Consider what happens if `awk'-level code associated with your + Well, consider what happens if `awk'-level code associated with your extension calls the `magic()' function (implemented in C by `do_magic()'), once per record, while processing hundreds of thousands or millions of records. The `MAGIC_VAR' variable is looked up in the @@ -23968,14 +24215,14 @@ File: gawk.info, Node: Cached values, Prev: Symbol table by cookie, Up: Symbo .......................................... The routines in this section allow you to create and release cached -values. As with scalar cookies, in theory, cached values are not +values. Like scalar cookies, in theory, cached values are not necessary. You can create numbers and strings using the functions in *note Constructor Functions::. You can then assign those values to variables using `sym_update()' or `sym_update_scalar()', as you like. However, you can understand the point of cached values if you remember that _every_ string value's storage _must_ come from -`gawk_malloc()', `gawk_calloc()' or `gawk_realloc()'. If you have 20 +`gawk_malloc()', `gawk_calloc()', or `gawk_realloc()'. If you have 20 variables, all of which have the same string value, you must create 20 identical copies of the string.(1) @@ -23987,8 +24234,8 @@ follows: `awk_bool_t create_value(awk_value_t *value, awk_value_cookie_t *result);' Create a cached string or numeric value from `value' for efficient later assignment. Only values of type `AWK_NUMBER' and - `AWK_STRING' are allowed. Any other type is rejected. While - `AWK_UNDEFINED' could be allowed, doing so would result in + `AWK_STRING' are allowed. Any other type is rejected. + `AWK_UNDEFINED' could be allowed, but doing so would result in inferior performance. `awk_bool_t release_value(awk_value_cookie_t vc);' @@ -24035,11 +24282,11 @@ of variables: ... } -Using value cookies in this way saves considerable storage, since all of +Using value cookies in this way saves considerable storage, as all of `VAR1' through `VAR100' share the same value. You might be wondering, "Is this sharing problematic? What happens -if `awk' code assigns a new value to `VAR1', are all the others changed +if `awk' code assigns a new value to `VAR1'; are all the others changed too?" That's a great question. The answer is that no, it's not a problem. @@ -24059,7 +24306,7 @@ using `release_value()'. `double' to store. -File: gawk.info, Node: Array Manipulation, Next: Extension API Variables, Prev: Symbol Table Access, Up: Extension API Description +File: gawk.info, Node: Array Manipulation, Next: Redirection API, Prev: Symbol Table Access, Up: Extension API Description 16.4.11 Array Manipulation -------------------------- @@ -24083,7 +24330,7 @@ arrays of arrays (*note General Data Types::). ---------- Footnotes ---------- - (1) Okay, the only data structure. + (1) OK, the only data structure. File: gawk.info, Node: Array Data Types, Next: Array Functions, Up: Array Manipulation @@ -24091,7 +24338,7 @@ File: gawk.info, Node: Array Data Types, Next: Array Functions, Up: Array Man 16.4.11.1 Array Data Types .......................... -The data types associated with arrays are listed below. +The data types associated with arrays are as follows: `typedef void *awk_array_t;' If you request the value of an array variable, you get back an @@ -24158,7 +24405,7 @@ File: gawk.info, Node: Array Functions, Next: Flattening Arrays, Prev: Array 16.4.11.2 Array Functions ......................... -The following functions relate to individual array elements. +The following functions relate to individual array elements: `awk_bool_t get_element_count(awk_array_t a_cookie, size_t *count);' For the array represented by `a_cookie', place in `*count' the @@ -24176,13 +24423,14 @@ The following functions relate to individual array elements. (*note Table 16.1: table-value-types-returned.). The value for `index' can be numeric, in which case `gawk' - converts it to a string. Using non-integral values is possible, but + converts it to a string. Using nonintegral values is possible, but requires that you understand how such values are converted to - strings (*note Conversion::); thus using integral values is safest. + strings (*note Conversion::); thus, using integral values is + safest. As with _all_ strings passed into `gawk' from an extension, the string value of `index' must come from `gawk_malloc()', - `gawk_calloc()' or `gawk_realloc()', and `gawk' releases the + `gawk_calloc()', or `gawk_realloc()', and `gawk' releases the storage. `awk_bool_t set_array_element(awk_array_t a_cookie,' @@ -24226,9 +24474,9 @@ The following functions relate to individual array elements. `awk_bool_t release_flattened_array(awk_array_t a_cookie,' ` awk_flat_array_t *data);' When done with a flattened array, release the storage using this - function. You must pass in both the original array cookie, and - the address of the created `awk_flat_array_t' structure. The - function returns true upon success, false otherwise. + function. You must pass in both the original array cookie and the + address of the created `awk_flat_array_t' structure. The function + returns true upon success, false otherwise. File: gawk.info, Node: Flattening Arrays, Next: Creating Arrays, Prev: Array Functions, Up: Array Manipulation @@ -24236,10 +24484,10 @@ File: gawk.info, Node: Flattening Arrays, Next: Creating Arrays, Prev: Array 16.4.11.3 Working With All The Elements of an Array ................................................... -To "flatten" an array is create a structure that represents the full +To "flatten" an array is to create a structure that represents the full array in a fashion that makes it easy for C code to traverse the entire -array. Test code in `extension/testext.c' does this, and also serves -as a nice example showing how to use the APIs. +array. Some of the code in `extension/testext.c' does this, and also +serves as a nice example showing how to use the APIs. We walk through that part of the code one step at a time. First, the `gawk' script that drives the test extension: @@ -24288,9 +24536,8 @@ number of arguments: } The function then proceeds in steps, as follows. First, retrieve the -name of the array, passed as the first argument. Then retrieve the -array itself. If either operation fails, print error messages and -return: +name of the array, passed as the first argument, followed by the array +itself. If either operation fails, print an error message and return: /* get argument named array as flat array and print it */ if (get_argument(0, AWK_STRING, & value)) { @@ -24320,9 +24567,9 @@ count of elements in the array and print it: printf("dump_array_and_delete: incoming size is %lu\n", (unsigned long) count); - The third step is to actually flatten the array, and then to double -check that the count in the `awk_flat_array_t' is the same as the count -just retrieved: + The third step is to actually flatten the array, and then to +double-check that the count in the `awk_flat_array_t' is the same as +the count just retrieved: if (! flatten_array(value2.array_cookie, & flat_array)) { printf("dump_array_and_delete: could not flatten array\n"); @@ -24339,7 +24586,7 @@ just retrieved: The fourth step is to retrieve the index of the element to be deleted, which was passed as the second argument. Remember that -argument counts passed to `get_argument()' are zero-based, thus the +argument counts passed to `get_argument()' are zero-based, and thus the second argument is numbered one: if (! get_argument(1, AWK_STRING, & value3)) { @@ -24352,7 +24599,7 @@ over every element in the array, printing the index and element values. In addition, upon finding the element with the index that is supposed to be deleted, the function sets the `AWK_ELEMENT_DELETE' bit in the `flags' field of the element. When the array is released, `gawk' -traverses the flattened array, and deletes any elements which have this +traverses the flattened array, and deletes any elements that have this flag bit set: for (i = 0; i < flat_array->count; i++) { @@ -24382,7 +24629,7 @@ this code) once you have called `release_flattened_array()': goto out; } - Finally, since everything was successful, the function sets the + Finally, because everything was successful, the function sets the return value to success, and returns: make_number(1.0, result); @@ -24417,7 +24664,7 @@ them and manipulate them. There are two important points about creating arrays from extension code: - 1. You must install a new array into `gawk''s symbol table + * You must install a new array into `gawk''s symbol table immediately upon creating it. Once you have done so, you can then populate the array. @@ -24431,7 +24678,7 @@ code: previously existing array using `set_array_element()'. We show example code shortly. - 2. Due to gawk internals, after using `sym_update()' to install an + * Due to `gawk' internals, after using `sym_update()' to install an array into `gawk', you have to retrieve the array cookie from the value passed in to `sym_update()' before doing anything else with it, like so: @@ -24515,7 +24762,7 @@ Note how `a_cookie' is reset from the `array_cookie' field in the } } - Here is sample script that loads the extension and then dumps the + Here is a sample script that loads the extension and then dumps the array: @load "subarray" @@ -24544,9 +24791,78 @@ array: environment variable.) -File: gawk.info, Node: Extension API Variables, Next: Extension API Boilerplate, Prev: Array Manipulation, Up: Extension API Description +File: gawk.info, Node: Redirection API, Next: Extension API Variables, Prev: Array Manipulation, Up: Extension API Description + +16.4.12 Accessing and Manipulating Redirections +----------------------------------------------- + +The following function allows extensions to access and manipulate +redirections. + +`awk_bool_t get_file(const char *name,' +` size_t name_len,' +` const char *filetype,' +` int fd,' +` const awk_input_buf_t **ibufp,' +` const awk_output_buf_t **obufp);' + Look up a file in `gawk''s internal redirection table. If `name' + is `NULL' or `name_len' is zero, return data for the currently + open input file corresponding to `FILENAME'. (This does not + access the `filetype' argument, so that may be undefined). If the + file is not already open, attempt to open it. The `filetype' + argument must be zero-terminated and should be one of: + + `">"' + A file opened for output. + + `">>"' + A file opened for append. + + `"<"' + A file opened for input. + + `"|>"' + A pipe opened for output. + + `"|<"' + A pipe opened for input. + + `"|&"' + A two-way coprocess. + + On error, return a `false' value. Otherwise, return `true', and + return additional information about the redirection in the `ibufp' + and `obufp' pointers. For input redirections, the `*ibufp' value + should be non-`NULL', and `*obufp' should be `NULL'. For output + redirections, the `*obufp' value should be non-`NULL', and `*ibufp' + should be `NULL'. For two-way coprocesses, both values should be + non-`NULL'. + + In the usual case, the extension is interested in `(*ibufp)->fd' + and/or `fileno((*obufp)->fp)'. If the file is not already open, + and the `fd' argument is non-negative, `gawk' will use that file + descriptor instead of opening the file in the usual way. If `fd' + is non-negative, but the file exists already, `gawk' ignores `fd' + and returns the existing file. It is the caller's responsibility + to notice that neither the `fd' in the returned `awk_input_buf_t' + nor the `fd' in the returned `awk_output_buf_t' matches the + requested value. + + Note that supplying a file descriptor is currently _not_ supported + for pipes. However, supplying a file descriptor should work for + input, output, append, and two-way (coprocess) sockets. If + `filetype' is two-way, `gawk' assumes that it is a socket! Note + that in the two-way case, the input and output file descriptors + may differ. To check for success, you must check whether either + matches. + + It is anticipated that this API function will be used to implement +I/O multiplexing and a socket library. + + +File: gawk.info, Node: Extension API Variables, Next: Extension API Boilerplate, Prev: Redirection API, Up: Extension API Description -16.4.12 API Variables +16.4.13 API Variables --------------------- The API provides two sets of variables. The first provides information @@ -24563,17 +24879,17 @@ information about how `gawk' was invoked. File: gawk.info, Node: Extension Versioning, Next: Extension API Informational Variables, Up: Extension API Variables -16.4.12.1 API Version Constants and Variables +16.4.13.1 API Version Constants and Variables ............................................. The API provides both a "major" and a "minor" version number. The API versions are available at compile time as constants: `GAWK_API_MAJOR_VERSION' - The major version of the API. + The major version of the API `GAWK_API_MINOR_VERSION' - The minor version of the API. + The minor version of the API The minor version increases when new functions are added to the API. Such new functions are always added to the end of the API `struct'. @@ -24588,13 +24904,13 @@ For this reason, the major and minor API versions of the running `gawk' are included in the API `struct' as read-only constant integers: `api->major_version' - The major version of the running `gawk'. + The major version of the running `gawk' `api->minor_version' - The minor version of the running `gawk'. + The minor version of the running `gawk' It is up to the extension to decide if there are API -incompatibilities. Typically a check like this is enough: +incompatibilities. Typically, a check like this is enough: if (api->major_version != GAWK_API_MAJOR_VERSION || api->minor_version < GAWK_API_MINOR_VERSION) { @@ -24606,13 +24922,13 @@ incompatibilities. Typically a check like this is enough: } Such code is included in the boilerplate `dl_load_func()' macro -provided in `gawkapi.h' (discussed later, in *note Extension API +provided in `gawkapi.h' (discussed in *note Extension API Boilerplate::). File: gawk.info, Node: Extension API Informational Variables, Prev: Extension Versioning, Up: Extension API Variables -16.4.12.2 Informational Variables +16.4.13.2 Informational Variables ................................. The API provides access to several variables that describe whether the @@ -24623,8 +24939,7 @@ invoked. The variables are: This variable is true if `gawk' was invoked with `--debug' option. `do_lint' - This variable is true if `gawk' was invoked with `--lint' option - (*note Options::). + This variable is true if `gawk' was invoked with `--lint' option. `do_mpfr' This variable is true if `gawk' was invoked with `--bignum' option. @@ -24648,17 +24963,17 @@ not change during execution. File: gawk.info, Node: Extension API Boilerplate, Prev: Extension API Variables, Up: Extension API Description -16.4.13 Boilerplate Code +16.4.14 Boilerplate Code ------------------------ As mentioned earlier (*note Extension Mechanism Outline::), the function definitions as presented are really macros. To use these macros, your extension must provide a small amount of boilerplate code (variables and -functions) towards the top of your source file, using pre-defined names -as described below. The boilerplate needed is also provided in comments +functions) toward the top of your source file, using predefined names +as described here. The boilerplate needed is also provided in comments in the `gawkapi.h' header file: - /* Boiler plate code: */ + /* Boilerplate code: */ int plugin_is_GPL_compatible; static gawk_api_t *const api; @@ -24708,7 +25023,7 @@ in the `gawkapi.h' header file: to point to a string giving the name and version of your extension. `static awk_ext_func_t func_table[] = { ... };' - This is an array of one or more `awk_ext_func_t' structures as + This is an array of one or more `awk_ext_func_t' structures, as described earlier (*note Extension Functions::). It can then be looped over for multiple calls to `add_ext_func()'. @@ -24729,9 +25044,9 @@ in the `gawkapi.h' header file: This macro expands to a `dl_load()' function that performs all the necessary initializations. - The point of the all the variables and arrays is to let the -`dl_load()' function (from the `dl_load_func()' macro) do all the -standard work. It does the following: + The point of all the variables and arrays is to let the `dl_load()' +function (from the `dl_load_func()' macro) do all the standard work. It +does the following: 1. Check the API versions. If the extension major version does not match `gawk''s, or if the extension minor version is greater than @@ -24765,7 +25080,7 @@ File: gawk.info, Node: Extension Example, Next: Extension Samples, Prev: Find 16.6 Example: Some File Functions ================================= - No matter where you go, there you are. -- Buckaroo Bonzai + No matter where you go, there you are. -- Buckaroo Banzai Two useful functions that are not in `awk' are `chdir()' (so that an `awk' program can change its directory) and `stat()' (so that an `awk' @@ -24821,7 +25136,7 @@ appropriate information: `stat()' fails. It fills in the following elements: `"name"' - The name of the file that was `stat()''ed. + The name of the file that was `stat()'ed. `"dev"' `"ino"' @@ -24869,7 +25184,7 @@ appropriate information: The file is a directory. `"fifo"' - The file is a named-pipe (also known as a FIFO). + The file is a named pipe (also known as a FIFO). `"file"' The file is just a regular file. @@ -24889,7 +25204,7 @@ appropriate information: systems, "a priori" knowledge is used to provide a value. Where no value can be determined, it defaults to 512. - Several additional elements may be present depending upon the + Several additional elements may be present, depending upon the operating system and the type of the file. You can test for them in your `awk' program by using the `in' operator (*note Reference to Elements::): @@ -24918,10 +25233,10 @@ File: gawk.info, Node: Internal File Ops, Next: Using Internal File Ops, Prev Here is the C code for these extensions.(1) The file includes a number of standard header files, and then -includes the `gawkapi.h' header file which provides the API definitions. -Those are followed by the necessary variable declarations to make use -of the API macros and boilerplate code (*note Extension API -Boilerplate::). +includes the `gawkapi.h' header file, which provides the API +definitions. Those are followed by the necessary variable declarations +to make use of the API macros and boilerplate code (*note Extension API +Boilerplate::): #ifdef HAVE_CONFIG_H #include <config.h> @@ -24956,9 +25271,9 @@ Boilerplate::). By convention, for an `awk' function `foo()', the C function that implements it is called `do_foo()'. The function should have two -arguments: the first is an `int' usually called `nargs', that +arguments. The first is an `int', usually called `nargs', that represents the number of actual arguments for the function. The second -is a pointer to an `awk_value_t', usually named `result'. +is a pointer to an `awk_value_t' structure, usually named `result': /* do_chdir --- provide dynamically loaded chdir() function for gawk */ @@ -24976,11 +25291,11 @@ is a pointer to an `awk_value_t', usually named `result'. "expecting 1")); The `newdir' variable represents the new directory to change to, -retrieved with `get_argument()'. Note that the first argument is -numbered zero. +which is retrieved with `get_argument()'. Note that the first argument +is numbered zero. If the argument is retrieved successfully, the function calls the -`chdir()' system call. If the `chdir()' fails, `ERRNO' is updated. +`chdir()' system call. If the `chdir()' fails, `ERRNO' is updated: if (get_argument(0, AWK_STRING, & newdir)) { ret = chdir(newdir.str_value.str); @@ -24994,8 +25309,8 @@ numbered zero. } The `stat()' extension is more involved. First comes a function -that turns a numeric mode into a printable representation (e.g., 644 -becomes `-rw-r--r--'). This is omitted here for brevity: +that turns a numeric mode into a printable representation (e.g., octal +`0644' becomes `-rw-r--r--'). This is omitted here for brevity: /* format_mode --- turn a stat mode field into something readable */ @@ -25045,8 +25360,8 @@ contain the result of the `stat()': The following function does most of the work to fill in the `awk_array_t' result array with values obtained from a valid `struct -stat'. It is done in a separate function to support the `stat()' -function for `gawk' and also to support the `fts()' extension which is +stat'. This work is done in a separate function to support the `stat()' +function for `gawk' and also to support the `fts()' extension, which is included in the same file but whose code is not shown here (*note Extension Sample File Functions::). @@ -25158,10 +25473,10 @@ argument is optional. If present, it causes `do_stat()' to use the `stat()' system call instead of the `lstat()' system call. This is done by using a function pointer: `statfunc'. `statfunc' is initialized to point to `lstat()' (instead of `stat()') to get the file -information, in case the file is a symbolic link. However, if there -were three arguments, `statfunc' is set point to `stat()', instead. +information, in case the file is a symbolic link. However, if the third +argument is included, `statfunc' is set to point to `stat()', instead. - Here is the `do_stat()' function. It starts with variable + Here is the `do_stat()' function, which starts with variable declarations and argument checking: /* do_stat --- provide a stat() function for gawk */ @@ -25208,7 +25523,7 @@ returns: /* always empty out the array */ clear_array(array); - /* stat the file, if error, set ERRNO and return */ + /* stat the file; if error, set ERRNO and return */ ret = statfunc(name, & sbuf); if (ret < 0) { update_ERRNO_int(errno); @@ -25227,7 +25542,8 @@ When done, the function returns the result from `fill_stat_array()': function(s) into `gawk'. The `filefuncs' extension also provides an `fts()' function, which -we omit here. For its sake there is an initialization function: +we omit here (*note Extension Sample File Functions::). For its sake, +there is an initialization function: /* init_filefuncs --- initialization routine */ @@ -25267,7 +25583,7 @@ version. File: gawk.info, Node: Using Internal File Ops, Prev: Internal File Ops, Up: Extension Example -16.6.3 Integrating The Extensions +16.6.3 Integrating the Extensions --------------------------------- Now that the code is written, it must be possible to add it at runtime @@ -25279,7 +25595,7 @@ create a GNU/Linux shared library: $ gcc -fPIC -shared -DHAVE_CONFIG_H -c -O -g -IIDIR filefuncs.c $ gcc -o filefuncs.so -shared filefuncs.o - Once the library exists, it is loaded by using the `@load' keyword. + Once the library exists, it is loaded by using the `@load' keyword: # file testff.awk @load "filefuncs" @@ -25340,21 +25656,21 @@ directory and run the program: ---------- Footnotes ---------- - (1) In practice, you would probably want to use the GNU -Autotools--Automake, Autoconf, Libtool, and `gettext'--to configure and -build your libraries. Instructions for doing so are beyond the scope of -this Info file. *Note gawkextlib::, for Internet links to the tools. + (1) In practice, you would probably want to use the GNU Autotools +(Automake, Autoconf, Libtool, and `gettext') to configure and build +your libraries. Instructions for doing so are beyond the scope of this +Info file. *Note gawkextlib::, for Internet links to the tools. File: gawk.info, Node: Extension Samples, Next: gawkextlib, Prev: Extension Example, Up: Dynamic Extensions -16.7 The Sample Extensions In The `gawk' Distribution +16.7 The Sample Extensions in the `gawk' Distribution ===================================================== -This minor node provides brief overviews of the sample extensions that +This minor node provides a brief overview of the sample extensions that come in the `gawk' distribution. Some of them are intended for -production use, such the `filefuncs', `readdir' and `inplace' -extensions. Others mainly provide example code that shows how to use +production use (e.g., the `filefuncs', `readdir', and `inplace' +extensions). Others mainly provide example code that shows how to use the extension API. * Menu: @@ -25378,11 +25694,11 @@ the extension API. File: gawk.info, Node: Extension Sample File Functions, Next: Extension Sample Fnmatch, Up: Extension Samples -16.7.1 File Related Functions +16.7.1 File-Related Functions ----------------------------- The `filefuncs' extension provides three different functions, as -follows: The usage is: +follows. The usage is: `@load "filefuncs"' This is how you load the extension. @@ -25390,13 +25706,13 @@ follows: The usage is: `result = chdir("/some/directory")' The `chdir()' function is a direct hook to the `chdir()' system call to change the current directory. It returns zero upon - success or less than zero upon error. In the latter case it - updates `ERRNO'. + success or a value less than zero upon error. In the latter case, + it updates `ERRNO'. `result = stat("/some/path", statdata' [`, follow']`)' The `stat()' function provides a hook into the `stat()' system - call. It returns zero upon success or less than zero upon error. - In the latter case it updates `ERRNO'. + call. It returns zero upon success or a value less than zero upon + error. In the latter case, it updates `ERRNO'. By default, it uses the `lstat()' system call. However, if passed a third argument, it uses `stat()' instead. @@ -25423,25 +25739,25 @@ follows: The usage is: `"minor"' `st_minor' Device files `"blksize"'`st_blksize' All `"pmode"' A human-readable version of the All - mode value, such as printed by - `ls'. For example, - `"-rwxr-xr-x"' + mode value, like that printed by + `ls' (for example, + `"-rwxr-xr-x"') `"linkval"'The value of the symbolic link Symbolic links - `"type"' The type of the file as a string. All - One of `"file"', `"blockdev"', - `"chardev"', `"directory"', - `"socket"', `"fifo"', `"symlink"', - `"door"', or `"unknown"'. Not - all systems support all file - types. + `"type"' The type of the file as a All + string--one of `"file"', + `"blockdev"', `"chardev"', + `"directory"', `"socket"', + `"fifo"', `"symlink"', `"door"', + or `"unknown"' (not all systems + support all file types) `flags = or(FTS_PHYSICAL, ...)' `result = fts(pathlist, flags, filedata)' Walk the file trees provided in `pathlist' and fill in the - `filedata' array as described below. `flags' is the bitwise OR of - several predefined values, also described below. Return zero if - there were no errors, otherwise return -1. + `filedata' array, as described next. `flags' is the bitwise OR of + several predefined values, also described in a moment. Return + zero if there were no errors, otherwise return -1. The `fts()' function provides a hook to the C library `fts()' routines for traversing file hierarchies. Instead of returning data @@ -25485,17 +25801,18 @@ requested hierarchies. By default, the C library `fts()' routines do not return entries for `.' (dot) and `..' (dot-dot). This option causes entries for dot-dot to also be included. (The extension - always includes an entry for dot, see below.) + always includes an entry for dot; more on this in a moment.) `FTS_XDEV' During a traversal, do not cross onto a different mounted filesystem. `filedata' - The `filedata' array is first cleared. Then, `fts()' creates an - element in `filedata' for every element in `pathlist'. The index - is the name of the directory or file given in `pathlist'. The - element for this index is itself an array. There are two cases. + The `filedata' array holds the results. `fts()' first clears it. + Then it creates an element in `filedata' for every element in + `pathlist'. The index is the name of the directory or file given + in `pathlist'. The element for this index is itself an array. + There are two cases: _The path is a file_ In this case, the array contains two or three elements: @@ -25518,10 +25835,10 @@ requested hierarchies. _The path is a directory_ In this case, the array contains one element for each entry - in the directory. If an entry is a file, that element is as - for files, just described. If the entry is a directory, that - element is (recursively), an array describing the - subdirectory. If `FTS_SEEDOT' was provided in the flags, + in the directory. If an entry is a file, that element is the + same as for files, just described. If the entry is a + directory, that element is (recursively) an array describing + the subdirectory. If `FTS_SEEDOT' was provided in the flags, then there will also be an element named `".."'. This element will be an array containing the data as provided by `stat()'. @@ -25531,14 +25848,14 @@ requested hierarchies. elements as for a file: `"path"', `"stat"', and `"error"'. The `fts()' function returns zero if there were no errors. -Otherwise it returns -1. +Otherwise, it returns -1. NOTE: The `fts()' extension does not exactly mimic the interface of the C library `fts()' routines, choosing instead to provide an interface that is based on associative arrays, which is more comfortable to use from an `awk' program. This includes the lack - of a comparison function, since `gawk' already provides powerful - array sorting facilities. While an `fts_read()'-like interface + of a comparison function, because `gawk' already provides powerful + array sorting facilities. Although an `fts_read()'-like interface could have been provided, this felt less natural than simply creating a multidimensional array to represent the file hierarchy and its information. @@ -25549,7 +25866,7 @@ the `fts()' extension function. File: gawk.info, Node: Extension Sample Fnmatch, Next: Extension Sample Fork, Prev: Extension Sample File Functions, Up: Extension Samples -16.7.2 Interface To `fnmatch()' +16.7.2 Interface to `fnmatch()' ------------------------------- This extension provides an interface to the C library `fnmatch()' @@ -25560,25 +25877,26 @@ function. The usage is: `result = fnmatch(pattern, string, flags)' The return value is zero on success, `FNM_NOMATCH' if the string - did not match the pattern, or a different non-zero value if an + did not match the pattern, or a different nonzero value if an error occurred. - Besides the `fnmatch()' function, the `fnmatch' extension adds one -constant (`FNM_NOMATCH'), and an array of flag values named `FNM'. + In addition to the `fnmatch()' function, the `fnmatch' extension +adds one constant (`FNM_NOMATCH'), and an array of flag values named +`FNM'. The arguments to `fnmatch()' are: `pattern' - The file name wildcard to match. + The file name wildcard to match `string' - The file name string. + The file name string `flag' Either zero, or the bitwise OR of one or more of the flags in the - `FNM' array. + `FNM' array - The flags are follows: + The flags are as follows: Array element Corresponding flag defined by `fnmatch()' -------------------------------------------------------------------------- @@ -25600,23 +25918,23 @@ Array element Corresponding flag defined by `fnmatch()' File: gawk.info, Node: Extension Sample Fork, Next: Extension Sample Inplace, Prev: Extension Sample Fnmatch, Up: Extension Samples -16.7.3 Interface To `fork()', `wait()' and `waitpid()' ------------------------------------------------------- +16.7.3 Interface to `fork()', `wait()', and `waitpid()' +------------------------------------------------------- -The `fork' extension adds three functions, as follows. +The `fork' extension adds three functions, as follows: `@load "fork"' This is how you load the extension. `pid = fork()' This function creates a new process. The return value is zero in - the child and the process-ID number of the child in the parent, or + the child and the process ID number of the child in the parent, or -1 upon error. In the latter case, `ERRNO' indicates the problem. In the child, `PROCINFO["pid"]' and `PROCINFO["ppid"]' are updated to reflect the correct values. `ret = waitpid(pid)' - This function takes a numeric argument, which is the process-ID to + This function takes a numeric argument, which is the process ID to wait for. The return value is that of the `waitpid()' system call. `ret = wait()' @@ -25640,8 +25958,8 @@ File: gawk.info, Node: Extension Sample Inplace, Next: Extension Sample Ord, 16.7.4 Enabling In-Place File Editing ------------------------------------- -The `inplace' extension emulates GNU `sed''s `-i' option which performs -"in place" editing of each input file. It uses the bundled +The `inplace' extension emulates GNU `sed''s `-i' option, which +performs "in-place" editing of each input file. It uses the bundled `inplace.awk' include file to invoke the extension properly: # inplace --- load and invoke the inplace extension. @@ -25651,11 +25969,16 @@ The `inplace' extension emulates GNU `sed''s `-i' option which performs # Please set INPLACE_SUFFIX to make a backup copy. For example, you may # want to set INPLACE_SUFFIX to .bak on the command line or in a BEGIN rule. + # N.B. We call inplace_end() in the BEGINFILE and END rules so that any + # actions in an ENDFILE rule will be redirected as expected. + BEGINFILE { - inplace_begin(FILENAME, INPLACE_SUFFIX) + if (_inplace_filename != "") + inplace_end(_inplace_filename, INPLACE_SUFFIX) + inplace_begin(_inplace_filename = FILENAME, INPLACE_SUFFIX) } - ENDFILE { + END { inplace_end(FILENAME, INPLACE_SUFFIX) } @@ -25667,6 +25990,10 @@ the extension restores standard output to its original destination. If a backup file name created by appending that suffix. Finally, the temporary file is renamed to the original file name. + The `_inplace_filename' variable serves to keep track of the current +filename so as to not invoke `inplace_end()' before processing the +first file. + If any error occurs, the extension issues a fatal error to terminate processing immediately without damaging the original file. @@ -25686,7 +26013,7 @@ File: gawk.info, Node: Extension Sample Ord, Next: Extension Sample Readdir, -------------------------------------------------------- The `ordchr' extension adds two functions, named `ord()' and `chr()', -as follows. +as follows: `@load "ordchr"' This is how you load the extension. @@ -25724,11 +26051,11 @@ returned as a record. The record consists of three fields. The first two are the inode number and the file name, separated by a forward slash character. On systems where the directory entry contains the file type, the record -has a third field (also separated by a slash) which is a single letter -indicating the type of the file. The letters are file types are shown -in *note table-readdir-file-types::. +has a third field (also separated by a slash), which is a single letter +indicating the type of the file. The letters and their corresponding +file types are shown in *note table-readdir-file-types::. -Letter File Type +Letter File type -------------------------------------------------------------------------- `b' Block device `c' Character device @@ -25739,7 +26066,7 @@ Letter File Type `s' Socket `u' Anything else (unknown) -Table 16.2: File Types Returned By The `readdir' Extension +Table 16.2: File types returned by the `readdir' extension On systems without the file type information, the third field is always `u'. @@ -25764,7 +26091,7 @@ File: gawk.info, Node: Extension Sample Revout, Next: Extension Sample Rev2way ----------------------- The `revoutput' extension adds a simple output wrapper that reverses -the characters in each output line. It's main purpose is to show how to +the characters in each output line. Its main purpose is to show how to write an output wrapper, although it may be mildly amusing for the unwary. Here is an example: @@ -25775,7 +26102,7 @@ unwary. Here is an example: print "don't panic" > "/dev/stdout" } - The output from this program is: `cinap t'nod'. + The output from this program is `cinap t'nod'. File: gawk.info, Node: Extension Sample Rev2way, Next: Extension Sample Read write array, Prev: Extension Sample Revout, Up: Extension Samples @@ -25785,9 +26112,9 @@ File: gawk.info, Node: Extension Sample Rev2way, Next: Extension Sample Read w The `revtwoway' extension adds a simple two-way processor that reverses the characters in each line sent to it for reading back by the `awk' -program. It's main purpose is to show how to write a two-way -processor, although it may also be mildly amusing. The following -example shows how to use it: +program. Its main purpose is to show how to write a two-way processor, +although it may also be mildly amusing. The following example shows +how to use it: @load "revtwoway" @@ -25804,7 +26131,7 @@ example shows how to use it: File: gawk.info, Node: Extension Sample Read write array, Next: Extension Sample Readfile, Prev: Extension Sample Rev2way, Up: Extension Samples -16.7.9 Dumping and Restoring An Array +16.7.9 Dumping and Restoring an Array ------------------------------------- The `rwarray' extension adds two functions, named `writea()' and @@ -25823,19 +26150,19 @@ The `rwarray' extension adds two functions, named `writea()' and `reada()' is the inverse of `writea()'; it reads the file named as its first argument, filling in the array named as the second argument. It clears the array first. Here too, the return value - is one on success and zero upon failure. + is one on success, or zero upon failure. The array created by `reada()' is identical to that written by `writea()' in the sense that the contents are the same. However, due to -implementation issues, the array traversal order of the recreated array -is likely to be different from that of the original array. As array -traversal order in `awk' is by default undefined, this is (technically) -not a problem. If you need to guarantee a particular traversal order, -use the array sorting features in `gawk' to do so (*note Array -Sorting::). +implementation issues, the array traversal order of the re-created +array is likely to be different from that of the original array. As +array traversal order in `awk' is by default undefined, this is +(technically) not a problem. If you need to guarantee a particular +traversal order, use the array sorting features in `gawk' to do so +(*note Array Sorting::). The file contains binary data. All integral values are written in -network byte order. However, double precision floating-point values +network byte order. However, double-precision floating-point values are written as native binary data. Thus, arrays containing only string data can theoretically be dumped on systems with one byte order and restored on systems with a different one, but this has not been tried. @@ -25851,7 +26178,7 @@ restored on systems with a different one, but this has not been tried. File: gawk.info, Node: Extension Sample Readfile, Next: Extension Sample Time, Prev: Extension Sample Read write array, Up: Extension Samples -16.7.10 Reading An Entire File +16.7.10 Reading an Entire File ------------------------------ The `readfile' extension adds a single function named `readfile()', and @@ -25895,7 +26222,7 @@ The `time' extension adds two functions, named `gettimeofday()' and `the_time = gettimeofday()' Return the time in seconds that has elapsed since 1970-01-01 UTC - as a floating point value. If the time is unavailable on this + as a floating-point value. If the time is unavailable on this platform, return -1 and set `ERRNO'. The returned time should have sub-second precision, but the actual precision may vary based on the platform. If the standard C `gettimeofday()' system call @@ -25907,7 +26234,7 @@ The `time' extension adds two functions, named `gettimeofday()' and Attempt to sleep for SECONDS seconds. If SECONDS is negative, or the attempt to sleep fails, return -1 and set `ERRNO'. Otherwise, return zero after sleeping for the indicated amount of time. Note - that SECONDS may be a floating-point (non-integral) value. + that SECONDS may be a floating-point (nonintegral) value. Implementation details: depending on platform availability, this function tries to use `nanosleep()' or `select()' to implement the delay. @@ -25935,19 +26262,25 @@ provides a number of `gawk' extensions, including one for processing XML files. This is the evolution of the original `xgawk' (XML `gawk') project. - As of this writing, there are five extensions: + As of this writing, there are seven extensions: + + * `errno' extension - * GD graphics library extension. + * GD graphics library extension - * PDF extension. + * MPFR library extension (this provides access to a number of MPFR + functions that `gawk''s native MPFR support does not) - * PostgreSQL extension. + * PDF extension - * MPFR library extension. This provides access to a number of MPFR - functions which `gawk''s native MPFR support does not. + * PostgreSQL extension + + * Redis extension + + * Select extension * XML parser extension, using the Expat - (http://expat.sourceforge.net) XML parsing library. + (http://expat.sourceforge.net) XML parsing library You can check out the code for the `gawkextlib' project using the Git (http://git-scm.com) distributed source code control system. The @@ -25983,12 +26316,12 @@ follows. First, build and install `gawk': If you have installed `gawk' in the standard way, then you will likely not need the `--with-gawk' option when configuring `gawkextlib'. -You may also need to use the `sudo' utility to install both `gawk' and +You may need to use the `sudo' utility to install both `gawk' and `gawkextlib', depending upon how your system works. If you write an extension that you wish to share with other `gawk' -users, please consider doing so through the `gawkextlib' project. See -the project's web site for more information. +users, consider doing so through the `gawkextlib' project. See the +project's website for more information. File: gawk.info, Node: Extension summary, Next: Extension Exercises, Prev: gawkextlib, Up: Dynamic Extensions @@ -25997,7 +26330,7 @@ File: gawk.info, Node: Extension summary, Next: Extension Exercises, Prev: ga ============ * You can write extensions (sometimes called plug-ins) for `gawk' in - C or C++ using the Application Programming Interface (API) defined + C or C++ using the application programming interface (API) defined by the `gawk' developers. * Extensions must have a license compatible with the GNU General @@ -26005,7 +26338,7 @@ File: gawk.info, Node: Extension summary, Next: Extension Exercises, Prev: ga a variable named `plugin_is_GPL_compatible'. * Communication between `gawk' and an extension is two-way. `gawk' - passes a `struct' to the extension which contains various data + passes a `struct' to the extension that contains various data fields and function pointers. The extension can then call into `gawk' via the supplied function pointers to accomplish certain tasks. @@ -26016,46 +26349,46 @@ File: gawk.info, Node: Extension summary, Next: Extension Exercises, Prev: ga convention, implementation functions are named `do_XXXX()' for some `awk'-level function `XXXX()'. - * The API is defined in a header file named `gawkpi.h'. You must + * The API is defined in a header file named `gawkapi.h'. You must include a number of standard header files _before_ including it in your source file. * API function pointers are provided for the following kinds of operations: - * Allocating, reallocating, and releasing memory. + * Allocating, reallocating, and releasing memory - * Registration functions. You may register extension functions, + * Registration functions (you may register extension functions, exit callbacks, a version string, input parsers, output - wrappers, and two-way processors. + wrappers, and two-way processors) - * Printing fatal, warning, and "lint" warning messages. + * Printing fatal, warning, and "lint" warning messages - * Updating `ERRNO', or unsetting it. + * Updating `ERRNO', or unsetting it * Accessing parameters, including converting an undefined - parameter into an array. + parameter into an array - * Symbol table access: retrieving a global variable, creating - one, or changing one. + * Symbol table access (retrieving a global variable, creating + one, or changing one) * Creating and releasing cached values; this provides an efficient way to use values for multiple variables and can be - a big performance win. + a big performance win - * Manipulating arrays: retrieving, adding, deleting, and + * Manipulating arrays (retrieving, adding, deleting, and modifying elements; getting the count of elements in an array; creating a new array; clearing an array; and flattening an - array for easy C style looping over all its indices and - elements. + array for easy C-style looping over all its indices and + elements) * The API defines a number of standard data types for representing `awk' values, array elements, and arrays. - * The API provide convenience functions for constructing values. It - also provides memory management functions to ensure compatibility - between memory allocated by `gawk' and memory allocated by an - extension. + * The API provides convenience functions for constructing values. + It also provides memory management functions to ensure + compatibility between memory allocated by `gawk' and memory + allocated by an extension. * _All_ memory passed from `gawk' to an extension must be treated as read-only by the extension. @@ -26069,12 +26402,12 @@ File: gawk.info, Node: Extension summary, Next: Extension Exercises, Prev: ga that loaded it. * It is easiest to start a new extension by copying the boilerplate - code described in this major node. Macros in the `gawkapi.h' make - this easier to do. + code described in this major node. Macros in the `gawkapi.h' + header file make this easier to do. * The `gawk' distribution includes a number of small but useful - sample extensions. The `gawkextlib' project includes several more, - larger, extensions. If you wish to write an extension and + sample extensions. The `gawkextlib' project includes several more + (larger) extensions. If you wish to write an extension and contribute it to the community of `gawk' users, the `gawkextlib' project is the place to do so. @@ -26089,12 +26422,26 @@ File: gawk.info, Node: Extension Exercises, Prev: Extension summary, Up: Dyna `chmod()', and `umask()' to the file operations extension presented in *note Internal File Ops::. - 2. (Hard.) How would you provide namespaces in `gawk', so that the + 2. Write an input parser that prints a prompt if the input is a from + a "terminal" device. You can use the `isatty()' function to tell + if the input file is a terminal. (Hint: this function is usually + expensive to call; try to call it just once.) The content of the + prompt should come from a variable settable by `awk'-level code. + You can write the prompt to stanard error. However, for best + results, open a new file descriptor (or file pointer) on + `/dev/tty' and print the prompt there, in case standard error has + been redirected. + + Why is standard error a better choice than standard output for + writing the prompt? Which reading mechanism should you replace, + the one to get a record, or the one to read raw bytes? + + 3. (Hard.) How would you provide namespaces in `gawk', so that the names of functions in different extensions don't conflict with each other? If you come up with a really good scheme, contact the `gawk' maintainer to tell him about it. - 3. Write a wrapper script that provides an interface similar to `sed + 4. Write a wrapper script that provides an interface similar to `sed -i' for the "inplace" extension presented in *note Extension Sample Inplace::. @@ -26106,7 +26453,7 @@ Appendix A The Evolution of the `awk' Language ********************************************** This Info file describes the GNU implementation of `awk', which follows -the POSIX specification. Many long-time `awk' users learned `awk' +the POSIX specification. Many longtime `awk' users learned `awk' programming with the original `awk' implementation in Version 7 Unix. (This implementation was the basis for `awk' in Berkeley Unix, through 4.3-Reno. Subsequent versions of Berkeley Unix, and, for a while, some @@ -26144,56 +26491,56 @@ available in System V Release 3.1 (1987). This minor node summarizes the changes, with cross-references to further details: * The requirement for `;' to separate rules on a line (*note - Statements/Lines::). + Statements/Lines::) * User-defined functions and the `return' statement (*note - User-defined::). + User-defined::) * The `delete' statement (*note Delete::). - * The `do'-`while' statement (*note Do Statement::). + * The `do'-`while' statement (*note Do Statement::) * The built-in functions `atan2()', `cos()', `sin()', `rand()', and - `srand()' (*note Numeric Functions::). + `srand()' (*note Numeric Functions::) * The built-in functions `gsub()', `sub()', and `match()' (*note - String Functions::). + String Functions::) * The built-in functions `close()' and `system()' (*note I/O - Functions::). + Functions::) * The `ARGC', `ARGV', `FNR', `RLENGTH', `RSTART', and `SUBSEP' - predefined variables (*note Built-in Variables::). + predefined variables (*note Built-in Variables::) - * Assignable `$0' (*note Changing Fields::). + * Assignable `$0' (*note Changing Fields::) * The conditional expression using the ternary operator `?:' (*note - Conditional Exp::). + Conditional Exp::) - * The expression `INDEX-VARIABLE in ARRAY' outside of `for' - statements (*note Reference to Elements::). + * The expression `INDX in ARRAY' outside of `for' statements (*note + Reference to Elements::) * The exponentiation operator `^' (*note Arithmetic Ops::) and its - assignment operator form `^=' (*note Assignment Ops::). + assignment operator form `^=' (*note Assignment Ops::) * C-compatible operator precedence, which breaks some old `awk' - programs (*note Precedence::). + programs (*note Precedence::) * Regexps as the value of `FS' (*note Field Separators::) and as the third argument to the `split()' function (*note String - Functions::), rather than using only the first character of `FS'. + Functions::), rather than using only the first character of `FS' * Dynamic regexps as operands of the `~' and `!~' operators (*note - Computed Regexps::). + Computed Regexps::) * The escape sequences `\b', `\f', and `\r' (*note Escape - Sequences::). + Sequences::) - * Redirection of input for the `getline' function (*note Getline::). + * Redirection of input for the `getline' function (*note Getline::) - * Multiple `BEGIN' and `END' rules (*note BEGIN/END::). + * Multiple `BEGIN' and `END' rules (*note BEGIN/END::) - * Multidimensional arrays (*note Multidimensional::). + * Multidimensional arrays (*note Multidimensional::) File: gawk.info, Node: SVR4, Next: POSIX, Prev: V7/SVR3.1, Up: Language History @@ -26204,37 +26551,37 @@ A.2 Changes Between SVR3.1 and SVR4 The System V Release 4 (1989) version of Unix `awk' added these features (some of which originated in `gawk'): - * The `ENVIRON' array (*note Built-in Variables::). + * The `ENVIRON' array (*note Built-in Variables::) - * Multiple `-f' options on the command line (*note Options::). + * Multiple `-f' options on the command line (*note Options::) * The `-v' option for assigning variables before program execution - begins (*note Options::). + begins (*note Options::) - * The `--' signal for terminating command-line options. + * The `--' signal for terminating command-line options * The `\a', `\v', and `\x' escape sequences (*note Escape - Sequences::). + Sequences::) * A defined return value for the `srand()' built-in function (*note - Numeric Functions::). + Numeric Functions::) * The `toupper()' and `tolower()' built-in string functions for case - translation (*note String Functions::). + translation (*note String Functions::) * A cleaner specification for the `%c' format-control letter in the - `printf' function (*note Control Letters::). + `printf' function (*note Control Letters::) * The ability to dynamically pass the field width and precision (`"%*.*d"') in the argument list of `printf' and `sprintf()' - (*note Control Letters::). + (*note Control Letters::) * The use of regexp constants, such as `/foo/', as expressions, where they are equivalent to using the matching operator, as in `$0 ~ - /foo/' (*note Using Constant Regexps::). + /foo/' (*note Using Constant Regexps::) * Processing of escape sequences inside command-line variable - assignments (*note Assignment Options::). + assignments (*note Assignment Options::) File: gawk.info, Node: POSIX, Next: BTL, Prev: SVR4, Up: Language History @@ -26246,30 +26593,30 @@ The POSIX Command Language and Utilities standard for `awk' (1992) introduced the following changes into the language: * The use of `-W' for implementation-specific options (*note - Options::). + Options::) * The use of `CONVFMT' for controlling the conversion of numbers to - strings (*note Conversion::). + strings (*note Conversion::) * The concept of a numeric string and tighter comparison rules to go - with it (*note Typing and Comparison::). + with it (*note Typing and Comparison::) * The use of predefined variables as function parameter names is - forbidden (*note Definition Syntax::). + forbidden (*note Definition Syntax::) * More complete documentation of many of the previously undocumented - features of the language. + features of the language In 2012, a number of extensions that had been commonly available for many years were finally added to POSIX. They are: * The `fflush()' built-in function for flushing buffered output - (*note I/O Functions::). + (*note I/O Functions::) - * The `nextfile' statement (*note Nextfile Statement::). + * The `nextfile' statement (*note Nextfile Statement::) * The ability to delete all of an array at once with `delete ARRAY' - (*note Delete::). + (*note Delete::) *Note Common Extensions::, for a list of common extensions not @@ -26288,16 +26635,16 @@ Brian Kernighan has made his version available via his home page (*note Other Versions::). This minor node describes common extensions that originally appeared -in his version of `awk'. +in his version of `awk': * The `**' and `**=' operators (*note Arithmetic Ops:: and *note - Assignment Ops::). + Assignment Ops::) * The use of `func' as an abbreviation for `function' (*note - Definition Syntax::). + Definition Syntax::) * The `fflush()' built-in function for flushing buffered output - (*note I/O Functions::). + (*note I/O Functions::) *Note Common Extensions::, for a full list of the extensions @@ -26319,104 +26666,108 @@ the current version of `gawk'. * Additional predefined variables: - - The `ARGIND' `BINMODE', `ERRNO', `FIELDWIDTHS', `FPAT', + - The `ARGIND', `BINMODE', `ERRNO', `FIELDWIDTHS', `FPAT', `IGNORECASE', `LINT', `PROCINFO', `RT', and `TEXTDOMAIN' - variables (*note Built-in Variables::). + variables (*note Built-in Variables::) * Special files in I/O redirections: - - The `/dev/stdin', `/dev/stdout', `/dev/stderr' and - `/dev/fd/N' special file names (*note Special Files::). + - The `/dev/stdin', `/dev/stdout', `/dev/stderr', and + `/dev/fd/N' special file names (*note Special Files::) - The `/inet', `/inet4', and `/inet6' special files for TCP/IP networking using `|&' to specify which version of the IP - protocol to use. (*note TCP/IP Networking::). + protocol to use (*note TCP/IP Networking::) * Changes and/or additions to the language: - - The `\x' escape sequence (*note Escape Sequences::). + - The `\x' escape sequence (*note Escape Sequences::) - - Full support for both POSIX and GNU regexps (*note Regexp::). + - Full support for both POSIX and GNU regexps (*note Regexp::) - The ability for `FS' and for the third argument to `split()' - to be null strings (*note Single Character Fields::). + to be null strings (*note Single Character Fields::) - - The ability for `RS' to be a regexp (*note Records::). + - The ability for `RS' to be a regexp (*note Records::) - The ability to use octal and hexadecimal constants in `awk' - program source code (*note Nondecimal-numbers::). + program source code (*note Nondecimal-numbers::) - The `|&' operator for two-way I/O to a coprocess (*note - Two-way I/O::). + Two-way I/O::) - - Indirect function calls (*note Indirect Calls::). + - Indirect function calls (*note Indirect Calls::) - Directories on the command line produce a warning and are - skipped (*note Command-line directories::). + skipped (*note Command-line directories::) + + - Output with `print' and `printf' need not be fatal (*note + Nonfatal::) * New keywords: - - The `BEGINFILE' and `ENDFILE' special patterns. (*note - BEGINFILE/ENDFILE::). + - The `BEGINFILE' and `ENDFILE' special patterns (*note + BEGINFILE/ENDFILE::) - - The `switch' statement (*note Switch Statement::). + - The `switch' statement (*note Switch Statement::) * Changes to standard `awk' functions: - The optional second argument to `close()' that allows closing - one end of a two-way pipe to a coprocess (*note Two-way - I/O::). + one end of a two-way pipe to a coprocess (*note Two-way I/O::) - - POSIX compliance for `gsub()' and `sub()' with `--posix'. + - POSIX compliance for `gsub()' and `sub()' with `--posix' - The `length()' function accepts an array argument and returns - the number of elements in the array (*note String - Functions::). + the number of elements in the array (*note String Functions::) - The optional third argument to the `match()' function for capturing text-matching subexpressions within a regexp (*note - String Functions::). + String Functions::) - Positional specifiers in `printf' formats for making - translations easier (*note Printf Ordering::). + translations easier (*note Printf Ordering::) - - The `split()' function's additional optional fourth argument - which is an array to hold the text of the field separators. - (*note String Functions::). + - The `split()' function's additional optional fourth argument, + which is an array to hold the text of the field separators + (*note String Functions::) * Additional functions only in `gawk': - The `gensub()', `patsplit()', and `strtonum()' functions for - more powerful text manipulation (*note String Functions::). + more powerful text manipulation (*note String Functions::) - The `asort()' and `asorti()' functions for sorting arrays - (*note Array Sorting::). + (*note Array Sorting::) - The `mktime()', `systime()', and `strftime()' functions for - working with timestamps (*note Time Functions::). + working with timestamps (*note Time Functions::) - The `and()', `compl()', `lshift()', `or()', `rshift()', and `xor()' functions for bit manipulation (*note Bitwise - Functions::). + Functions::) - The `isarray()' function to check if a variable is an array - or not (*note Type Functions::). + or not (*note Type Functions::) - - The `bindtextdomain()', `dcgettext()' and `dcngettext()' - functions for internationalization (*note Programmer i18n::). + - The `bindtextdomain()', `dcgettext()', and `dcngettext()' + functions for internationalization (*note Programmer i18n::) + + - The `intdiv()' function for doing integer division and + remainder (*note Numeric Functions::) * Changes and/or additions in the command-line options: - The `AWKPATH' environment variable for specifying a path - search for the `-f' command-line option (*note Options::). + search for the `-f' command-line option (*note Options::) - The `AWKLIBPATH' environment variable for specifying a path - search for the `-l' command-line option (*note Options::). + search for the `-l' command-line option (*note Options::) - The `-b', `-c', `-C', `-d', `-D', `-e', `-E', `-g', `-h', `-i', `-l', `-L', `-M', `-n', `-N', `-o', `-O', `-p', `-P', `-r', `-S', `-t', and `-V' short options. Also, the ability - to use GNU-style long-named options that start with `--' and + to use GNU-style long-named options that start with `--', and the `--assign', `--bignum', `--characters-as-bytes', `--copyright', `--debug', `--dump-variables', `--exec', `--field-separator', `--file', `--gen-pot', `--help', @@ -26454,12 +26805,15 @@ the current version of `gawk'. - GCC for VAX and Alpha has not been tested for a while. - * Support for the following obsolete systems was removed from the - code for `gawk' version 4.1: + * Support for the following obsolete system was removed from the code + for `gawk' version 4.1: - Ultrix - * Support for MirBSD was removed at `gawk' version 4.2. + * Support for the following systems was removed from the code for + `gawk' version 4.2: + + - MirBSD @@ -26830,7 +27184,7 @@ in POSIX `awk', in the order they were added to `gawk'. - The `-M' and `--bignum' options enable MPFR. - - The `-o' only does pretty-printing. + - The `-o' option only does pretty-printing. - The `-p' option is used for profiling. @@ -26846,6 +27200,28 @@ in POSIX `awk', in the order they were added to `gawk'. * The dynamic extension interface was completely redone (*note Dynamic Extensions::). + * Support for Ultrix was removed. + + + Version 4.2 introduced the following changes: + + * Changes to `ENVIRON' are reflected into `gawk''s environment and + that of programs that it runs. *Note Auto-set::. + + * The `--pretty-print' option no longer runs the `awk' program too. + *Note Options::. + + * The `igawk' program and its manual page are no longer installed + when `gawk' is built. *Note Igawk Program::. + + * The `intdiv()' function. *Note Numeric Functions::. + + * The maximum number of hexdecimal digits in `\x' escapes is now two. + *Note Escape Sequences::. + + * Nonfatal output with `print' and `printf'. *Note Nonfatal::. + + * Support for MirBSD was removed. File: gawk.info, Node: Common Extensions, Next: Ranges and Locales, Prev: Feature History, Up: Language History @@ -26854,25 +27230,25 @@ A.7 Common Extensions Summary ============================= The following table summarizes the common extensions supported by -`gawk', Brian Kernighan's `awk', and `mawk', the three most widely-used +`gawk', Brian Kernighan's `awk', and `mawk', the three most widely used freely available versions of `awk' (*note Other Versions::). -Feature BWK Awk Mawk GNU Awk Now standard ------------------------------------------------------------------------ -`\x' Escape sequence X X X -`FS' as null string X X X -`/dev/stdin' special file X X X -`/dev/stdout' special file X X X -`/dev/stderr' special file X X X -`delete' without subscript X X X X -`fflush()' function X X X X -`length()' of an array X X X -`nextfile' statement X X X X -`**' and `**=' operators X X -`func' keyword X X -`BINMODE' variable X X -`RS' as regexp X X -Time related functions X X +Feature BWK `awk' `mawk' `gawk' Now standard +-------------------------------------------------------------------------- +`\x' escape sequence X X X +`FS' as null string X X X +`/dev/stdin' special file X X X +`/dev/stdout' special file X X X +`/dev/stderr' special file X X X +`delete' without subscript X X X X +`fflush()' function X X X X +`length()' of an array X X X +`nextfile' statement X X X X +`**' and `**=' operators X X +`func' keyword X X +`BINMODE' variable X X +`RS' as regexp X X +Time-related functions X X File: gawk.info, Node: Ranges and Locales, Next: Contributors, Prev: Common Extensions, Up: Language History @@ -26890,9 +27266,9 @@ first character in the range and the last character in the range, inclusive. Ordering was based on the numeric value of each character in the machine's native character set. Thus, on ASCII-based systems, `[a-z]' matched all the lowercase letters, and only the lowercase -letters, since the numeric values for the letters from `a' through `z' +letters, as the numeric values for the letters from `a' through `z' were contiguous. (On an EBCDIC system, the range `[a-z]' includes -additional, non-alphabetic characters as well.) +additional nonalphabetic characters as well.) Almost all introductory Unix literature explained range expressions as working in this fashion, and in particular, would teach that the @@ -26901,9 +27277,9 @@ as working in this fashion, and in particular, would teach that the this was true.(1) The 1992 POSIX standard introduced the idea of locales (*note -Locales::). Since many locales include other letters besides the plain -twenty-six letters of the American English alphabet, the POSIX standard -added character classes (*note Bracket Expressions::) as a way to match +Locales::). Because many locales include other letters besides the +plain 26 letters of the English alphabet, the POSIX standard added +character classes (*note Bracket Expressions::) as a way to match different kinds of characters besides the traditional ones in the ASCII character set. @@ -26916,9 +27292,9 @@ outside those locales, the ordering was defined to be based on What does that mean? In many locales, `A' and `a' are both less than `B'. In other words, these locales sort characters in dictionary order, and `[a-dx-z]' is typically not equivalent to `[abcdxyz]'; -instead it might be equivalent to `[ABCXYabcdxyz]', for example. +instead, it might be equivalent to `[ABCXYabcdxyz]', for example. - This point needs to be emphasized: Much literature teaches that you + This point needs to be emphasized: much literature teaches that you should use `[a-z]' to match a lowercase character. But on systems with non-ASCII locales, this also matches all of the uppercase characters except `A' or `Z'! This was a continuous cause of confusion, even well @@ -26931,22 +27307,22 @@ the intent is to remove trailing uppercase characters: $ echo something1234abc | gawk-3.1.8 '{ sub("[A-Z]*$", ""); print }' -| something1234a -This output is unexpected, since the `bc' at the end of -`something1234abc' should not normally match `[A-Z]*'. This result is -due to the locale setting (and thus you may not see it on your system). +This output is unexpected, as the `bc' at the end of `something1234abc' +should not normally match `[A-Z]*'. This result is due to the locale +setting (and thus you may not see it on your system). Similar considerations apply to other ranges. For example, `["-/]' is perfectly valid in ASCII, but is not valid in many Unicode locales, such as `en_US.UTF-8'. Early versions of `gawk' used regexp matching code that was not -locale aware, so ranges had their traditional interpretation. +locale-aware, so ranges had their traditional interpretation. When `gawk' switched to using locale-aware regexp matchers, the problems began; especially as both GNU/Linux and commercial Unix vendors started implementing non-ASCII locales, _and making them the default_. Perhaps the most frequently asked question became something -like "why does `[A-Z]' match lowercase letters?!?" +like, "Why does `[A-Z]' match lowercase letters?!?" This situation existed for close to 10 years, if not more, and the `gawk' maintainer grew weary of trying to explain that `gawk' was being @@ -26962,18 +27338,18 @@ of range expressions was _undefined_.(3) By using this lovely technical term, the standard gives license to implementors to implement ranges in whatever way they choose. The -`gawk' maintainer chose to apply the pre-POSIX meaning in all cases: -the default regexp matching; with `--traditional' and with `--posix'; -in all cases, `gawk' remains POSIX compliant. +`gawk' maintainer chose to apply the pre-POSIX meaning both with the +default regexp matching and when `--traditional' or `--posix' are used. +In all cases `gawk' remains POSIX-compliant. ---------- Footnotes ---------- (1) And Life was good. (2) And thus was born the Campaign for Rational Range Interpretation -(or RRI). A number of GNU tools have either implemented this change, or -will soon. Thanks to Karl Berry for coining the phrase "Rational Range -Interpretation." +(or RRI). A number of GNU tools have already implemented this change, +or will soon. Thanks to Karl Berry for coining the phrase "Rational +Range Interpretation." (3) See the standard (http://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html#tag_09_03_05) @@ -27005,7 +27381,7 @@ Info file, in approximate chronological order: * Richard Stallman helped finish the implementation and the initial draft of this Info file. He is also the founder of the FSF and - the GNU project. + the GNU Project. * John Woods contributed parts of the code (mostly fixes) in the initial version of `gawk'. @@ -27081,7 +27457,7 @@ Info file, in approximate chronological order: statements. * Patrick T.J. McPhee contributed the code for dynamic loading in - Windows32 environments. (This is no longer supported) + Windows32 environments. (This is no longer supported.) * Anders Wallin helped keep the VMS port going for several years. @@ -27091,22 +27467,22 @@ Info file, in approximate chronological order: * John Haque made the following contributions: - The modifications to convert `gawk' into a byte-code - interpreter, including the debugger. + interpreter, including the debugger - - The addition of true arrays of arrays. + - The addition of true arrays of arrays - - The additional modifications for support of arbitrary - precision arithmetic. + - The additional modifications for support of + arbitrary-precision arithmetic - - The initial text of *note Arbitrary Precision Arithmetic::. + - The initial text of *note Arbitrary Precision Arithmetic:: - The work to merge the three versions of `gawk' into one, for - the 4.1 release. + the 4.1 release - - Improved array internals for arrays indexed by integers. + - Improved array internals for arrays indexed by integers - - The improved array sorting features were driven by John - together with Pat Rankin. + - The improved array sorting features were also driven by John, + together with Pat Rankin * Panos Papadopoulos contributed the original text for *note Include Files::. @@ -27135,11 +27511,11 @@ A.10 Summary ============ * The `awk' language has evolved over time. The first release was - with V7 Unix circa 1978. In 1987 for System V Release 3.1, major - additions, including user-defined functions, were made to the - language. Additional changes were made for System V Release 4, in - 1989. Since then, further minor changes happen under the auspices - of the POSIX standard. + with V7 Unix, circa 1978. In 1987, for System V Release 3.1, + major additions, including user-defined functions, were made to + the language. Additional changes were made for System V Release + 4, in 1989. Since then, further minor changes have happened under + the auspices of the POSIX standard. * Brian Kernighan's `awk' provides a small number of extensions that are implemented in common with other versions of `awk'. @@ -27152,7 +27528,7 @@ A.10 Summary been confusing over the years. Today, `gawk' implements Rational Range Interpretation, where ranges of the form `[a-z]' match _only_ the characters numerically between `a' through `z' in the - machine's native character set. Usually this is ASCII but it can + machine's native character set. Usually this is ASCII, but it can be EBCDIC on IBM S/390 systems. * Many people have contributed to `gawk' development over the years. @@ -27169,8 +27545,8 @@ Appendix B Installing `gawk' This appendix provides instructions for installing `gawk' on the various platforms that are supported by the developers. The primary developer supports GNU/Linux (and Unix), whereas the other ports are -contributed. *Note Bugs::, for the electronic mail addresses of the -people who maintain the respective ports. +contributed. *Note Bugs::, for the email addresses of the people who +maintain the respective ports. * Menu: @@ -27216,7 +27592,7 @@ There are two ways to get GNU software: wget http://ftp.gnu.org/gnu/gawk/gawk-4.1.2.tar.gz The GNU software archive is mirrored around the world. The -up-to-date list of mirror sites is available from the main FSF web site +up-to-date list of mirror sites is available from the main FSF website (http://www.gnu.org/order/ftp.html). Try to use one of the mirrors; they will be less busy, and you can usually find one closer to your site. @@ -27230,11 +27606,11 @@ B.1.2 Extracting the Distribution `gawk' is distributed as several `tar' files compressed with different compression programs: `gzip', `bzip2', and `xz'. For simplicity, the rest of these instructions assume you are using the one compressed with -the GNU Zip program, `gzip'. +the GNU Gzip program (`gzip'). - Once you have the distribution (for example, `gawk-4.1.2.tar.gz'), -use `gzip' to expand the file and then use `tar' to extract it. You -can use the following pipeline to produce the `gawk' distribution: + Once you have the distribution (e.g., `gawk-4.1.2.tar.gz'), use +`gzip' to expand the file and then use `tar' to extract it. You can +use the following pipeline to produce the `gawk' distribution: gzip -d -c gawk-4.1.2.tar.gz | tar -xvpf - @@ -27270,10 +27646,10 @@ files, subdirectories, and files related to the configuration process to different non-Unix operating systems: Various `.c', `.y', and `.h' files - The actual `gawk' source code. + These files contain the actual `gawk' source code. `ABOUT-NLS' - Information about GNU `gettext' and translations. + A file containing information about GNU `gettext' and translations. `AUTHORS' A file with some information about the authorship of `gawk'. It @@ -27305,14 +27681,14 @@ Various `.c', `.y', and `.h' files The GNU General Public License. `POSIX.STD' - A description of behaviors in the POSIX standard for `awk' which + A description of behaviors in the POSIX standard for `awk' that are left undefined, or where `gawk' may not comply fully, as well as a list of things that the POSIX standard should describe but does not. `doc/awkforai.txt' Pointers to the original draft of a short article describing why - `gawk' is a good language for Artificial Intelligence (AI) + `gawk' is a good language for artificial intelligence (AI) programming. `doc/bc_notes' @@ -27417,6 +27793,13 @@ Various `.c', `.y', and `.h' files sample extensions included with `gawk'. *Note Dynamic Extensions::, for more information. +`extras/*' + Additional non-essential files. Currently, this directory + contains some shell startup files to be installed in + `/etc/profile.d' to aid in manipulating the `AWKPATH' and + `AWKLIBPATH' environment variables. *Note Shell Startup Files::, + for more information. + `posix/*' Files needed for building `gawk' on POSIX-compliant systems. @@ -27437,7 +27820,7 @@ Various `.c', `.y', and `.h' files File: gawk.info, Node: Unix Installation, Next: Non-Unix Installation, Prev: Gawk Distribution, Up: Installation -B.2 Compiling and Installing `gawk' on Unix-like Systems +B.2 Compiling and Installing `gawk' on Unix-Like Systems ======================================================== Usually, you can compile and install `gawk' by typing only two @@ -27447,13 +27830,14 @@ configure `gawk' for your system yourself. * Menu: * Quick Installation:: Compiling `gawk' under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. -File: gawk.info, Node: Quick Installation, Next: Additional Configuration Options, Up: Unix Installation +File: gawk.info, Node: Quick Installation, Next: Shell Startup Files, Up: Unix Installation -B.2.1 Compiling `gawk' for Unix-like Systems +B.2.1 Compiling `gawk' for Unix-Like Systems -------------------------------------------- The normal installation steps should work on all modern commercial @@ -27496,8 +27880,7 @@ That's all there is to it! To verify that `gawk' is working properly, run `make check'. All of the tests should succeed. If these steps do not work, or if any of the tests fail, check the files in the `README_d' directory to see if you've found a known problem. If the -failure is not described there, please send in a bug report (*note -Bugs::). +failure is not described there, send in a bug report (*note Bugs::). Of course, once you've built `gawk', it is likely that you will wish to install it. To do so, you need to run the command `make install', @@ -27508,9 +27891,43 @@ will be asked for your password, and you will have to have been set up previously as a user who is allowed to run the `sudo' command. -File: gawk.info, Node: Additional Configuration Options, Next: Configuration Philosophy, Prev: Quick Installation, Up: Unix Installation +File: gawk.info, Node: Shell Startup Files, Next: Additional Configuration Options, Prev: Quick Installation, Up: Unix Installation + +B.2.2 Shell Startup Files +------------------------- + +The distribution contains shell startup files `gawk.sh' and `gawk.csh' +containing functions to aid in manipulating the `AWKPATH' and +`AWKLIBPATH' environment variables. On a Fedora system, these files +should be installed in `/etc/profile.d'; on other platforms, the +appropriate location may be different. + +`gawkpath_default' + Reset the `AWKPATH' environment variable to its default value. + +`gawkpath_prepend' + Add the argument to the front of the `AWKPATH' environment + variable. + +`gawkpath_append' + Add the argument to the end of the `AWKPATH' environment variable. + +`gawklibpath_default' + Reset the `AWKLIBPATH' environment variable to its default value. -B.2.2 Additional Configuration Options +`gawklibpath_prepend' + Add the argument to the front of the `AWKLIBPATH' environment + variable. + +`gawklibpath_append' + Add the argument to the end of the `AWKLIBPATH' environment + variable. + + + +File: gawk.info, Node: Additional Configuration Options, Next: Configuration Philosophy, Prev: Shell Startup Files, Up: Unix Installation + +B.2.3 Additional Configuration Options -------------------------------------- There are several additional options you may use on the `configure' @@ -27528,14 +27945,16 @@ command line when compiling `gawk' from scratch, including: do nothing. Similarly, setting the `LINT' variable (*note User-modified::) has no effect on the running `awk' program. - When used with GCC's automatic dead-code-elimination, this option - cuts almost 23K bytes off the size of the `gawk' executable on - GNU/Linux x86_64 systems. Results on other systems and with other - compilers are likely to vary. Using this option may bring you - some slight performance improvement. + When used with the GNU Compiler Collection's (GCC's) automatic + dead-code-elimination, this option cuts almost 23K bytes off the + size of the `gawk' executable on GNU/Linux x86_64 systems. + Results on other systems and with other compilers are likely to + vary. Using this option may bring you some slight performance + improvement. - Using this option will cause some of the tests in the test suite - to fail. This option may be removed at a later date. + CAUTION: Using this option will cause some of the tests in + the test suite to fail. This option may be removed at a + later date. `--disable-nls' Disable all message-translation facilities. This is usually not @@ -27547,12 +27966,12 @@ command line when compiling `gawk' from scratch, including: for deficient systems. Use the command `./configure --help' to see the full list of options -that `configure' supplies. +supplied by `configure'. File: gawk.info, Node: Configuration Philosophy, Prev: Additional Configuration Options, Up: Unix Installation -B.2.3 The Configuration Process +B.2.4 The Configuration Process ------------------------------- This minor node is of interest only if you know something about using @@ -27581,15 +28000,15 @@ element in the `stat' structure. In this case, It is possible for your C compiler to lie to `configure'. It may do so by not exiting with an error when a library function is not -available. To get around this, edit the file `custom.h'. Use an +available. To get around this, edit the `custom.h' file. Use an `#ifdef' that is appropriate for your system, and either `#define' any constants that `configure' should have defined but didn't, or `#undef' -any constants that `configure' defined and should not have. `custom.h' -is automatically included by `config.h'. +any constants that `configure' defined and should not have. The +`custom.h' file is automatically included by the `config.h' file. It is also possible that the `configure' program generated by Autoconf will not work on your system in some other fashion. If you do -have a problem, the file `configure.ac' is the input for Autoconf. You +have a problem, the `configure.ac' file is the input for Autoconf. You may be able to change this file and generate a new version of `configure' that works on your system (*note Bugs::, for information on how to report problems in configuring `gawk'). The same mechanism may @@ -27619,14 +28038,14 @@ B.3.1 Installation on PC Operating Systems This minor node covers installation and usage of `gawk' on Intel architecture machines running MS-DOS, any version of MS-Windows, or OS/2. In this minor node, the term "Windows32" refers to any of -Microsoft Windows-95/98/ME/NT/2000/XP/Vista/7/8. +Microsoft Windows 95/98/ME/NT/2000/XP/Vista/7/8. The limitations of MS-DOS (and MS-DOS shells under the other -operating systems) has meant that various "DOS extenders" are often +operating systems) have meant that various "DOS extenders" are often used with programs such as `gawk'. The varying capabilities of Microsoft Windows 3.1 and Windows32 can add to the confusion. For an -overview of the considerations, please refer to `README_d/README.pc' in -the distribution. +overview of the considerations, refer to `README_d/README.pc' in the +distribution. * Menu: @@ -27755,8 +28174,8 @@ other set of (self-consistent) environment variables and compiler flags. ---------- Footnotes ---------- - (1) As of May, 2014, this site is still there, but the author could -not find a package for GNU Make. + (1) As of November 2014, this site is still there, but the author +could not find a package for GNU Make. File: gawk.info, Node: PC Testing, Next: PC Using, Prev: PC Compiling, Up: PC Installation @@ -27817,7 +28236,7 @@ The DJGPP collection of tools includes an MS-DOS port of Bash, and several shells are available for OS/2, including `ksh'. Under MS-Windows, OS/2 and MS-DOS, `gawk' (and many other text -programs) silently translate end-of-line `\r\n' to `\n' on input and +programs) silently translates end-of-line `\r\n' to `\n' on input and `\n' to `\r\n' on output. A special `BINMODE' variable (c.e.) allows control over these translations and is interpreted as follows: @@ -27839,7 +28258,7 @@ The modes for standard input and standard output are set one time only program). Setting `BINMODE' for standard input or standard output is accomplished by using an appropriate `-v BINMODE=N' option on the command line. `BINMODE' is set at the time a file or pipe is opened -and cannot be changed mid-stream. +and cannot be changed midstream. The name `BINMODE' was chosen to match `mawk' (*note Other Versions::). `mawk' and `gawk' handle `BINMODE' similarly; however, @@ -27883,10 +28302,9 @@ B.3.1.5 Using `gawk' In The Cygwin Environment `gawk' can be built and used "out of the box" under MS-Windows if you are using the Cygwin environment (http://www.cygwin.com). This -environment provides an excellent simulation of GNU/Linux, using the -GNU tools, such as Bash, the GNU Compiler Collection (GCC), GNU Make, -and other GNU programs. Compilation and installation for Cygwin is the -same as for a Unix system: +environment provides an excellent simulation of GNU/Linux, using Bash, +GCC, GNU Make, and other GNU programs. Compilation and installation +for Cygwin is the same as for a Unix system: tar -xvpzf gawk-4.1.2.tar.gz cd gawk-4.1.2 @@ -27904,12 +28322,12 @@ B.3.1.6 Using `gawk' In The MSYS Environment ............................................ In the MSYS environment under MS-Windows, `gawk' automatically uses -binary mode for reading and writing files. Thus there is no need to +binary mode for reading and writing files. Thus, there is no need to use the `BINMODE' variable. This can cause problems with other Unix-like components that have been ported to MS-Windows that expect `gawk' to do automatic -translation of `"\r\n"', since it won't. +translation of `"\r\n"', because it won't. File: gawk.info, Node: VMS Installation, Prev: PC Installation, Up: Non-Unix Installation @@ -27959,9 +28377,9 @@ available from `https://github.com/endlesssoftware/mmk'. target parameter may need to be exact. `gawk' has been tested under VAX/VMS 7.3 and Alpha/VMS 7.3-1 using -Compaq C V6.4, and Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3. -The most recent builds used HP C V7.3 on Alpha VMS 8.3 and both Alpha -and IA64 VMS 8.4 used HP C 7.3.(1) +Compaq C V6.4, and under Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS +8.3. The most recent builds used HP C V7.3 on Alpha VMS 8.3 and both +Alpha and IA64 VMS 8.4 used HP C 7.3.(1) *Note VMS GNV::, for information on building `gawk' as a PCSI kit that is compatible with the GNV product. @@ -27977,7 +28395,7 @@ B.3.2.2 Compiling `gawk' Dynamic Extensions on VMS .................................................. The extensions that have been ported to VMS can be built using one of -the following commands. +the following commands: $ MMS/DESCRIPTION=[.vms]descrip.mms extensions @@ -27989,7 +28407,7 @@ or: logical name to find the dynamic extensions. Dynamic extensions need to be compiled with the same compiler -options for floating point, pointer size, and symbol name handling as +options for floating-point, pointer size, and symbol name handling as were used to compile `gawk' itself. Alpha and Itanium should use IEEE floating point. The pointer size is 32 bits, and the symbol name handling should be exact case with CRC shortening for symbols longer @@ -28004,7 +28422,7 @@ than 32 bits. /name=(as_is,short) - Compile time macros need to be defined before the first VMS-supplied + Compile-time macros need to be defined before the first VMS-supplied header file is included, as follows: #if (__CRTL_VER >= 70200000) && !defined (__VAX) @@ -28048,14 +28466,14 @@ directory tree, the program will be known as `GNV$GNU:[vms_help]gawk.hlp'. The PCSI kit also installs a `GNV$GNU:[vms_bin]gawk_verb.cld' file -which can be used to add `gawk' and `awk' as DCL commands. +that can be used to add `gawk' and `awk' as DCL commands. For just the current process you can use: $ set command gnv$gnu:[vms_bin]gawk_verb.cld Or the system manager can use `GNV$GNU:[vms_bin]gawk_verb.cld' to -add the `gawk' and `awk' to the system wide `DCLTABLES'. +add the `gawk' and `awk' to the system-wide `DCLTABLES'. The DCL syntax is documented in the `gawk.hlp' file. @@ -28105,25 +28523,24 @@ Note that uppercase and mixed-case text must be quoted. The VMS port of `gawk' includes a `DCL'-style interface in addition to the original shell-style interface (see the help entry for details). One side effect of dual command-line parsing is that if there is only a -single parameter (as in the quoted string program above), the command -becomes ambiguous. To work around this, the normally optional `--' -flag is required to force Unix-style parsing rather than `DCL' parsing. -If any other dash-type options (or multiple parameters such as data -files to process) are present, there is no ambiguity and `--' can be -omitted. +single parameter (as in the quoted string program), the command becomes +ambiguous. To work around this, the normally optional `--' flag is +required to force Unix-style parsing rather than `DCL' parsing. If any +other dash-type options (or multiple parameters such as data files to +process) are present, there is no ambiguity and `--' can be omitted. The `exit' value is a Unix-style value and is encoded into a VMS exit status value when the program exits. The VMS severity bits will be set based on the `exit' value. A -failure is indicated by 1 and VMS sets the `ERROR' status. A fatal -error is indicated by 2 and VMS sets the `FATAL' status. All other +failure is indicated by 1, and VMS sets the `ERROR' status. A fatal +error is indicated by 2, and VMS sets the `FATAL' status. All other values will have the `SUCCESS' status. The exit value is encoded to comply with VMS coding standards and will have the `C_FACILITY_NO' of `0x350000' with the constant `0xA000' added to the number shifted over by 3 bits to make room for the severity codes. - To extract the actual `gawk' exit code from the VMS status use: + To extract the actual `gawk' exit code from the VMS status, use: unix_status = (vms_status .and. &x7f8) / 8 @@ -28139,7 +28556,7 @@ Function::. VMS reports time values in GMT unless one of the `SYS$TIMEZONE_RULE' or `TZ' logical names is set. Older versions of VMS, such as VAX/VMS -7.3 do not set these logical names. +7.3, do not set these logical names. The default search path, when looking for `awk' program files specified by the `-f' option, is `"SYS$DISK:[],AWK_LIBRARY:"'. The @@ -28156,7 +28573,7 @@ B.3.2.5 The VMS GNV Project The VMS GNV package provides a build environment similar to POSIX with ports of a collection of open source tools. The `gawk' found in the GNV -base kit is an older port. Currently the GNV project is being +base kit is an older port. Currently, the GNV project is being reorganized to supply individual PCSI packages for each component. See `https://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/'. @@ -28186,81 +28603,80 @@ File: gawk.info, Node: Bugs, Next: Other Versions, Prev: Non-Unix Installatio B.4 Reporting Problems and Bugs =============================== - There is nothing more dangerous than a bored archeologist. -- The - Hitchhiker's Guide to the Galaxy + There is nothing more dangerous than a bored archaeologist. -- + Douglas Adams, `The Hitchhiker's Guide to the Galaxy' If you have problems with `gawk' or think that you have found a bug, -please report it to the developers; we cannot promise to do anything -but we might well want to fix it. +report it to the developers; we cannot promise to do anything, but we +might well want to fix it. - Before reporting a bug, please make sure you have really found a -genuine bug. Carefully reread the documentation and see if it says you -can do what you're trying to do. If it's not clear whether you should -be able to do something or not, report that too; it's a bug in the + Before reporting a bug, make sure you have really found a genuine +bug. Carefully reread the documentation and see if it says you can do +what you're trying to do. If it's not clear whether you should be able +to do something or not, report that too; it's a bug in the documentation! Before reporting a bug or trying to fix it yourself, try to isolate it to the smallest possible `awk' program and input data file that -reproduces the problem. Then send us the program and data file, some +reproduce the problem. Then send us the program and data file, some idea of what kind of Unix system you're using, the compiler you used to compile `gawk', and the exact results `gawk' gave you. Also say what you expected to occur; this helps us decide whether the problem is really in the documentation. - Please include the version number of `gawk' you are using. You can -get this information with the command `gawk --version'. + Make sure to include the version number of `gawk' you are using. +You can get this information with the command `gawk --version'. Once you have a precise problem description, send email to <bug-gawk@gnu.org>. - The `gawk' maintainers subscribe to this address and thus they will + The `gawk' maintainers subscribe to this address, and thus they will receive your bug report. Although you can send mail to the maintainers -directly, the bug reporting address is preferred since the email list +directly, the bug reporting address is preferred because the email list is archived at the GNU Project. _All email must be in English. This is the only language understood in common by all the maintainers._ CAUTION: Do _not_ try to report bugs in `gawk' by posting to the - Usenet/Internet newsgroup `comp.lang.awk'. While the `gawk' - developers do occasionally read this newsgroup, there is no - guarantee that we will see your posting. The steps described - above are the only official recognized way for reporting bugs. - Really. + Usenet/Internet newsgroup `comp.lang.awk'. The `gawk' developers + do occasionally read this newsgroup, but there is no guarantee + that we will see your posting. The steps described here are the + only officially recognized way for reporting bugs. Really. NOTE: Many distributions of GNU/Linux and the various BSD-based operating systems have their own bug reporting systems. If you - report a bug using your distribution's bug reporting system, - _please_ also send a copy to <bug-gawk@gnu.org>. + report a bug using your distribution's bug reporting system, you + should also send a copy to <bug-gawk@gnu.org>. - This is for two reasons. First, while some distributions forward - bug reports "upstream" to the GNU mailing list, many don't, so - there is a good chance that the `gawk' maintainers won't even see - the bug report! Second, mail to the GNU list is archived, and - having everything at the GNU project keeps things self-contained - and not dependant on other organizations. + This is for two reasons. First, although some distributions + forward bug reports "upstream" to the GNU mailing list, many + don't, so there is a good chance that the `gawk' maintainers + won't even see the bug report! Second, mail to the GNU list is + archived, and having everything at the GNU Project keeps things + self-contained and not dependent on other organizations. Non-bug suggestions are always welcome as well. If you have questions about things that are unclear in the documentation or are just obscure features, ask on the bug list; we will try to help you out if we can. - If you find bugs in one of the non-Unix ports of `gawk', please send -an electronic mail message to the bug list, with a copy to the person -who maintains that port. They are named in the following list, as well -as in the `README' file in the `gawk' distribution. Information in the + If you find bugs in one of the non-Unix ports of `gawk', send an +email to the bug list, with a copy to the person who maintains that +port. The maintainers are named in the following list, as well as in +the `README' file in the `gawk' distribution. Information in the `README' file should be considered authoritative if it conflicts with this Info file. The people maintaining the various `gawk' ports are: -Unix and POSIX systems Arnold Robbins, <arnold@skeeve.com>. -MS-DOS with DJGPP Scott Deifik, <scottd.mail@sbcglobal.net>. -MS-Windows with MinGW Eli Zaretskii, <eliz@gnu.org>. -OS/2 Andreas Buening, <andreas.buening@nexgo.de>. -VMS John Malmberg, <wb8tyw@qsl.net>. -z/OS (OS/390) Dave Pitts, <dpitts@cozx.com>. +Unix and POSIX systems Arnold Robbins, <arnold@skeeve.com> +MS-DOS with DJGPP Scott Deifik, <scottd.mail@sbcglobal.net> +MS-Windows with MinGW Eli Zaretskii, <eliz@gnu.org> +OS/2 Andreas Buening, <andreas.buening@nexgo.de> +VMS John Malmberg, <wb8tyw@qsl.net> +z/OS (OS/390) Dave Pitts, <dpitts@cozx.com> - If your bug is also reproducible under Unix, please send a copy of -your report to the <bug-gawk@gnu.org> email list as well. + If your bug is also reproducible under Unix, send a copy of your +report to the <bug-gawk@gnu.org> email list as well. File: gawk.info, Node: Other Versions, Next: Installation summary, Prev: Bugs, Up: Installation @@ -28268,7 +28684,7 @@ File: gawk.info, Node: Other Versions, Next: Installation summary, Prev: Bugs B.5 Other Freely Available `awk' Implementations ================================================ - It's kind of fun to put comments like this in your awk code. + It's kind of fun to put comments like this in your awk code: `// Do C++ comments work? answer: yes! of course' -- Michael Brennan @@ -28278,7 +28694,7 @@ This minor node briefly describes where to get them: Unix `awk' Brian Kernighan, one of the original designers of Unix `awk', has made his implementation of `awk' freely available. You can - retrieve this version via the World Wide Web from his home page + retrieve this version via his home page (http://www.cs.princeton.edu/~bwk). It is available in several archive formats: @@ -28291,14 +28707,14 @@ Unix `awk' Zip file `http://www.cs.princeton.edu/~bwk/btl.mirror/awk.zip' - You can also retrieve it from Git Hub: + You can also retrieve it from GitHub: git clone git://github.com/onetrueawk/awk bwkawk - The above command creates a copy of the Git - (http://www.git-scm.com) repository in a directory named `bwkawk'. - If you leave that argument off the `git' command line, the - repository copy is created in a directory named `awk'. + This command creates a copy of the Git (http://git-scm.com) + repository in a directory named `bwkawk'. If you leave that + argument off the `git' command line, the repository copy is + created in a directory named `awk'. This version requires an ISO C (1990 standard) compiler; the C compiler from GCC (the GNU Compiler Collection) works quite nicely. @@ -28306,6 +28722,10 @@ Unix `awk' *Note Common Extensions::, for a list of extensions in this `awk' that are not in POSIX `awk'. + As a side note, Dan Bornstein has created a Git repository tracking + all the versions of BWK `awk' that he could find. It's available + at `git://github.com/danfuzz/one-true-awk'. + `mawk' Michael Brennan wrote an independent implementation of `awk', called `mawk'. It is available under the GPL (*note Copying::), @@ -28329,7 +28749,7 @@ Unix `awk' `awka' Written by Andrew Sumner, `awka' translates `awk' programs into C, compiles them, and links them with a library of functions that - provides the core `awk' functionality. It also has a number of + provide the core `awk' functionality. It also has a number of extensions. The `awk' translator is released under the GPL, and the library is @@ -28338,19 +28758,19 @@ Unix `awk' To get `awka', go to `http://sourceforge.net/projects/awka'. The project seems to be frozen; no new code changes have been made - since approximately 2003. + since approximately 2001. `pawk' Nelson H.F. Beebe at the University of Utah has modified BWK `awk' to provide timing and profiling information. It is different from - `gawk' with the `--profile' option (*note Profiling::), in that it + `gawk' with the `--profile' option (*note Profiling::) in that it uses CPU-based profiling, not line-count profiling. You may find it at either `ftp://ftp.math.utah.edu/pub/pawk/pawk-20030606.tar.gz' or `http://www.math.utah.edu/pub/pawk/pawk-20030606.tar.gz'. -Busybox Awk - Busybox is a GPL-licensed program providing small versions of many +BusyBox `awk' + BusyBox is a GPL-licensed program providing small versions of many applications within a single executable. It is aimed at embedded systems. It includes a full implementation of POSIX `awk'. When building it, be careful not to do `make install' as it will @@ -28360,15 +28780,15 @@ Busybox Awk The OpenSolaris POSIX `awk' The versions of `awk' in `/usr/xpg4/bin' and `/usr/xpg6/bin' on - Solaris are more-or-less POSIX-compliant. They are based on the + Solaris are more or less POSIX-compliant. They are based on the `awk' from Mortice Kern Systems for PCs. We were able to make this code compile and work under GNU/Linux with 1-2 hours of work. Making it more generally portable (using GNU Autoconf and/or Automake) would take more work, and this has not been done, at least to our knowledge. - The source code used to be available from the OpenSolaris web site. - However, that project was ended and the web site shut down. + The source code used to be available from the OpenSolaris website. + However, that project was ended and the website shut down. Fortunately, the Illumos project (http://wiki.illumos.org/display/illumos/illumos+Home) makes this implementation available. You can view the files one at a time @@ -28384,7 +28804,7 @@ The OpenSolaris POSIX `awk' Libmawk This is an embeddable `awk' interpreter derived from `mawk'. For - more information see `http://repo.hu/projects/libmawk/'. + more information, see `http://repo.hu/projects/libmawk/'. `pawk' This is a Python module that claims to bring `awk'-like features @@ -28392,8 +28812,8 @@ Libmawk information. (This is not related to Nelson Beebe's modified version of BWK `awk', described earlier.) -QSE Awk - This is an embeddable `awk' interpreter. For more information see +QSE `awk' + This is an embeddable `awk' interpreter. For more information, see `http://code.google.com/p/qse/' and `http://awk.info/?tools/qse'. `QTawk' @@ -28406,10 +28826,11 @@ QSE Awk The project may also be frozen; no new code changes have been made since approximately 2008. -Other Versions - See also the Wikipedia article - (http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations), - for information on additional versions. +Other versions + See also the "Versions and implementations" section of the + Wikipedia article + (http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations) + on `awk' for information on additional versions. @@ -28418,7 +28839,7 @@ File: gawk.info, Node: Installation summary, Prev: Other Versions, Up: Instal B.6 Summary =========== - * The `gawk' distribution is available from GNU project's main + * The `gawk' distribution is available from the GNU Project's main distribution site, `ftp.gnu.org'. The canonical build recipe is: wget http://ftp.gnu.org/gnu/gawk/gawk-4.1.2.tar.gz @@ -28427,17 +28848,17 @@ B.6 Summary ./configure && make && make check * `gawk' may be built on non-POSIX systems as well. The currently - supported systems are MS-Windows using DJGPP, MSYS, MinGW and + supported systems are MS-Windows using DJGPP, MSYS, MinGW, and Cygwin, OS/2 using EMX, and both Vax/VMS and OpenVMS. Instructions for each system are included in this major node. * Bug reports should be sent via email to <bug-gawk@gnu.org>. Bug - reports should be in English, and should include the version of + reports should be in English and should include the version of `gawk', how it was compiled, and a short program and data file - which demonstrate the problem. + that demonstrate the problem. * There are a number of other freely available `awk' - implementations. Many are POSIX compliant; others are less so. + implementations. Many are POSIX-compliant; others are less so. @@ -28522,7 +28943,7 @@ released versions of `gawk'. changes, you will probably wish to work with the development version. To do so, you will need to access the `gawk' source code repository. The code is maintained using the Git distributed version control system -(http://git-scm.com/). You will need to install it if your system +(http://git-scm.com). You will need to install it if your system doesn't have it. Once you have done so, use the command: git clone git://git.savannah.gnu.org/gawk.git @@ -28577,9 +28998,8 @@ possible to include them: document describes how GNU software should be written. If you haven't read it, please do so, preferably _before_ starting to modify `gawk'. (The `GNU Coding Standards' are available from the - GNU Project's web site - (http://www.gnu.org/prep/standards_toc.html). Texinfo, Info, and - DVI versions are also available.) + GNU Project's website (http://www.gnu.org/prep/standards/). + Texinfo, Info, and DVI versions are also available.) 5. Use the `gawk' coding style. The C code for `gawk' follows the instructions in the `GNU Coding Standards', with minor exceptions. @@ -29431,6 +29851,11 @@ Action pattern matches an input record, `awk' executes the rule's action. Actions are always enclosed in braces. (*Note Action Overview::.) +Ada + A programming language originally defined by the U.S. Department of + Defense for embedded programming. It was designed to enforce good + Software Engineering practices. + Amazing `awk' Assembler Henry Spencer at the University of Toronto wrote a retargetable assembler completely as `sed' and `awk' scripts. It is thousands @@ -29439,11 +29864,6 @@ Amazing `awk' Assembler been better written in another language. You can get it from `http://awk.info/?awk100/aaa'. -Ada - A programming language originally defined by the U.S. Department of - Defense for embedded programming. It was designed to enforce good - Software Engineering practices. - Amazingly Workable Formatter (`awf') Henry Spencer at the University of Toronto wrote a formatter that accepts a large subset of the `nroff -ms' and `nroff -man' @@ -29460,6 +29880,21 @@ ANSI C++ programming languages. These standards often become international standards as well. See also "ISO." +Argument + An argument can be two different things. It can be an option or a + file name passed to a command while invoking it from the command + line, or it can be something passed to a "function" inside a + program, e.g. inside `awk'. + + In the latter case, an argument can be passed to a function in two + ways. Either it is given to the called function by value, i.e., a + copy of the value of the variable is made available to the called + function, but the original variable cannot be modified by the + function itself; or it is given by reference, i.e., a pointer to + the interested variable is passed to the function, which can then + directly modify it. In `awk' scalars are passed by value, and + arrays are passed by reference. See "Pass By Value/Reference." + Array A grouping of multiple values under the same name. Most languages just provide sequential arrays. `awk' provides associative arrays. @@ -29495,6 +29930,26 @@ Bash The GNU version of the standard shell (the Bourne-Again SHell). See also "Bourne Shell." +Binary + Base-two notation, where the digits are `0'-`1'. Since electronic + circuitry works "naturally" in base 2 (just think of Off/On), + everything inside a computer is calculated using base 2. Each digit + represents the presence (or absence) of a power of 2 and is called + a "bit". So, for example, the base-two number `10101' is the same + as decimal 21, ((1 x 16) + (1 x 4) + (1 x 1)). + + Since base-two numbers quickly become very long to read and write, + they are usually grouped by 3 (i.e., they are read as octal + numbers), or by 4 (i.e., they are read as hexadecimal numbers). + There is no direct way to insert base 2 numbers in a C program. + If need arises, such numbers are usually inserted as octal or + hexadecimal numbers. The number of base-two digits that fit into + registers used for representing integer numbers in computers is a + rough indication of the computing power of the computer itself. + Most computers nowadays use 64 bits for representing integer + numbers in their registers, but 32-bit, 16-bit and 8-bit registers + have been widely used in the past. *Note Nondecimal-numbers::. + Bit Short for "Binary Digit." All values in computer memory ultimately reduce to binary digits: values that are either zero or @@ -29519,6 +29974,23 @@ Bourne Shell shells (Bash, `ksh', `pdksh', `zsh') are generally upwardly compatible with the Bourne shell. +Braces + The characters `{' and `}'. Braces are used in `awk' for + delimiting actions, compound statements, and function bodies. + +Bracket Expression + Inside a "regular expression", an expression included in square + brackets, meant to designate a single character as belonging to a + specified character class. A bracket expression can contain a list + of one or more characters, like `[abc]', a range of characters, + like `[A-Z]', or a name, delimited by `:', that designates a known + set of characters, like `[:digit:]'. The form of bracket expression + enclosed between `:' is independent of the underlying + representation of the character themselves, which could utilize + the ASCII, ECBDIC, or Unicode codesets, depending on the + architecture of the computer system, and on localization. See + also "Regular Expression." + Built-in Function The `awk' language provides built-in functions that perform various numerical, I/O-related, and string computations. Examples are @@ -29537,10 +30009,6 @@ Built-in Variable them affects `awk''s running environment. (*Note Built-in Variables::.) -Braces - The characters `{' and `}'. Braces are used in `awk' for - delimiting actions, compound statements, and function bodies. - C The system programming language that most GNU software is written in. The `awk' programming language has C-like syntax, and this @@ -29550,9 +30018,25 @@ C In general, `gawk' attempts to be as similar to the 1990 version of ISO C as makes sense. +C Shell + The C Shell (`csh' or its improved version, `tcsh') is a Unix + shell that was created by Bill Joy in the late 1970s. The C shell + was differentiated from other shells by its interactive features + and overall style, which looks more like C. The C Shell is not + backward compatible with the Bourne Shell, so special attention is + required when converting scripts written for other Unix shells to + the C shell, especially with regard to the management of shell + variables. See also "Bourne Shell." + C++ A popular object-oriented programming language derived from C. +Character Class + See "Bracket Expression." + +Character List + See "Bracket Expression." + Character Set The set of numeric codes used by a computer system to represent the characters (letters, numbers, punctuation, etc.) of a particular @@ -29567,25 +30051,34 @@ CHEM A preprocessor for `pic' that reads descriptions of molecules and produces `pic' input for drawing them. It was written in `awk' by Brian Kernighan and Jon Bentley, and is available from - `http://netlib.sandia.gov/netlib/typesetting/chem.gz'. + `http://netlib.org/typesetting/chem'. -Cookie - A peculiar goodie, token, saying or remembrance produced by or - presented to a program. (With thanks to Professor Doug McIlroy.) - -Coprocess - A subordinate program with which two-way communications is - possible. +Comparison Expression + A relation that is either true or false, such as `a < b'. + Comparison expressions are used in `if', `while', `do', and `for' + statements, and in patterns to select which input records to + process. (*Note Typing and Comparison::.) Compiler A program that translates human-readable source code into machine-executable object code. The object code is then executed directly by the computer. See also "Interpreter." +Complemented Bracket Expression + The negation of a "bracket expression". All that is _not_ + described by a given bracket expression. The symbol `^' precedes + the negated bracket expression. E.g.: `[[^:digit:]' designates + whatever character is not a digit. `[^bad]' designates whatever + character is not one of the letters `b', `a', or `d'. See + "Bracket Expression." + Compound Statement A series of `awk' statements, enclosed in curly braces. Compound statements may be nested. (*Note Statements::.) +Computed Regexps + See "Dynamic Regular Expressions." + Concatenation Concatenating two strings means sticking them together, one after another, producing a new string. For example, the string `foo' @@ -29599,11 +30092,19 @@ Conditional Expression otherwise the value is EXPR3. In either case, only one of EXPR2 and EXPR3 is evaluated. (*Note Conditional Exp::.) -Comparison Expression - A relation that is either true or false, such as `a < b'. - Comparison expressions are used in `if', `while', `do', and `for' - statements, and in patterns to select which input records to - process. (*Note Typing and Comparison::.) +Control Statement + A control statement is an instruction to perform a given operation + or a set of operations inside an `awk' program, if a given + condition is true. Control statements are: `if', `for', `while', + and `do' (*note Statements::). + +Cookie + A peculiar goodie, token, saying or remembrance produced by or + presented to a program. (With thanks to Professor Doug McIlroy.) + +Coprocess + A subordinate program with which two-way communications is + possible. Curly Braces See "Braces." @@ -29645,15 +30146,15 @@ Dynamic Regular Expression `"foo"', but it may also be an expression whose value can vary. (*Note Computed Regexps::.) +Empty String + See "Null String." + Environment A collection of strings, of the form `NAME=VAL', that each program has available to it. Users generally place values into the environment in order to provide information to various programs. Typical examples are the environment variables `HOME' and `PATH'. -Empty String - See "Null String." - Epoch The date used as the "beginning of time" for timestamps. Time values in most systems are represented as seconds since the epoch, @@ -29703,25 +30204,38 @@ Format are controlled by the format strings contained in the predefined variables `CONVFMT' and `OFMT'. (*Note Control Letters::.) +Fortran + Shorthand for FORmula TRANslator, one of the first programming + languages available for scientific calculations. It was created by + John Backus, and has been available since 1957. It is still in use + today. + Free Documentation License This document describes the terms under which this Info file is published and may be copied. (*Note GNU Free Documentation License::.) -Function - A specialized group of statements used to encapsulate general or - program-specific tasks. `awk' has a number of built-in functions, - and also allows you to define your own. (*Note Functions::.) - -FSF - See "Free Software Foundation." - Free Software Foundation A nonprofit organization dedicated to the production and distribution of freely distributable software. It was founded by Richard M. Stallman, the author of the original Emacs editor. GNU Emacs is the most widely used version of Emacs today. +FSF + See "Free Software Foundation." + +Function + A part of an `awk' program that can be invoked from every point of + the program, to perform a task. `awk' has several built-in + functions. Users can define their own functions in every part of + the program. Function can be recursive, i.e., they may invoke + themselves. *Note Functions::. In `gawk' it is also possible to + have functions shared among different programs, and included where + required using the `@include' directive (*note Include Files::). + In `gawk' the name of the function that should be invoked can be + generated at run time, i.e., dynamically. The `gawk' extension + API provides constructor functions (*note Constructor Functions::). + `gawk' The GNU implementation of `awk'. @@ -29816,17 +30330,23 @@ Keyword `else', `exit', `for...in', `for', `function', `func', `if', `next', `nextfile', `switch', and `while'. +Korn Shell + The Korn Shell (`ksh') is a Unix shell which was developed by + David Korn at Bell Laboratories in the early 1980s. The Korn Shell + is backward-compatible with the Bourne shell and includes many + features of the C shell. See also "Bourne Shell." + Lesser General Public License This document describes the terms under which binary library archives or shared objects, and their source code may be distributed. -Linux - See "GNU/Linux." - LGPL See "Lesser General Public License." +Linux + See "GNU/Linux." + Localization The process of providing the data necessary for an internationalized program to work in a particular language. @@ -29853,6 +30373,13 @@ Metacharacters Instead, they denote regular expression operations, such as repetition, grouping, or alternation. +Nesting + Nesting is where information is organized in layers, or where + objects contain other similar objects. In `gawk' the `@include' + directive can be nested. The "natural" nesting of arithmetic and + logical operations can be changed using parentheses (*note + Precedence::). + No-op An operation that does nothing. @@ -29872,6 +30399,11 @@ Octal are written in C using a leading `0', to indicate their base. Thus, `013' is 11 ((1 x 8) + 3). *Note Nondecimal-numbers::. +Output Record + A single chunk of data that is written out by `awk'. Usually, an + `awk' output record consists of one or more lines of text. *Note + Records::. + Pattern Patterns tell `awk' which input records are interesting to which rules. @@ -29887,6 +30419,9 @@ PEBKAC computer usage problems. (Problem Exists Between Keyboard And Chair.) +Plug-in + See "Extensions." + POSIX The name for a series of standards that specify a Portable Operating System interface. The "IX" denotes the Unix heritage of @@ -29910,6 +30445,9 @@ Range (of input lines) can specify ranges of input lines for `awk' to process or it can specify single lines. (*Note Pattern Overview::.) +Record + See "Input record" and "Output record." + Recursion When a function calls itself, either directly or indirectly. If this is clear, stop, and proceed to the next entry. Otherwise, @@ -29926,6 +30464,16 @@ Redirection using the `>', `>>', `|', and `|&' operators. (*Note Getline::, and *note Redirection::.) +Reference Counts + An internal mechanism in `gawk' to minimize the amount of memory + needed to store the value of string variables. If the value + assumed by a variable is used in more than one place, only one + copy of the value itself is kept, and the associated reference + count is increased when the same value is used by an additional + variable, and decresed when the related variable is no longer in + use. When the reference count goes to zero, the memory space used + to store the value of the variable is freed. + Regexp See "Regular Expression." @@ -29944,6 +30492,15 @@ Regular Expression Constant when you write the `awk' program and cannot be changed during its execution. (*Note Regexp Usage::.) +Regular Expression Operators + See "Metacharacters." + +Rounding + Rounding the result of an arithmetic operation can be tricky. + More than one way of rounding exists, and in `gawk' it is possible + to choose which method should be used in a program. *Note Setting + the rounding mode::. + Rule A segment of an `awk' program that specifies how to process single input records. A rule consists of a "pattern" and an "action". @@ -29965,13 +30522,13 @@ Search Path source files. In the shell, a list of directories to search for executable programs. +`sed' + See "Stream Editor." + Seed The initial value, or starting point, for a sequence of random numbers. -`sed' - See "Stream Editor." - Shell The command interpreter for Unix and POSIX-compliant systems. The shell works both interactively, and as a programming language for @@ -30005,6 +30562,11 @@ Special File handed directly to the underlying operating system--for example, `/dev/stderr'. (*Note Special Files::.) +Statement + An expression inside an `awk' program in the action part of a + pattern-action rule, or inside an `awk' function. A statement can + be a variable assignment, an array operation, a loop, etc. + Stream Editor A program that reads records from an input stream and processes them one or more at a time. This is in contrast with batch @@ -30047,10 +30609,15 @@ UTC reference time for day and date calculations. See also "Epoch" and "GMT." +Variable + A name for a value. In `awk', variables may be either scalars or + arrays. + Whitespace A sequence of space, TAB, or newline characters occurring inside an input record or a string. + File: gawk.info, Node: Copying, Next: GNU Free Documentation License, Prev: Glossary, Up: Top @@ -31283,7 +31850,7 @@ Index * ! (exclamation point), !~ operator <5>: Case-sensitivity. (line 26) * ! (exclamation point), !~ operator <6>: Computed Regexps. (line 6) * ! (exclamation point), !~ operator: Regexp Usage. (line 19) -* " (double quote), in regexp constants: Computed Regexps. (line 29) +* " (double quote), in regexp constants: Computed Regexps. (line 30) * " (double quote), in shell commands: Quoting. (line 54) * # (number sign), #! (executable scripts): Executable Scripts. (line 6) @@ -31312,7 +31879,7 @@ Index * * (asterisk), * operator, as regexp operator: Regexp Operators. (line 89) * * (asterisk), * operator, null strings, matching: String Functions. - (line 535) + (line 537) * * (asterisk), ** operator <1>: Precedence. (line 49) * * (asterisk), ** operator: Arithmetic Ops. (line 81) * * (asterisk), **= operator <1>: Precedence. (line 95) @@ -31343,7 +31910,7 @@ Index * --disable-lint configuration option: Additional Configuration Options. (line 15) * --disable-nls configuration option: Additional Configuration Options. - (line 30) + (line 32) * --dump-variables option: Options. (line 93) * --dump-variables option, using for library functions: Library Names. (line 45) @@ -31362,26 +31929,26 @@ Index * --non-decimal-data option: Options. (line 211) * --non-decimal-data option, strtonum() function and: Nondecimal Data. (line 35) -* --optimize option: Options. (line 237) +* --optimize option: Options. (line 236) * --posix option: Options. (line 254) * --posix option, --traditional option and: Options. (line 273) -* --pretty-print option: Options. (line 226) +* --pretty-print option: Options. (line 225) * --profile option <1>: Profiling. (line 12) * --profile option: Options. (line 242) * --re-interval option: Options. (line 279) * --sandbox option: Options. (line 286) * --sandbox option, disabling system() function: I/O Functions. - (line 96) + (line 129) * --sandbox option, input redirection with getline: Getline. (line 19) * --sandbox option, output redirection with print, printf: Redirection. (line 6) * --source option: Options. (line 117) * --traditional option: Options. (line 81) * --traditional option, --posix option and: Options. (line 273) -* --use-lc-numeric option: Options. (line 221) +* --use-lc-numeric option: Options. (line 220) * --version option: Options. (line 300) * --with-whiny-user-strftime configuration option: Additional Configuration Options. - (line 35) + (line 37) * -b option: Options. (line 68) * -C option: Options. (line 88) * -c option: Options. (line 81) @@ -31403,10 +31970,10 @@ Index * -L option: Options. (line 295) * -l option: Options. (line 173) * -M option: Options. (line 205) -* -N option: Options. (line 221) +* -N option: Options. (line 220) * -n option: Options. (line 211) -* -O option: Options. (line 237) -* -o option: Options. (line 226) +* -O option: Options. (line 236) +* -o option: Options. (line 225) * -P option: Options. (line 254) * -p option: Options. (line 242) * -r option: Options. (line 279) @@ -31417,7 +31984,7 @@ Index * -W option: Options. (line 46) * . (period), regexp operator: Regexp Operators. (line 44) * .gmo files: Explaining gettext. (line 42) -* .gmo files, specifying directory of <1>: Programmer i18n. (line 47) +* .gmo files, specifying directory of <1>: Programmer i18n. (line 48) * .gmo files, specifying directory of: Explaining gettext. (line 54) * .mo files, converting from .po: I18N Example. (line 64) * .po files <1>: Translator i18n. (line 6) @@ -31474,10 +32041,10 @@ Index (line 8) * [] (square brackets), regexp operator: Regexp Operators. (line 56) * \ (backslash): Comments. (line 50) -* \ (backslash), \" escape sequence: Escape Sequences. (line 84) +* \ (backslash), \" escape sequence: Escape Sequences. (line 85) * \ (backslash), \' operator (gawk): GNU Regexp Operators. (line 56) -* \ (backslash), \/ escape sequence: Escape Sequences. (line 75) +* \ (backslash), \/ escape sequence: Escape Sequences. (line 76) * \ (backslash), \< operator (gawk): GNU Regexp Operators. (line 30) * \ (backslash), \> operator (gawk): GNU Regexp Operators. @@ -31517,8 +32084,8 @@ Index * \ (backslash), in bracket expressions: Bracket Expressions. (line 17) * \ (backslash), in escape sequences: Escape Sequences. (line 6) * \ (backslash), in escape sequences, POSIX and: Escape Sequences. - (line 120) -* \ (backslash), in regexp constants: Computed Regexps. (line 29) + (line 108) +* \ (backslash), in regexp constants: Computed Regexps. (line 30) * \ (backslash), in shell commands: Quoting. (line 48) * \ (backslash), regexp operator: Regexp Operators. (line 18) * ^ (caret), ^ operator: Precedence. (line 49) @@ -31546,17 +32113,17 @@ Index * actions, control statements in: Statements. (line 6) * actions, default: Very Simple. (line 34) * actions, empty: Very Simple. (line 39) -* Ada programming language: Glossary. (line 19) +* Ada programming language: Glossary. (line 11) * adding, features to gawk: Adding Code. (line 6) * adding, fields: Changing Fields. (line 53) -* advanced features, fixed-width data: Constant Size. (line 10) +* advanced features, fixed-width data: Constant Size. (line 6) * advanced features, gawk: Advanced Features. (line 6) * advanced features, network programming: TCP/IP Networking. (line 6) * advanced features, nondecimal input data: Nondecimal Data. (line 6) * advanced features, processes, communicating with: Two-way I/O. (line 6) * advanced features, specifying field content: Splitting By Content. - (line 10) + (line 9) * Aho, Alfred <1>: Contributors. (line 11) * Aho, Alfred: History. (line 17) * alarm clock example program: Alarm Program. (line 11) @@ -31564,7 +32131,7 @@ Index * algorithms: Basic High Level. (line 68) * allocating memory for extensions: Memory Allocation Functions. (line 6) -* amazing awk assembler (aaa): Glossary. (line 11) +* amazing awk assembler (aaa): Glossary. (line 16) * amazingly workable formatter (awf): Glossary. (line 24) * ambiguity, syntactic: /= operator vs. /=.../ regexp constant: Assignment Ops. (line 148) @@ -31586,7 +32153,7 @@ Index (line 6) * arbitrary precision integers: Arbitrary Precision Integers. (line 6) -* archeologists: Bugs. (line 6) +* archaeologists: Bugs. (line 6) * arctangent: Numeric Functions. (line 11) * ARGC/ARGV variables: Auto-set. (line 15) * ARGC/ARGV variables, command-line arguments: Other Arguments. @@ -31608,13 +32175,13 @@ Index (line 6) * array scanning order, controlling: Controlling Scanning. (line 14) -* array, number of elements: String Functions. (line 200) +* array, number of elements: String Functions. (line 201) * arrays: Arrays. (line 6) * arrays of arrays: Arrays of Arrays. (line 6) * arrays, an example of using: Array Example. (line 6) -* arrays, and IGNORECASE variable: Array Intro. (line 94) +* arrays, and IGNORECASE variable: Array Intro. (line 100) * arrays, as parameters to functions: Pass By Value/Reference. - (line 47) + (line 44) * arrays, associative: Array Intro. (line 50) * arrays, associative, library functions and: Library Names. (line 58) * arrays, deleting entire contents: Delete. (line 39) @@ -31624,7 +32191,7 @@ Index * arrays, elements, deleting: Delete. (line 6) * arrays, elements, order of access by in operator: Scanning an Array. (line 48) -* arrays, elements, retrieving number of: String Functions. (line 41) +* arrays, elements, retrieving number of: String Functions. (line 42) * arrays, for statement and: Scanning an Array. (line 20) * arrays, indexing: Array Intro. (line 50) * arrays, merging into strings: Join Function. (line 6) @@ -31639,23 +32206,23 @@ Index (line 6) * arrays, sorting, and IGNORECASE variable: Array Sorting Functions. (line 83) -* arrays, sparse: Array Intro. (line 72) +* arrays, sparse: Array Intro. (line 76) * arrays, subscripts, uninitialized variables as: Uninitialized Subscripts. (line 6) * arrays, unassigned elements: Reference to Elements. (line 18) * artificial intelligence, gawk and: Distribution contents. (line 52) -* ASCII <1>: Glossary. (line 133) +* ASCII <1>: Glossary. (line 197) * ASCII: Ordinal Functions. (line 45) * asort <1>: Array Sorting Functions. (line 6) -* asort: String Functions. (line 41) +* asort: String Functions. (line 42) * asort() function (gawk), arrays, sorting: Array Sorting Functions. (line 6) * asorti <1>: Array Sorting Functions. (line 6) -* asorti: String Functions. (line 41) +* asorti: String Functions. (line 42) * asorti() function (gawk), arrays, sorting: Array Sorting Functions. (line 6) * assert() function (C library): Assert Function. (line 6) @@ -31673,7 +32240,7 @@ Index * asterisk (*), * operator, as regexp operator: Regexp Operators. (line 89) * asterisk (*), * operator, null strings, matching: String Functions. - (line 535) + (line 537) * asterisk (*), ** operator <1>: Precedence. (line 49) * asterisk (*), ** operator: Arithmetic Ops. (line 81) * asterisk (*), **= operator <1>: Precedence. (line 95) @@ -31734,7 +32301,7 @@ Index * awk, versions of, See Also Brian Kernighan's awk <1>: Other Versions. (line 13) * awk, versions of, See Also Brian Kernighan's awk: BTL. (line 6) -* awka compiler for awk: Other Versions. (line 64) +* awka compiler for awk: Other Versions. (line 68) * AWKLIBPATH environment variable: AWKLIBPATH Variable. (line 6) * AWKPATH environment variable <1>: PC Using. (line 10) * AWKPATH environment variable: AWKPATH Variable. (line 6) @@ -31743,10 +32310,10 @@ Index * awkvars.out file: Options. (line 93) * b debugger command (alias for break): Breakpoint Control. (line 11) * backslash (\): Comments. (line 50) -* backslash (\), \" escape sequence: Escape Sequences. (line 84) +* backslash (\), \" escape sequence: Escape Sequences. (line 85) * backslash (\), \' operator (gawk): GNU Regexp Operators. (line 56) -* backslash (\), \/ escape sequence: Escape Sequences. (line 75) +* backslash (\), \/ escape sequence: Escape Sequences. (line 76) * backslash (\), \< operator (gawk): GNU Regexp Operators. (line 30) * backslash (\), \> operator (gawk): GNU Regexp Operators. @@ -31786,12 +32353,12 @@ Index * backslash (\), in bracket expressions: Bracket Expressions. (line 17) * backslash (\), in escape sequences: Escape Sequences. (line 6) * backslash (\), in escape sequences, POSIX and: Escape Sequences. - (line 120) -* backslash (\), in regexp constants: Computed Regexps. (line 29) + (line 108) +* backslash (\), in regexp constants: Computed Regexps. (line 30) * backslash (\), in shell commands: Quoting. (line 48) * backslash (\), regexp operator: Regexp Operators. (line 18) * backtrace debugger command: Execution Stack. (line 13) -* Beebe, Nelson H.F. <1>: Other Versions. (line 78) +* Beebe, Nelson H.F. <1>: Other Versions. (line 82) * Beebe, Nelson H.F.: Acknowledgments. (line 60) * BEGIN pattern <1>: Using BEGIN/END. (line 6) * BEGIN pattern <2>: BEGIN/END. (line 6) @@ -31808,7 +32375,7 @@ Index * BEGIN pattern, next/nextfile statements and: I/O And BEGIN/END. (line 37) * BEGIN pattern, OFS/ORS variables, assigning values to: Output Separators. - (line 20) + (line 21) * BEGIN pattern, operators and: Using BEGIN/END. (line 17) * BEGIN pattern, print statement and: I/O And BEGIN/END. (line 16) * BEGIN pattern, pwcat program: Passwd Functions. (line 143) @@ -31817,13 +32384,13 @@ Index * BEGINFILE pattern: BEGINFILE/ENDFILE. (line 6) * BEGINFILE pattern, Boolean patterns and: Expression Patterns. (line 70) -* beginfile() user-defined function: Filetrans Function. (line 61) -* Bentley, Jon: Glossary. (line 143) +* beginfile() user-defined function: Filetrans Function. (line 62) +* Bentley, Jon: Glossary. (line 207) * Benzinger, Michael: Contributors. (line 97) * Berry, Karl <1>: Ranges and Locales. (line 74) * Berry, Karl: Acknowledgments. (line 33) * binary input/output: User-modified. (line 15) -* bindtextdomain <1>: Programmer i18n. (line 47) +* bindtextdomain <1>: Programmer i18n. (line 48) * bindtextdomain: I18N Functions. (line 12) * bindtextdomain() function (C library): Explaining gettext. (line 50) * bindtextdomain() function (gawk), portability and: I18N Portability. @@ -31876,10 +32443,11 @@ Index * Brennan, Michael <2>: Simple Sed. (line 25) * Brennan, Michael <3>: Delete. (line 56) * Brennan, Michael <4>: Acknowledgments. (line 78) -* Brennan, Michael: Foreword. (line 83) +* Brennan, Michael <5>: Foreword4. (line 33) +* Brennan, Michael: Foreword3. (line 84) * Brian Kernighan's awk <1>: I/O Functions. (line 43) * Brian Kernighan's awk <2>: Gory Details. (line 19) -* Brian Kernighan's awk <3>: String Functions. (line 491) +* Brian Kernighan's awk <3>: String Functions. (line 493) * Brian Kernighan's awk <4>: Delete. (line 51) * Brian Kernighan's awk <5>: Nextfile Statement. (line 47) * Brian Kernighan's awk <6>: Continue Statement. (line 44) @@ -31890,8 +32458,8 @@ Index * Brian Kernighan's awk <11>: Regexp Field Splitting. (line 67) * Brian Kernighan's awk <12>: GNU Regexp Operators. - (line 83) -* Brian Kernighan's awk <13>: Escape Sequences. (line 124) + (line 82) +* Brian Kernighan's awk <13>: Escape Sequences. (line 112) * Brian Kernighan's awk: When. (line 21) * Brian Kernighan's awk, extensions: BTL. (line 6) * Brian Kernighan's awk, source code: Other Versions. (line 13) @@ -31899,14 +32467,14 @@ Index * Brink, Jeroen: DOS Quoting. (line 10) * Broder, Alan J.: Contributors. (line 88) * Brown, Martin: Contributors. (line 82) -* BSD-based operating systems: Glossary. (line 611) +* BSD-based operating systems: Glossary. (line 753) * bt debugger command (alias for backtrace): Execution Stack. (line 13) -* Buening, Andreas <1>: Bugs. (line 72) +* Buening, Andreas <1>: Bugs. (line 71) * Buening, Andreas <2>: Contributors. (line 92) * Buening, Andreas: Acknowledgments. (line 60) * buffering, input/output <1>: Two-way I/O. (line 52) -* buffering, input/output: I/O Functions. (line 139) -* buffering, interactive vs. noninteractive: I/O Functions. (line 108) +* buffering, input/output: I/O Functions. (line 141) +* buffering, interactive vs. noninteractive: I/O Functions. (line 76) * buffers, flushing: I/O Functions. (line 32) * buffers, operators for: GNU Regexp Operators. (line 48) @@ -31914,12 +32482,12 @@ Index * bug-gawk@gnu.org bug reporting address: Bugs. (line 30) * built-in functions: Functions. (line 6) * built-in functions, evaluation order: Calling Built-in. (line 30) -* Busybox Awk: Other Versions. (line 88) +* BusyBox Awk: Other Versions. (line 92) * c.e., See common extensions: Conventions. (line 51) * call by reference: Pass By Value/Reference. - (line 47) + (line 44) * call by value: Pass By Value/Reference. - (line 18) + (line 15) * call stack, display in debugger: Execution Stack. (line 13) * caret (^), ^ operator: Precedence. (line 49) * caret (^), ^= operator <1>: Precedence. (line 95) @@ -31931,8 +32499,8 @@ Index * case keyword: Switch Statement. (line 6) * case sensitivity, and regexps: User-modified. (line 76) * case sensitivity, and string comparisons: User-modified. (line 76) -* case sensitivity, array indices and: Array Intro. (line 94) -* case sensitivity, converting case: String Functions. (line 521) +* case sensitivity, array indices and: Array Intro. (line 100) +* case sensitivity, converting case: String Functions. (line 523) * case sensitivity, example programs: Library Functions. (line 53) * case sensitivity, gawk: Case-sensitivity. (line 26) * case sensitivity, regexps and: Case-sensitivity. (line 6) @@ -31941,7 +32509,7 @@ Index (line 56) * character lists in regular expression: Bracket Expressions. (line 6) * character lists, See bracket expressions: Regexp Operators. (line 56) -* character sets (machine character encodings) <1>: Glossary. (line 133) +* character sets (machine character encodings) <1>: Glossary. (line 197) * character sets (machine character encodings): Ordinal Functions. (line 45) * character sets, See Also bracket expressions: Regexp Operators. @@ -31952,7 +32520,7 @@ Index * Chassell, Robert J.: Acknowledgments. (line 33) * chdir() extension function: Extension Sample File Functions. (line 12) -* chem utility: Glossary. (line 143) +* chem utility: Glossary. (line 207) * chr() extension function: Extension Sample Ord. (line 15) * chr() user-defined function: Ordinal Functions. (line 16) @@ -31968,7 +32536,7 @@ Index * close() function, portability: Close Files And Pipes. (line 81) * close() function, return value: Close Files And Pipes. - (line 132) + (line 133) * close() function, two-way pipes and: Two-way I/O. (line 59) * Close, Diane <1>: Contributors. (line 20) * Close, Diane: Manual History. (line 34) @@ -32010,9 +32578,9 @@ Index * common extensions, \x escape sequence: Escape Sequences. (line 61) * common extensions, BINMODE variable: PC Using. (line 33) * common extensions, delete to delete entire arrays: Delete. (line 39) -* common extensions, func keyword: Definition Syntax. (line 93) +* common extensions, func keyword: Definition Syntax. (line 98) * common extensions, length() applied to an array: String Functions. - (line 200) + (line 201) * common extensions, RS as a regexp: gawk split records. (line 6) * common extensions, single character fields: Single Character Fields. (line 6) @@ -32029,7 +32597,7 @@ Index * compatibility mode (gawk), octal numbers: Nondecimal-numbers. (line 60) * compatibility mode (gawk), specifying: Options. (line 81) -* compiled programs <1>: Glossary. (line 157) +* compiled programs <1>: Glossary. (line 219) * compiled programs: Basic High Level. (line 15) * compiling gawk for Cygwin: Cygwin. (line 6) * compiling gawk for MS-DOS and MS-Windows: PC Compiling. (line 13) @@ -32046,9 +32614,9 @@ Index * configuration option, --disable-lint: Additional Configuration Options. (line 15) * configuration option, --disable-nls: Additional Configuration Options. - (line 30) + (line 32) * configuration option, --with-whiny-user-strftime: Additional Configuration Options. - (line 35) + (line 37) * configuration options, gawk: Additional Configuration Options. (line 6) * constant regexps: Regexp Usage. (line 57) @@ -32061,9 +32629,9 @@ Index * control statements: Statements. (line 6) * controlling array scanning order: Controlling Scanning. (line 14) -* convert string to lower case: String Functions. (line 522) -* convert string to number: String Functions. (line 389) -* convert string to upper case: String Functions. (line 528) +* convert string to lower case: String Functions. (line 524) +* convert string to number: String Functions. (line 391) +* convert string to upper case: String Functions. (line 530) * converting integer array subscripts: Numeric Array Subscripts. (line 31) * converting, dates to timestamps: Time Functions. (line 76) @@ -32075,7 +32643,7 @@ Index * CONVFMT variable: Strings And Numbers. (line 29) * CONVFMT variable, and array subscripts: Numeric Array Subscripts. (line 6) -* cookie: Glossary. (line 149) +* cookie: Glossary. (line 258) * coprocesses <1>: Two-way I/O. (line 25) * coprocesses: Redirection. (line 96) * coprocesses, closing: Close Files And Pipes. @@ -32099,7 +32667,7 @@ Index * cut.awk program: Cut Program. (line 45) * d debugger command (alias for delete): Breakpoint Control. (line 64) * d.c., See dark corner: Conventions. (line 42) -* dark corner <1>: Glossary. (line 188) +* dark corner <1>: Glossary. (line 269) * dark corner: Conventions. (line 42) * dark corner, "0" is actually true: Truth Values. (line 24) * dark corner, /= operator vs. /=.../ regexp constant: Assignment Ops. @@ -32110,40 +32678,41 @@ Index (line 43) * dark corner, break statement: Break Statement. (line 51) * dark corner, close() function: Close Files And Pipes. - (line 132) + (line 133) * dark corner, command-line arguments: Assignment Options. (line 43) * dark corner, continue statement: Continue Statement. (line 44) * dark corner, CONVFMT variable: Strings And Numbers. (line 40) * dark corner, escape sequences: Other Arguments. (line 38) * dark corner, escape sequences, for metacharacters: Escape Sequences. - (line 142) + (line 143) * dark corner, exit statement: Exit Statement. (line 30) -* dark corner, field separators: Field Splitting Summary. - (line 46) -* dark corner, FILENAME variable <1>: Auto-set. (line 98) +* dark corner, field separators: Full Line Fields. (line 22) +* dark corner, FILENAME variable <1>: Auto-set. (line 109) * dark corner, FILENAME variable: Getline Notes. (line 19) -* dark corner, FNR/NR variables: Auto-set. (line 321) +* dark corner, FNR/NR variables: Auto-set. (line 341) * dark corner, format-control characters: Control Letters. (line 18) * dark corner, FS as null string: Single Character Fields. (line 20) * dark corner, input files: awk split records. (line 111) * dark corner, invoking awk: Command Line. (line 16) -* dark corner, length() function: String Functions. (line 186) +* dark corner, length() function: String Functions. (line 187) * dark corner, locale's decimal point character: Locale influences conversions. (line 17) * dark corner, multiline records: Multiple Line. (line 35) * dark corner, NF variable, decrementing: Changing Fields. (line 107) * dark corner, OFMT variable: OFMT. (line 27) +* dark corner, regexp as second argument to index(): String Functions. + (line 165) * dark corner, regexp constants: Using Constant Regexps. (line 6) * dark corner, regexp constants, /= operator and: Assignment Ops. (line 148) * dark corner, regexp constants, as arguments to user-defined functions: Using Constant Regexps. (line 43) -* dark corner, split() function: String Functions. (line 360) +* dark corner, split() function: String Functions. (line 362) * dark corner, strings, storing: gawk split records. (line 83) * dark corner, value of ARGV[0]: Auto-set. (line 39) -* data, fixed-width: Constant Size. (line 10) +* data, fixed-width: Constant Size. (line 6) * data-driven languages: Basic High Level. (line 85) * database, group, reading: Group Functions. (line 6) * database, users, reading: Passwd Functions. (line 6) @@ -32155,11 +32724,11 @@ Index * Davies, Stephen <1>: Contributors. (line 74) * Davies, Stephen: Acknowledgments. (line 60) * Day, Robert P.J.: Acknowledgments. (line 78) -* dcgettext <1>: Programmer i18n. (line 19) +* dcgettext <1>: Programmer i18n. (line 20) * dcgettext: I18N Functions. (line 22) * dcgettext() function (gawk), portability and: I18N Portability. (line 33) -* dcngettext <1>: Programmer i18n. (line 36) +* dcngettext <1>: Programmer i18n. (line 37) * dcngettext: I18N Functions. (line 28) * dcngettext() function (gawk), portability and: I18N Portability. (line 33) @@ -32246,7 +32815,7 @@ Index * debugger commands, t (tbreak): Breakpoint Control. (line 90) * debugger commands, tbreak: Breakpoint Control. (line 90) * debugger commands, trace: Miscellaneous Debugger Commands. - (line 108) + (line 107) * debugger commands, u (until): Debugger Execution Control. (line 83) * debugger commands, undisplay: Viewing And Changing Data. @@ -32262,18 +32831,18 @@ Index (line 67) * debugger commands, where (backtrace): Execution Stack. (line 13) * debugger default list amount: Debugger Info. (line 69) -* debugger history file: Debugger Info. (line 80) +* debugger history file: Debugger Info. (line 81) * debugger history size: Debugger Info. (line 65) * debugger options: Debugger Info. (line 57) -* debugger prompt: Debugger Info. (line 77) +* debugger prompt: Debugger Info. (line 78) * debugger, how to start: Debugger Invocation. (line 6) -* debugger, read commands from a file: Debugger Info. (line 96) +* debugger, read commands from a file: Debugger Info. (line 97) * debugging awk programs: Debugger. (line 6) * debugging gawk, bug reports: Bugs. (line 9) * decimal point character, locale specific: Options. (line 270) * decrement operators: Increment Ops. (line 35) * default keyword: Switch Statement. (line 6) -* Deifik, Scott <1>: Bugs. (line 72) +* Deifik, Scott <1>: Bugs. (line 71) * Deifik, Scott <2>: Contributors. (line 53) * Deifik, Scott: Acknowledgments. (line 60) * delete ARRAY: Delete. (line 39) @@ -32287,7 +32856,7 @@ Index * deleting entire arrays: Delete. (line 39) * Demaille, Akim: Acknowledgments. (line 60) * describe call stack frame, in debugger: Debugger Info. (line 27) -* differences between gawk and awk: String Functions. (line 200) +* differences between gawk and awk: String Functions. (line 201) * differences in awk and gawk, ARGC/ARGV variables: ARGC and ARGV. (line 90) * differences in awk and gawk, ARGIND variable: Auto-set. (line 44) @@ -32309,12 +32878,12 @@ Index (line 81) * differences in awk and gawk, command-line directories: Command-line directories. (line 6) -* differences in awk and gawk, ERRNO variable: Auto-set. (line 82) +* differences in awk and gawk, ERRNO variable: Auto-set. (line 88) * differences in awk and gawk, error messages: Special FD. (line 19) * differences in awk and gawk, FIELDWIDTHS variable: User-modified. (line 37) * differences in awk and gawk, FPAT variable: User-modified. (line 43) -* differences in awk and gawk, FUNCTAB variable: Auto-set. (line 123) +* differences in awk and gawk, FUNCTAB variable: Auto-set. (line 135) * differences in awk and gawk, function arguments (gawk): Calling Built-in. (line 16) * differences in awk and gawk, getline command: Getline. (line 19) @@ -32332,12 +32901,12 @@ Index (line 6) * differences in awk and gawk, line continuations: Conditional Exp. (line 34) -* differences in awk and gawk, LINT variable: User-modified. (line 88) +* differences in awk and gawk, LINT variable: User-modified. (line 87) * differences in awk and gawk, match() function: String Functions. - (line 262) + (line 263) * differences in awk and gawk, print/printf statements: Format Modifiers. (line 13) -* differences in awk and gawk, PROCINFO array: Auto-set. (line 137) +* differences in awk and gawk, PROCINFO array: Auto-set. (line 149) * differences in awk and gawk, read timeouts: Read Timeout. (line 6) * differences in awk and gawk, record separators: awk split records. (line 125) @@ -32345,17 +32914,19 @@ Index (line 43) * differences in awk and gawk, regular expressions: Case-sensitivity. (line 26) +* differences in awk and gawk, retrying input: Retrying Input. + (line 6) * differences in awk and gawk, RS/RT variables: gawk split records. (line 58) -* differences in awk and gawk, RT variable: Auto-set. (line 272) +* differences in awk and gawk, RT variable: Auto-set. (line 292) * differences in awk and gawk, single-character fields: Single Character Fields. (line 6) * differences in awk and gawk, split() function: String Functions. - (line 348) + (line 349) * differences in awk and gawk, strings: Scalar Constants. (line 20) * differences in awk and gawk, strings, storing: gawk split records. (line 77) -* differences in awk and gawk, SYMTAB variable: Auto-set. (line 276) +* differences in awk and gawk, SYMTAB variable: Auto-set. (line 296) * differences in awk and gawk, TEXTDOMAIN variable: User-modified. (line 151) * differences in awk and gawk, trunc-mod operation: Arithmetic Ops. @@ -32371,7 +32942,6 @@ Index * display debugger command: Viewing And Changing Data. (line 8) * display debugger options: Debugger Info. (line 57) -* div: Numeric Functions. (line 18) * division: Arithmetic Ops. (line 44) * do-while statement: Do Statement. (line 6) * do-while statement, use of regexps in: Regexp Usage. (line 19) @@ -32383,7 +32953,7 @@ Index * dollar sign ($), incrementing fields and arrays: Increment Ops. (line 30) * dollar sign ($), regexp operator: Regexp Operators. (line 35) -* double quote ("), in regexp constants: Computed Regexps. (line 29) +* double quote ("), in regexp constants: Computed Regexps. (line 30) * double quote ("), in shell commands: Quoting. (line 54) * down debugger command: Execution Stack. (line 23) * Drepper, Ulrich: Acknowledgments. (line 52) @@ -32396,8 +32966,8 @@ Index * dynamically loaded extensions: Dynamic Extensions. (line 6) * e debugger command (alias for enable): Breakpoint Control. (line 73) * EBCDIC: Ordinal Functions. (line 45) -* effective group ID of gawk user: Auto-set. (line 142) -* effective user ID of gawk user: Auto-set. (line 146) +* effective group ID of gawk user: Auto-set. (line 154) +* effective user ID of gawk user: Auto-set. (line 162) * egrep utility <1>: Egrep Program. (line 6) * egrep utility: Bracket Expressions. (line 26) * egrep.awk program: Egrep Program. (line 54) @@ -32435,9 +33005,9 @@ Index * END pattern, print statement and: I/O And BEGIN/END. (line 16) * ENDFILE pattern: BEGINFILE/ENDFILE. (line 6) * ENDFILE pattern, Boolean patterns and: Expression Patterns. (line 70) -* endfile() user-defined function: Filetrans Function. (line 61) -* endgrent() function (C library): Group Functions. (line 211) -* endgrent() user-defined function: Group Functions. (line 214) +* endfile() user-defined function: Filetrans Function. (line 62) +* endgrent() function (C library): Group Functions. (line 212) +* endgrent() user-defined function: Group Functions. (line 215) * endpwent() function (C library): Passwd Functions. (line 207) * endpwent() user-defined function: Passwd Functions. (line 210) * English, Steve: Advanced Features. (line 6) @@ -32445,20 +33015,20 @@ Index * environment variables used by gawk: Environment Variables. (line 6) * environment variables, in ENVIRON array: Auto-set. (line 60) -* epoch, definition of: Glossary. (line 234) +* epoch, definition of: Glossary. (line 315) * equals sign (=), = operator: Assignment Ops. (line 6) * equals sign (=), == operator <1>: Precedence. (line 65) * equals sign (=), == operator: Comparison Operators. (line 11) * EREs (Extended Regular Expressions): Bracket Expressions. (line 26) * ERRNO variable <1>: TCP/IP Networking. (line 54) -* ERRNO variable: Auto-set. (line 82) +* ERRNO variable: Auto-set. (line 88) * ERRNO variable, with BEGINFILE pattern: BEGINFILE/ENDFILE. (line 26) * ERRNO variable, with close() function: Close Files And Pipes. - (line 140) + (line 141) * ERRNO variable, with getline command: Getline. (line 19) * error handling: Special FD. (line 19) -* error handling, ERRNO variable and: Auto-set. (line 82) +* error handling, ERRNO variable and: Auto-set. (line 88) * error output: Special FD. (line 6) * escape processing, gsub()/gensub()/sub() functions: Gory Details. (line 6) @@ -32488,13 +33058,13 @@ Index * exclamation point (!), !~ operator: Regexp Usage. (line 19) * exit statement: Exit Statement. (line 6) * exit status, of gawk: Exit Status. (line 6) -* exit status, of VMS: VMS Running. (line 29) +* exit status, of VMS: VMS Running. (line 28) * exit the debugger: Miscellaneous Debugger Commands. (line 99) -* exp: Numeric Functions. (line 33) -* expand utility: Very Simple. (line 72) -* Expat XML parser library: gawkextlib. (line 31) -* exponent: Numeric Functions. (line 33) +* exp: Numeric Functions. (line 18) +* expand utility: Very Simple. (line 73) +* Expat XML parser library: gawkextlib. (line 37) +* exponent: Numeric Functions. (line 18) * expressions: Expressions. (line 6) * expressions, as patterns: Expression Patterns. (line 6) * expressions, assignment: Assignment Ops. (line 6) @@ -32512,7 +33082,7 @@ Index (line 6) * extension API version: Extension Versioning. (line 6) -* extension API, version number: Auto-set. (line 239) +* extension API, version number: Auto-set. (line 255) * extension example: Extension Example. (line 6) * extension registration: Registration Functions. (line 6) @@ -32531,9 +33101,9 @@ Index * extensions, common, BINMODE variable: PC Using. (line 33) * extensions, common, delete to delete entire arrays: Delete. (line 39) * extensions, common, fflush() function: I/O Functions. (line 43) -* extensions, common, func keyword: Definition Syntax. (line 93) +* extensions, common, func keyword: Definition Syntax. (line 98) * extensions, common, length() applied to an array: String Functions. - (line 200) + (line 201) * extensions, common, RS as a regexp: gawk split records. (line 6) * extensions, common, single character fields: Single Character Fields. (line 6) @@ -32561,8 +33131,7 @@ Index * field separator, in multiline records: Multiple Line. (line 41) * field separator, on command line: Command Line Field Separator. (line 6) -* field separator, POSIX and: Field Splitting Summary. - (line 40) +* field separator, POSIX and: Full Line Fields. (line 16) * field separators <1>: User-modified. (line 50) * field separators: Field Separators. (line 15) * field separators, choice of: Field Separators. (line 51) @@ -32588,18 +33157,18 @@ Index * fields, single-character: Single Character Fields. (line 6) * FIELDWIDTHS variable <1>: User-modified. (line 37) -* FIELDWIDTHS variable: Constant Size. (line 23) +* FIELDWIDTHS variable: Constant Size. (line 22) * file descriptors: Special FD. (line 6) * file inclusion, @include directive: Include Files. (line 8) * file names, distinguishing: Auto-set. (line 56) * file names, in compatibility mode: Special Caveats. (line 9) * file names, standard streams in gawk: Special FD. (line 48) -* FILENAME variable <1>: Auto-set. (line 98) +* FILENAME variable <1>: Auto-set. (line 109) * FILENAME variable: Reading Files. (line 6) * FILENAME variable, getline, setting with: Getline Notes. (line 19) * filenames, assignments as: Ignoring Assigns. (line 6) * files, .gmo: Explaining gettext. (line 42) -* files, .gmo, specifying directory of <1>: Programmer i18n. (line 47) +* files, .gmo, specifying directory of <1>: Programmer i18n. (line 48) * files, .gmo, specifying directory of: Explaining gettext. (line 54) * files, .mo, converting from .po: I18N Example. (line 64) * files, .po <1>: Translator i18n. (line 6) @@ -32626,7 +33195,7 @@ Index * files, message object, converting from portable object files: I18N Example. (line 64) * files, message object, specifying directory of <1>: Programmer i18n. - (line 47) + (line 48) * files, message object, specifying directory of: Explaining gettext. (line 54) * files, multiple passes over: Other Arguments. (line 56) @@ -32648,23 +33217,23 @@ Index * files, source, search path for: Programs Exercises. (line 70) * files, splitting: Split Program. (line 6) * files, Texinfo, extracting programs from: Extract Program. (line 6) -* find substring in string: String Functions. (line 155) +* find substring in string: String Functions. (line 156) * finding extensions: Finding Extensions. (line 6) * finish debugger command: Debugger Execution Control. (line 39) * Fish, Fred: Contributors. (line 50) -* fixed-width data: Constant Size. (line 10) +* fixed-width data: Constant Size. (line 6) * flag variables <1>: Tee Program. (line 20) * flag variables: Boolean Ops. (line 69) * floating-point, numbers, arbitrary precision: Arbitrary Precision Arithmetic. (line 6) -* floating-point, VAX/VMS: VMS Running. (line 51) +* floating-point, VAX/VMS: VMS Running. (line 50) * flush buffered output: I/O Functions. (line 28) * fnmatch() extension function: Extension Sample Fnmatch. (line 12) -* FNR variable <1>: Auto-set. (line 107) +* FNR variable <1>: Auto-set. (line 119) * FNR variable: Records. (line 6) -* FNR variable, changing: Auto-set. (line 321) +* FNR variable, changing: Auto-set. (line 341) * for statement: For Statement. (line 6) * for statement, looping over arrays: Scanning an Array. (line 20) * fork() extension function: Extension Sample Fork. @@ -32678,7 +33247,7 @@ Index * format time string: Time Functions. (line 48) * formats, numeric output: OFMT. (line 6) * formatting output: Printf. (line 6) -* formatting strings: String Functions. (line 382) +* formatting strings: String Functions. (line 384) * forward slash (/) to enclose regular expressions: Regexp. (line 10) * forward slash (/), / operator: Precedence. (line 55) * forward slash (/), /= operator <1>: Precedence. (line 95) @@ -32688,14 +33257,14 @@ Index * forward slash (/), patterns and: Expression Patterns. (line 24) * FPAT variable <1>: User-modified. (line 43) * FPAT variable: Splitting By Content. - (line 27) + (line 25) * frame debugger command: Execution Stack. (line 27) * Free Documentation License (FDL): GNU Free Documentation License. (line 7) -* Free Software Foundation (FSF) <1>: Glossary. (line 296) +* Free Software Foundation (FSF) <1>: Glossary. (line 375) * Free Software Foundation (FSF) <2>: Getting. (line 10) * Free Software Foundation (FSF): Manual History. (line 6) -* FreeBSD: Glossary. (line 611) +* FreeBSD: Glossary. (line 753) * FS variable <1>: User-modified. (line 50) * FS variable: Field Separators. (line 15) * FS variable, --field-separator option and: Options. (line 21) @@ -32709,19 +33278,19 @@ Index * FS, containing ^: Regexp Field Splitting. (line 59) * FS, in multiline records: Multiple Line. (line 41) -* FSF (Free Software Foundation) <1>: Glossary. (line 296) +* FSF (Free Software Foundation) <1>: Glossary. (line 375) * FSF (Free Software Foundation) <2>: Getting. (line 10) * FSF (Free Software Foundation): Manual History. (line 6) * fts() extension function: Extension Sample File Functions. (line 61) -* FUNCTAB array: Auto-set. (line 123) +* FUNCTAB array: Auto-set. (line 135) * function calls: Function Calls. (line 6) * function calls, indirect: Indirect Calls. (line 6) * function calls, indirect, @-notation for: Indirect Calls. (line 47) * function definition example: Function Example. (line 6) * function pointers: Indirect Calls. (line 6) * functions, arrays as parameters to: Pass By Value/Reference. - (line 47) + (line 44) * functions, built-in <1>: Functions. (line 6) * functions, built-in: Function Calls. (line 10) * functions, built-in, evaluation order: Calling Built-in. (line 30) @@ -32749,10 +33318,10 @@ Index * functions, library, user database, reading: Passwd Functions. (line 6) * functions, names of: Definition Syntax. (line 23) -* functions, recursive: Definition Syntax. (line 83) +* functions, recursive: Definition Syntax. (line 88) * functions, string-translation: I18N Functions. (line 6) * functions, undefined: Pass By Value/Reference. - (line 71) + (line 68) * functions, user-defined: User-defined. (line 6) * functions, user-defined, calling: Function Caveats. (line 6) * functions, user-defined, counts, in a profile: Profiling. (line 137) @@ -32764,14 +33333,14 @@ Index * G-d: Acknowledgments. (line 94) * Garfinkle, Scott: Contributors. (line 34) * gawk program, dynamic profiling: Profiling. (line 178) -* gawk version: Auto-set. (line 214) +* gawk version: Auto-set. (line 230) * gawk, ARGIND variable in: Other Arguments. (line 15) * gawk, awk and <1>: This Manual. (line 14) * gawk, awk and: Preface. (line 21) * gawk, bitwise operations in: Bitwise Functions. (line 40) * gawk, break statement in: Break Statement. (line 51) -* gawk, character classes and: Bracket Expressions. (line 100) -* gawk, coding style in: Adding Code. (line 39) +* gawk, character classes and: Bracket Expressions. (line 101) +* gawk, coding style in: Adding Code. (line 38) * gawk, command-line options, and regular expressions: GNU Regexp Operators. (line 70) * gawk, configuring: Configuration Philosophy. @@ -32782,30 +33351,30 @@ Index * gawk, distribution: Distribution contents. (line 6) * gawk, ERRNO variable in <1>: TCP/IP Networking. (line 54) -* gawk, ERRNO variable in <2>: Auto-set. (line 82) +* gawk, ERRNO variable in <2>: Auto-set. (line 88) * gawk, ERRNO variable in <3>: BEGINFILE/ENDFILE. (line 26) * gawk, ERRNO variable in <4>: Close Files And Pipes. - (line 140) + (line 141) * gawk, ERRNO variable in: Getline. (line 19) -* gawk, escape sequences: Escape Sequences. (line 132) +* gawk, escape sequences: Escape Sequences. (line 120) * gawk, extensions, disabling: Options. (line 254) * gawk, features, adding: Adding Code. (line 6) * gawk, features, advanced: Advanced Features. (line 6) * gawk, field separators and: User-modified. (line 71) * gawk, FIELDWIDTHS variable in <1>: User-modified. (line 37) -* gawk, FIELDWIDTHS variable in: Constant Size. (line 23) +* gawk, FIELDWIDTHS variable in: Constant Size. (line 22) * gawk, file names in: Special Files. (line 6) * gawk, format-control characters: Control Letters. (line 18) * gawk, FPAT variable in <1>: User-modified. (line 43) * gawk, FPAT variable in: Splitting By Content. - (line 27) -* gawk, FUNCTAB array in: Auto-set. (line 123) + (line 25) +* gawk, FUNCTAB array in: Auto-set. (line 135) * gawk, function arguments and: Calling Built-in. (line 16) * gawk, hexadecimal numbers and: Nondecimal-numbers. (line 42) * gawk, IGNORECASE variable in <1>: Array Sorting Functions. (line 83) -* gawk, IGNORECASE variable in <2>: String Functions. (line 57) -* gawk, IGNORECASE variable in <3>: Array Intro. (line 94) +* gawk, IGNORECASE variable in <2>: String Functions. (line 58) +* gawk, IGNORECASE variable in <3>: Array Intro. (line 100) * gawk, IGNORECASE variable in <4>: User-modified. (line 76) * gawk, IGNORECASE variable in: Case-sensitivity. (line 26) * gawk, implementation issues: Notes. (line 6) @@ -32821,7 +33390,7 @@ Index (line 6) * gawk, interval expressions and: Regexp Operators. (line 139) * gawk, line continuation in: Conditional Exp. (line 34) -* gawk, LINT variable in: User-modified. (line 88) +* gawk, LINT variable in: User-modified. (line 87) * gawk, list of contributors to: Contributors. (line 6) * gawk, MS-DOS version of: PC Using. (line 10) * gawk, MS-Windows version of: PC Using. (line 10) @@ -32831,7 +33400,7 @@ Index * gawk, predefined variables and: Built-in Variables. (line 14) * gawk, PROCINFO array in <1>: Two-way I/O. (line 99) * gawk, PROCINFO array in <2>: Time Functions. (line 47) -* gawk, PROCINFO array in: Auto-set. (line 137) +* gawk, PROCINFO array in: Auto-set. (line 149) * gawk, regexp constants and: Using Constant Regexps. (line 28) * gawk, regular expressions, case sensitivity: Case-sensitivity. @@ -32839,14 +33408,14 @@ Index * gawk, regular expressions, operators: GNU Regexp Operators. (line 6) * gawk, regular expressions, precedence: Regexp Operators. (line 161) -* gawk, RT variable in <1>: Auto-set. (line 272) +* gawk, RT variable in <1>: Auto-set. (line 292) * gawk, RT variable in <2>: Multiple Line. (line 129) * gawk, RT variable in: awk split records. (line 125) * gawk, See Also awk: Preface. (line 34) * gawk, source code, obtaining: Getting. (line 6) -* gawk, splitting fields and: Constant Size. (line 88) +* gawk, splitting fields and: Constant Size. (line 87) * gawk, string-translation functions: I18N Functions. (line 6) -* gawk, SYMTAB array in: Auto-set. (line 276) +* gawk, SYMTAB array in: Auto-set. (line 296) * gawk, TEXTDOMAIN variable in: User-modified. (line 151) * gawk, timestamps: Time Functions. (line 6) * gawk, uses for: Preface. (line 34) @@ -32856,22 +33425,28 @@ Index (line 63) * gawkextlib: gawkextlib. (line 6) * gawkextlib project: gawkextlib. (line 6) -* General Public License (GPL): Glossary. (line 305) +* gawklibpath_append shell function: Shell Startup Files. (line 29) +* gawklibpath_default shell function: Shell Startup Files. (line 22) +* gawklibpath_prepend shell function: Shell Startup Files. (line 25) +* gawkpath_append shell function: Shell Startup Files. (line 19) +* gawkpath_default shell function: Shell Startup Files. (line 12) +* gawkpath_prepend shell function: Shell Startup Files. (line 15) +* General Public License (GPL): Glossary. (line 399) * General Public License, See GPL: Manual History. (line 11) * generate time values: Time Functions. (line 25) -* gensub <1>: String Functions. (line 89) +* gensub <1>: String Functions. (line 90) * gensub: Using Constant Regexps. (line 43) * gensub() function (gawk), escape processing: Gory Details. (line 6) * getaddrinfo() function (C library): TCP/IP Networking. (line 38) * getgrent() function (C library): Group Functions. (line 6) * getgrent() user-defined function: Group Functions. (line 6) -* getgrgid() function (C library): Group Functions. (line 182) -* getgrgid() user-defined function: Group Functions. (line 185) -* getgrnam() function (C library): Group Functions. (line 171) -* getgrnam() user-defined function: Group Functions. (line 176) -* getgruser() function (C library): Group Functions. (line 191) -* getgruser() function, user-defined: Group Functions. (line 194) +* getgrgid() function (C library): Group Functions. (line 183) +* getgrgid() user-defined function: Group Functions. (line 186) +* getgrnam() function (C library): Group Functions. (line 172) +* getgrnam() user-defined function: Group Functions. (line 177) +* getgruser() function (C library): Group Functions. (line 192) +* getgruser() function, user-defined: Group Functions. (line 195) * getline command: Reading Files. (line 20) * getline command, _gr_init() user-defined function: Group Functions. (line 83) @@ -32888,7 +33463,7 @@ Index * getline from a file: Getline/File. (line 6) * getline into a variable: Getline/Variable. (line 6) * getline statement, BEGINFILE/ENDFILE patterns and: BEGINFILE/ENDFILE. - (line 54) + (line 53) * getlocaltime() user-defined function: Getlocaltime Function. (line 16) * getopt() function (C library): Getopt Function. (line 15) @@ -32904,40 +33479,40 @@ Index * gettext() function (C library): Explaining gettext. (line 63) * gettimeofday() extension function: Extension Sample Time. (line 12) -* git utility <1>: Adding Code. (line 112) +* git utility <1>: Adding Code. (line 111) * git utility <2>: Accessing The Source. (line 10) * git utility <3>: Other Versions. (line 29) -* git utility: gawkextlib. (line 25) +* git utility: gawkextlib. (line 31) * Git, use of for gawk source code: Derived Files. (line 6) * GNITS mailing list: Acknowledgments. (line 52) * GNU awk, See gawk: Preface. (line 51) * GNU Free Documentation License: GNU Free Documentation License. (line 7) -* GNU General Public License: Glossary. (line 305) -* GNU Lesser General Public License: Glossary. (line 396) +* GNU General Public License: Glossary. (line 399) +* GNU Lesser General Public License: Glossary. (line 496) * GNU long options <1>: Options. (line 6) * GNU long options: Command Line. (line 13) * GNU long options, printing list of: Options. (line 154) -* GNU Project <1>: Glossary. (line 314) +* GNU Project <1>: Glossary. (line 408) * GNU Project: Manual History. (line 11) -* GNU/Linux <1>: Glossary. (line 611) +* GNU/Linux <1>: Glossary. (line 753) * GNU/Linux <2>: I18N Example. (line 55) * GNU/Linux: Manual History. (line 28) * Gordon, Assaf: Contributors. (line 105) -* GPL (General Public License) <1>: Glossary. (line 305) +* GPL (General Public License) <1>: Glossary. (line 399) * GPL (General Public License): Manual History. (line 11) * GPL (General Public License), printing: Options. (line 88) * grcat program: Group Functions. (line 16) * Grigera, Juan: Contributors. (line 57) * group database, reading: Group Functions. (line 6) * group file: Group Functions. (line 6) -* group ID of gawk user: Auto-set. (line 187) +* group ID of gawk user: Auto-set. (line 203) * groups, information about: Group Functions. (line 6) -* gsub <1>: String Functions. (line 139) +* gsub <1>: String Functions. (line 140) * gsub: Using Constant Regexps. (line 43) -* gsub() function, arguments of: String Functions. (line 461) +* gsub() function, arguments of: String Functions. (line 463) * gsub() function, escape processing: Gory Details. (line 6) * h debugger command (alias for help): Miscellaneous Debugger Commands. (line 66) @@ -32964,7 +33539,7 @@ Index * hyphen (-), in bracket expressions: Bracket Expressions. (line 17) * i debugger command (alias for info): Debugger Info. (line 13) * id utility: Id Program. (line 6) -* id.awk program: Id Program. (line 30) +* id.awk program: Id Program. (line 31) * if statement: If Statement. (line 6) * if statement, actions, changing: Ranges. (line 25) * if statement, use of regexps in: Regexp Usage. (line 19) @@ -32972,15 +33547,15 @@ Index * ignore breakpoint: Breakpoint Control. (line 87) * ignore debugger command: Breakpoint Control. (line 87) * IGNORECASE variable: User-modified. (line 76) -* IGNORECASE variable, and array indices: Array Intro. (line 94) +* IGNORECASE variable, and array indices: Array Intro. (line 100) * IGNORECASE variable, and array sorting functions: Array Sorting Functions. (line 83) * IGNORECASE variable, in example programs: Library Functions. (line 53) * IGNORECASE variable, with ~ and !~ operators: Case-sensitivity. (line 26) -* Illumos: Other Versions. (line 105) -* Illumos, POSIX-compliant awk: Other Versions. (line 105) +* Illumos: Other Versions. (line 109) +* Illumos, POSIX-compliant awk: Other Versions. (line 109) * implementation issues, gawk: Notes. (line 6) * implementation issues, gawk, debugging: Compatibility Mode. (line 6) * implementation issues, gawk, limits <1>: Redirection. (line 129) @@ -32997,7 +33572,7 @@ Index * in operator, use in loops: Scanning an Array. (line 17) * including files, @include directive: Include Files. (line 8) * increment operators: Increment Ops. (line 6) -* index: String Functions. (line 155) +* index: String Functions. (line 156) * indexing arrays: Array Intro. (line 50) * indirect function calls: Indirect Calls. (line 6) * indirect function calls, @-notation: Indirect Calls. (line 47) @@ -33016,7 +33591,7 @@ Index * input files, running awk without: Read Terminal. (line 6) * input files, variable assignments and: Other Arguments. (line 26) * input pipeline: Getline/Pipe. (line 9) -* input record, length of: String Functions. (line 177) +* input record, length of: String Functions. (line 178) * input redirection: Getline/File. (line 6) * input, data, nondecimal: Nondecimal Data. (line 6) * input, explicit: Getline. (line 6) @@ -33032,46 +33607,47 @@ Index * insomnia, cure for: Alarm Program. (line 6) * installation, VMS: VMS Installation. (line 6) * installing gawk: Installation. (line 6) -* instruction tracing, in debugger: Debugger Info. (line 89) -* int: Numeric Functions. (line 38) +* instruction tracing, in debugger: Debugger Info. (line 90) +* int: Numeric Functions. (line 23) * INT signal (MS-Windows): Profiling. (line 213) +* intdiv: Numeric Functions. (line 28) * integer array indices: Numeric Array Subscripts. (line 31) * integers, arbitrary precision: Arbitrary Precision Integers. (line 6) * integers, unsigned: Computer Arithmetic. (line 41) -* interacting with other programs: I/O Functions. (line 74) +* interacting with other programs: I/O Functions. (line 107) * internationalization <1>: I18N and L10N. (line 6) * internationalization: I18N Functions. (line 6) * internationalization, localization <1>: Internationalization. (line 13) * internationalization, localization: User-modified. (line 151) * internationalization, localization, character classes: Bracket Expressions. - (line 100) + (line 101) * internationalization, localization, gawk and: Internationalization. (line 13) * internationalization, localization, locale categories: Explaining gettext. (line 81) * internationalization, localization, marked strings: Programmer i18n. - (line 14) + (line 13) * internationalization, localization, portability and: I18N Portability. (line 6) * internationalizing a program: Explaining gettext. (line 6) -* interpreted programs <1>: Glossary. (line 356) +* interpreted programs <1>: Glossary. (line 450) * interpreted programs: Basic High Level. (line 15) * interval expressions, regexp operator: Regexp Operators. (line 116) * inventory-shipped file: Sample Data Files. (line 32) -* invoke shell command: I/O Functions. (line 74) +* invoke shell command: I/O Functions. (line 107) * isarray: Type Functions. (line 11) -* ISO: Glossary. (line 367) -* ISO 8859-1: Glossary. (line 133) -* ISO Latin-1: Glossary. (line 133) +* ISO: Glossary. (line 461) +* ISO 8859-1: Glossary. (line 197) +* ISO Latin-1: Glossary. (line 197) * Jacobs, Andrew: Passwd Functions. (line 90) * Jaegermann, Michal <1>: Contributors. (line 45) * Jaegermann, Michal: Acknowledgments. (line 60) -* Java implementation of awk: Other Versions. (line 113) -* Java programming language: Glossary. (line 379) -* jawk: Other Versions. (line 113) +* Java implementation of awk: Other Versions. (line 117) +* Java programming language: Glossary. (line 473) +* jawk: Other Versions. (line 117) * Jedi knights: Undocumented. (line 6) * Johansen, Chris: Signature Program. (line 25) * join() user-defined function: Join Function. (line 18) @@ -33079,7 +33655,7 @@ Index * Kahrs, Ju"rgen: Acknowledgments. (line 60) * Kasal, Stepan: Acknowledgments. (line 60) * Kenobi, Obi-Wan: Undocumented. (line 6) -* Kernighan, Brian <1>: Glossary. (line 143) +* Kernighan, Brian <1>: Glossary. (line 207) * Kernighan, Brian <2>: Basic Data Typing. (line 54) * Kernighan, Brian <3>: Other Versions. (line 13) * Kernighan, Brian <4>: Contributors. (line 11) @@ -33117,12 +33693,12 @@ Index * left shift: Bitwise Functions. (line 47) * left shift, bitwise: Bitwise Functions. (line 32) * leftmost longest match: Multiple Line. (line 26) -* length: String Functions. (line 170) -* length of input record: String Functions. (line 177) -* length of string: String Functions. (line 170) -* Lesser General Public License (LGPL): Glossary. (line 396) -* LGPL (Lesser General Public License): Glossary. (line 396) -* libmawk: Other Versions. (line 121) +* length: String Functions. (line 171) +* length of input record: String Functions. (line 178) +* length of string: String Functions. (line 171) +* Lesser General Public License (LGPL): Glossary. (line 496) +* LGPL (Lesser General Public License): Glossary. (line 496) +* libmawk: Other Versions. (line 125) * libraries of awk functions: Library Functions. (line 6) * libraries of awk functions, assertions: Assert Function. (line 6) * libraries of awk functions, associative arrays and: Library Names. @@ -33155,7 +33731,7 @@ Index * lines, duplicate, removing: History Sorting. (line 6) * lines, matching ranges of: Ranges. (line 6) * lines, skipping between markers: Ranges. (line 43) -* lint checking: User-modified. (line 88) +* lint checking: User-modified. (line 87) * lint checking, array elements: Delete. (line 34) * lint checking, array subscripts: Uninitialized Subscripts. (line 43) @@ -33164,9 +33740,9 @@ Index * lint checking, POSIXLY_CORRECT environment variable: Options. (line 339) * lint checking, undefined functions: Pass By Value/Reference. - (line 88) -* LINT variable: User-modified. (line 88) -* Linux <1>: Glossary. (line 611) + (line 85) +* LINT variable: User-modified. (line 87) +* Linux <1>: Glossary. (line 753) * Linux <2>: I18N Example. (line 55) * Linux: Manual History. (line 28) * list all global variables, in debugger: Debugger Info. (line 48) @@ -33183,9 +33759,9 @@ Index * localization: I18N and L10N. (line 6) * localization, See internationalization, localization: I18N and L10N. (line 6) -* log: Numeric Functions. (line 45) +* log: Numeric Functions. (line 43) * log files, timestamps in: Time Functions. (line 6) -* logarithm: Numeric Functions. (line 45) +* logarithm: Numeric Functions. (line 43) * logical false/true: Truth Values. (line 6) * logical operators, See Boolean expressions: Boolean Ops. (line 6) * login information: Passwd Functions. (line 16) @@ -33206,7 +33782,7 @@ Index * mail-list file: Sample Data Files. (line 6) * mailing labels, printing: Labels Program. (line 6) * mailing list, GNITS: Acknowledgments. (line 52) -* Malmberg, John <1>: Bugs. (line 72) +* Malmberg, John <1>: Bugs. (line 71) * Malmberg, John: Acknowledgments. (line 60) * Malmberg, John E.: Contributors. (line 137) * mark parity: Ordinal Functions. (line 45) @@ -33214,33 +33790,33 @@ Index (line 6) * marked strings, extracting: String Extraction. (line 6) * Marx, Groucho: Increment Ops. (line 60) -* match: String Functions. (line 210) -* match regexp in string: String Functions. (line 210) +* match: String Functions. (line 211) +* match regexp in string: String Functions. (line 211) * match() function, RSTART/RLENGTH variables: String Functions. - (line 227) + (line 228) * matching, expressions, See comparison expressions: Typing and Comparison. (line 9) * matching, leftmost longest: Multiple Line. (line 26) -* matching, null strings: String Functions. (line 535) -* mawk utility <1>: Other Versions. (line 44) +* matching, null strings: String Functions. (line 537) +* mawk utility <1>: Other Versions. (line 48) * mawk utility <2>: Nextfile Statement. (line 47) * mawk utility <3>: Concatenation. (line 36) * mawk utility <4>: Getline/Pipe. (line 62) -* mawk utility: Escape Sequences. (line 132) -* maximum precision supported by MPFR library: Auto-set. (line 228) -* McIlroy, Doug: Glossary. (line 149) +* mawk utility: Escape Sequences. (line 120) +* maximum precision supported by MPFR library: Auto-set. (line 244) +* McIlroy, Doug: Glossary. (line 258) * McPhee, Patrick: Contributors. (line 100) * message object files: Explaining gettext. (line 42) * message object files, converting from portable object files: I18N Example. (line 64) * message object files, specifying directory of <1>: Programmer i18n. - (line 47) + (line 48) * message object files, specifying directory of: Explaining gettext. (line 54) * messages from extensions: Printing Messages. (line 6) * metacharacters in regular expressions: Regexp Operators. (line 6) -* metacharacters, escape sequences for: Escape Sequences. (line 138) -* minimum precision supported by MPFR library: Auto-set. (line 231) +* metacharacters, escape sequences for: Escape Sequences. (line 139) +* minimum precision required by MPFR library: Auto-set. (line 247) * mktime: Time Functions. (line 25) * modifiers, in format specifiers: Format Modifiers. (line 6) * monetary information, localization: Explaining gettext. (line 104) @@ -33256,7 +33832,7 @@ Index * names, functions: Definition Syntax. (line 23) * namespace issues: Library Names. (line 6) * namespace issues, functions: Definition Syntax. (line 23) -* NetBSD: Glossary. (line 611) +* NetBSD: Glossary. (line 753) * networks, programming: TCP/IP Networking. (line 6) * networks, support for: Special Network. (line 6) * newlines <1>: Boolean Ops. (line 69) @@ -33265,8 +33841,8 @@ Index * newlines, as field separators: Default Field Splitting. (line 6) * newlines, as record separators: awk split records. (line 12) -* newlines, in dynamic regexps: Computed Regexps. (line 59) -* newlines, in regexp constants: Computed Regexps. (line 69) +* newlines, in dynamic regexps: Computed Regexps. (line 60) +* newlines, in regexp constants: Computed Regexps. (line 70) * newlines, printing: Print Examples. (line 12) * newlines, separating statements in actions <1>: Statements. (line 10) * newlines, separating statements in actions: Action Overview. @@ -33289,7 +33865,7 @@ Index (line 47) * nexti debugger command: Debugger Execution Control. (line 49) -* NF variable <1>: Auto-set. (line 112) +* NF variable <1>: Auto-set. (line 124) * NF variable: Fields. (line 33) * NF variable, decrementing: Changing Fields. (line 107) * ni debugger command (alias for nexti): Debugger Execution Control. @@ -33298,9 +33874,9 @@ Index * non-existent array elements: Reference to Elements. (line 23) * not Boolean-logic operator: Boolean Ops. (line 6) -* NR variable <1>: Auto-set. (line 132) +* NR variable <1>: Auto-set. (line 144) * NR variable: Records. (line 6) -* NR variable, changing: Auto-set. (line 321) +* NR variable, changing: Auto-set. (line 341) * null strings <1>: Basic Data Typing. (line 26) * null strings <2>: Truth Values. (line 6) * null strings <3>: Regexp Field Splitting. @@ -33312,9 +33888,9 @@ Index (line 43) * null strings, converting numbers to strings: Strings And Numbers. (line 21) -* null strings, matching: String Functions. (line 535) +* null strings, matching: String Functions. (line 537) * number as string of bits: Bitwise Functions. (line 110) -* number of array elements: String Functions. (line 200) +* number of array elements: String Functions. (line 201) * number sign (#), #! (executable scripts): Executable Scripts. (line 6) * number sign (#), commenting: Comments. (line 6) @@ -33337,15 +33913,15 @@ Index * obsolete features: Obsolete. (line 6) * octal numbers: Nondecimal-numbers. (line 6) * octal values, enabling interpretation of: Options. (line 211) -* OFMT variable <1>: User-modified. (line 105) +* OFMT variable <1>: User-modified. (line 104) * OFMT variable <2>: Strings And Numbers. (line 57) * OFMT variable: OFMT. (line 15) * OFMT variable, POSIX awk and: OFMT. (line 27) * OFS variable <1>: User-modified. (line 113) * OFS variable <2>: Output Separators. (line 6) * OFS variable: Changing Fields. (line 64) -* OpenBSD: Glossary. (line 611) -* OpenSolaris: Other Versions. (line 96) +* OpenBSD: Glossary. (line 753) +* OpenSolaris: Other Versions. (line 100) * operating systems, BSD-based: Manual History. (line 28) * operating systems, PC, gawk on: PC Using. (line 6) * operating systems, PC, gawk on, installing: PC Installation. @@ -33395,10 +33971,10 @@ Index * ord() user-defined function: Ordinal Functions. (line 16) * order of evaluation, concatenation: Concatenation. (line 41) * ORS variable <1>: User-modified. (line 118) -* ORS variable: Output Separators. (line 20) +* ORS variable: Output Separators. (line 21) * output field separator, See OFS variable: Changing Fields. (line 64) * output record separator, See ORS variable: Output Separators. - (line 20) + (line 21) * output redirection: Redirection. (line 6) * output wrapper: Output Wrappers. (line 6) * output, buffering: I/O Functions. (line 32) @@ -33409,16 +33985,16 @@ Index * output, formatted: Printf. (line 6) * output, pipes: Redirection. (line 57) * output, printing, See printing: Printing. (line 6) -* output, records: Output Separators. (line 20) +* output, records: Output Separators. (line 21) * output, standard: Special FD. (line 6) * p debugger command (alias for print): Viewing And Changing Data. (line 36) * Papadopoulos, Panos: Contributors. (line 128) -* parent process ID of gawk process: Auto-set. (line 196) +* parent process ID of gawk process: Auto-set. (line 212) * parentheses (), in a profile: Profiling. (line 146) * parentheses (), regexp operator: Regexp Operators. (line 81) * password file: Passwd Functions. (line 16) -* patsplit: String Functions. (line 296) +* patsplit: String Functions. (line 297) * patterns: Patterns and Actions. (line 6) * patterns, comparison expressions as: Expression Patterns. (line 14) @@ -33430,8 +34006,8 @@ Index * patterns, regexp constants as: Expression Patterns. (line 34) * patterns, types of: Pattern Overview. (line 15) * pawk (profiling version of Brian Kernighan's awk): Other Versions. - (line 78) -* pawk, awk-like facilities for Python: Other Versions. (line 125) + (line 82) +* pawk, awk-like facilities for Python: Other Versions. (line 129) * PC operating systems, gawk on: PC Using. (line 6) * PC operating systems, gawk on, installing: PC Installation. (line 6) * percent sign (%), % operator: Precedence. (line 55) @@ -33445,7 +34021,7 @@ Index (line 6) * pipe, input: Getline/Pipe. (line 9) * pipe, output: Redirection. (line 57) -* Pitts, Dave <1>: Bugs. (line 72) +* Pitts, Dave <1>: Bugs. (line 71) * Pitts, Dave: Acknowledgments. (line 60) * Plauger, P.J.: Library Functions. (line 12) * plug-in: Extension Intro. (line 6) @@ -33456,33 +34032,33 @@ Index * plus sign (+), += operator: Assignment Ops. (line 82) * plus sign (+), regexp operator: Regexp Operators. (line 105) * pointers to functions: Indirect Calls. (line 6) -* portability: Escape Sequences. (line 102) +* portability: Escape Sequences. (line 103) * portability, #! (executable scripts): Executable Scripts. (line 33) * portability, ** operator and: Arithmetic Ops. (line 81) * portability, **= operator and: Assignment Ops. (line 143) * portability, ARGV variable: Executable Scripts. (line 59) * portability, backslash continuation and: Statements/Lines. (line 30) * portability, backslash in escape sequences: Escape Sequences. - (line 120) + (line 108) * portability, close() function and: Close Files And Pipes. (line 81) * portability, data files as single record: gawk split records. (line 65) * portability, deleting array elements: Delete. (line 56) * portability, example programs: Library Functions. (line 42) -* portability, functions, defining: Definition Syntax. (line 109) +* portability, functions, defining: Definition Syntax. (line 114) * portability, gawk: New Ports. (line 6) * portability, gettext library and: Explaining gettext. (line 11) * portability, internationalization and: I18N Portability. (line 6) -* portability, length() function: String Functions. (line 179) +* portability, length() function: String Functions. (line 180) * portability, new awk vs. old awk: Strings And Numbers. (line 57) * portability, next statement in user-defined functions: Pass By Value/Reference. - (line 91) + (line 88) * portability, NF variable, decrementing: Changing Fields. (line 115) * portability, operators: Increment Ops. (line 60) * portability, operators, not in POSIX awk: Precedence. (line 98) * portability, POSIXLY_CORRECT environment variable: Options. (line 359) -* portability, substr() function: String Functions. (line 511) +* portability, substr() function: String Functions. (line 513) * portable object files <1>: Translator i18n. (line 6) * portable object files: Explaining gettext. (line 37) * portable object files, converting to message object files: I18N Example. @@ -33502,7 +34078,7 @@ Index * POSIX awk, < operator and: Getline/File. (line 26) * POSIX awk, arithmetic operators and: Arithmetic Ops. (line 30) * POSIX awk, backslashes in string constants: Escape Sequences. - (line 120) + (line 108) * POSIX awk, BEGIN/END patterns: I/O And BEGIN/END. (line 16) * POSIX awk, bracket expressions and: Bracket Expressions. (line 26) * POSIX awk, bracket expressions and, character classes: Bracket Expressions. @@ -33512,13 +34088,12 @@ Index * POSIX awk, continue statement and: Continue Statement. (line 44) * POSIX awk, CONVFMT variable and: User-modified. (line 30) * POSIX awk, date utility and: Time Functions. (line 254) -* POSIX awk, field separators and <1>: Field Splitting Summary. - (line 40) +* POSIX awk, field separators and <1>: Full Line Fields. (line 16) * POSIX awk, field separators and: Fields. (line 6) * POSIX awk, FS variable and: User-modified. (line 60) -* POSIX awk, function keyword in: Definition Syntax. (line 93) +* POSIX awk, function keyword in: Definition Syntax. (line 98) * POSIX awk, functions and, gsub()/sub(): Gory Details. (line 90) -* POSIX awk, functions and, length(): String Functions. (line 179) +* POSIX awk, functions and, length(): String Functions. (line 180) * POSIX awk, GNU long options and: Options. (line 15) * POSIX awk, interval expressions in: Regexp Operators. (line 135) * POSIX awk, next/nextfile statements and: Next Statement. (line 44) @@ -33581,24 +34156,24 @@ Index * printing, unduplicated lines of text: Uniq Program. (line 6) * printing, user information: Id Program. (line 6) * private variables: Library Names. (line 11) -* process group idIDof gawk process: Auto-set. (line 190) -* process ID of gawk process: Auto-set. (line 193) +* process group ID of gawk process: Auto-set. (line 206) +* process ID of gawk process: Auto-set. (line 209) * processes, two-way communications with: Two-way I/O. (line 6) * processing data: Basic High Level. (line 6) * PROCINFO array <1>: Passwd Functions. (line 6) * PROCINFO array <2>: Time Functions. (line 47) -* PROCINFO array: Auto-set. (line 137) +* PROCINFO array: Auto-set. (line 149) * PROCINFO array, and communications via ptys: Two-way I/O. (line 99) * PROCINFO array, and group membership: Group Functions. (line 6) * PROCINFO array, and user and group ID numbers: Id Program. (line 15) * PROCINFO array, testing the field splitting: Passwd Functions. (line 154) -* PROCINFO array, uses: Auto-set. (line 249) +* PROCINFO array, uses: Auto-set. (line 265) * PROCINFO, values of sorted_in: Controlling Scanning. (line 26) * profiling awk programs: Profiling. (line 6) * profiling awk programs, dynamically: Profiling. (line 178) -* program identifiers: Auto-set. (line 155) +* program identifiers: Auto-set. (line 171) * program, definition of: Getting Started. (line 21) * programming conventions, --non-decimal-data option: Nondecimal Data. (line 35) @@ -33609,28 +34184,28 @@ Index * programming conventions, functions, calling: Calling Built-in. (line 10) * programming conventions, functions, writing: Definition Syntax. - (line 65) + (line 70) * programming conventions, gawk extensions: Internal File Ops. (line 45) * programming conventions, private variable names: Library Names. (line 23) * programming language, recipe for: History. (line 6) -* programming languages, Ada: Glossary. (line 19) +* programming languages, Ada: Glossary. (line 11) * programming languages, data-driven vs. procedural: Getting Started. (line 12) -* programming languages, Java: Glossary. (line 379) +* programming languages, Java: Glossary. (line 473) * programming, basic steps: Basic High Level. (line 20) * programming, concepts: Basic Concepts. (line 6) * pwcat program: Passwd Functions. (line 23) * q debugger command (alias for quit): Miscellaneous Debugger Commands. (line 99) -* QSE Awk: Other Versions. (line 131) +* QSE awk: Other Versions. (line 135) * Quanstrom, Erik: Alarm Program. (line 8) * question mark (?), ?: operator: Precedence. (line 92) * question mark (?), regexp operator <1>: GNU Regexp Operators. (line 59) * question mark (?), regexp operator: Regexp Operators. (line 111) -* QuikTrim Awk: Other Versions. (line 135) +* QuikTrim Awk: Other Versions. (line 139) * quit debugger command: Miscellaneous Debugger Commands. (line 99) * QUIT signal (MS-Windows): Profiling. (line 213) @@ -33642,12 +34217,12 @@ Index * Rakitzis, Byron: History Sorting. (line 25) * Ramey, Chet <1>: General Data Types. (line 6) * Ramey, Chet: Acknowledgments. (line 60) -* rand: Numeric Functions. (line 50) +* rand: Numeric Functions. (line 48) * random numbers, Cliff: Cliff Random Function. (line 6) * random numbers, rand()/srand() functions: Numeric Functions. - (line 50) -* random numbers, seed of: Numeric Functions. (line 80) + (line 48) +* random numbers, seed of: Numeric Functions. (line 78) * range expressions (regexps): Bracket Expressions. (line 6) * range patterns: Ranges. (line 6) * range patterns, line continuation and: Ranges. (line 65) @@ -33678,8 +34253,8 @@ Index * records, splitting input into: Records. (line 6) * records, terminating: awk split records. (line 125) * records, treating files as: gawk split records. (line 93) -* recursive functions: Definition Syntax. (line 83) -* redirect gawk output, in debugger: Debugger Info. (line 72) +* recursive functions: Definition Syntax. (line 88) +* redirect gawk output, in debugger: Debugger Info. (line 73) * redirection of input: Getline/File. (line 6) * redirection of output: Redirection. (line 6) * reference counting, sorting arrays: Array Sorting Functions. @@ -33693,8 +34268,8 @@ Index * regexp constants, as patterns: Expression Patterns. (line 34) * regexp constants, in gawk: Using Constant Regexps. (line 28) -* regexp constants, slashes vs. quotes: Computed Regexps. (line 29) -* regexp constants, vs. string constants: Computed Regexps. (line 39) +* regexp constants, slashes vs. quotes: Computed Regexps. (line 30) +* regexp constants, vs. string constants: Computed Regexps. (line 40) * register extension: Registration Functions. (line 6) * regular expressions: Regexp. (line 6) @@ -33713,7 +34288,7 @@ Index (line 57) * regular expressions, dynamic: Computed Regexps. (line 6) * regular expressions, dynamic, with embedded newlines: Computed Regexps. - (line 59) + (line 60) * regular expressions, gawk, command-line options: GNU Regexp Operators. (line 70) * regular expressions, interval expressions and: Options. (line 279) @@ -33732,18 +34307,19 @@ Index * regular expressions, searching for: Egrep Program. (line 6) * relational operators, See comparison operators: Typing and Comparison. (line 9) -* replace in string: String Functions. (line 407) +* replace in string: String Functions. (line 409) +* retrying input: Retrying Input. (line 6) * return debugger command: Debugger Execution Control. (line 54) * return statement, user-defined functions: Return Statement. (line 6) * return value, close() function: Close Files And Pipes. - (line 132) + (line 133) * rev() user-defined function: Function Example. (line 54) * revoutput extension: Extension Sample Revout. (line 11) * revtwoway extension: Extension Sample Rev2way. (line 12) -* rewind() user-defined function: Rewind Function. (line 16) +* rewind() user-defined function: Rewind Function. (line 15) * right angle bracket (>), > operator <1>: Precedence. (line 65) * right angle bracket (>), > operator: Comparison Operators. (line 11) @@ -33756,10 +34332,10 @@ Index * right shift: Bitwise Functions. (line 53) * right shift, bitwise: Bitwise Functions. (line 32) * Ritchie, Dennis: Basic Data Typing. (line 54) -* RLENGTH variable: Auto-set. (line 259) -* RLENGTH variable, match() function and: String Functions. (line 227) +* RLENGTH variable: Auto-set. (line 279) +* RLENGTH variable, match() function and: String Functions. (line 228) * Robbins, Arnold <1>: Future Extensions. (line 6) -* Robbins, Arnold <2>: Bugs. (line 72) +* Robbins, Arnold <2>: Bugs. (line 71) * Robbins, Arnold <3>: Contributors. (line 144) * Robbins, Arnold <4>: General Data Types. (line 6) * Robbins, Arnold <5>: Alarm Program. (line 6) @@ -33774,7 +34350,7 @@ Index * Robbins, Miriam <2>: Getline/Pipe. (line 39) * Robbins, Miriam: Acknowledgments. (line 94) * Rommel, Kai Uwe: Contributors. (line 42) -* round to nearest integer: Numeric Functions. (line 38) +* round to nearest integer: Numeric Functions. (line 23) * round() user-defined function: Round Function. (line 16) * rounding numbers: Round Function. (line 6) * ROUNDMODE variable: User-modified. (line 127) @@ -33782,9 +34358,9 @@ Index * RS variable: awk split records. (line 12) * RS variable, multiline records and: Multiple Line. (line 17) * rshift: Bitwise Functions. (line 53) -* RSTART variable: Auto-set. (line 265) -* RSTART variable, match() function and: String Functions. (line 227) -* RT variable <1>: Auto-set. (line 272) +* RSTART variable: Auto-set. (line 285) +* RSTART variable, match() function and: String Functions. (line 228) +* RT variable <1>: Auto-set. (line 292) * RT variable <2>: Multiple Line. (line 129) * RT variable: awk split records. (line 125) * Rubin, Paul <1>: Contributors. (line 15) @@ -33798,33 +34374,32 @@ Index * sample debugging session: Sample Debugging Session. (line 6) * sandbox mode: Options. (line 286) -* save debugger options: Debugger Info. (line 84) +* save debugger options: Debugger Info. (line 85) * scalar or array: Type Functions. (line 11) * scalar values: Basic Data Typing. (line 13) * scanning arrays: Scanning an Array. (line 6) * scanning multidimensional arrays: Multiscanning. (line 11) * Schorr, Andrew <1>: Contributors. (line 133) -* Schorr, Andrew <2>: Auto-set. (line 304) +* Schorr, Andrew <2>: Auto-set. (line 324) * Schorr, Andrew: Acknowledgments. (line 60) * Schreiber, Bert: Acknowledgments. (line 38) * Schreiber, Rita: Acknowledgments. (line 38) -* search and replace in strings: String Functions. (line 89) -* search in string: String Functions. (line 155) -* search paths <1>: VMS Running. (line 58) +* search and replace in strings: String Functions. (line 90) +* search in string: String Functions. (line 156) +* search paths <1>: VMS Running. (line 57) * search paths <2>: PC Using. (line 10) * search paths: Programs Exercises. (line 70) * search paths, for loadable extensions: AWKLIBPATH Variable. (line 6) -* search paths, for source files <1>: VMS Running. (line 58) +* search paths, for source files <1>: VMS Running. (line 57) * search paths, for source files <2>: PC Using. (line 10) * search paths, for source files <3>: Programs Exercises. (line 70) * search paths, for source files: AWKPATH Variable. (line 6) * searching, files for regular expressions: Egrep Program. (line 6) * searching, for words: Dupword Program. (line 6) -* sed utility <1>: Glossary. (line 11) +* sed utility <1>: Glossary. (line 16) * sed utility <2>: Simple Sed. (line 6) -* sed utility: Field Splitting Summary. - (line 46) -* seeding random number generator: Numeric Functions. (line 80) +* sed utility: Full Line Fields. (line 22) +* seeding random number generator: Numeric Functions. (line 78) * semicolon (;), AWKPATH variable and: PC Using. (line 10) * semicolon (;), separating statements in actions <1>: Statements. (line 10) @@ -33848,7 +34423,7 @@ Index * set directory of message catalogs: I18N Functions. (line 12) * set watchpoint: Viewing And Changing Data. (line 67) -* shadowing of variable values: Definition Syntax. (line 71) +* shadowing of variable values: Definition Syntax. (line 76) * shell quoting, rules for: Quoting. (line 6) * shells, piping commands into: Redirection. (line 136) * shells, quoting: Using Shell Variables. @@ -33885,35 +34460,35 @@ Index * sidebar, A Constant's Base Does Not Affect Its Value: Nondecimal-numbers. (line 64) * sidebar, Backslash Before Regular Characters: Escape Sequences. - (line 118) -* sidebar, Changing FS Does Not Affect the Fields: Field Splitting Summary. - (line 38) -* sidebar, Changing NR and FNR: Auto-set. (line 319) + (line 106) +* sidebar, Changing FS Does Not Affect the Fields: Full Line Fields. + (line 14) +* sidebar, Changing NR and FNR: Auto-set. (line 339) * sidebar, Controlling Output Buffering with system(): I/O Functions. - (line 137) + (line 139) * sidebar, Escape Sequences for Metacharacters: Escape Sequences. - (line 136) + (line 137) * sidebar, FS and IGNORECASE: Field Splitting Summary. - (line 64) + (line 38) * sidebar, Interactive Versus Noninteractive Buffering: I/O Functions. - (line 106) -* sidebar, Matching the Null String: String Functions. (line 533) + (line 74) +* sidebar, Matching the Null String: String Functions. (line 535) * sidebar, Operator Evaluation Order: Increment Ops. (line 58) * sidebar, Piping into sh: Redirection. (line 134) -* sidebar, Pre-POSIX awk Used OFMT For String Conversion: Strings And Numbers. +* sidebar, Pre-POSIX awk Used OFMT for String Conversion: Strings And Numbers. (line 55) -* sidebar, Recipe For A Programming Language: History. (line 6) +* sidebar, Recipe for a Programming Language: History. (line 6) * sidebar, RS = "\0" Is Not Portable: gawk split records. (line 63) -* sidebar, So Why Does gawk have BEGINFILE and ENDFILE?: Filetrans Function. - (line 82) +* sidebar, So Why Does gawk Have BEGINFILE and ENDFILE?: Filetrans Function. + (line 83) * sidebar, Syntactic Ambiguities Between /= and Regular Expressions: Assignment Ops. (line 146) * sidebar, Understanding #!: Executable Scripts. (line 31) * sidebar, Understanding $0: Changing Fields. (line 134) * sidebar, Using \n in Bracket Expressions of Dynamic Regexps: Computed Regexps. - (line 57) + (line 58) * sidebar, Using close()'s Return Value: Close Files And Pipes. - (line 130) + (line 131) * SIGHUP signal, for dynamic profiling: Profiling. (line 210) * SIGINT signal (MS-Windows): Profiling. (line 213) * signals, HUP/SIGHUP, for profiling: Profiling. (line 210) @@ -33925,8 +34500,8 @@ Index * SIGUSR1 signal, for dynamic profiling: Profiling. (line 187) * silent debugger command: Debugger Execution Control. (line 10) -* sin: Numeric Functions. (line 91) -* sine: Numeric Functions. (line 91) +* sin: Numeric Functions. (line 89) +* sine: Numeric Functions. (line 89) * single quote ('): One-shot. (line 15) * single quote (') in gawk command lines: Long. (line 35) * single quote ('), in shell commands: Quoting. (line 48) @@ -33940,48 +34515,48 @@ Index * sleep utility: Alarm Program. (line 110) * sleep() extension function: Extension Sample Time. (line 22) -* Solaris, POSIX-compliant awk: Other Versions. (line 96) -* sort array: String Functions. (line 41) -* sort array indices: String Functions. (line 41) +* Solaris, POSIX-compliant awk: Other Versions. (line 100) +* sort array: String Functions. (line 42) +* sort array indices: String Functions. (line 42) * sort function, arrays, sorting: Array Sorting Functions. (line 6) * sort utility: Word Sorting. (line 50) * sort utility, coprocesses and: Two-way I/O. (line 65) * sorting characters in different languages: Explaining gettext. (line 94) -* source code, awka: Other Versions. (line 64) +* source code, awka: Other Versions. (line 68) * source code, Brian Kernighan's awk: Other Versions. (line 13) -* source code, Busybox Awk: Other Versions. (line 88) +* source code, BusyBox Awk: Other Versions. (line 92) * source code, gawk: Gawk Distribution. (line 6) -* source code, Illumos awk: Other Versions. (line 105) -* source code, jawk: Other Versions. (line 113) -* source code, libmawk: Other Versions. (line 121) -* source code, mawk: Other Versions. (line 44) +* source code, Illumos awk: Other Versions. (line 109) +* source code, jawk: Other Versions. (line 117) +* source code, libmawk: Other Versions. (line 125) +* source code, mawk: Other Versions. (line 48) * source code, mixing: Options. (line 117) -* source code, pawk: Other Versions. (line 78) -* source code, pawk (Python version): Other Versions. (line 125) -* source code, QSE Awk: Other Versions. (line 131) -* source code, QuikTrim Awk: Other Versions. (line 135) -* source code, Solaris awk: Other Versions. (line 96) +* source code, pawk: Other Versions. (line 82) +* source code, pawk (Python version): Other Versions. (line 129) +* source code, QSE awk: Other Versions. (line 135) +* source code, QuikTrim Awk: Other Versions. (line 139) +* source code, Solaris awk: Other Versions. (line 100) * source files, search path for: Programs Exercises. (line 70) -* sparse arrays: Array Intro. (line 72) -* Spencer, Henry: Glossary. (line 11) -* split: String Functions. (line 315) -* split string into array: String Functions. (line 296) +* sparse arrays: Array Intro. (line 76) +* Spencer, Henry: Glossary. (line 16) +* split: String Functions. (line 316) +* split string into array: String Functions. (line 297) * split utility: Split Program. (line 6) * split() function, array elements, deleting: Delete. (line 61) * split.awk program: Split Program. (line 30) -* sprintf <1>: String Functions. (line 382) +* sprintf <1>: String Functions. (line 384) * sprintf: OFMT. (line 15) * sprintf() function, OFMT variable and: User-modified. (line 113) * sprintf() function, print/printf statements and: Round Function. (line 6) -* sqrt: Numeric Functions. (line 94) +* sqrt: Numeric Functions. (line 92) * square brackets ([]), regexp operator: Regexp Operators. (line 56) -* square root: Numeric Functions. (line 94) -* srand: Numeric Functions. (line 98) +* square root: Numeric Functions. (line 92) +* srand: Numeric Functions. (line 96) * stack frame: Debugging Terms. (line 10) -* Stallman, Richard <1>: Glossary. (line 296) +* Stallman, Richard <1>: Glossary. (line 375) * Stallman, Richard <2>: Contributors. (line 23) * Stallman, Richard <3>: Acknowledgments. (line 18) * Stallman, Richard: Manual History. (line 6) @@ -34002,39 +34577,38 @@ Index * stop automatic display, in debugger: Viewing And Changing Data. (line 80) * stream editors <1>: Simple Sed. (line 6) -* stream editors: Field Splitting Summary. - (line 46) +* stream editors: Full Line Fields. (line 22) * strftime: Time Functions. (line 48) * string constants: Scalar Constants. (line 15) -* string constants, vs. regexp constants: Computed Regexps. (line 39) +* string constants, vs. regexp constants: Computed Regexps. (line 40) * string extraction (internationalization): String Extraction. (line 6) -* string length: String Functions. (line 170) +* string length: String Functions. (line 171) * string operators: Concatenation. (line 8) -* string, regular expression match: String Functions. (line 210) +* string, regular expression match: String Functions. (line 211) * string-manipulation functions: String Functions. (line 6) * string-matching operators: Regexp Usage. (line 19) * string-translation functions: I18N Functions. (line 6) -* strings splitting, example: String Functions. (line 334) +* strings splitting, example: String Functions. (line 335) * strings, converting <1>: Bitwise Functions. (line 110) * strings, converting: Strings And Numbers. (line 6) -* strings, converting letter case: String Functions. (line 521) +* strings, converting letter case: String Functions. (line 523) * strings, converting, numbers to: User-modified. (line 30) * strings, empty, See null strings: awk split records. (line 115) * strings, extracting: String Extraction. (line 6) -* strings, for localization: Programmer i18n. (line 14) +* strings, for localization: Programmer i18n. (line 13) * strings, length limitations: Scalar Constants. (line 20) * strings, merging arrays into: Join Function. (line 6) * strings, null: Regexp Field Splitting. (line 43) * strings, numeric: Variable Typing. (line 6) -* strtonum: String Functions. (line 389) +* strtonum: String Functions. (line 391) * strtonum() function (gawk), --non-decimal-data option and: Nondecimal Data. (line 35) -* sub <1>: String Functions. (line 407) +* sub <1>: String Functions. (line 409) * sub: Using Constant Regexps. (line 43) -* sub() function, arguments of: String Functions. (line 461) +* sub() function, arguments of: String Functions. (line 463) * sub() function, escape processing: Gory Details. (line 6) * subscript separators: User-modified. (line 145) * subscripts in arrays, multidimensional: Multidimensional. (line 10) @@ -34047,16 +34621,16 @@ Index * SUBSEP variable: User-modified. (line 145) * SUBSEP variable, and multidimensional arrays: Multidimensional. (line 16) -* substitute in string: String Functions. (line 89) -* substr: String Functions. (line 480) -* substring: String Functions. (line 480) -* Sumner, Andrew: Other Versions. (line 64) -* supplementary groups of gawk process: Auto-set. (line 244) +* substitute in string: String Functions. (line 90) +* substr: String Functions. (line 482) +* substring: String Functions. (line 482) +* Sumner, Andrew: Other Versions. (line 68) +* supplementary groups of gawk process: Auto-set. (line 260) * switch statement: Switch Statement. (line 6) -* SYMTAB array: Auto-set. (line 276) +* SYMTAB array: Auto-set. (line 296) * syntactic ambiguity: /= operator vs. /=.../ regexp constant: Assignment Ops. (line 148) -* system: I/O Functions. (line 74) +* system: I/O Functions. (line 107) * systime: Time Functions. (line 66) * t debugger command (alias for tbreak): Breakpoint Control. (line 90) * tbreak debugger command: Breakpoint Control. (line 90) @@ -34070,7 +34644,7 @@ Index * testbits.awk program: Bitwise Functions. (line 71) * testext extension: Extension Sample API Tests. (line 6) -* Texinfo <1>: Adding Code. (line 100) +* Texinfo <1>: Adding Code. (line 99) * Texinfo <2>: Distribution contents. (line 77) * Texinfo <3>: Extract Program. (line 12) @@ -34082,7 +34656,7 @@ Index (line 6) * text, printing: Print. (line 22) * text, printing, unduplicated lines of: Uniq Program. (line 6) -* TEXTDOMAIN variable <1>: Programmer i18n. (line 9) +* TEXTDOMAIN variable <1>: Programmer i18n. (line 8) * TEXTDOMAIN variable: User-modified. (line 151) * TEXTDOMAIN variable, BEGIN pattern and: Programmer i18n. (line 60) * TEXTDOMAIN variable, portability and: I18N Portability. (line 20) @@ -34106,11 +34680,11 @@ Index * timestamps, converting dates to: Time Functions. (line 76) * timestamps, formatted: Getlocaltime Function. (line 6) -* tolower: String Functions. (line 522) -* toupper: String Functions. (line 528) +* tolower: String Functions. (line 524) +* toupper: String Functions. (line 530) * tr utility: Translate Program. (line 6) * trace debugger command: Miscellaneous Debugger Commands. - (line 108) + (line 107) * traceback, display in debugger: Execution Stack. (line 13) * translate string: I18N Functions. (line 22) * translate.awk program: Translate Program. (line 55) @@ -34120,31 +34694,31 @@ Index (line 37) * troubleshooting, awk uses FS not IFS: Field Separators. (line 30) * troubleshooting, backslash before nonspecial character: Escape Sequences. - (line 120) + (line 108) * troubleshooting, division: Arithmetic Ops. (line 44) * troubleshooting, fatal errors, field widths, specifying: Constant Size. - (line 23) + (line 22) * troubleshooting, fatal errors, printf format strings: Format Modifiers. (line 158) -* troubleshooting, fflush() function: I/O Functions. (line 62) +* troubleshooting, fflush() function: I/O Functions. (line 63) * troubleshooting, function call syntax: Function Calls. (line 30) * troubleshooting, gawk: Compatibility Mode. (line 6) * troubleshooting, gawk, bug reports: Bugs. (line 9) * troubleshooting, gawk, fatal errors, function arguments: Calling Built-in. (line 16) * troubleshooting, getline function: File Checking. (line 25) -* troubleshooting, gsub()/sub() functions: String Functions. (line 471) -* troubleshooting, match() function: String Functions. (line 291) +* troubleshooting, gsub()/sub() functions: String Functions. (line 473) +* troubleshooting, match() function: String Functions. (line 292) * troubleshooting, print statement, omitting commas: Print Examples. (line 31) * troubleshooting, printing: Redirection. (line 112) * troubleshooting, quotes with file names: Special FD. (line 62) * troubleshooting, readable data files: File Checking. (line 6) * troubleshooting, regexp constants vs. string constants: Computed Regexps. - (line 39) + (line 40) * troubleshooting, string concatenation: Concatenation. (line 26) -* troubleshooting, substr() function: String Functions. (line 498) -* troubleshooting, system() function: I/O Functions. (line 96) +* troubleshooting, substr() function: String Functions. (line 500) +* troubleshooting, system() function: I/O Functions. (line 129) * troubleshooting, typographical errors, global variables: Options. (line 98) * true, logical: Truth Values. (line 6) @@ -34159,7 +34733,7 @@ Index * unassigned array elements: Reference to Elements. (line 18) * undefined functions: Pass By Value/Reference. - (line 71) + (line 68) * underscore (_), C macro: Explaining gettext. (line 71) * underscore (_), in names of private variables: Library Names. (line 29) @@ -34167,18 +34741,18 @@ Index * undisplay debugger command: Viewing And Changing Data. (line 80) * undocumented features: Undocumented. (line 6) -* Unicode <1>: Glossary. (line 133) +* Unicode <1>: Glossary. (line 197) * Unicode <2>: Ranges and Locales. (line 61) * Unicode: Ordinal Functions. (line 45) * uninitialized variables, as array subscripts: Uninitialized Subscripts. (line 6) * uniq utility: Uniq Program. (line 6) * uniq.awk program: Uniq Program. (line 65) -* Unix: Glossary. (line 611) +* Unix: Glossary. (line 753) * Unix awk, backslashes in escape sequences: Escape Sequences. - (line 132) + (line 120) * Unix awk, close() function and: Close Files And Pipes. - (line 132) + (line 133) * Unix awk, password files, field separators and: Command Line Field Separator. (line 62) * Unix, awk scripts and: Executable Scripts. (line 6) @@ -34223,17 +34797,17 @@ Index * variables, predefined conveying information: Auto-set. (line 6) * variables, private: Library Names. (line 11) * variables, setting: Options. (line 32) -* variables, shadowing: Definition Syntax. (line 71) +* variables, shadowing: Definition Syntax. (line 76) * variables, types of: Assignment Ops. (line 40) * variables, types of, comparison expressions and: Typing and Comparison. (line 9) * variables, uninitialized, as array subscripts: Uninitialized Subscripts. (line 6) * variables, user-defined: Variables. (line 6) -* version of gawk: Auto-set. (line 214) -* version of gawk extension API: Auto-set. (line 239) -* version of GNU MP library: Auto-set. (line 225) -* version of GNU MPFR library: Auto-set. (line 221) +* version of gawk: Auto-set. (line 230) +* version of gawk extension API: Auto-set. (line 255) +* version of GNU MP library: Auto-set. (line 241) +* version of GNU MPFR library: Auto-set. (line 237) * vertical bar (|): Regexp Operators. (line 70) * vertical bar (|), | operator (I/O) <1>: Precedence. (line 65) * vertical bar (|), | operator (I/O): Getline/Pipe. (line 9) @@ -34245,7 +34819,7 @@ Index * Vinschen, Corinna: Acknowledgments. (line 60) * w debugger command (alias for watch): Viewing And Changing Data. (line 67) -* w utility: Constant Size. (line 23) +* w utility: Constant Size. (line 22) * wait() extension function: Extension Sample Fork. (line 22) * waitpid() extension function: Extension Sample Fork. @@ -34290,7 +34864,7 @@ Index * xor: Bitwise Functions. (line 56) * XOR bitwise operation: Bitwise Functions. (line 6) * Yawitz, Efraim: Contributors. (line 131) -* Zaretskii, Eli <1>: Bugs. (line 72) +* Zaretskii, Eli <1>: Bugs. (line 71) * Zaretskii, Eli <2>: Contributors. (line 55) * Zaretskii, Eli: Acknowledgments. (line 60) * zerofile.awk program: Empty Files. (line 21) @@ -34307,7 +34881,7 @@ Index * | (vertical bar), |& operator (I/O) <3>: Redirection. (line 96) * | (vertical bar), |& operator (I/O): Getline/Coprocess. (line 6) * | (vertical bar), |& operator (I/O), pipes, closing: Close Files And Pipes. - (line 120) + (line 121) * | (vertical bar), || operator <1>: Precedence. (line 89) * | (vertical bar), || operator: Boolean Ops. (line 59) * ~ (tilde), ~ operator <1>: Expression Patterns. (line 24) @@ -34323,557 +34897,563 @@ Index Tag Table: Node: Top1204 -Node: Foreword42103 -Node: Preface46450 -Ref: Preface-Footnote-149320 -Ref: Preface-Footnote-249427 -Ref: Preface-Footnote-349660 -Node: History49802 -Node: Names52150 -Ref: Names-Footnote-153244 -Node: This Manual53390 -Ref: This Manual-Footnote-159219 -Node: Conventions59319 -Node: Manual History61659 -Ref: Manual History-Footnote-164650 -Ref: Manual History-Footnote-264691 -Node: How To Contribute64765 -Node: Acknowledgments66004 -Node: Getting Started70812 -Node: Running gawk73246 -Node: One-shot74436 -Node: Read Terminal75661 -Node: Long77688 -Node: Executable Scripts79204 -Ref: Executable Scripts-Footnote-181993 -Node: Comments82095 -Node: Quoting84568 -Node: DOS Quoting90074 -Node: Sample Data Files90749 -Node: Very Simple93342 -Node: Two Rules98233 -Node: More Complex100119 -Node: Statements/Lines102981 -Ref: Statements/Lines-Footnote-1107437 -Node: Other Features107702 -Node: When108633 -Ref: When-Footnote-1110389 -Node: Intro Summary110454 -Node: Invoking Gawk111337 -Node: Command Line112852 -Node: Options113643 -Ref: Options-Footnote-1129409 -Node: Other Arguments129434 -Node: Naming Standard Input132395 -Node: Environment Variables133488 -Node: AWKPATH Variable134046 -Ref: AWKPATH Variable-Footnote-1136898 -Ref: AWKPATH Variable-Footnote-2136943 -Node: AWKLIBPATH Variable137203 -Node: Other Environment Variables137962 -Node: Exit Status141453 -Node: Include Files142128 -Node: Loading Shared Libraries145716 -Node: Obsolete147143 -Node: Undocumented147840 -Node: Invoking Summary148107 -Node: Regexp149773 -Node: Regexp Usage151232 -Node: Escape Sequences153265 -Node: Regexp Operators159365 -Ref: Regexp Operators-Footnote-1166799 -Ref: Regexp Operators-Footnote-2166946 -Node: Bracket Expressions167044 -Ref: table-char-classes169061 -Node: Leftmost Longest172001 -Node: Computed Regexps173303 -Node: GNU Regexp Operators176700 -Node: Case-sensitivity180402 -Ref: Case-sensitivity-Footnote-1183292 -Ref: Case-sensitivity-Footnote-2183527 -Node: Regexp Summary183635 -Node: Reading Files185104 -Node: Records187198 -Node: awk split records187930 -Node: gawk split records192844 -Ref: gawk split records-Footnote-1197383 -Node: Fields197420 -Ref: Fields-Footnote-1200218 -Node: Nonconstant Fields200304 -Ref: Nonconstant Fields-Footnote-1202540 -Node: Changing Fields202742 -Node: Field Separators208674 -Node: Default Field Splitting211378 -Node: Regexp Field Splitting212495 -Node: Single Character Fields215845 -Node: Command Line Field Separator216904 -Node: Full Line Fields220116 -Ref: Full Line Fields-Footnote-1220624 -Node: Field Splitting Summary220670 -Ref: Field Splitting Summary-Footnote-1223801 -Node: Constant Size223902 -Node: Splitting By Content228508 -Ref: Splitting By Content-Footnote-1232581 -Node: Multiple Line232621 -Ref: Multiple Line-Footnote-1238510 -Node: Getline238689 -Node: Plain Getline240900 -Node: Getline/Variable243540 -Node: Getline/File244687 -Node: Getline/Variable/File246071 -Ref: Getline/Variable/File-Footnote-1247672 -Node: Getline/Pipe247759 -Node: Getline/Variable/Pipe250442 -Node: Getline/Coprocess251573 -Node: Getline/Variable/Coprocess252825 -Node: Getline Notes253564 -Node: Getline Summary256356 -Ref: table-getline-variants256768 -Node: Read Timeout257597 -Ref: Read Timeout-Footnote-1261411 -Node: Command-line directories261469 -Node: Input Summary262373 -Node: Input Exercises265625 -Node: Printing266353 -Node: Print268130 -Node: Print Examples269587 -Node: Output Separators272366 -Node: OFMT274384 -Node: Printf275738 -Node: Basic Printf276523 -Node: Control Letters278094 -Node: Format Modifiers282078 -Node: Printf Examples288085 -Node: Redirection290567 -Node: Special FD297406 -Ref: Special FD-Footnote-1300563 -Node: Special Files300637 -Node: Other Inherited Files301253 -Node: Special Network302253 -Node: Special Caveats303114 -Node: Close Files And Pipes304065 -Ref: Close Files And Pipes-Footnote-1311244 -Ref: Close Files And Pipes-Footnote-2311392 -Node: Output Summary311542 -Node: Output Exercises312538 -Node: Expressions313218 -Node: Values314403 -Node: Constants315079 -Node: Scalar Constants315759 -Ref: Scalar Constants-Footnote-1316618 -Node: Nondecimal-numbers316868 -Node: Regexp Constants319868 -Node: Using Constant Regexps320393 -Node: Variables323531 -Node: Using Variables324186 -Node: Assignment Options326096 -Node: Conversion327971 -Node: Strings And Numbers328495 -Ref: Strings And Numbers-Footnote-1331559 -Node: Locale influences conversions331668 -Ref: table-locale-affects334413 -Node: All Operators335001 -Node: Arithmetic Ops335631 -Node: Concatenation338136 -Ref: Concatenation-Footnote-1340955 -Node: Assignment Ops341061 -Ref: table-assign-ops346044 -Node: Increment Ops347322 -Node: Truth Values and Conditions350760 -Node: Truth Values351843 -Node: Typing and Comparison352892 -Node: Variable Typing353685 -Node: Comparison Operators357337 -Ref: table-relational-ops357747 -Node: POSIX String Comparison361262 -Ref: POSIX String Comparison-Footnote-1362334 -Node: Boolean Ops362472 -Ref: Boolean Ops-Footnote-1366951 -Node: Conditional Exp367042 -Node: Function Calls368769 -Node: Precedence372649 -Node: Locales376317 -Node: Expressions Summary377948 -Node: Patterns and Actions380522 -Node: Pattern Overview381642 -Node: Regexp Patterns383321 -Node: Expression Patterns383864 -Node: Ranges387644 -Node: BEGIN/END390750 -Node: Using BEGIN/END391512 -Ref: Using BEGIN/END-Footnote-1394249 -Node: I/O And BEGIN/END394355 -Node: BEGINFILE/ENDFILE396669 -Node: Empty399570 -Node: Using Shell Variables399887 -Node: Action Overview402163 -Node: Statements404490 -Node: If Statement406338 -Node: While Statement407836 -Node: Do Statement409864 -Node: For Statement411006 -Node: Switch Statement414161 -Node: Break Statement416549 -Node: Continue Statement418590 -Node: Next Statement420415 -Node: Nextfile Statement422795 -Node: Exit Statement425425 -Node: Built-in Variables427828 -Node: User-modified428961 -Ref: User-modified-Footnote-1436641 -Node: Auto-set436703 -Ref: Auto-set-Footnote-1450070 -Ref: Auto-set-Footnote-2450275 -Node: ARGC and ARGV450331 -Node: Pattern Action Summary454535 -Node: Arrays456962 -Node: Array Basics458291 -Node: Array Intro459135 -Ref: figure-array-elements461099 -Ref: Array Intro-Footnote-1463623 -Node: Reference to Elements463751 -Node: Assigning Elements466201 -Node: Array Example466692 -Node: Scanning an Array468450 -Node: Controlling Scanning471466 -Ref: Controlling Scanning-Footnote-1476655 -Node: Numeric Array Subscripts476971 -Node: Uninitialized Subscripts479156 -Node: Delete480773 -Ref: Delete-Footnote-1483517 -Node: Multidimensional483574 -Node: Multiscanning486669 -Node: Arrays of Arrays488258 -Node: Arrays Summary493019 -Node: Functions495124 -Node: Built-in495997 -Node: Calling Built-in497075 -Node: Numeric Functions499063 -Ref: Numeric Functions-Footnote-1503887 -Ref: Numeric Functions-Footnote-2504244 -Ref: Numeric Functions-Footnote-3504292 -Node: String Functions504561 -Ref: String Functions-Footnote-1528033 -Ref: String Functions-Footnote-2528162 -Ref: String Functions-Footnote-3528410 -Node: Gory Details528497 -Ref: table-sub-escapes530278 -Ref: table-sub-proposed531798 -Ref: table-posix-sub533162 -Ref: table-gensub-escapes534702 -Ref: Gory Details-Footnote-1535534 -Node: I/O Functions535685 -Ref: I/O Functions-Footnote-1542786 -Node: Time Functions542933 -Ref: Time Functions-Footnote-1553402 -Ref: Time Functions-Footnote-2553470 -Ref: Time Functions-Footnote-3553628 -Ref: Time Functions-Footnote-4553739 -Ref: Time Functions-Footnote-5553851 -Ref: Time Functions-Footnote-6554078 -Node: Bitwise Functions554344 -Ref: table-bitwise-ops554906 -Ref: Bitwise Functions-Footnote-1559214 -Node: Type Functions559383 -Node: I18N Functions560532 -Node: User-defined562177 -Node: Definition Syntax562981 -Ref: Definition Syntax-Footnote-1568387 -Node: Function Example568456 -Ref: Function Example-Footnote-1571373 -Node: Function Caveats571395 -Node: Calling A Function571913 -Node: Variable Scope572868 -Node: Pass By Value/Reference575856 -Node: Return Statement579366 -Node: Dynamic Typing582350 -Node: Indirect Calls583279 -Ref: Indirect Calls-Footnote-1594583 -Node: Functions Summary594711 -Node: Library Functions597410 -Ref: Library Functions-Footnote-1601028 -Ref: Library Functions-Footnote-2601171 -Node: Library Names601342 -Ref: Library Names-Footnote-1604802 -Ref: Library Names-Footnote-2605022 -Node: General Functions605108 -Node: Strtonum Function606211 -Node: Assert Function609231 -Node: Round Function612555 -Node: Cliff Random Function614096 -Node: Ordinal Functions615112 -Ref: Ordinal Functions-Footnote-1618177 -Ref: Ordinal Functions-Footnote-2618429 -Node: Join Function618640 -Ref: Join Function-Footnote-1620411 -Node: Getlocaltime Function620611 -Node: Readfile Function624352 -Node: Shell Quoting626322 -Node: Data File Management627723 -Node: Filetrans Function628355 -Node: Rewind Function632414 -Node: File Checking633799 -Ref: File Checking-Footnote-1635127 -Node: Empty Files635328 -Node: Ignoring Assigns637307 -Node: Getopt Function638858 -Ref: Getopt Function-Footnote-1650318 -Node: Passwd Functions650521 -Ref: Passwd Functions-Footnote-1659372 -Node: Group Functions659460 -Ref: Group Functions-Footnote-1667363 -Node: Walking Arrays667576 -Node: Library Functions Summary669179 -Node: Library Exercises670580 -Node: Sample Programs671860 -Node: Running Examples672630 -Node: Clones673358 -Node: Cut Program674582 -Node: Egrep Program684312 -Ref: Egrep Program-Footnote-1691816 -Node: Id Program691926 -Node: Split Program695570 -Ref: Split Program-Footnote-1699016 -Node: Tee Program699144 -Node: Uniq Program701931 -Node: Wc Program709352 -Ref: Wc Program-Footnote-1713600 -Node: Miscellaneous Programs713692 -Node: Dupword Program714905 -Node: Alarm Program716936 -Node: Translate Program721740 -Ref: Translate Program-Footnote-1726304 -Node: Labels Program726574 -Ref: Labels Program-Footnote-1729923 -Node: Word Sorting730007 -Node: History Sorting734077 -Node: Extract Program735913 -Node: Simple Sed743445 -Node: Igawk Program746507 -Ref: Igawk Program-Footnote-1760833 -Ref: Igawk Program-Footnote-2761034 -Ref: Igawk Program-Footnote-3761156 -Node: Anagram Program761271 -Node: Signature Program764333 -Node: Programs Summary765580 -Node: Programs Exercises766773 -Ref: Programs Exercises-Footnote-1770904 -Node: Advanced Features770995 -Node: Nondecimal Data772943 -Node: Array Sorting774533 -Node: Controlling Array Traversal775230 -Ref: Controlling Array Traversal-Footnote-1783561 -Node: Array Sorting Functions783679 -Ref: Array Sorting Functions-Footnote-1787571 -Node: Two-way I/O787765 -Ref: Two-way I/O-Footnote-1792709 -Ref: Two-way I/O-Footnote-2792895 -Node: TCP/IP Networking792977 -Node: Profiling795849 -Node: Advanced Features Summary804123 -Node: Internationalization806056 -Node: I18N and L10N807536 -Node: Explaining gettext808222 -Ref: Explaining gettext-Footnote-1813251 -Ref: Explaining gettext-Footnote-2813435 -Node: Programmer i18n813600 -Ref: Programmer i18n-Footnote-1818466 -Node: Translator i18n818515 -Node: String Extraction819309 -Ref: String Extraction-Footnote-1820440 -Node: Printf Ordering820526 -Ref: Printf Ordering-Footnote-1823312 -Node: I18N Portability823376 -Ref: I18N Portability-Footnote-1825825 -Node: I18N Example825888 -Ref: I18N Example-Footnote-1828688 -Node: Gawk I18N828760 -Node: I18N Summary829398 -Node: Debugger830737 -Node: Debugging831759 -Node: Debugging Concepts832200 -Node: Debugging Terms834057 -Node: Awk Debugging836632 -Node: Sample Debugging Session837524 -Node: Debugger Invocation838044 -Node: Finding The Bug839428 -Node: List of Debugger Commands845903 -Node: Breakpoint Control847235 -Node: Debugger Execution Control850927 -Node: Viewing And Changing Data854291 -Node: Execution Stack857656 -Node: Debugger Info859294 -Node: Miscellaneous Debugger Commands863311 -Node: Readline Support868503 -Node: Limitations869395 -Node: Debugging Summary871492 -Node: Arbitrary Precision Arithmetic872660 -Node: Computer Arithmetic874076 -Ref: table-numeric-ranges877677 -Ref: Computer Arithmetic-Footnote-1878536 -Node: Math Definitions878593 -Ref: table-ieee-formats881880 -Ref: Math Definitions-Footnote-1882484 -Node: MPFR features882589 -Node: FP Math Caution884260 -Ref: FP Math Caution-Footnote-1885310 -Node: Inexactness of computations885679 -Node: Inexact representation886627 -Node: Comparing FP Values887982 -Node: Errors accumulate889055 -Node: Getting Accuracy890488 -Node: Try To Round893147 -Node: Setting precision894046 -Ref: table-predefined-precision-strings894730 -Node: Setting the rounding mode896524 -Ref: table-gawk-rounding-modes896888 -Ref: Setting the rounding mode-Footnote-1900342 -Node: Arbitrary Precision Integers900521 -Ref: Arbitrary Precision Integers-Footnote-1905425 -Node: POSIX Floating Point Problems905574 -Ref: POSIX Floating Point Problems-Footnote-1909450 -Node: Floating point summary909488 -Node: Dynamic Extensions911680 -Node: Extension Intro913232 -Node: Plugin License914498 -Node: Extension Mechanism Outline915295 -Ref: figure-load-extension915723 -Ref: figure-register-new-function917203 -Ref: figure-call-new-function918207 -Node: Extension API Description920193 -Node: Extension API Functions Introduction921643 -Node: General Data Types926479 -Ref: General Data Types-Footnote-1932166 -Node: Memory Allocation Functions932465 -Ref: Memory Allocation Functions-Footnote-1935295 -Node: Constructor Functions935391 -Node: Registration Functions937125 -Node: Extension Functions937810 -Node: Exit Callback Functions940106 -Node: Extension Version String941354 -Node: Input Parsers942004 -Node: Output Wrappers951819 -Node: Two-way processors956335 -Node: Printing Messages958539 -Ref: Printing Messages-Footnote-1959616 -Node: Updating `ERRNO'959768 -Node: Requesting Values960508 -Ref: table-value-types-returned961236 -Node: Accessing Parameters962194 -Node: Symbol Table Access963425 -Node: Symbol table by name963939 -Node: Symbol table by cookie965919 -Ref: Symbol table by cookie-Footnote-1970058 -Node: Cached values970121 -Ref: Cached values-Footnote-1973625 -Node: Array Manipulation973716 -Ref: Array Manipulation-Footnote-1974814 -Node: Array Data Types974853 -Ref: Array Data Types-Footnote-1977510 -Node: Array Functions977602 -Node: Flattening Arrays981456 -Node: Creating Arrays988343 -Node: Extension API Variables993110 -Node: Extension Versioning993746 -Node: Extension API Informational Variables995647 -Node: Extension API Boilerplate996735 -Node: Finding Extensions1000551 -Node: Extension Example1001111 -Node: Internal File Description1001883 -Node: Internal File Ops1005950 -Ref: Internal File Ops-Footnote-11017608 -Node: Using Internal File Ops1017748 -Ref: Using Internal File Ops-Footnote-11020131 -Node: Extension Samples1020404 -Node: Extension Sample File Functions1021928 -Node: Extension Sample Fnmatch1029530 -Node: Extension Sample Fork1031012 -Node: Extension Sample Inplace1032225 -Node: Extension Sample Ord1033900 -Node: Extension Sample Readdir1034736 -Ref: table-readdir-file-types1035592 -Node: Extension Sample Revout1036403 -Node: Extension Sample Rev2way1036994 -Node: Extension Sample Read write array1037735 -Node: Extension Sample Readfile1039674 -Node: Extension Sample Time1040769 -Node: Extension Sample API Tests1042118 -Node: gawkextlib1042609 -Node: Extension summary1045259 -Node: Extension Exercises1048941 -Node: Language History1049663 -Node: V7/SVR3.11051320 -Node: SVR41053501 -Node: POSIX1054946 -Node: BTL1056335 -Node: POSIX/GNU1057069 -Node: Feature History1062698 -Node: Common Extensions1075789 -Node: Ranges and Locales1077113 -Ref: Ranges and Locales-Footnote-11081752 -Ref: Ranges and Locales-Footnote-21081779 -Ref: Ranges and Locales-Footnote-31082013 -Node: Contributors1082234 -Node: History summary1087774 -Node: Installation1089143 -Node: Gawk Distribution1090099 -Node: Getting1090583 -Node: Extracting1091407 -Node: Distribution contents1093049 -Node: Unix Installation1098819 -Node: Quick Installation1099436 -Node: Additional Configuration Options1101867 -Node: Configuration Philosophy1103607 -Node: Non-Unix Installation1105958 -Node: PC Installation1106416 -Node: PC Binary Installation1107742 -Node: PC Compiling1109590 -Ref: PC Compiling-Footnote-11112611 -Node: PC Testing1112716 -Node: PC Using1113892 -Node: Cygwin1118007 -Node: MSYS1118830 -Node: VMS Installation1119328 -Node: VMS Compilation1120120 -Ref: VMS Compilation-Footnote-11121342 -Node: VMS Dynamic Extensions1121400 -Node: VMS Installation Details1123084 -Node: VMS Running1125336 -Node: VMS GNV1128177 -Node: VMS Old Gawk1128911 -Node: Bugs1129381 -Node: Other Versions1133285 -Node: Installation summary1139498 -Node: Notes1140554 -Node: Compatibility Mode1141419 -Node: Additions1142201 -Node: Accessing The Source1143126 -Node: Adding Code1144562 -Node: New Ports1150734 -Node: Derived Files1155216 -Ref: Derived Files-Footnote-11160691 -Ref: Derived Files-Footnote-21160725 -Ref: Derived Files-Footnote-31161321 -Node: Future Extensions1161435 -Node: Implementation Limitations1162041 -Node: Extension Design1163289 -Node: Old Extension Problems1164443 -Ref: Old Extension Problems-Footnote-11165960 -Node: Extension New Mechanism Goals1166017 -Ref: Extension New Mechanism Goals-Footnote-11169377 -Node: Extension Other Design Decisions1169566 -Node: Extension Future Growth1171674 -Node: Old Extension Mechanism1172510 -Node: Notes summary1174272 -Node: Basic Concepts1175458 -Node: Basic High Level1176139 -Ref: figure-general-flow1176411 -Ref: figure-process-flow1177010 -Ref: Basic High Level-Footnote-11180239 -Node: Basic Data Typing1180424 -Node: Glossary1183752 -Node: Copying1208910 -Node: GNU Free Documentation License1246466 -Node: Index1271602 +Node: Foreword342451 +Node: Foreword446895 +Node: Preface48426 +Ref: Preface-Footnote-151297 +Ref: Preface-Footnote-251404 +Ref: Preface-Footnote-351637 +Node: History51779 +Node: Names54130 +Ref: Names-Footnote-155224 +Node: This Manual55370 +Ref: This Manual-Footnote-161870 +Node: Conventions61970 +Node: Manual History64307 +Ref: Manual History-Footnote-167300 +Ref: Manual History-Footnote-267341 +Node: How To Contribute67415 +Node: Acknowledgments68544 +Node: Getting Started73410 +Node: Running gawk75849 +Node: One-shot77039 +Node: Read Terminal78303 +Node: Long80334 +Node: Executable Scripts81847 +Ref: Executable Scripts-Footnote-184636 +Node: Comments84739 +Node: Quoting87221 +Node: DOS Quoting92739 +Node: Sample Data Files93414 +Node: Very Simple96009 +Node: Two Rules100908 +Node: More Complex102794 +Node: Statements/Lines105656 +Ref: Statements/Lines-Footnote-1110111 +Node: Other Features110376 +Node: When111312 +Ref: When-Footnote-1113066 +Node: Intro Summary113131 +Node: Invoking Gawk114015 +Node: Command Line115529 +Node: Options116327 +Ref: Options-Footnote-1132122 +Ref: Options-Footnote-2132351 +Node: Other Arguments132376 +Node: Naming Standard Input135324 +Node: Environment Variables136417 +Node: AWKPATH Variable136975 +Ref: AWKPATH Variable-Footnote-1140382 +Ref: AWKPATH Variable-Footnote-2140427 +Node: AWKLIBPATH Variable140687 +Node: Other Environment Variables141943 +Node: Exit Status145574 +Node: Include Files146250 +Node: Loading Shared Libraries149839 +Node: Obsolete151266 +Node: Undocumented151958 +Node: Invoking Summary152225 +Node: Regexp153888 +Node: Regexp Usage155342 +Node: Escape Sequences157379 +Node: Regexp Operators163608 +Ref: Regexp Operators-Footnote-1171018 +Ref: Regexp Operators-Footnote-2171165 +Node: Bracket Expressions171263 +Ref: table-char-classes173278 +Node: Leftmost Longest176220 +Node: Computed Regexps177522 +Node: GNU Regexp Operators180951 +Node: Case-sensitivity184623 +Ref: Case-sensitivity-Footnote-1187508 +Ref: Case-sensitivity-Footnote-2187743 +Node: Regexp Summary187851 +Node: Reading Files189318 +Node: Records191480 +Node: awk split records192213 +Node: gawk split records197142 +Ref: gawk split records-Footnote-1201681 +Node: Fields201718 +Ref: Fields-Footnote-1204496 +Node: Nonconstant Fields204582 +Ref: Nonconstant Fields-Footnote-1206820 +Node: Changing Fields207023 +Node: Field Separators212954 +Node: Default Field Splitting215658 +Node: Regexp Field Splitting216775 +Node: Single Character Fields220125 +Node: Command Line Field Separator221184 +Node: Full Line Fields224401 +Ref: Full Line Fields-Footnote-1225922 +Ref: Full Line Fields-Footnote-2225968 +Node: Field Splitting Summary226069 +Node: Constant Size228143 +Node: Splitting By Content232722 +Ref: Splitting By Content-Footnote-1236687 +Node: Multiple Line236850 +Ref: Multiple Line-Footnote-1242731 +Node: Getline242910 +Node: Plain Getline245380 +Node: Getline/Variable248020 +Node: Getline/File249169 +Node: Getline/Variable/File250554 +Ref: Getline/Variable/File-Footnote-1252157 +Node: Getline/Pipe252244 +Node: Getline/Variable/Pipe254922 +Node: Getline/Coprocess256053 +Node: Getline/Variable/Coprocess257317 +Node: Getline Notes258056 +Node: Getline Summary260850 +Ref: table-getline-variants261262 +Node: Read Timeout262091 +Ref: Read Timeout-Footnote-1265994 +Node: Retrying Input266052 +Node: Command-line directories267251 +Node: Input Summary268158 +Node: Input Exercises271543 +Node: Printing272271 +Node: Print274106 +Node: Print Examples275563 +Node: Output Separators278342 +Node: OFMT280360 +Node: Printf281715 +Node: Basic Printf282500 +Node: Control Letters284072 +Node: Format Modifiers288057 +Node: Printf Examples294063 +Node: Redirection296549 +Node: Special FD303387 +Ref: Special FD-Footnote-1306553 +Node: Special Files306627 +Node: Other Inherited Files307244 +Node: Special Network308244 +Node: Special Caveats309106 +Node: Close Files And Pipes310055 +Ref: Close Files And Pipes-Footnote-1317240 +Ref: Close Files And Pipes-Footnote-2317388 +Node: Nonfatal317538 +Node: Output Summary319863 +Node: Output Exercises321084 +Node: Expressions321764 +Node: Values322953 +Node: Constants323630 +Node: Scalar Constants324321 +Ref: Scalar Constants-Footnote-1325183 +Node: Nondecimal-numbers325433 +Node: Regexp Constants328443 +Node: Using Constant Regexps328969 +Node: Variables332132 +Node: Using Variables332789 +Node: Assignment Options334700 +Node: Conversion336575 +Node: Strings And Numbers337099 +Ref: Strings And Numbers-Footnote-1340164 +Node: Locale influences conversions340273 +Ref: table-locale-affects343019 +Node: All Operators343611 +Node: Arithmetic Ops344240 +Node: Concatenation346745 +Ref: Concatenation-Footnote-1349564 +Node: Assignment Ops349671 +Ref: table-assign-ops354650 +Node: Increment Ops355960 +Node: Truth Values and Conditions359391 +Node: Truth Values360474 +Node: Typing and Comparison361523 +Node: Variable Typing362339 +Node: Comparison Operators366006 +Ref: table-relational-ops366416 +Node: POSIX String Comparison369911 +Ref: POSIX String Comparison-Footnote-1370983 +Node: Boolean Ops371122 +Ref: Boolean Ops-Footnote-1375600 +Node: Conditional Exp375691 +Node: Function Calls377429 +Node: Precedence381309 +Node: Locales384969 +Node: Expressions Summary386601 +Node: Patterns and Actions389172 +Node: Pattern Overview390292 +Node: Regexp Patterns391971 +Node: Expression Patterns392514 +Node: Ranges396294 +Node: BEGIN/END399401 +Node: Using BEGIN/END400162 +Ref: Using BEGIN/END-Footnote-1402898 +Node: I/O And BEGIN/END403004 +Node: BEGINFILE/ENDFILE405319 +Node: Empty408216 +Node: Using Shell Variables408533 +Node: Action Overview410806 +Node: Statements413132 +Node: If Statement414980 +Node: While Statement416475 +Node: Do Statement418503 +Node: For Statement419651 +Node: Switch Statement422809 +Node: Break Statement425191 +Node: Continue Statement427284 +Node: Next Statement429111 +Node: Nextfile Statement431492 +Node: Exit Statement434120 +Node: Built-in Variables436531 +Node: User-modified437664 +Ref: User-modified-Footnote-1445298 +Node: Auto-set445360 +Ref: Auto-set-Footnote-1459593 +Ref: Auto-set-Footnote-2459798 +Node: ARGC and ARGV459854 +Node: Pattern Action Summary464072 +Node: Arrays466505 +Node: Array Basics467834 +Node: Array Intro468678 +Ref: figure-array-elements470615 +Ref: Array Intro-Footnote-1473238 +Node: Reference to Elements473366 +Node: Assigning Elements475828 +Node: Array Example476319 +Node: Scanning an Array478078 +Node: Controlling Scanning481101 +Ref: Controlling Scanning-Footnote-1486495 +Node: Numeric Array Subscripts486811 +Node: Uninitialized Subscripts488996 +Node: Delete490613 +Ref: Delete-Footnote-1493362 +Node: Multidimensional493419 +Node: Multiscanning496516 +Node: Arrays of Arrays498105 +Node: Arrays Summary502859 +Node: Functions504950 +Node: Built-in505989 +Node: Calling Built-in507067 +Node: Numeric Functions509062 +Ref: Numeric Functions-Footnote-1513895 +Ref: Numeric Functions-Footnote-2514252 +Ref: Numeric Functions-Footnote-3514300 +Node: String Functions514572 +Ref: String Functions-Footnote-1538073 +Ref: String Functions-Footnote-2538202 +Ref: String Functions-Footnote-3538450 +Node: Gory Details538537 +Ref: table-sub-escapes540318 +Ref: table-sub-proposed541833 +Ref: table-posix-sub543195 +Ref: table-gensub-escapes544732 +Ref: Gory Details-Footnote-1545565 +Node: I/O Functions545716 +Ref: I/O Functions-Footnote-1552952 +Node: Time Functions553099 +Ref: Time Functions-Footnote-1563608 +Ref: Time Functions-Footnote-2563676 +Ref: Time Functions-Footnote-3563834 +Ref: Time Functions-Footnote-4563945 +Ref: Time Functions-Footnote-5564057 +Ref: Time Functions-Footnote-6564284 +Node: Bitwise Functions564550 +Ref: table-bitwise-ops565112 +Ref: Bitwise Functions-Footnote-1569440 +Node: Type Functions569612 +Node: I18N Functions570764 +Node: User-defined572411 +Node: Definition Syntax573216 +Ref: Definition Syntax-Footnote-1578875 +Node: Function Example578946 +Ref: Function Example-Footnote-1581867 +Node: Function Caveats581889 +Node: Calling A Function582407 +Node: Variable Scope583365 +Node: Pass By Value/Reference586358 +Node: Return Statement589855 +Node: Dynamic Typing592834 +Node: Indirect Calls593763 +Ref: Indirect Calls-Footnote-1604006 +Node: Functions Summary604134 +Node: Library Functions606836 +Ref: Library Functions-Footnote-1610444 +Ref: Library Functions-Footnote-2610587 +Node: Library Names610758 +Ref: Library Names-Footnote-1614216 +Ref: Library Names-Footnote-2614439 +Node: General Functions614525 +Node: Strtonum Function615628 +Node: Assert Function618650 +Node: Round Function621974 +Node: Cliff Random Function623515 +Node: Ordinal Functions624531 +Ref: Ordinal Functions-Footnote-1627594 +Ref: Ordinal Functions-Footnote-2627846 +Node: Join Function628057 +Ref: Join Function-Footnote-1629827 +Node: Getlocaltime Function630027 +Node: Readfile Function633771 +Node: Shell Quoting635743 +Node: Data File Management637144 +Node: Filetrans Function637776 +Node: Rewind Function641872 +Node: File Checking643258 +Ref: File Checking-Footnote-1644591 +Node: Empty Files644792 +Node: Ignoring Assigns646771 +Node: Getopt Function648321 +Ref: Getopt Function-Footnote-1659785 +Node: Passwd Functions659985 +Ref: Passwd Functions-Footnote-1668825 +Node: Group Functions668913 +Ref: Group Functions-Footnote-1676810 +Node: Walking Arrays677015 +Node: Library Functions Summary680021 +Node: Library Exercises681423 +Node: Sample Programs682703 +Node: Running Examples683473 +Node: Clones684201 +Node: Cut Program685425 +Node: Egrep Program695145 +Ref: Egrep Program-Footnote-1702648 +Node: Id Program702758 +Node: Split Program706434 +Ref: Split Program-Footnote-1709888 +Node: Tee Program710016 +Node: Uniq Program712805 +Node: Wc Program720224 +Ref: Wc Program-Footnote-1724474 +Node: Miscellaneous Programs724568 +Node: Dupword Program725781 +Node: Alarm Program727812 +Node: Translate Program732617 +Ref: Translate Program-Footnote-1737180 +Node: Labels Program737450 +Ref: Labels Program-Footnote-1740801 +Node: Word Sorting740885 +Node: History Sorting744955 +Node: Extract Program746790 +Node: Simple Sed754314 +Node: Igawk Program757384 +Ref: Igawk Program-Footnote-1771710 +Ref: Igawk Program-Footnote-2771911 +Ref: Igawk Program-Footnote-3772033 +Node: Anagram Program772148 +Node: Signature Program775209 +Node: Programs Summary776456 +Node: Programs Exercises777677 +Ref: Programs Exercises-Footnote-1781808 +Node: Advanced Features781899 +Node: Nondecimal Data783881 +Node: Array Sorting785471 +Node: Controlling Array Traversal786171 +Ref: Controlling Array Traversal-Footnote-1794537 +Node: Array Sorting Functions794655 +Ref: Array Sorting Functions-Footnote-1798541 +Node: Two-way I/O798737 +Ref: Two-way I/O-Footnote-1803682 +Ref: Two-way I/O-Footnote-2803868 +Node: TCP/IP Networking803950 +Node: Profiling806822 +Node: Advanced Features Summary815093 +Node: Internationalization817026 +Node: I18N and L10N818506 +Node: Explaining gettext819192 +Ref: Explaining gettext-Footnote-1824217 +Ref: Explaining gettext-Footnote-2824401 +Node: Programmer i18n824566 +Ref: Programmer i18n-Footnote-1829442 +Node: Translator i18n829491 +Node: String Extraction830285 +Ref: String Extraction-Footnote-1831416 +Node: Printf Ordering831502 +Ref: Printf Ordering-Footnote-1834288 +Node: I18N Portability834352 +Ref: I18N Portability-Footnote-1836808 +Node: I18N Example836871 +Ref: I18N Example-Footnote-1839674 +Node: Gawk I18N839746 +Node: I18N Summary840390 +Node: Debugger841730 +Node: Debugging842752 +Node: Debugging Concepts843193 +Node: Debugging Terms845003 +Node: Awk Debugging847575 +Node: Sample Debugging Session848481 +Node: Debugger Invocation849015 +Node: Finding The Bug850400 +Node: List of Debugger Commands856879 +Node: Breakpoint Control858211 +Node: Debugger Execution Control861888 +Node: Viewing And Changing Data865247 +Node: Execution Stack868623 +Node: Debugger Info870258 +Node: Miscellaneous Debugger Commands874303 +Node: Readline Support879304 +Node: Limitations880198 +Node: Debugging Summary882313 +Node: Arbitrary Precision Arithmetic883487 +Node: Computer Arithmetic884903 +Ref: table-numeric-ranges888480 +Ref: Computer Arithmetic-Footnote-1889004 +Node: Math Definitions889061 +Ref: table-ieee-formats892356 +Ref: Math Definitions-Footnote-1892960 +Node: MPFR features893065 +Node: FP Math Caution894736 +Ref: FP Math Caution-Footnote-1895786 +Node: Inexactness of computations896155 +Node: Inexact representation897114 +Node: Comparing FP Values898472 +Node: Errors accumulate899554 +Node: Getting Accuracy900986 +Node: Try To Round903690 +Node: Setting precision904589 +Ref: table-predefined-precision-strings905273 +Node: Setting the rounding mode907102 +Ref: table-gawk-rounding-modes907466 +Ref: Setting the rounding mode-Footnote-1910918 +Node: Arbitrary Precision Integers911097 +Ref: Arbitrary Precision Integers-Footnote-1916013 +Node: POSIX Floating Point Problems916162 +Ref: POSIX Floating Point Problems-Footnote-1920041 +Node: Floating point summary920079 +Node: Dynamic Extensions922266 +Node: Extension Intro923818 +Node: Plugin License925083 +Node: Extension Mechanism Outline925880 +Ref: figure-load-extension926308 +Ref: figure-register-new-function927788 +Ref: figure-call-new-function928792 +Node: Extension API Description930779 +Node: Extension API Functions Introduction932313 +Node: General Data Types937182 +Ref: General Data Types-Footnote-1943082 +Node: Memory Allocation Functions943381 +Ref: Memory Allocation Functions-Footnote-1946220 +Node: Constructor Functions946319 +Node: Registration Functions948058 +Node: Extension Functions948743 +Node: Exit Callback Functions951040 +Node: Extension Version String952288 +Node: Input Parsers952951 +Node: Output Wrappers962826 +Node: Two-way processors967339 +Node: Printing Messages969602 +Ref: Printing Messages-Footnote-1970678 +Node: Updating `ERRNO'970830 +Node: Requesting Values971570 +Ref: table-value-types-returned972297 +Node: Accessing Parameters973254 +Node: Symbol Table Access974488 +Node: Symbol table by name975002 +Node: Symbol table by cookie977022 +Ref: Symbol table by cookie-Footnote-1981167 +Node: Cached values981230 +Ref: Cached values-Footnote-1984726 +Node: Array Manipulation984817 +Ref: Array Manipulation-Footnote-1985907 +Node: Array Data Types985944 +Ref: Array Data Types-Footnote-1988599 +Node: Array Functions988691 +Node: Flattening Arrays992550 +Node: Creating Arrays999452 +Node: Redirection API1004223 +Node: Extension API Variables1007048 +Node: Extension Versioning1007681 +Node: Extension API Informational Variables1009572 +Node: Extension API Boilerplate1010637 +Node: Finding Extensions1014446 +Node: Extension Example1015006 +Node: Internal File Description1015778 +Node: Internal File Ops1019845 +Ref: Internal File Ops-Footnote-11031596 +Node: Using Internal File Ops1031736 +Ref: Using Internal File Ops-Footnote-11034119 +Node: Extension Samples1034392 +Node: Extension Sample File Functions1035920 +Node: Extension Sample Fnmatch1043601 +Node: Extension Sample Fork1045089 +Node: Extension Sample Inplace1046304 +Node: Extension Sample Ord1048390 +Node: Extension Sample Readdir1049226 +Ref: table-readdir-file-types1050103 +Node: Extension Sample Revout1050914 +Node: Extension Sample Rev2way1051503 +Node: Extension Sample Read write array1052243 +Node: Extension Sample Readfile1054183 +Node: Extension Sample Time1055278 +Node: Extension Sample API Tests1056626 +Node: gawkextlib1057117 +Node: Extension summary1059818 +Node: Extension Exercises1063507 +Node: Language History1065003 +Node: V7/SVR3.11066659 +Node: SVR41068812 +Node: POSIX1070246 +Node: BTL1071627 +Node: POSIX/GNU1072358 +Node: Feature History1078197 +Node: Common Extensions1091994 +Node: Ranges and Locales1093366 +Ref: Ranges and Locales-Footnote-11097985 +Ref: Ranges and Locales-Footnote-21098012 +Ref: Ranges and Locales-Footnote-31098247 +Node: Contributors1098468 +Node: History summary1104008 +Node: Installation1105387 +Node: Gawk Distribution1106333 +Node: Getting1106817 +Node: Extracting1107640 +Node: Distribution contents1109277 +Node: Unix Installation1115379 +Node: Quick Installation1116062 +Node: Shell Startup Files1118473 +Node: Additional Configuration Options1119552 +Node: Configuration Philosophy1121356 +Node: Non-Unix Installation1123725 +Node: PC Installation1124183 +Node: PC Binary Installation1125503 +Node: PC Compiling1127351 +Ref: PC Compiling-Footnote-11130372 +Node: PC Testing1130481 +Node: PC Using1131657 +Node: Cygwin1135772 +Node: MSYS1136542 +Node: VMS Installation1137043 +Node: VMS Compilation1137835 +Ref: VMS Compilation-Footnote-11139064 +Node: VMS Dynamic Extensions1139122 +Node: VMS Installation Details1140806 +Node: VMS Running1143057 +Node: VMS GNV1145897 +Node: VMS Old Gawk1146632 +Node: Bugs1147102 +Node: Other Versions1150991 +Node: Installation summary1157425 +Node: Notes1158484 +Node: Compatibility Mode1159349 +Node: Additions1160131 +Node: Accessing The Source1161056 +Node: Adding Code1162491 +Node: New Ports1168648 +Node: Derived Files1173130 +Ref: Derived Files-Footnote-11178605 +Ref: Derived Files-Footnote-21178639 +Ref: Derived Files-Footnote-31179235 +Node: Future Extensions1179349 +Node: Implementation Limitations1179955 +Node: Extension Design1181203 +Node: Old Extension Problems1182357 +Ref: Old Extension Problems-Footnote-11183874 +Node: Extension New Mechanism Goals1183931 +Ref: Extension New Mechanism Goals-Footnote-11187291 +Node: Extension Other Design Decisions1187480 +Node: Extension Future Growth1189588 +Node: Old Extension Mechanism1190424 +Node: Notes summary1192186 +Node: Basic Concepts1193372 +Node: Basic High Level1194053 +Ref: figure-general-flow1194325 +Ref: figure-process-flow1194924 +Ref: Basic High Level-Footnote-11198153 +Node: Basic Data Typing1198338 +Node: Glossary1201666 +Node: Copying1233595 +Node: GNU Free Documentation License1271151 +Node: Index1296287 End Tag Table diff --git a/doc/gawk.texi b/doc/gawk.texi index 0f2b2985..ca378cca 100644 --- a/doc/gawk.texi +++ b/doc/gawk.texi @@ -37,13 +37,11 @@ @ifnotdocbook @set BULLET @bullet{} @set MINUS @minus{} -@set NUL @sc{nul} @end ifnotdocbook @ifdocbook @set BULLET @set MINUS -@set NUL NUL @end ifdocbook @set xref-automatic-section-title @@ -53,12 +51,13 @@ @c applies to and all the info about who's publishing this edition @c These apply across the board. -@set UPDATE-MONTH September, 2014 +@set UPDATE-MONTH February, 2015 @set VERSION 4.1 @set PATCHLEVEL 2 +@set GAWKINETTITLE TCP/IP Internetworking with @command{gawk} @ifset FOR_PRINT -@set TITLE Effective AWK Programming +@set TITLE Effective awk Programming @end ifset @ifclear FOR_PRINT @set TITLE GAWK: Effective AWK Programming @@ -177,19 +176,31 @@ @macro DBREF{text} @ref{\text\} @end macro +@macro DBXREF{text} +@xref{\text\} +@end macro +@macro DBPXREF{text} +@pxref{\text\} +@end macro @end ifdocbook @ifnotdocbook @macro DBREF{text} @ref{\text\}, @end macro +@macro DBXREF{text} +@xref{\text\}, +@end macro +@macro DBPXREF{text} +@pxref{\text\}, +@end macro @end ifnotdocbook @ifclear FOR_PRINT @set FN file name -@set FFN File Name +@set FFN File name @set DF data file -@set DDF Data File +@set DDF Data file @set PVERSION version @end ifclear @ifset FOR_PRINT @@ -197,7 +208,7 @@ @set FFN Filename @set DF datafile @set DDF Datafile -@set PVERSION Version +@set PVERSION version @end ifset @c For HTML, spell out email addresses, to avoid problems with @@ -288,13 +299,13 @@ Fax: +1-617-542-2652 Email: <email>gnu@@gnu.org</email> URL: <ulink url="http://www.gnu.org">http://www.gnu.org/</ulink></literallayout> -<literallayout class="normal">Copyright © 1989, 1991, 1992, 1993, 1996–2005, 2007, 2009–2014 +<literallayout class="normal">Copyright © 1989, 1991, 1992, 1993, 1996–2005, 2007, 2009–2015 Free Software Foundation, Inc. All Rights Reserved.</literallayout> @end docbook @ifnotdocbook -Copyright @copyright{} 1989, 1991, 1992, 1993, 1996--2005, 2007, 2009--2014 @* +Copyright @copyright{} 1989, 1991, 1992, 1993, 1996--2005, 2007, 2009--2015 @* Free Software Foundation, Inc. @end ifnotdocbook @sp 2 @@ -317,7 +328,7 @@ A copy of the license is included in the section entitled A copy of the license may be found on the Internet at @uref{http://www.gnu.org/software/gawk/manual/html_node/GNU-Free-Documentation-License.html, -the GNU Project's web site}. +the GNU Project's website}. @end ifset @enumerate a @@ -385,10 +396,10 @@ ISBN 1-882114-28-0 @* @sp 9 @center @i{To my parents, for their love, and for the wonderful example they set for me.} @sp 1 -@center @i{To my wife Miriam, for making me complete. +@center @i{To my wife, Miriam, for making me complete. Thank you for building your life together with me.} @sp 1 -@center @i{To our children Chana, Rivka, Nachum and Malka, for enrichening our lives in innumerable ways.} +@center @i{To our children, Chana, Rivka, Nachum, and Malka, for enrichening our lives in innumerable ways.} @sp 1 @w{ } @page @@ -433,8 +444,9 @@ particular records in a file and perform operations upon them. @end ifnottex @menu -* Foreword:: Some nice words about this +* Foreword3:: Some nice words about this @value{DOCUMENT}. +* Foreword4:: More nice words. * Preface:: What this @value{DOCUMENT} is about; brief history and acknowledgments. * Getting Started:: A basic introduction to using @@ -461,7 +473,7 @@ particular records in a file and perform operations upon them. @command{gawk}. * Internationalization:: Getting @command{gawk} to speak your language. -* Debugger:: The @code{gawk} debugger. +* Debugger:: The @command{gawk} debugger. * Arbitrary Precision Arithmetic:: Arbitrary precision arithmetic with @command{gawk}. * Dynamic Extensions:: Adding new built-in functions to @@ -592,6 +604,7 @@ particular records in a file and perform operations upon them. @code{getline}. * Getline Summary:: Summary of @code{getline} Variants. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -621,6 +634,7 @@ particular records in a file and perform operations upon them. * Special Caveats:: Things to watch out for. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. * Values:: Constants, Variables, and Regular @@ -932,6 +946,7 @@ particular records in a file and perform operations upon them. * Array Functions:: Functions for working with arrays. * Flattening Arrays:: How to flatten arrays. * Creating Arrays:: How to create and populate arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension Versioning:: API Version information. * Extension API Informational Variables:: Variables providing information about @@ -944,7 +959,7 @@ particular records in a file and perform operations upon them. * Internal File Ops:: The code for internal file operations. * Using Internal File Ops:: How to use an external extension. * Extension Samples:: The sample extensions that ship with - @code{gawk}. + @command{gawk}. * Extension Sample File Functions:: The file functions sample. * Extension Sample Fnmatch:: An interface to @code{fnmatch()}. * Extension Sample Fork:: An interface to @code{fork()} and @@ -990,6 +1005,7 @@ particular records in a file and perform operations upon them. * Unix Installation:: Installing @command{gawk} under various versions of Unix. * Quick Installation:: Compiling @command{gawk} under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. * Non-Unix Installation:: Installation on Other Operating @@ -1062,8 +1078,8 @@ for enrichening our lives in innumerable ways. @summarycontents @contents -@node Foreword -@unnumbered Foreword +@node Foreword3 +@unnumbered Foreword to the Third Edition @c This bit is post-processed by a script which turns the chapter @c tag into a preface tag, and moves this stuff to before the title. @@ -1076,7 +1092,7 @@ for enrichening our lives in innumerable ways. <!-- can't put mawk into command tags. sigh. --> <affiliation><jobtitle>Author of mawk</jobtitle></affiliation> </author> - <date>March, 2001</date> + <date>March 2001</date> </prefaceinfo> @end docbook @@ -1088,21 +1104,23 @@ The circumstances started a couple of years earlier. I was working at a new job and noticed an unplugged Unix computer sitting in the corner. No one knew how to use it, and neither did I. However, -a couple of days later it was running, and +a couple of days later, it was running, and I was @code{root} and the one-and-only user. That day, I began the transition from statistician to Unix programmer. On one of many trips to the library or bookstore in search of -books on Unix, I found the gray AWK book, a.k.a.@: Aho, Kernighan and -Weinberger, @cite{The AWK Programming Language}, Addison-Wesley, -1988. AWK's simple programming paradigm---find a pattern in the +books on Unix, I found the gray AWK book, a.k.a.@: +Alfred V.@: Aho, Brian W.@: Kernighan, and +Peter J.@: Weinberger's @cite{The AWK Programming Language} (Addison-Wesley, +1988). @command{awk}'s simple programming paradigm---find a pattern in the input and then perform an action---often reduced complex or tedious data manipulations to a few lines of code. I was excited to try my hand at programming in AWK. Alas, the @command{awk} on my computer was a limited version of the -language described in the AWK book. I discovered that my computer -had ``old @command{awk}'' and the AWK book described ``new @command{awk}.'' +language described in the gray book. I discovered that my computer +had ``old @command{awk}'' and the book described +``new @command{awk}.'' I learned that this was typical; the old version refused to step aside or relinquish its name. If a system had a new @command{awk}, it was invariably called @command{nawk}, and few systems had it. @@ -1120,7 +1138,7 @@ My Unix system started out unplugged from the wall; it certainly was not plugged into a network. So, oblivious to the existence of @command{gawk} and the Unix community in general, and desiring a new @command{awk}, I wrote my own, called @command{mawk}. -Before I was finished I knew about @command{gawk}, +Before I was finished, I knew about @command{gawk}, but it was too late to stop, so I eventually posted to a @code{comp.sources} newsgroup. @@ -1129,7 +1147,7 @@ from Arnold introducing himself. He suggested we share design and algorithms and attached a draft of the POSIX standard so that I could update @command{mawk} to support language extensions added -after publication of the AWK book. +after publication of @cite{The AWK Programming Language}. Frankly, if our roles had been reversed, I would not have been so open and we probably would @@ -1148,7 +1166,7 @@ standard. On the other hand, the novice AWK programmer can study a wealth of practical programs that emphasize the power of AWK's basic idioms: -data driven control-flow, pattern matching with regular expressions, +data-driven control flow, pattern matching with regular expressions, and associative arrays. Those looking for something new can try out @command{gawk}'s interface to network protocols via special @file{/inet} files. @@ -1156,7 +1174,7 @@ interface to network protocols via special @file{/inet} files. The programs in this book make clear that an AWK program is typically much smaller and faster to develop than a counterpart written in C. -Consequently, there is often a payoff to prototype an +Consequently, there is often a payoff to prototyping an algorithm or design in AWK to get it running quickly and expose problems early. Often, the interpreted performance is adequate and the AWK prototype becomes the product. @@ -1209,7 +1227,62 @@ AWK or want to learn how, then read this book. @display Michael Brennan Author of @command{mawk} -March, 2001 +March 2001 +@end display +@end ifnotdocbook + +@node Foreword4 +@unnumbered Foreword to the Fourth Edition + +@c This bit is post-processed by a script which turns the chapter +@c tag into a preface tag, and moves this stuff to before the title. +@c Bleah. +@docbook + <prefaceinfo> + <author> + <firstname>Michael</firstname> + <surname>Brennan</surname> + <!-- can't put mawk into command tags. sigh. --> + <affiliation><jobtitle>Author of mawk</jobtitle></affiliation> + </author> + <date>October 2014</date> + </prefaceinfo> +@end docbook + +Some things don't change. Thirteen years ago I wrote: +``If you use AWK or want to learn how, then read this book.'' +True then, and still true today. + +Learning to use a programming language is about more than mastering the +syntax. One needs to acquire an understanding of how to use the +features of the language to solve practical programming problems. +A focus of this book is many examples that show how to use AWK. + +Some things do change. Our computers are much faster and have more memory. +Consequently, speed and storage inefficiencies of a high-level language +matter less. Prototyping in AWK and then rewriting in C for performance +reasons happens less, because more often the prototype is fast enough. + +Of course, there are computing operations that are best done in C or C++. +With @command{gawk} 4.1 and later, you do not have to choose between writing +your program in AWK or in C/C++. You can write most of your +program in AWK and the aspects that require C/C++ capabilities can be written +in C/C++, and then the pieces glued together when the @command{gawk} module loads +the C/C++ module as a dynamic plug-in. +@c Chapter 16 +@ref{Dynamic Extensions}, +has all the +details, and, as expected, many examples to help you learn the ins and outs. + +I enjoy programming in AWK and had fun (re)reading this book. +I think you will too. + +@ifnotdocbook +@cindex Brennan, Michael +@display +Michael Brennan +Author of @command{mawk} +October 2014 @end display @end ifnotdocbook @@ -1229,9 +1302,9 @@ March, 2001 <firstname>Arnold</firstname> <surname>Robbins</surname> <affiliation><jobtitle>Nof Ayalon</jobtitle></affiliation> - <affiliation><jobtitle>ISRAEL</jobtitle></affiliation> + <affiliation><jobtitle>Israel</jobtitle></affiliation> </author> - <date>December, 2014</date> + <date>February 2015</date> </prefaceinfo> @end docbook @@ -1243,7 +1316,7 @@ The @command{awk} utility interprets a special-purpose programming language that makes it easy to handle simple data-reformatting jobs. The GNU implementation of @command{awk} is called @command{gawk}; if you -invoke it with the proper options or environment variables +invoke it with the proper options or environment variables, it is fully compatible with the POSIX@footnote{The 2008 POSIX standard is accessible online at @w{@url{http://www.opengroup.org/onlinepubs/9699919799/}.}} @@ -1274,7 +1347,7 @@ Generate reports Validate data @item -Produce indexes and perform other document preparation tasks +Produce indexes and perform other document-preparation tasks @item Experiment with algorithms that you can adapt later to other computer @@ -1350,10 +1423,10 @@ has been removed.} @unnumberedsec History of @command{awk} and @command{gawk} @cindex recipe for a programming language @cindex programming language, recipe for -@cindex sidebar, Recipe For A Programming Language +@cindex sidebar, Recipe for a Programming Language @ifdocbook @docbook -<sidebar><title>Recipe For A Programming Language</title> +<sidebar><title>Recipe for a Programming Language</title> @end docbook @@ -1375,7 +1448,7 @@ more parts C. Document very well and release. @ifnotdocbook @cartouche -@center @b{Recipe For A Programming Language} +@center @b{Recipe for a Programming Language} @@ -1397,7 +1470,7 @@ more parts C. Document very well and release. @cindex Kernighan, Brian @cindex @command{awk}, history of The name @command{awk} comes from the initials of its designers: Alfred V.@: -Aho, Peter J.@: Weinberger and Brian W.@: Kernighan. The original version of +Aho, Peter J.@: Weinberger, and Brian W.@: Kernighan. The original version of @command{awk} was written in 1977 at AT&T Bell Laboratories. In 1985, a new version made the programming language more powerful, introducing user-defined functions, multiple input @@ -1421,23 +1494,23 @@ help from me, thoroughly reworked @command{gawk} for compatibility with the newer @command{awk}. Circa 1994, I became the primary maintainer. Current development focuses on bug fixes, -performance improvements, standards compliance and, occasionally, new features. +performance improvements, standards compliance, and, occasionally, new features. -In May of 1997, J@"urgen Kahrs felt the need for network access +In May 1997, J@"urgen Kahrs felt the need for network access from @command{awk}, and with a little help from me, set about adding features to do this for @command{gawk}. At that time, he also wrote the bulk of -@cite{TCP/IP Internetworking with @command{gawk}} +@cite{@value{GAWKINETTITLE}} (a separate document, available as part of the @command{gawk} distribution). His code finally became part of the main @command{gawk} distribution with @command{gawk} @value{PVERSION} 3.1. John Haque rewrote the @command{gawk} internals, in the process providing an @command{awk}-level debugger. This version became available as -@command{gawk} @value{PVERSION} 4.0, in 2011. +@command{gawk} @value{PVERSION} 4.0 in 2011. -@xref{Contributors}, -for a full list of those who made important contributions to @command{gawk}. +@DBXREF{Contributors} +for a full list of those who have made important contributions to @command{gawk}. @node Names @unnumberedsec A Rose by Any Other Name @@ -1446,11 +1519,11 @@ for a full list of those who made important contributions to @command{gawk}. The @command{awk} language has evolved over the years. Full details are provided in @ref{Language History}. The language described in this @value{DOCUMENT} -is often referred to as ``new @command{awk}''. +is often referred to as ``new @command{awk}.'' By analogy, the original version of @command{awk} is referred to as ``old @command{awk}.'' -Today, on most systems, when you run the @command{awk} utility, +On most current systems, when you run the @command{awk} utility you get some version of new @command{awk}.@footnote{Only Solaris systems still use an old @command{awk} for the default @command{awk} utility. A more modern @command{awk} lives in @@ -1510,7 +1583,9 @@ the POSIX standard for @command{awk}. This @value{DOCUMENT} has the difficult task of being both a tutorial and a reference. If you are a novice, feel free to skip over details that seem too complex. You should also ignore the many cross-references; they are for the -expert user and for the online Info and HTML versions of the @value{DOCUMENT}. +expert user and for the Info and +@uref{http://www.gnu.org/software/gawk/manual/, HTML} +versions of the @value{DOCUMENT}. @end ifnotinfo There are sidebars @@ -1525,12 +1600,15 @@ Most of the time, the examples use complete @command{awk} programs. Some of the more advanced sections show only the part of the @command{awk} program that illustrates the concept being described. -While this @value{DOCUMENT} is aimed principally at people who have not been +Although this @value{DOCUMENT} is aimed principally at people who have not been exposed to @command{awk}, there is a lot of information here that even the @command{awk} expert should find useful. In particular, the description of POSIX @command{awk} and the example programs in -@ref{Library Functions}, and in +@ref{Library Functions}, and +@ifnotdocbook +in +@end ifnotdocbook @ref{Sample Programs}, should be of interest. @@ -1538,22 +1616,30 @@ This @value{DOCUMENT} is split into several parts, as follows: @c FULLXREF ON -Part I describes the @command{awk} language and @command{gawk} program in detail. +@itemize @value{BULLET} +@item +Part I describes the @command{awk} language and the @command{gawk} program in detail. It starts with the basics, and continues through all of the features of @command{awk}. It contains the following chapters: +@c nested +@itemize @value{MINUS} +@item @ref{Getting Started}, provides the essentials you need to know to begin using @command{awk}. +@item @ref{Invoking Gawk}, describes how to run @command{gawk}, the meaning of its command-line options, and how it finds @command{awk} program source files. +@item @ref{Regexp}, introduces regular expressions in general, and in particular the flavors supported by POSIX @command{awk} and @command{gawk}. +@item @ref{Reading Files}, describes how @command{awk} reads your data. It introduces the concepts of records and fields, as well @@ -1561,46 +1647,62 @@ as the @code{getline} command. I/O redirection is first described here. Network I/O is also briefly introduced here. +@item @ref{Printing}, describes how @command{awk} programs can produce output with @code{print} and @code{printf}. +@item @ref{Expressions}, describes expressions, which are the basic building blocks for getting most things done in a program. +@item @ref{Patterns and Actions}, describes how to write patterns for matching records, actions for doing something when a record is matched, and the predefined variables @command{awk} and @command{gawk} use. +@item @ref{Arrays}, -covers @command{awk}'s one-and-only data structure: associative arrays. -Deleting array elements and whole arrays is also described, as well as -sorting arrays in @command{gawk}. It also describes how @command{gawk} -provides arrays of arrays. +covers @command{awk}'s one-and-only data structure: the associative array. +Deleting array elements and whole arrays is described, as well as +sorting arrays in @command{gawk}. The @value{CHAPTER} also describes how +@command{gawk} provides arrays of arrays. +@item @ref{Functions}, describes the built-in functions @command{awk} and @command{gawk} provide, as well as how to define your own functions. It also discusses how @command{gawk} lets you call functions indirectly. +@end itemize +@item Part II shows how to use @command{awk} and @command{gawk} for problem solving. There is lots of code here for you to read and learn from. -It contains the following chapters: +This part contains the following chapters: -@ref{Library Functions}, which provides a number of functions meant to +@c nested +@itemize @value{MINUS} +@item +@ref{Library Functions}, provides a number of functions meant to be used from main @command{awk} programs. +@item @ref{Sample Programs}, -which provides many sample @command{awk} programs. +provides many sample @command{awk} programs. +@end itemize Reading these two chapters allows you to see @command{awk} solving real problems. +@item Part III focuses on features specific to @command{gawk}. It contains the following chapters: +@c nested +@itemize @value{MINUS} +@item @ref{Advanced Features}, describes a number of advanced features. Of particular note @@ -1609,18 +1711,24 @@ have two-way communications with another process, perform TCP/IP networking, and profile your @command{awk} programs. +@item @ref{Internationalization}, describes special features for translating program messages into different languages at runtime. +@item @ref{Debugger}, describes the @command{gawk} debugger. +@item @ref{Arbitrary Precision Arithmetic}, describes advanced arithmetic facilities. +@item @ref{Dynamic Extensions}, describes how to add new variables and functions to @command{gawk} by writing extensions in C or C++. +@end itemize +@item @ifclear FOR_PRINT Part IV provides the appendices, the Glossary, and two licenses that cover the @command{gawk} source code and this @value{DOCUMENT}, respectively. @@ -1631,11 +1739,14 @@ Part IV provides the following appendices, including the GNU General Public License: @end ifset +@itemize @value{MINUS} +@item @ref{Language History}, describes how the @command{awk} language has evolved since -its first release to present. It also describes how @command{gawk} +its first release to the present. It also describes how @command{gawk} has acquired features over time. +@item @ref{Installation}, describes how to get @command{gawk}, how to compile it on POSIX-compatible systems, @@ -1645,15 +1756,44 @@ in @command{gawk} and where to get other freely available @command{awk} implementations. @ifset FOR_PRINT - +@item @ref{Copying}, presents the license that covers the @command{gawk} source code. +@end ifset +@ifclear FOR_PRINT +@item +@ref{Notes}, +describes how to disable @command{gawk}'s extensions, as +well as how to contribute new code to @command{gawk}, +and some possible future directions for @command{gawk} development. + +@item +@ref{Basic Concepts}, +provides some very cursory background material for those who +are completely unfamiliar with computer programming. + +The @ref{Glossary}, defines most, if not all, of the significant terms used +throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, +try looking them up here. + +@item +@ref{Copying}, and +@ref{GNU Free Documentation License}, +present the licenses that cover the @command{gawk} source code +and this @value{DOCUMENT}, respectively. +@end ifclear +@end itemize +@end itemize + +@ifset FOR_PRINT The version of this @value{DOCUMENT} distributed with @command{gawk} contains additional appendices and other end material. To save space, we have omitted them from the printed edition. You may find them online, as follows: +@itemize @value{BULLET} +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Notes.html, The appendix on implementation notes} describes how to disable @command{gawk}'s extensions, how to contribute @@ -1661,45 +1801,29 @@ new code to @command{gawk}, where to find information on some possible future directions for @command{gawk} development, and the design decisions behind the extension API. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Basic-Concepts.html, The appendix on basic concepts} provides some very cursory background material for those who are completely unfamiliar with computer programming. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Glossary.html, The Glossary} -defines most, if not all, the significant terms used +defines most, if not all, of the significant terms used throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, try looking them up here. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/GNU-Free-Documentation-License.html, The GNU FDL} is the license that covers this @value{DOCUMENT}. +@end itemize Some of the chapters have exercise sections; these have also been omitted from the print edition but are available online. @end ifset -@ifclear FOR_PRINT -@ref{Notes}, -describes how to disable @command{gawk}'s extensions, as -well as how to contribute new code to @command{gawk}, -and some possible future directions for @command{gawk} development. - -@ref{Basic Concepts}, -provides some very cursory background material for those who -are completely unfamiliar with computer programming. - -The @ref{Glossary}, defines most, if not all, the significant terms used -throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, -try looking them up here. - -@ref{Copying}, and -@ref{GNU Free Documentation License}, -present the licenses that cover the @command{gawk} source code -and this @value{DOCUMENT}, respectively. -@end ifclear - @c FULLXREF OFF @node Conventions @@ -1730,7 +1854,7 @@ This typically represents the command's standard output. Output from the command, usually its standard output, appears @code{like this}. @end ifset -Error messages, and other output on the command's standard error, are preceded +Error messages and other output on the command's standard error are preceded by the glyph ``@error{}''. For example: @example @@ -1741,15 +1865,23 @@ $ @kbd{echo hello on stderr 1>&2} @end example @ifnotinfo -In the text, command names appear in @code{this font}, while code segments +In the text, almost anything related to programming, such as +command names, +variable and function names, and string, numeric and regexp constants +appear in @code{this font}. Code fragments appear in the same font and quoted, @samp{like this}. +Things that are replaced by the user or programmer +appear in @var{this font}. Options look like this: @option{-f}. +@value{FFN}s are indicated like this: @file{/path/to/ourfile}. +@ifclear FOR_PRINT Some things are emphasized @emph{like this}, and if a point needs to be made -strongly, it is done @strong{like this}. The first occurrence of +strongly, it is done @strong{like this}. +@end ifclear +The first occurrence of a new term is usually its @dfn{definition} and appears in the same font as the previous occurrence of ``definition'' in this sentence. -Finally, @value{FN}s are indicated like this: @file{/path/to/ourfile}. @end ifnotinfo Characters that you type at the keyboard look @kbd{like this}. In particular, @@ -1757,11 +1889,11 @@ there are special characters called ``control characters.'' These are characters that you type by holding down both the @kbd{CONTROL} key and another key, at the same time. For example, a @kbd{Ctrl-d} is typed by first pressing and holding the @kbd{CONTROL} key, next -pressing the @kbd{d} key and finally releasing both keys. +pressing the @kbd{d} key, and finally releasing both keys. For the sake of brevity, throughout this @value{DOCUMENT}, we refer to Brian Kernighan's version of @command{awk} as ``BWK @command{awk}.'' -(@xref{Other Versions}, for information on his and other versions.) +(@DBXREF{Other Versions} for information on his and other versions.) @ifset FOR_PRINT @quotation NOTE @@ -1777,7 +1909,7 @@ Cautionary or warning notes look like this. @unnumberedsubsec Dark Corners @cindex Kernighan, Brian @quotation -@i{Dark corners are basically fractal --- no matter how much +@i{Dark corners are basically fractal---no matter how much you illuminate, there's always a smaller but darker one.} @author Brian Kernighan @end quotation @@ -1793,7 +1925,7 @@ the picture of a flashlight in the margin, as shown here. @value{DARKCORNER} @end iftex @ifnottex -``(d.c.)''. +``(d.c.).'' @end ifnottex @ifclear FOR_PRINT They also appear in the index under the heading ``dark corner.'' @@ -1828,12 +1960,12 @@ Emacs editor. GNU Emacs is the most widely used version of Emacs today. @cindex GPL (General Public License) @cindex General Public License, See GPL @cindex documentation, online -The GNU@footnote{GNU stands for ``GNU's not Unix.''} +The GNU@footnote{GNU stands for ``GNU's Not Unix.''} Project is an ongoing effort on the part of the Free Software Foundation to create a complete, freely distributable, POSIX-compliant computing environment. -The FSF uses the ``GNU General Public License'' (GPL) to ensure that -their software's +The FSF uses the GNU General Public License (GPL) to ensure that +its software's source code is always available to the end user. @ifclear FOR_PRINT A copy of the GPL is included @@ -1847,7 +1979,7 @@ The GPL applies to the C language source code for @command{gawk}. To find out more about the FSF and the GNU Project online, see @uref{http://www.gnu.org, the GNU Project's home page}. This @value{DOCUMENT} may also be read from -@uref{http://www.gnu.org/software/gawk/manual/, their web site}. +@uref{http://www.gnu.org/software/gawk/manual/, GNU's website}. @ifclear FOR_PRINT A shell, an editor (Emacs), highly portable optimizing C, C++, and @@ -1884,16 +2016,16 @@ License in @ref{GNU Free Documentation License}.) @cindex Close, Diane The @value{DOCUMENT} itself has gone through multiple previous editions. Paul Rubin wrote the very first draft of @cite{The GAWK Manual}; -it was around 40 pages in size. +it was around 40 pages long. Diane Close and Richard Stallman improved it, yielding a version that was -around 90 pages long and barely described the original, ``old'' +around 90 pages and barely described the original, ``old'' version of @command{awk}. I started working with that version in the fall of 1988. As work on it progressed, the FSF published several preliminary versions (numbered 0.@var{x}). -In 1996, Edition 1.0 was released with @command{gawk} 3.0.0. +In 1996, edition 1.0 was released with @command{gawk} 3.0.0. The FSF published the first two editions under the title @cite{The GNU Awk User's Guide}. @ifset FOR_PRINT @@ -1905,7 +2037,7 @@ the third edition in 2001. This edition maintains the basic structure of the previous editions. For FSF edition 4.0, the content was thoroughly reviewed and updated. All references to @command{gawk} versions prior to 4.0 were removed. -Of significant note for that edition was @ref{Debugger}. +Of significant note for that edition was the addition of @ref{Debugger}. For FSF edition @ifclear FOR_PRINT @@ -1920,17 +2052,17 @@ and the major new additions are @ref{Arbitrary Precision Arithmetic}, and @ref{Dynamic Extensions}. This @value{DOCUMENT} will undoubtedly continue to evolve. If you -find an error in this @value{DOCUMENT}, please report it! @xref{Bugs}, +find an error in the @value{DOCUMENT}, please report it! @DBXREF{Bugs} for information on submitting problem reports electronically. @ifset FOR_PRINT @c fakenode --- for prepinfo @unnumberedsec How to Stay Current -It may be you have a version of @command{gawk} which is newer than the +You may have a newer version of @command{gawk} than the one described here. To find out what has changed, you should first look at the @file{NEWS} file in the @command{gawk} -distribution, which provides a high level summary of what changed in +distribution, which provides a high-level summary of the changes in each release. You can then look at the @uref{http://www.gnu.org/software/gawk/manual/, @@ -1955,16 +2087,18 @@ contributed code: the archive did not grow and the domain went unused for several years. Late in 2008, a volunteer took on the task of setting up -an @command{awk}-related web site---@uref{http://awk.info}---and did a very +an @command{awk}-related website---@uref{http://awk.info}---and did a very nice job. If you have written an interesting @command{awk} program, or have written a @command{gawk} extension that you would like to share with the rest of the world, please see @uref{http://awk.info/?contribute} for how to -contribute it to the web site. +contribute it to the website. +@ignore As of this writing, this website is in search of a maintainer; please contact me if you are interested. +@end ignore @ignore Other links: @@ -1982,7 +2116,7 @@ The initial draft of @cite{The GAWK Manual} had the following acknowledgments: Many people need to be thanked for their assistance in producing this manual. Jay Fenlason contributed many ideas and sample programs. Richard Mlynarik and Robert Chassell gave helpful comments on drafts of this -manual. The paper @cite{A Supplemental Document for @command{awk}} by John W.@: +manual. The paper @cite{A Supplemental Document for AWK} by John W.@: Pierce of the Chemistry Department at UC San Diego, pinpointed several issues relevant both to @command{awk} implementation and to this manual, that would otherwise have escaped us. @@ -1993,12 +2127,18 @@ I would like to acknowledge Richard M.@: Stallman, for his vision of a better world and for his courage in founding the FSF and starting the GNU Project. +@ifclear FOR_PRINT Earlier editions of this @value{DOCUMENT} had the following acknowledgements: +@end ifclear +@ifset FOR_PRINT +The previous edition of this @value{DOCUMENT} had +the following acknowledgements: +@end ifset @quotation The following people (in alphabetical order) provided helpful comments on various -versions of this book, +versions of this book: Rick Adams, Dr.@: Nelson H.F. Beebe, Karl Berry, @@ -2026,7 +2166,7 @@ Robert J.@: Chassell provided much valuable advice on the use of Texinfo. He also deserves special thanks for convincing me @emph{not} to title this @value{DOCUMENT} -@cite{How To Gawk Politely}. +@cite{How to Gawk Politely}. Karl Berry helped significantly with the @TeX{} part of Texinfo. @cindex Hartholz, Marshall @@ -2106,38 +2246,39 @@ portable program it is today. It has been and continues to be a pleasure working with this team of fine people. Notable code and documentation contributions were made by -a number of people. @xref{Contributors}, for the full list. +a number of people. @DBXREF{Contributors} for the full list. @ifset FOR_PRINT @cindex Oram, Andy -Thanks to Andy Oram, of O'Reilly Media, for initiating +Thanks to Andy Oram of O'Reilly Media for initiating the fourth edition and for his support during the work. +Thanks to Jasmine Kwityn for her copyediting work. @end ifset -Thanks to Michael Brennan for the Foreword. +Thanks to Michael Brennan for the Forewords. @cindex Duman, Patrice @cindex Berry, Karl Thanks to Patrice Dumas for the new @command{makeinfo} program. -Thanks to Karl Berry who continues to work to keep +Thanks to Karl Berry, who continues to work to keep the Texinfo markup language sane. @cindex Kernighan, Brian @cindex Brennan, Michael @cindex Day, Robert P.J.@: -Robert P.J.@: Day, Michael Brennan and Brian Kernighan kindly acted as +Robert P.J.@: Day, Michael Brennan, and Brian Kernighan kindly acted as reviewers for the 2015 edition of this @value{DOCUMENT}. Their feedback helped improve the final work. -I would like to thank Brian Kernighan for invaluable assistance during the -testing and debugging of @command{gawk}, and for ongoing +I would also like to thank Brian Kernighan for his invaluable assistance during the +testing and debugging of @command{gawk}, and for his ongoing help and advice in clarifying numerous points about the language. We could not have done nearly as good a job on either @command{gawk} or its documentation without his help. Brian is in a class by himself as a programmer and technical author. I have to thank him (yet again) for his ongoing friendship -and the role model he has been for me for close to 30 years! +and for being a role model to me for close to 30 years! Having him as a reviewer is an exciting privilege. It has also been extremely humbling@enddots{} @@ -2153,14 +2294,14 @@ which they raised and educated me. Finally, I also must acknowledge my gratitude to G-d, for the many opportunities He has sent my way, as well as for the gifts He has given me with which to take advantage of those opportunities. -@iftex +@ifnotdocbook @sp 2 @noindent Arnold Robbins @* Nof Ayalon @* -ISRAEL @* -December, 2014 -@end iftex +Israel @* +February 2015 +@end ifnotdocbook @ifnotinfo @part @value{PART1}The @command{awk} Language @@ -2176,31 +2317,31 @@ following chapters: @itemize @value{BULLET} @item -@ref{Getting Started}. +@ref{Getting Started} @item -@ref{Invoking Gawk}. +@ref{Invoking Gawk} @item -@ref{Regexp}. +@ref{Regexp} @item -@ref{Reading Files}. +@ref{Reading Files} @item -@ref{Printing}. +@ref{Printing} @item -@ref{Expressions}. +@ref{Expressions} @item -@ref{Patterns and Actions}. +@ref{Patterns and Actions} @item -@ref{Arrays}. +@ref{Arrays} @item -@ref{Functions}. +@ref{Functions} @end itemize @end ifdocbook @@ -2215,17 +2356,17 @@ following chapters: The basic function of @command{awk} is to search files for lines (or other units of text) that contain certain patterns. When a line matches one of the patterns, @command{awk} performs specified actions on that line. -@command{awk} keeps processing input lines in this way until it reaches +@command{awk} continues to process input lines in this way until it reaches the end of the input files. @cindex @command{awk}, uses for @cindex programming languages@comma{} data-driven vs.@: procedural @cindex @command{awk} programs Programs in @command{awk} are different from programs in most other languages, -because @command{awk} programs are @dfn{data-driven}; that is, you describe -the data you want to work with and then what to do when you find it. +because @command{awk} programs are @dfn{data driven} (i.e., you describe +the data you want to work with and then what to do when you find it). Most other languages are @dfn{procedural}; you have to describe, in great -detail, every step the program is to take. When working with procedural +detail, every step the program should take. When working with procedural languages, it is usually much harder to clearly describe the data your program will process. For this reason, @command{awk} programs are often refreshingly easy to @@ -2235,15 +2376,15 @@ read and write. @cindex rule, definition of When you run @command{awk}, you specify an @command{awk} @dfn{program} that tells @command{awk} what to do. The program consists of a series of -@dfn{rules}. (It may also contain @dfn{function definitions}, -an advanced feature that we will ignore for now. -@xref{User-defined}.) Each rule specifies one +@dfn{rules} (it may also contain @dfn{function definitions}, +an advanced feature that we will ignore for now; +@pxref{User-defined}). Each rule specifies one pattern to search for and one action to perform upon finding the pattern. -Syntactically, a rule consists of a pattern followed by an action. The -action is enclosed in braces to separate it from the pattern. -Newlines usually separate rules. Therefore, an @command{awk} +Syntactically, a rule consists of a @dfn{pattern} followed by an +@dfn{action}. The action is enclosed in braces to separate it from the +pattern. Newlines usually separate rules. Therefore, an @command{awk} program looks like this: @example @@ -2317,8 +2458,8 @@ awk '@var{program}' @var{input-file1} @var{input-file2} @dots{} @end example @noindent -where @var{program} consists of a series of @var{patterns} and -@var{actions}, as described earlier. +where @var{program} consists of a series of patterns and +actions, as described earlier. @cindex single quote (@code{'}) @cindex @code{'} (single quote) @@ -2337,11 +2478,12 @@ programs from shell scripts, because it avoids the need for a separate file for the @command{awk} program. A self-contained shell script is more reliable because there are no other files to misplace. +Later in this chapter, in +@ifdocbook +the section +@end ifdocbook @ref{Very Simple}, -@ifnotinfo -later in this @value{CHAPTER}, -@end ifnotinfo -presents several short, +we'll see examples of several short, self-contained programs. @node Read Terminal @@ -2362,10 +2504,10 @@ awk '@var{program}' which usually means whatever you type on the keyboard. This continues until you indicate end-of-file by typing @kbd{Ctrl-d}. @ifset FOR_PRINT -(On other operating systems, the end-of-file character may be different.) +(On non-POSIX operating systems, the end-of-file character may be different.) @end ifset @ifclear FOR_PRINT -(On other operating systems, the end-of-file character may be different. +(On non-POSIX operating systems, the end-of-file character may be different. For example, on OS/2, it is @kbd{Ctrl-z}.) @end ifclear @@ -2399,7 +2541,7 @@ startup file. This next simple @command{awk} program emulates the @command{cat} utility; it copies whatever you type on the -keyboard to its standard output (why this works is explained shortly). +keyboard to its standard output (why this works is explained shortly): @example $ @kbd{awk '@{ print @}'} @@ -2465,11 +2607,9 @@ for programs that are provided on the @command{awk} command line. (Also, placing the program in a file allows us to use a literal single quote in the program text, instead of the magic @samp{\47}.) -@c STARTOFRANGE sq1x @cindex single quote (@code{'}) in @command{gawk} command lines -@c STARTOFRANGE qs2x @cindex @code{'} (single quote) in @command{gawk} command lines -If you want to clearly identify your @command{awk} program files as such, +If you want to clearly identify an @command{awk} program file as such, you can add the extension @file{.awk} to the @value{FN}. This doesn't affect the execution of the @command{awk} program but it does make ``housekeeping'' easier. @@ -2484,7 +2624,7 @@ affect the execution of the @command{awk} program but it does make Once you have learned @command{awk}, you may want to write self-contained @command{awk} scripts, using the @samp{#!} script mechanism. You can do this on many systems.@footnote{The @samp{#!} mechanism works on -GNU/Linux systems, BSD-based systems and commercial Unix systems.} +GNU/Linux systems, BSD-based systems, and commercial Unix systems.} For example, you could update the file @file{advice} to look like this: @example @@ -2528,7 +2668,7 @@ according to the instructions in your program. (This is different from a @dfn{compiled} language such as C, where your program is first compiled into machine code that is executed directly by your system's processor.) The @command{awk} utility is thus termed an @dfn{interpreter}. -Many modern languages are interperted. +Many modern languages are interpreted. The line beginning with @samp{#!} lists the full @value{FN} of an interpreter to run and a single optional initial command-line argument @@ -2578,7 +2718,7 @@ according to the instructions in your program. (This is different from a @dfn{compiled} language such as C, where your program is first compiled into machine code that is executed directly by your system's processor.) The @command{awk} utility is thus termed an @dfn{interpreter}. -Many modern languages are interperted. +Many modern languages are interpreted. The line beginning with @samp{#!} lists the full @value{FN} of an interpreter to run and a single optional initial command-line argument @@ -2625,14 +2765,14 @@ can explain what the program does and how it works. Nearly all programming languages have provisions for comments, as programs are typically hard to understand without them. -In the @command{awk} language, a comment starts with the sharp sign +In the @command{awk} language, a comment starts with the number sign character (@samp{#}) and continues to the end of the line. The @samp{#} does not have to be the first character on the line. The -@command{awk} language ignores the rest of a line following a sharp sign. +@command{awk} language ignores the rest of a line following a number sign. For example, we could have put the following into @file{advice}: @example -# This program prints a nice friendly message. It helps +# This program prints a nice, friendly message. It helps # keep novice users from being afraid of the computer. BEGIN @{ print "Don't Panic!" @} @end example @@ -2648,7 +2788,8 @@ when reading it at a later time. @quotation CAUTION As mentioned in @ref{One-shot}, -you can enclose small to medium programs in single quotes, in order to keep +you can enclose short to medium-sized programs in single quotes, +in order to keep your shell scripts self-contained. When doing so, @emph{don't} put an apostrophe (i.e., a single quote) into a comment (or anywhere else in your program). The shell interprets the quote as the closing @@ -2677,19 +2818,19 @@ $ @kbd{awk '@{ print "hello" @} # let's be cute'} @cindex @code{\} (backslash) @cindex backslash (@code{\}) Putting a backslash before the single quote in @samp{let's} wouldn't help, -since backslashes are not special inside single quotes. +because backslashes are not special inside single quotes. The next @value{SUBSECTION} describes the shell's quoting rules. @end quotation @node Quoting -@subsection Shell-Quoting Issues +@subsection Shell Quoting Issues @cindex shell quoting, rules for @menu * DOS Quoting:: Quoting in Windows Batch Files. @end menu -For short to medium length @command{awk} programs, it is most convenient +For short to medium-length @command{awk} programs, it is most convenient to enter the program on the @command{awk} command line. This is best done by enclosing the entire program in single quotes. This is true whether you are entering the program interactively at @@ -2713,8 +2854,8 @@ or empty, string. The null string is character data that has no value. In other words, it is empty. It is written in @command{awk} programs like this: @code{""}. In the shell, it can be written using single -or double quotes: @code{""} or @code{''}. While the null string has -no characters in it, it does exist. Consider this command: +or double quotes: @code{""} or @code{''}. Although the null string has +no characters in it, it does exist. For example, consider this command: @example $ @kbd{echo ""} @@ -2724,8 +2865,7 @@ $ @kbd{echo ""} Here, the @command{echo} utility receives a single argument, even though that argument has no characters in it. In the rest of this @value{DOCUMENT}, we use the terms @dfn{null string} and @dfn{empty string} -interchangeably. Now, on to the quoting rules. - +interchangeably. Now, on to the quoting rules: @itemize @value{BULLET} @item @@ -2748,7 +2888,7 @@ The shell does no interpretation of the quoted text, passing it on verbatim to the command. It is @emph{impossible} to embed a single quote inside single-quoted text. Refer back to -@ref{Comments}, +@DBREF{Comments} for an example of what happens if you try. @item @@ -2758,7 +2898,7 @@ Double quotes protect most things between the opening and closing quotes. The shell does at least variable and command substitution on the quoted text. Different shells may do additional kinds of processing on double-quoted text. -Since certain characters within double-quoted text are processed by the shell, +Because certain characters within double-quoted text are processed by the shell, they must be @dfn{escaped} within the text. Of note are the characters @samp{$}, @samp{`}, @samp{\}, and @samp{"}, all of which must be preceded by a backslash within double-quoted text if they are to be passed on literally @@ -2820,7 +2960,7 @@ $ @kbd{awk 'BEGIN @{ print "Here is a single quote <'"'"'>" @}'} @noindent This program consists of three concatenated quoted strings. The first and the -third are single-quoted, the second is double-quoted. +third are single-quoted, and the second is double-quoted. This can be ``simplified'' to: @@ -2841,8 +2981,6 @@ $ @kbd{awk "BEGIN @{ print \"Here is a single quote <'>\" @}"} @end example @noindent -@c ENDOFRANGE sq1x -@c ENDOFRANGE qs2x This option is also painful, because double quotes, backslashes, and dollar signs are very common in more advanced @command{awk} programs. @@ -2859,22 +2997,22 @@ $ @kbd{awk 'BEGIN @{ print "Here is a double quote <\42>" @}'} @end example @noindent -This works nicely, except that you should comment clearly what the +This works nicely, but you should comment clearly what the escapes mean. A fourth option is to use command-line variable assignment, like this: @example -$ awk -v sq="'" 'BEGIN @{ print "Here is a single quote <" sq ">" @}' +$ @kbd{awk -v sq="'" 'BEGIN @{ print "Here is a single quote <" sq ">" @}'} @print{} Here is a single quote <'> @end example (Here, the two string constants and the value of @code{sq} are concatenated -into a single string which is printed by @code{print}.) +into a single string that is printed by @code{print}.) If you really need both single and double quotes in your @command{awk} program, it is probably best to move it into a separate file, where -the shell won't be part of the picture, and you can say what you mean. +the shell won't be part of the picture and you can say what you mean. @node DOS Quoting @subsubsection Quoting in MS-Windows Batch Files @@ -2936,7 +3074,7 @@ information about monthly shipments. In both files, each line is considered to be one @dfn{record}. In @file{mail-list}, each record contains the name of a person, -his/her phone number, his/her email-address, and a code for their relationship +his/her phone number, his/her email address, and a code for his/her relationship with the author of the list. The columns are aligned using spaces. An @samp{A} in the last column @@ -2973,7 +3111,7 @@ of green crates shipped, the number of red boxes shipped, the number of orange bags shipped, and the number of blue packages shipped, respectively. There are 16 entries, covering the 12 months of last year and the first four months of the current year. -An empty line separates the data for the two years. +An empty line separates the data for the two years: @example @c file eg/data/inventory-shipped @@ -3007,7 +3145,7 @@ The following command runs a simple @command{awk} program that searches the input file @file{mail-list} for the character string @samp{li} (a grouping of characters is usually called a @dfn{string}; the term @dfn{string} is based on similar usage in English, such -as ``a string of pearls,'' or ``a string of cars in a train''): +as ``a string of pearls'' or ``a string of cars in a train''): @example awk '/li/ @{ print $0 @}' mail-list @@ -3054,11 +3192,11 @@ omitting the @code{print} statement but retaining the braces makes an empty action that does nothing (i.e., no lines are printed). @cindex @command{awk} programs, one-line examples -Many practical @command{awk} programs are just a line or two. Following is a +Many practical @command{awk} programs are just a line or two long. Following is a collection of useful, short programs to get you started. Some of these programs contain constructs that haven't been covered yet. (The description -of the program will give you a good idea of what is going on, but please -read the rest of the @value{DOCUMENT} to become an @command{awk} expert!) +of the program will give you a good idea of what is going on, but you'll +need to read the rest of the @value{DOCUMENT} to become an @command{awk} expert!) Most of the examples use a @value{DF} named @file{data}. This is just a placeholder; if you use these programs yourself, substitute your own @value{FN}s for @file{data}. @@ -3075,7 +3213,7 @@ Print every line that is longer than 80 characters: awk 'length($0) > 80' data @end example -The sole rule has a relational expression as its pattern and it has no +The sole rule has a relational expression as its pattern and has no action---so it uses the default action, printing the record. @item @@ -3099,7 +3237,7 @@ expand data | awk '@{ if (x < length($0)) x = length($0) @} @end example This example differs slightly from the previous one: -The input is processed by the @command{expand} utility to change TABs +the input is processed by the @command{expand} utility to change TABs into spaces, so the widths compared are actually the right-margin columns, as opposed to the number of input characters on each line. @@ -3162,7 +3300,7 @@ Print the even-numbered lines in the @value{DF}: awk 'NR % 2 == 0' data @end example -If you use the expression @samp{NR % 2 == 1} instead, +If you used the expression @samp{NR % 2 == 1} instead, the program would print the odd-numbered lines. @end itemize @@ -3178,8 +3316,13 @@ no actions run. After processing all the rules that match the line (and perhaps there are none), @command{awk} reads the next line. (However, -@pxref{Next Statement}, +@DBPXREF{Next Statement} +@ifdocbook +and @DBREF{Nextfile Statement}.) +@end ifdocbook +@ifnotdocbook and also @pxref{Nextfile Statement}.) +@end ifnotdocbook This continues until the program reaches the end of the file. For example, the following @command{awk} program contains two rules: @@ -3353,7 +3496,7 @@ lines in the middle of a regular expression or a string. with the C shell.} It works for @command{awk} programs in files and for one-shot programs, @emph{provided} you are using a POSIX-compliant shell, such as the Unix Bourne shell or Bash. But the C shell behaves -differently! There, you must use two backslashes in a row, followed by +differently! There you must use two backslashes in a row, followed by a newline. Note also that when using the C shell, @emph{every} newline in your @command{awk} program must be escaped with a backslash. To illustrate: @@ -3394,9 +3537,9 @@ starts a comment, it ignores @emph{everything} on the rest of the line. For example: @example -$ gawk 'BEGIN @{ print "dont panic" # a friendly \ -> BEGIN rule -> @}' +$ @kbd{gawk 'BEGIN @{ print "dont panic" # a friendly \} +> @kbd{ BEGIN rule} +> @kbd{@}'} @error{} gawk: cmd. line:2: BEGIN rule @error{} gawk: cmd. line:2: ^ syntax error @end example @@ -3444,9 +3587,9 @@ performing bit manipulation, for runtime string translation (internationalizatio determining the type of a variable, and array sorting. -As we develop our presentation of the @command{awk} language, we introduce +As we develop our presentation of the @command{awk} language, we will introduce most of the variables and many of the functions. They are described -systematically in @ref{Built-in Variables}, and in +systematically in @DBREF{Built-in Variables} and in @ref{Built-in}. @node When @@ -3480,8 +3623,8 @@ eight-bit microprocessors, @end ifset and a microcode assembler for a special-purpose Prolog computer. -While the original @command{awk}'s capabilities were strained by tasks -of such complexity, modern versions are more capable. +The original @command{awk}'s capabilities were strained by tasks +of such complexity, but modern versions are more capable. @cindex @command{awk} programs, complex If you find yourself writing @command{awk} scripts of more than, say, @@ -3498,7 +3641,7 @@ and Perl.} @c FIXME: Review this chapter for summary of builtin functions called. @itemize @value{BULLET} @item -Programs in @command{awk} consist of @var{pattern}-@var{action} pairs. +Programs in @command{awk} consist of @var{pattern}--@var{action} pairs. @item An @var{action} without a @var{pattern} always runs. The default @@ -3527,7 +3670,7 @@ part of a larger shell script (or MS-Windows batch file). You may use backslash continuation to continue a source line. Lines are automatically continued after a comma, open brace, question mark, colon, -@samp{||}, @samp{&&}, @code{do} and @code{else}. +@samp{||}, @samp{&&}, @code{do}, and @code{else}. @end itemize @node Invoking Gawk @@ -3536,7 +3679,7 @@ a comma, open brace, question mark, colon, This @value{CHAPTER} covers how to run @command{awk}, both POSIX-standard and @command{gawk}-specific command-line options, and what @command{awk} and -@command{gawk} do with non-option arguments. +@command{gawk} do with nonoption arguments. It then proceeds to cover how @command{gawk} searches for source files, reading standard input along with other files, @command{gawk}'s environment variables, @command{gawk}'s exit status, using include files, @@ -3580,7 +3723,7 @@ enclosed in [@dots{}] in these templates are optional: @cindex GNU long options @cindex long options @cindex options, long -Besides traditional one-letter POSIX-style options, @command{gawk} also +In addition to traditional one-letter POSIX-style options, @command{gawk} also supports GNU long options. @cindex dark corner, invoking @command{awk} @@ -3602,20 +3745,16 @@ warning that the program is empty. @node Options @section Command-Line Options -@c STARTOFRANGE ocl @cindex options, command-line -@c STARTOFRANGE clo @cindex command line, options -@c STARTOFRANGE gnulo @cindex GNU long options -@c STARTOFRANGE longo @cindex options, long Options begin with a dash and consist of a single character. GNU-style long options consist of two dashes and a keyword. The keyword can be abbreviated, as long as the abbreviation allows the option -to be uniquely identified. If the option takes an argument, then the -keyword is either immediately followed by an equals sign (@samp{=}) and the +to be uniquely identified. If the option takes an argument, either the +keyword is immediately followed by an equals sign (@samp{=}) and the argument's value, or the keyword and the argument's value are separated by whitespace. If a particular option with a value is given more than once, it is the @@ -3642,8 +3781,8 @@ Set the @code{FS} variable to @var{fs} @cindex @option{-f} option @cindex @option{--file} option @cindex @command{awk} programs, location of -Read @command{awk} program source from @var{source-file} -instead of in the first non-option argument. +Read the @command{awk} program source from @var{source-file} +instead of in the first nonoption argument. This option may be given multiple times; the @command{awk} program consists of the concatenation of the contents of each specified @var{source-file}. @@ -3697,8 +3836,6 @@ by the user that could start with @samp{-}. It is also useful for passing options on to the @command{awk} program; see @ref{Getopt Function}. @end table -@c ENDOFRANGE gnulo -@c ENDOFRANGE longo The following list describes @command{gawk}-specific options: @@ -3710,14 +3847,14 @@ The following list describes @command{gawk}-specific options: @cindex @option{--characters-as-bytes} option Cause @command{gawk} to treat all input data as single-byte characters. In addition, all output written with @code{print} or @code{printf} -are treated as single-byte characters. +is treated as single-byte characters. Normally, @command{gawk} follows the POSIX standard and attempts to process its input data according to the current locale (@pxref{Locales}). This can often involve converting multibyte characters into wide characters (internally), and can lead to problems or confusion if the input data does not contain valid -multibyte characters. This option is an easy way to tell @command{gawk}: -``hands off my data!''. +multibyte characters. This option is an easy way to tell @command{gawk}, +``Hands off my data!'' @item @option{-c} @itemx @option{--traditional} @@ -3771,9 +3908,10 @@ names like @code{i}, @code{j}, etc.) @cindex @command{awk} debugging, enabling Enable debugging of @command{awk} programs (@pxref{Debugging}). -By default, the debugger reads commands interactively from the keyboard. +By default, the debugger reads commands interactively from the keyboard +(standard input). The optional @var{file} argument allows you to specify a file with a list -of commands for the debugger to execute non-interactively. +of commands for the debugger to execute noninteractively. No space is allowed between the @option{-D} and @var{file}, if @var{file} is supplied. @@ -3811,7 +3949,13 @@ Command-line variable assignments of the form This option is particularly necessary for World Wide Web CGI applications that pass arguments through the URL; using this option prevents a malicious (or other) user from passing in options, assignments, or @command{awk} source -code (via @option{-e}) to the CGI application. This option should be used +code (via @option{-e}) to the CGI application.@footnote{For more detail, +please see Section 4.4 of @uref{http://www.ietf.org/rfc/rfc3875, +RFC 3875}. Also see the +@uref{http://lists.gnu.org/archive/html/bug-gawk/2014-11/msg00022.html, +explanatory note sent to the @command{gawk} bug +mailing list}.} +This option should be used with @samp{#!} scripts (@pxref{Executable Scripts}), like so: @example @@ -3827,7 +3971,7 @@ with @samp{#!} scripts (@pxref{Executable Scripts}), like so: @cindex portable object files, generating @cindex files, portable object, generating Analyze the source program and -generate a GNU @command{gettext} Portable Object Template file on standard +generate a GNU @command{gettext} portable object template file on standard output for all string constants that have been marked for translation. @xref{Internationalization}, for information about this option. @@ -3839,7 +3983,7 @@ for information about this option. @cindex GNU long options, printing list of @cindex options, printing list of @cindex printing, list of options -Print a ``usage'' message summarizing the short and long style options +Print a ``usage'' message summarizing the short- and long-style options that @command{gawk} accepts and then exit. @item @option{-i} @var{source-file} @@ -3849,7 +3993,7 @@ that @command{gawk} accepts and then exit. @cindex @command{awk} programs, location of Read an @command{awk} source library from @var{source-file}. This option is completely equivalent to using the @code{@@include} directive inside -your program. This option is very similar to the @option{-f} option, +your program. It is very similar to the @option{-f} option, but there are two important differences. First, when @option{-i} is used, the program source is not loaded if it has been previously loaded, whereas with @option{-f}, @command{gawk} always loads the file. @@ -3902,7 +4046,7 @@ care to search for all occurrences of each inappropriate construct. As @itemx @option{--bignum} @cindex @option{-M} option @cindex @option{--bignum} option -Force arbitrary precision arithmetic on numbers. This option has no effect +Force arbitrary-precision arithmetic on numbers. This option has no effect if @command{gawk} is not compiled to use the GNU MPFR and MP libraries (@pxref{Arbitrary Precision Arithmetic}). @@ -3918,10 +4062,8 @@ values in input data (@pxref{Nondecimal Data}). @quotation CAUTION -This option can severely break old programs. -Use with care. - -This option may disappear in a future version of @command{gawk}. +This option can severely break old programs. Use with care. Also note +that this option may disappear in a future version of @command{gawk}. @end quotation @item @option{-N} @@ -3936,7 +4078,7 @@ when parsing numeric input data (@pxref{Locales}). @cindex @option{-o} option @cindex @option{--pretty-print} option Enable pretty-printing of @command{awk} programs. -By default, output program is created in a file named @file{awkprof.out} +By default, the output program is created in a file named @file{awkprof.out} (@pxref{Profiling}). The optional @var{file} argument allows you to specify a different @value{FN} for the output. @@ -3953,7 +4095,7 @@ This is no longer the case. @cindex @option{--optimize} option @cindex @option{-O} option Enable some optimizations on the internal representation of the program. -At the moment this includes just simple constant folding. +At the moment, this includes just simple constant folding. @item @option{-p}[@var{file}] @itemx @option{--profile}[@code{=}@var{file}] @@ -3980,7 +4122,7 @@ in the left margin, and function call counts for each function. Operate in strict POSIX mode. This disables all @command{gawk} extensions (just like @option{--traditional}) and disables all extensions not allowed by POSIX. -@xref{Common Extensions}, for a summary of the extensions +@DBXREF{Common Extensions} for a summary of the extensions in @command{gawk} that are disabled by this option. Also, the following additional @@ -4030,8 +4172,8 @@ Allow interval expressions (@pxref{Regexp Operators}) in regexps. This is now @command{gawk}'s default behavior. -Nevertheless, this option remains both for backward compatibility, -and for use in combination with @option{--traditional}. +Nevertheless, this option remains (both for backward compatibility +and for use in combination with @option{--traditional}). @item @option{-S} @itemx @option{--sandbox} @@ -4084,7 +4226,7 @@ If it is, @command{awk} reads its program source from all of the named files, as if they had been concatenated together into one big file. This is useful for creating libraries of @command{awk} functions. These functions can be written once and then retrieved from a standard place, instead -of having to be included into each individual program. +of having to be included in each individual program. The @option{-i} option is similar in this regard. (As mentioned in @ref{Definition Syntax}, @@ -4095,20 +4237,20 @@ if the program is entered at the keyboard, by specifying @samp{-f /dev/tty}. After typing your program, type @kbd{Ctrl-d} (the end-of-file character) to terminate it. (You may also use @samp{-f -} to read program source from the standard -input but then you will not be able to also use the standard input as a +input, but then you will not be able to also use the standard input as a source of data.) Because it is clumsy using the standard @command{awk} mechanisms to mix source file and command-line @command{awk} programs, @command{gawk} provides the @option{-e} option. This does not require you to -pre-empt the standard input for your source code; it allows you to easily +preempt the standard input for your source code; it allows you to easily mix command-line and library source code (@pxref{AWKPATH Variable}). As with @option{-f}, the @option{-e} and @option{-i} options may also be used multiple times on the command line. @cindex @option{-e} option If no @option{-f} or @option{-e} option is specified, then @command{gawk} -uses the first non-option command-line argument as the text of the +uses the first nonoption command-line argument as the text of the program source code. @cindex @env{POSIXLY_CORRECT} environment variable @@ -4147,8 +4289,6 @@ setenv POSIXLY_CORRECT true Having @env{POSIXLY_CORRECT} set is not recommended for daily use, but it is good for testing the portability of your programs to other environments. -@c ENDOFRANGE ocl -@c ENDOFRANGE clo @node Other Arguments @section Other Command-Line Arguments @@ -4175,7 +4315,7 @@ All the command-line arguments are made available to your @command{awk} program and the program text (if present) are omitted from @code{ARGV}. All other arguments, including variable assignments, are included. As each element of @code{ARGV} is processed, @command{gawk} -sets the variable @code{ARGIND} to the index in @code{ARGV} of the +sets @code{ARGIND} to the index in @code{ARGV} of the current element. @c FIXME: One day, move the ARGC and ARGV node closer to here. @@ -4280,30 +4420,26 @@ behaves. @cindex @env{AWKPATH} environment variable @cindex directories, searching for source files @cindex search paths, for source files -@cindex differences in @command{awk} and @command{gawk}, @code{AWKPATH} environment variable +@cindex differences in @command{awk} and @command{gawk}, @env{AWKPATH} environment variable @ifinfo The previous @value{SECTION} described how @command{awk} program files can be named on the command line with the @option{-f} option. @end ifinfo In most @command{awk} -implementations, you must supply a precise path name for each program +implementations, you must supply a precise pathname for each program file, unless the file is in the current directory. -But in @command{gawk}, if the @value{FN} supplied to the @option{-f} +But with @command{gawk}, if the @value{FN} supplied to the @option{-f} or @option{-i} options does not contain a directory separator @samp{/}, then @command{gawk} searches a list of -directories (called the @dfn{search path}), one by one, looking for a +directories (called the @dfn{search path}) one by one, looking for a file with the specified name. The search path is a string consisting of directory names -separated by colons@footnote{Semicolons on MS-Windows and MS-DOS.}. @command{gawk} gets its search path from the +separated by colons.@footnote{Semicolons on MS-Windows and MS-DOS.} +@command{gawk} gets its search path from the @env{AWKPATH} environment variable. If that variable does not exist, -@command{gawk} uses a default path, -@samp{.:/usr/local/share/awk}.@footnote{Your version of @command{gawk} -may use a different directory; it -will depend upon how @command{gawk} was built and installed. The actual -directory is the value of @code{$(datadir)} generated when -@command{gawk} was configured. You probably don't need to worry about this, -though.} +or if it has an empty value, +@command{gawk} uses a default path (described shortly). The search path feature is particularly helpful for building libraries of useful @command{awk} functions. The library files can be placed in a @@ -4311,7 +4447,7 @@ standard directory in the default path and then specified on the command line with a short @value{FN}. Otherwise, you would have to type the full @value{FN} for each file. -By using the @option{-i} option, or the @option{-e} and @option{-f} options, your command-line +By using the @option{-i} or @option{-f} options, your command-line @command{awk} programs can use facilities in @command{awk} library files (@pxref{Library Functions}). Path searching is not done if @command{gawk} is in compatibility mode. @@ -4319,7 +4455,7 @@ This is true for both @option{--traditional} and @option{--posix}. @xref{Options}. If the source code file is not found after the initial search, the path is searched -again after adding the default @samp{.awk} suffix to the @value{FN}. +again after adding the suffix @samp{.awk} to the @value{FN}. @command{gawk}'s path search mechanism is similar to the shell's. @@ -4331,21 +4467,35 @@ directory. colon or by placing two colons next to each other [@samp{::}].) @quotation NOTE -@command{gawk} always looks in the current directory @emph{before} -searching @env{AWKPATH}. Thus, while you can include the current directory -in the search path, either explicitly or with a null entry, there is no -real reason to do so. -@c Prior to 4.0, gawk searched the current directory after the -@c path search, but it's not worth documenting it. +To include the current directory in the path, either place @file{.} +as an entry in the path or write a null entry in the path. + +Different past versions of @command{gawk} would also look explicitly in +the current directory, either before or after the path search. As of +@value{PVERSION} 4.1.2, this no longer happens; if you wish to look +in the current directory, you must include @file{.} either as a separate +entry or as a null entry in the search path. @end quotation -If @env{AWKPATH} is not defined in the -environment, @command{gawk} places its default search path into -@code{ENVIRON["AWKPATH"]}. This makes it easy to determine -the actual search path that @command{gawk} used -from within an @command{awk} program. +The default value for @env{AWKPATH} is +@samp{.:/usr/local/share/awk}.@footnote{Your version of @command{gawk} +may use a different directory; it +will depend upon how @command{gawk} was built and installed. The actual +directory is the value of @code{$(datadir)} generated when +@command{gawk} was configured. You probably don't need to worry about this, +though.} Since @file{.} is included at the beginning, @command{gawk} +searches first in the current directory and then in @file{/usr/local/share/awk}. +In practice, this means that you will rarely need to change the +value of @env{AWKPATH}. + +@xref{Shell Startup Files}, for information on functions that help to +manipulate the @env{AWKPATH} variable. + +@command{gawk} places the value of the search path that it used into +@code{ENVIRON["AWKPATH"]}. This provides access to the actual search +path value from within an @command{awk} program. -While you can change @code{ENVIRON["AWKPATH"]} within your @command{awk} +Although you can change @code{ENVIRON["AWKPATH"]} within your @command{awk} program, this has no effect on the running program's behavior. This makes sense: the @env{AWKPATH} environment variable is used to find the program source files. Once your program is running, all the files have been @@ -4367,12 +4517,24 @@ the platform. For example, on GNU/Linux systems, the suffix @samp{.so} is used. The search path specified is also used for extensions loaded via the @code{@@load} keyword (@pxref{Loading Shared Libraries}). +If @env{AWKLIBPATH} does not exist in the environment, or if it has +an empty value, @command{gawk} uses a default path; this +is typically @samp{/usr/local/lib/gawk}, although it can vary depending +upon how @command{gawk} was built. + +@xref{Shell Startup Files}, for information on functions that help to +manipulate the @env{AWKLIBPATH} variable. + +@command{gawk} places the value of the search path that it used into +@code{ENVIRON["AWKLIBPATH"]}. This provides access to the actual search +path value from within an @command{awk} program. + @node Other Environment Variables @subsection Other Environment Variables A number of other environment variables affect @command{gawk}'s behavior, but they are more specialized. Those in the following -list are meant to be used by regular users. +list are meant to be used by regular users: @table @env @item GAWK_MSEC_SLEEP @@ -4390,9 +4552,11 @@ wait for input before returning with an error. Controls the number of times @command{gawk} attempts to retry a two-way TCP/IP (socket) connection before giving up. @xref{TCP/IP Networking}. +Note that when nonfatal I/O is enabled (@pxref{Nonfatal}), +@command{gawk} only tries to open a TCP/IP socket once. @item POSIXLY_CORRECT -Causes @command{gawk} to switch to POSIX compatibility +Causes @command{gawk} to switch to POSIX-compatibility mode, disabling all traditional and GNU extensions. @xref{Options}. @end table @@ -4424,11 +4588,11 @@ for debugging problems on filesystems on non-POSIX operating systems where I/O is performed in records, not in blocks. @item GAWK_MSG_SRC -If this variable exists, @command{gawk} includes the file -name and line number within the @command{gawk} source code +If this variable exists, @command{gawk} includes the @value{FN} +and line number within the @command{gawk} source code from which warning and/or fatal messages are generated. Its purpose is to help isolate the source of a -message, since there are multiple places which produce the +message, as there are multiple places that produce the same warning or error message. @item GAWK_NO_DFA @@ -4444,16 +4608,16 @@ This specifies the amount by which @command{gawk} should grow its internal evaluation stack, when needed. @item INT_CHAIN_MAX -The intended maximum number of items @command{gawk} will maintain on a +This specifies intended maximum number of items @command{gawk} will maintain on a hash chain for managing arrays indexed by integers. @item STR_CHAIN_MAX -The intended maximum number of items @command{gawk} will maintain on a +This specifies intended maximum number of items @command{gawk} will maintain on a hash chain for managing arrays indexed by strings. @item TIDYMEM If this variable exists, @command{gawk} uses the @code{mtrace()} library -calls from GNU LIBC to help track down possible memory leaks. +calls from the GNU C library to help track down possible memory leaks. @end table @node Exit Status @@ -4472,11 +4636,11 @@ If an error occurs, @command{gawk} exits with the value of the C constant @code{EXIT_FAILURE}. This is usually one. If @command{gawk} exits because of a fatal error, the exit -status is 2. On non-POSIX systems, this value may be mapped +status is two. On non-POSIX systems, this value may be mapped to @code{EXIT_FAILURE}. @node Include Files -@section Including Other Files Into Your Program +@section Including Other Files into Your Program @c Panos Papadopoulos <panos1962@gmail.com> contributed the original @c text for this section. @@ -4490,7 +4654,7 @@ The @code{@@include} keyword can be used to read external @command{awk} source files. This gives you the ability to split large @command{awk} source files into smaller, more manageable pieces, and also lets you reuse common @command{awk} code from various @command{awk} scripts. In other words, you can group -together @command{awk} functions, used to carry out specific tasks, +together @command{awk} functions used to carry out specific tasks into external files. These files can be used just like function libraries, using the @code{@@include} keyword in conjunction with the @env{AWKPATH} environment variable. Note that source files may also be included @@ -4525,9 +4689,9 @@ $ @kbd{gawk -f test2} @print{} This is script test2. @end example -@code{gawk} runs the @file{test2} script which includes @file{test1} +@command{gawk} runs the @file{test2} script, which includes @file{test1} using the @code{@@include} -keyword. So, to include external @command{awk} source files you just +keyword. So, to include external @command{awk} source files, you just use @code{@@include} followed by the name of the file to be included, enclosed in double quotes. @@ -4564,27 +4728,28 @@ The @value{FN} can, of course, be a pathname. For example: @end example @noindent -or: +and: @example @@include "/usr/awklib/network" @end example @noindent -are valid. The @code{AWKPATH} environment variable can be of great +are both valid. The @env{AWKPATH} environment variable can be of great value when using @code{@@include}. The same rules for the use -of the @code{AWKPATH} variable in command-line file searches +of the @env{AWKPATH} variable in command-line file searches (@pxref{AWKPATH Variable}) apply to @code{@@include} also. This is very helpful in constructing @command{gawk} function libraries. -If you have a large script with useful, general purpose @command{awk} +If you have a large script with useful, general-purpose @command{awk} functions, you can break it down into library files and put those files -in a special directory. You can then include those ``libraries,'' using -either the full pathnames of the files, or by setting the @code{AWKPATH} +in a special directory. You can then include those ``libraries,'' +either by using the full pathnames of the files, or by setting the @env{AWKPATH} environment variable accordingly and then using @code{@@include} with -just the file part of the full pathname. Of course you can have more -than one directory to keep library files; the more complex the working +just the file part of the full pathname. Of course, +you can keep library files in more than one directory; +the more complex the working environment is, the more directories you may need to organize the files to be included. @@ -4597,11 +4762,11 @@ In particular, @code{@@include} is very useful for writing CGI scripts to be run from web pages. As mentioned in @ref{AWKPATH Variable}, the current directory is always -searched first for source files, before searching in @env{AWKPATH}, -and this also applies to files named with @code{@@include}. +searched first for source files, before searching in @env{AWKPATH}; +this also applies to files named with @code{@@include}. @node Loading Shared Libraries -@section Loading Dynamic Extensions Into Your Program +@section Loading Dynamic Extensions into Your Program This @value{SECTION} describes a feature that is specific to @command{gawk}. @@ -4611,7 +4776,7 @@ This @value{SECTION} describes a feature that is specific to @command{gawk}. The @code{@@load} keyword can be used to read external @command{awk} extensions (stored as system shared libraries). This allows you to link in compiled code that may offer superior -performance and/or give you access to extended capabilities not supported +performance and/or give you access to extended capabilities not supported by the @command{awk} language. The @env{AWKLIBPATH} variable is used to search for the extension. Using @code{@@load} is completely equivalent to using the @option{-l} command-line option. @@ -4619,7 +4784,7 @@ to using the @option{-l} command-line option. If the extension is not initially found in @env{AWKLIBPATH}, another search is conducted after appending the platform's default shared library suffix to the @value{FN}. For example, on GNU/Linux systems, the suffix -@samp{.so} is used. +@samp{.so} is used: @example $ @kbd{gawk '@@load "ordchr"; BEGIN @{print chr(65)@}'} @@ -4652,8 +4817,8 @@ It also describes the @code{ordchr} extension. @cindex features, deprecated @cindex obsolete features This @value{SECTION} describes features and/or command-line options from -previous releases of @command{gawk} that are either not available in the -current version or that are still supported but deprecated (meaning that +previous releases of @command{gawk} that either are not available in the +current version or are still supported but deprecated (meaning that they will @emph{not} be in the next release). The process-related special files @file{/dev/pid}, @file{/dev/ppid}, @@ -4733,7 +4898,7 @@ This seems to have been a long-undocumented feature in Unix @command{awk}. Similarly, you may use @code{print} or @code{printf} statements in the @var{init} and @var{increment} parts of a @code{for} loop. This is another -long-undocumented ``feature'' of Unix @code{awk}. +long-undocumented ``feature'' of Unix @command{awk}. @end ignore @@ -4750,17 +4915,17 @@ to run @command{awk}. @item The three standard options for all versions of @command{awk} are -@option{-f}, @option{-F} and @option{-v}. @command{gawk} supplies these +@option{-f}, @option{-F}, and @option{-v}. @command{gawk} supplies these and many others, as well as corresponding GNU-style long options. @item -Non-option command-line arguments are usually treated as @value{FN}s, +Nonoption command-line arguments are usually treated as @value{FN}s, unless they have the form @samp{@var{var}=@var{value}}, in which case they are taken as variable assignments to be performed at that point in processing the input. @item -All non-option command-line arguments, excluding the program text, +All nonoption command-line arguments, excluding the program text, are placed in the @code{ARGV} array. Adjusting @code{ARGC} and @code{ARGV} affects how @command{awk} processes input. @@ -4787,13 +4952,12 @@ and @option{-f} command-line options. @item @command{gawk} allows you to load additional functions written in C or C++ using the @code{@@load} statement and/or the @option{-l} option. -(This advanced feature is described later on in @ref{Dynamic Extensions}.) +(This advanced feature is described later, in @ref{Dynamic Extensions}.) @end itemize @node Regexp @chapter Regular Expressions @cindex regexp -@c STARTOFRANGE regexp @cindex regular expressions A @dfn{regular expression}, or @dfn{regexp}, is a way of describing a @@ -4809,7 +4973,7 @@ belongs to that set. The simplest regular expression is a sequence of letters, numbers, or both. Such a regexp matches any string that contains that sequence. Thus, the regexp @samp{foo} matches any string containing @samp{foo}. -Therefore, the pattern @code{/foo/} matches any input record containing +Thus, the pattern @code{/foo/} matches any input record containing the three adjacent characters @samp{foo} @emph{anywhere} in the record. Other kinds of regexps let you specify more complicated classes of strings. @@ -4872,17 +5036,16 @@ and @samp{!~} perform regular expression comparisons. Expressions using these operators can be used as patterns, or in @code{if}, @code{while}, @code{for}, and @code{do} statements. (@xref{Statements}.) -For example: +For example, the following is true if the expression @var{exp} (taken +as a string) matches @var{regexp}: @example @var{exp} ~ /@var{regexp}/ @end example @noindent -is true if the expression @var{exp} (taken as a string) -matches @var{regexp}. The following example matches, or selects, -all input records with the uppercase letter @samp{J} somewhere in the -first field: +This example matches, or selects, all input records with the uppercase +letter @samp{J} somewhere in the first field: @example $ @kbd{awk '$1 ~ /J/' inventory-shipped} @@ -4952,9 +5115,9 @@ string or regexp. Thus, the string whose contents are the two characters @samp{"} and @samp{\} must be written @code{"\"\\"}. Other escape sequences represent unprintable characters -such as TAB or newline. While there is nothing to stop you from entering most +such as TAB or newline. There is nothing to stop you from entering most unprintable characters directly in a string constant or regexp constant, -they may look ugly. +but they may look ugly. The following list presents all the escape sequences used in @command{awk} and @@ -5001,7 +5164,7 @@ Horizontal TAB, @kbd{Ctrl-i}, ASCII code 9 (HT). @cindex @code{\} (backslash), @code{\v} escape sequence @cindex backslash (@code{\}), @code{\v} escape sequence @item \v -Vertical tab, @kbd{Ctrl-k}, ASCII code 11 (VT). +Vertical TAB, @kbd{Ctrl-k}, ASCII code 11 (VT). @cindex @code{\} (backslash), @code{\}@var{nnn} escape sequence @cindex backslash (@code{\}), @code{\}@var{nnn} escape sequence @@ -5022,15 +5185,18 @@ of hexadecimal digits (@samp{0}--@samp{9}, and either @samp{A}--@samp{F} or @samp{a}--@samp{f}). A maximum of two digts are allowed after the @samp{\x}. Any further hexadecimal digits are treated as simple letters or numbers. @value{COMMONEXT} +(The @samp{\x} escape sequence is not allowed in POSIX awk.) @quotation CAUTION In ISO C, the escape sequence continues until the first nonhexadecimal digit is seen. -@c FIXME: Add exact version here. For many years, @command{gawk} would continue incorporating hexadecimal digits into the value until a non-hexadecimal digit or the end of the string was encountered. -However, using more than two hexadecimal digits produces +However, using more than two hexadecimal digits produced +undefined results. +As of @value{PVERSION} 4.2, only two digits +are processed. @end quotation @cindex @code{\} (backslash), @code{\/} escape sequence @@ -5040,7 +5206,7 @@ A literal slash (necessary for regexp constants only). This sequence is used when you want to write a regexp constant that contains a slash (such as @code{/.*:\/home\/[[:alnum:]]+:.*/}; the @samp{[[:alnum:]]} -notation is discussed shortly, in @ref{Bracket Expressions}). +notation is discussed in @ref{Bracket Expressions}). Because the regexp is delimited by slashes, you need to escape any slash that is part of the pattern, in order to tell @command{awk} to keep processing the rest of the regexp. @@ -5063,7 +5229,7 @@ with a backslash have special meaning in regexps. In a regexp, a backslash before any character that is not in the previous list and not listed in -@ref{GNU Regexp Operators}, +@DBREF{GNU Regexp Operators} means that the next character should be taken literally, even if it would normally be a regexp operator. For example, @code{/a\+b/} matches the three characters @samp{a+b}. @@ -5072,27 +5238,9 @@ characters @samp{a+b}. @cindex @code{\} (backslash), in escape sequences @cindex portability For complete portability, do not use a backslash before any character not -shown in the previous list and that is not an operator. - -To summarize: - -@itemize @value{BULLET} -@item -The escape sequences in the list above are always processed first, -for both string constants and regexp constants. This happens very early, -as soon as @command{awk} reads your program. - -@item -@command{gawk} processes both regexp constants and dynamic regexps -(@pxref{Computed Regexps}), -for the special operators listed in -@ref{GNU Regexp Operators}. - -@item -A backslash before any other character means to treat that character -literally. -@end itemize +shown in the previous list or that is not an operator. +@c 11/2014: Moved so as to not stack sidebars @cindex sidebar, Backslash Before Regular Characters @ifdocbook @docbook @@ -5177,6 +5325,25 @@ In such implementations, typing @code{"a\qc"} is the same as typing @end cartouche @end ifnotdocbook +To summarize: + +@itemize @value{BULLET} +@item +The escape sequences in the preceding list are always processed first, +for both string constants and regexp constants. This happens very early, +as soon as @command{awk} reads your program. + +@item +@command{gawk} processes both regexp constants and dynamic regexps +(@pxref{Computed Regexps}), +for the special operators listed in +@ref{GNU Regexp Operators}. + +@item +A backslash before any other character means to treat that character +literally. +@end itemize + @cindex sidebar, Escape Sequences for Metacharacters @ifdocbook @docbook @@ -5233,7 +5400,6 @@ escape sequences literally when used in regexp constants. Thus, @node Regexp Operators @section Regular Expression Operators -@c STARTOFRANGE regexpo @cindex regular expressions, operators @cindex metacharacters in regular expressions @@ -5251,14 +5417,14 @@ are recognized and converted into corresponding real characters as the very first step in processing regexps. Here is a list of metacharacters. All characters that are not escape -sequences and that are not listed in the following stand for themselves: +sequences and that are not listed here stand for themselves: @c Use @asis so the docbook comes out ok. Sigh. @table @asis @cindex backslash (@code{\}), regexp operator @cindex @code{\} (backslash), regexp operator @item @code{\} -This is used to suppress the special meaning of a character when +This suppresses the special meaning of a character when matching. For example, @samp{\$} matches the character @samp{$}. @@ -5267,8 +5433,9 @@ matches the character @samp{$}. @cindex @code{^} (caret), regexp operator @cindex caret (@code{^}), regexp operator @item @code{^} -This matches the beginning of a string. For example, @samp{^@@chapter} -matches @samp{@@chapter} at the beginning of a string and can be used +This matches the beginning of a string. @samp{^@@chapter} +matches @samp{@@chapter} at the beginning of a string, +for example, and can be used to identify chapter beginnings in Texinfo source files. The @samp{^} is known as an @dfn{anchor}, because it anchors the pattern to match only at the beginning of the string. @@ -5308,10 +5475,10 @@ with @samp{A}. @cindex POSIX @command{awk}, period (@code{.})@comma{} using In strict POSIX mode (@pxref{Options}), -@samp{.} does not match the @value{NUL} +@samp{.} does not match the @sc{nul} character, which is a character with all bits equal to zero. -Otherwise, @value{NUL} is just another character. Other versions of @command{awk} -may not be able to match the @value{NUL} character. +Otherwise, @sc{nul} is just another character. Other versions of @command{awk} +may not be able to match the @sc{nul} character. @cindex @code{[]} (square brackets), regexp operator @cindex square brackets (@code{[]}), regexp operator @@ -5373,8 +5540,8 @@ just @samp{p} if no @samp{h}s are present. There are two subtle points to understand about how @samp{*} works. First, the @samp{*} applies only to the single preceding regular expression component (e.g., in @samp{ph*}, it applies just to the @samp{h}). -To cause @samp{*} to apply to a larger sub-expression, use parentheses: -@samp{(ph)*} matches @samp{ph}, @samp{phph}, @samp{phphph} and so on. +To cause @samp{*} to apply to a larger subexpression, use parentheses: +@samp{(ph)*} matches @samp{ph}, @samp{phph}, @samp{phphph}, and so on. Second, @samp{*} finds as many repetitions as possible. If the text to be matched is @samp{phhhhhhhhhhhhhhooey}, @samp{ph*} matches all of @@ -5412,10 +5579,10 @@ is repeated at least @var{n} times: Matches @samp{whhhy}, but not @samp{why} or @samp{whhhhy}. @item wh@{3,5@}y -Matches @samp{whhhy}, @samp{whhhhy}, or @samp{whhhhhy}, only. +Matches @samp{whhhy}, @samp{whhhhy}, or @samp{whhhhhy} only. @item wh@{2,@}y -Matches @samp{whhy} or @samp{whhhy}, and so on. +Matches @samp{whhy}, @samp{whhhy}, and so on. @end table @cindex POSIX @command{awk}, interval expressions in @@ -5464,17 +5631,15 @@ usage as a syntax error. If @command{gawk} is in compatibility mode (@pxref{Options}), interval expressions are not available in regular expressions. -@c ENDOFRANGE regexpo @node Bracket Expressions @section Using Bracket Expressions -@c STARTOFRANGE charlist @cindex bracket expressions @cindex bracket expressions, range expressions @cindex range expressions (regexps) @cindex character lists in regular expression -As mentioned earlier, a bracket expression matches any character amongst +As mentioned earlier, a bracket expression matches any character among those listed between the opening and closing square brackets. Within a bracket expression, a @dfn{range expression} consists of two @@ -5532,23 +5697,23 @@ a keyword denoting the class, and @samp{:]}. POSIX standard. @float Table,table-char-classes -@caption{POSIX Character Classes} +@caption{POSIX character classes} @multitable @columnfractions .15 .85 @headitem Class @tab Meaning -@item @code{[:alnum:]} @tab Alphanumeric characters. -@item @code{[:alpha:]} @tab Alphabetic characters. -@item @code{[:blank:]} @tab Space and TAB characters. -@item @code{[:cntrl:]} @tab Control characters. -@item @code{[:digit:]} @tab Numeric characters. -@item @code{[:graph:]} @tab Characters that are both printable and visible. -(A space is printable but not visible, whereas an @samp{a} is both.) -@item @code{[:lower:]} @tab Lowercase alphabetic characters. -@item @code{[:print:]} @tab Printable characters (characters that are not control characters). +@item @code{[:alnum:]} @tab Alphanumeric characters +@item @code{[:alpha:]} @tab Alphabetic characters +@item @code{[:blank:]} @tab Space and TAB characters +@item @code{[:cntrl:]} @tab Control characters +@item @code{[:digit:]} @tab Numeric characters +@item @code{[:graph:]} @tab Characters that are both printable and visible +(a space is printable but not visible, whereas an @samp{a} is both) +@item @code{[:lower:]} @tab Lowercase alphabetic characters +@item @code{[:print:]} @tab Printable characters (characters that are not control characters) @item @code{[:punct:]} @tab Punctuation characters (characters that are not letters, digits, -control characters, or space characters). -@item @code{[:space:]} @tab Space characters (such as space, TAB, and formfeed, to name a few). -@item @code{[:upper:]} @tab Uppercase alphabetic characters. -@item @code{[:xdigit:]} @tab Characters that are hexadecimal digits. +control characters, or space characters) +@item @code{[:space:]} @tab Space characters (such as space, TAB, and formfeed, to name a few) +@item @code{[:upper:]} @tab Uppercase alphabetic characters +@item @code{[:xdigit:]} @tab Characters that are hexadecimal digits @end multitable @end float @@ -5563,12 +5728,12 @@ and numeric characters in your character set. @c Thanks to @c Date: Tue, 01 Jul 2014 07:39:51 +0200 @c From: Hermann Peifer <peifer@gmx.eu> -Some utilities that match regular expressions provide a non-standard -@code{[:ascii:]} character class; @command{awk} does not. However, you -can simulate such a construct using @code{[\x00-\x7F]}. This matches +Some utilities that match regular expressions provide a nonstandard +@samp{[:ascii:]} character class; @command{awk} does not. However, you +can simulate such a construct using @samp{[\x00-\x7F]}. This matches all values numerically between zero and 127, which is the defined range of the ASCII character set. Use a complemented character list -(@code{[^\x00-\x7F]}) to match any single-byte characters that are not +(@samp{[^\x00-\x7F]}) to match any single-byte characters that are not in the ASCII range. @cindex bracket expressions, collating elements @@ -5597,8 +5762,8 @@ Locale-specific names for a list of characters that are equal. The name is enclosed between @samp{[=} and @samp{=]}. For example, the name @samp{e} might be used to represent all of -``e,'' ``@`e,'' and ``@'e.'' In this case, @samp{[[=e=]]} is a regexp -that matches any of @samp{e}, @samp{@'e}, or @samp{@`e}. +``e,'' ``@^e,'' ``@`e,'' and ``@'e.'' In this case, @samp{[[=e=]]} is a regexp +that matches any of @samp{e}, @samp{@^e}, @samp{@'e}, or @samp{@`e}. @end table These features are very valuable in non-English-speaking locales. @@ -5612,7 +5777,6 @@ expression matching currently recognize only POSIX character classes; they do not recognize collating symbols or equivalence classes. @end quotation @c maybe one day ... -@c ENDOFRANGE charlist @node Leftmost Longest @section How Much Text Matches? @@ -5628,7 +5792,7 @@ echo aaaabcd | awk '@{ sub(/a+/, "<A>"); print @}' This example uses the @code{sub()} function to make a change to the input record. (@code{sub()} replaces the first instance of any text matched by the first argument with the string provided as the second argument; -@pxref{String Functions}). Here, the regexp @code{/a+/} indicates ``one +@pxref{String Functions}.) Here, the regexp @code{/a+/} indicates ``one or more @samp{a} characters,'' and the replacement text is @samp{<A>}. The input contains four @samp{a} characters. @@ -5656,9 +5820,7 @@ and also @pxref{Field Separators}). @node Computed Regexps @section Using Dynamic Regexps -@c STARTOFRANGE dregexp @cindex regular expressions, computed -@c STARTOFRANGE regexpd @cindex regular expressions, dynamic @cindex @code{~} (tilde), @code{~} operator @cindex tilde (@code{~}), @code{~} operator @@ -5684,14 +5846,14 @@ and tests whether the input record matches this regexp. @quotation NOTE When using the @samp{~} and @samp{!~} -operators, there is a difference between a regexp constant +operators, be aware that there is a difference between a regexp constant enclosed in slashes and a string constant enclosed in double quotes. If you are going to use a string constant, you have to understand that the string is, in essence, scanned @emph{twice}: the first time when @command{awk} reads your program, and the second time when it goes to match the string on the lefthand side of the operator with the pattern on the right. This is true of any string-valued expression (such as -@code{digits_regexp}, shown previously), not just string constants. +@code{digits_regexp}, shown in the previous example), not just string constants. @end quotation @cindex regexp constants, slashes vs.@: quotes @@ -5754,7 +5916,7 @@ $ @kbd{awk '$0 ~ "[ \t\n]"'} @error{} ]... @error{} source line number 1 @error{} context is -@error{} $0 ~ "[ >>> \t\n]" <<< +@error{} $0 ~ "[ >>> \t\n]" <<< @end example @cindex newlines, in regexp constants @@ -5792,7 +5954,7 @@ $ @kbd{awk '$0 ~ "[ \t\n]"'} @error{} ]... @error{} source line number 1 @error{} context is -@error{} $0 ~ "[ >>> \t\n]" <<< +@error{} $0 ~ "[ >>> \t\n]" <<< @end example @cindex newlines, in regexp constants @@ -5809,17 +5971,13 @@ $ @kbd{awk '$0 ~ /[ \t\n]/'} occur often in practice, but it's worth noting for future reference. @end cartouche @end ifnotdocbook -@c ENDOFRANGE dregexp -@c ENDOFRANGE regexpd @node GNU Regexp Operators @section @command{gawk}-Specific Regexp Operators @c This section adapted (long ago) from the regex-0.12 manual -@c STARTOFRANGE regexpg @cindex regular expressions, operators, @command{gawk} -@c STARTOFRANGE gregexp @cindex @command{gawk}, regular expressions, operators @cindex operators, GNU-specific @cindex regular expressions, operators, for words @@ -5895,7 +6053,7 @@ matches either @samp{ball} or @samp{balls}, as a separate word. @item \B Matches the empty string that occurs between two word-constituent characters. For example, -@code{/\Brat\B/} matches @samp{crate} but it does not match @samp{dirty rat}. +@code{/\Brat\B/} matches @samp{crate}, but it does not match @samp{dirty rat}. @samp{\B} is essentially the opposite of @samp{\y}. @end table @@ -5903,9 +6061,9 @@ word-constituent characters. For example, @cindex regular expressions, operators, for buffers @cindex operators, string-matching, for buffers There are two other operators that work on buffers. In Emacs, a -@dfn{buffer} is, naturally, an Emacs buffer. For other programs, -@command{gawk}'s regexp library routines consider the entire -string to match as the buffer. +@dfn{buffer} is, naturally, an Emacs buffer. +Other GNU programs, including @command{gawk}, +consider the entire string to match as the buffer. The operators are: @table @code @@ -5914,14 +6072,14 @@ The operators are: @cindex backslash (@code{\}), @code{\`} operator (@command{gawk}) @cindex @code{\} (backslash), @code{\`} operator (@command{gawk}) Matches the empty string at the -beginning of a buffer (string). +beginning of a buffer (string) @c @cindex operators, @code{\'} (@command{gawk}) @cindex backslash (@code{\}), @code{\'} operator (@command{gawk}) @cindex @code{\} (backslash), @code{\'} operator (@command{gawk}) @item \' Matches the empty string at the -end of a buffer (string). +end of a buffer (string) @end table @cindex @code{^} (caret), regexp operator @@ -5966,16 +6124,16 @@ in @ref{Regexp Operators}. @end ifnottex @item @code{--posix} -Only POSIX regexps are supported; the GNU operators are not special +Match only POSIX regexps; the GNU operators are not special (e.g., @samp{\w} matches a literal @samp{w}). Interval expressions are allowed. @cindex Brian Kernighan's @command{awk} @item @code{--traditional} -Traditional Unix @command{awk} regexps are matched. The GNU operators +Match traditional Unix @command{awk} regexps. The GNU operators are not special, and interval expressions are not available. -The POSIX character classes (@samp{[[:alnum:]]}, etc.) are supported, -as BWK @command{awk} supports them. +Because BWK @command{awk} supports them, +the POSIX character classes (@samp{[[:alnum:]]}, etc.) are available. Characters described by octal and hexadecimal escape sequences are treated literally, even if they represent regexp metacharacters. @@ -5984,15 +6142,11 @@ Allow interval expressions in regexps, if @option{--traditional} has been provided. Otherwise, interval expressions are available by default. @end table -@c ENDOFRANGE gregexp -@c ENDOFRANGE regexpg @node Case-sensitivity @section Case Sensitivity in Matching -@c STARTOFRANGE regexpcs @cindex regular expressions, case sensitivity -@c STARTOFRANGE csregexp @cindex case sensitivity, regexps and Case is normally significant in regular expressions, both when matching ordinary characters (i.e., not metacharacters) and inside bracket @@ -6035,7 +6189,7 @@ When @code{IGNORECASE} is not zero, @emph{all} regexp and string operations ignore case. Changing the value of @code{IGNORECASE} dynamically controls the -case-sensitivity of the program as it runs. Case is significant by +case sensitivity of the program as it runs. Case is significant by default because @code{IGNORECASE} (like most variables) is initialized to zero: @@ -6048,7 +6202,7 @@ if (x ~ /ab/) @dots{} # now it will succeed @end example In general, you cannot use @code{IGNORECASE} to make certain rules -case-insensitive and other rules case-sensitive, because there is no +case insensitive and other rules case sensitive, as there is no straightforward way to set @code{IGNORECASE} just for the pattern of a particular rule.@footnote{Experienced C and C++ programmers will note @@ -6059,13 +6213,13 @@ and However, this is somewhat obscure and we don't recommend it.} To do this, use either bracket expressions or @code{tolower()}. However, one thing you can do with @code{IGNORECASE} only is dynamically turn -case-sensitivity on or off for all the rules at once. +case sensitivity on or off for all the rules at once. @code{IGNORECASE} can be set on the command line or in a @code{BEGIN} rule (@pxref{Other Arguments}; also @pxref{Using BEGIN/END}). Setting @code{IGNORECASE} from the command line is a way to make -a program case-insensitive without having to edit it. +a program case insensitive without having to edit it. @c @cindex ISO 8859-1 @c @cindex ISO Latin-1 @@ -6084,8 +6238,6 @@ the right thing.} The value of @code{IGNORECASE} has no effect if @command{gawk} is in compatibility mode (@pxref{Options}). Case is always significant in compatibility mode. -@c ENDOFRANGE csregexp -@c ENDOFRANGE regexpcs @node Regexp Summary @section Summary @@ -6102,12 +6254,12 @@ in conditional expressions, or as part of matching expressions using the @samp{~} and @samp{!~} operators. @item -Escape sequences let you represent non-printable characters and +Escape sequences let you represent nonprintable characters and also let you represent regexp metacharacters as literal characters to be matched. @item -Regexp operators provide grouping, alternation and repetition. +Regexp operators provide grouping, alternation, and repetition. @item Bracket expressions give you a shorthand for specifying sets @@ -6122,8 +6274,8 @@ the match, such as for text substitution and when the record separator is a regexp. @item -Matching expressions may use dynamic regexps, that is, string values -treated as regular expressions. +Matching expressions may use dynamic regexps (i.e., string values +treated as regular expressions). @item @command{gawk}'s @code{IGNORECASE} variable lets you control the @@ -6132,12 +6284,10 @@ versions, use @code{tolower()} or @code{toupper()}. @end itemize -@c ENDOFRANGE regexp @node Reading Files @chapter Reading Input Files -@c STARTOFRANGE infir @cindex reading input files @cindex input files, reading @cindex input files @@ -6162,7 +6312,7 @@ This makes it more convenient for programs to work on the parts of a record. @cindex @code{getline} command On rare occasions, you may need to use the @code{getline} command. -The @code{getline} command is valuable, both because it +The @code{getline} command is valuable both because it can do explicit input from any number of files, and because the files used with it do not have to be named on the @command{awk} command line (@pxref{Getline}). @@ -6179,6 +6329,7 @@ used with it do not have to be named on the @command{awk} command line * Getline:: Reading files under explicit program control using the @code{getline} function. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -6188,16 +6339,14 @@ used with it do not have to be named on the @command{awk} command line @node Records @section How Input Is Split into Records -@c STARTOFRANGE inspl @cindex input, splitting into records -@c STARTOFRANGE recspl @cindex records, splitting input into @cindex @code{NR} variable @cindex @code{FNR} variable @command{awk} divides the input for your program into records and fields. It keeps track of the number of records that have been read so far from the current input file. This value is stored in a predefined variable -called @code{FNR} which is reset to zero every time a new file is started. +called @code{FNR}, which is reset to zero every time a new file is started. Another predefined variable, @code{NR}, records the total number of input records read so far from all @value{DF}s. It starts at zero, but is never automatically reset to zero. @@ -6208,15 +6357,15 @@ never automatically reset to zero. @end menu @node awk split records -@subsection Record Splitting With Standard @command{awk} +@subsection Record Splitting with Standard @command{awk} @cindex separators, for records @cindex record separators Records are separated by a character called the @dfn{record separator}. By default, the record separator is the newline character. This is why records are, by default, single lines. -A different character can be used for the record separator by -assigning the character to the predefined variable @code{RS}. +To use a different character for the record separator, +simply assign that character to the predefined variable @code{RS}. @cindex newlines, as record separators @cindex @code{RS} variable @@ -6225,7 +6374,7 @@ the value of @code{RS} can be changed in the @command{awk} program with the assignment operator, @samp{=} (@pxref{Assignment Ops}). The new record-separator character should be enclosed in quotation marks, -which indicate a string constant. Often the right time to do this is +which indicate a string constant. Often, the right time to do this is at the beginning of execution, before any input is processed, so that the very first record is read with the proper separator. To do this, use the special @code{BEGIN} pattern @@ -6239,8 +6388,8 @@ awk 'BEGIN @{ RS = "u" @} @noindent changes the value of @code{RS} to @samp{u}, before reading any input. -This is a string whose first character is the letter ``u;'' as a result, records -are separated by the letter ``u.'' Then the input file is read, and the second +The new value is a string whose first character is the letter ``u''; as a result, records +are separated by the letter ``u''. Then the input file is read, and the second rule in the @command{awk} program (the action with no pattern) prints each record. Because each @code{print} statement adds a newline at the end of its output, this @command{awk} program copies the input @@ -6287,7 +6436,7 @@ $ @kbd{awk 'BEGIN @{ RS = "u" @}} @print{} m@@ny @print{} .ed @print{} R -@print{} +@print{} @end example @noindent @@ -6301,8 +6450,8 @@ Bill 555-1675 bill.drowning@@hotmail.com A @end example @noindent -It contains no @samp{u} so there is no reason to split the record, -unlike the others which have one or more occurrences of the @samp{u}. +It contains no @samp{u}, so there is no reason to split the record, +unlike the others, which each have one or more occurrences of the @samp{u}. In fact, this record is treated as part of the previous record; the newline separating them in the output is the original newline in the @value{DF}, not the one added by @@ -6333,7 +6482,7 @@ being fully POSIX-compliant (@pxref{Options}). Then, the following (extreme) pipeline prints a surprising @samp{1}: @example -$ echo | gawk --posix 'BEGIN @{ RS = "a" @} ; @{ print NF @}' +$ @kbd{echo | gawk --posix 'BEGIN @{ RS = "a" @} ; @{ print NF @}'} @print{} 1 @end example @@ -6355,7 +6504,7 @@ The empty string @code{""} (a string without any characters) has a special meaning as the value of @code{RS}. It means that records are separated by one or more blank lines and nothing else. -@xref{Multiple Line}, for more details. +@DBXREF{Multiple Line} for more details. If you change the value of @code{RS} in the middle of an @command{awk} run, the new value is used to delimit subsequent records, but the record @@ -6375,7 +6524,7 @@ sets the variable @code{RT} to the text in the input that matched @code{RS}. @node gawk split records -@subsection Record Splitting With @command{gawk} +@subsection Record Splitting with @command{gawk} @cindex common extensions, @code{RS} as a regexp @cindex extensions, common@comma{} @code{RS} as a regexp @@ -6397,7 +6546,7 @@ contains the same single character. However, when @code{RS} is a regular expression, @code{RT} contains the actual input text that matched the regular expression. -If the input file ended without any text that matches @code{RS}, +If the input file ends without any text matching @code{RS}, @command{gawk} sets @code{RT} to the null string. The following example illustrates both of these features. @@ -6419,11 +6568,11 @@ $ @kbd{echo record 1 AAAA record 2 BBBB record 3 |} The square brackets delineate the contents of @code{RT}, letting you see the leading and trailing whitespace. The final value of @code{RT} is a newline. -@xref{Simple Sed}, for a more useful example +@DBXREF{Simple Sed} for a more useful example of @code{RS} as a regexp and @code{RT}. If you set @code{RS} to a regular expression that allows optional -trailing text, such as @samp{RS = "abc(XYZ)?"} it is possible, due +trailing text, such as @samp{RS = "abc(XYZ)?"}, it is possible, due to implementation constraints, that @command{gawk} may match the leading part of the regular expression, but not the trailing part, particularly if the input text that could match the trailing part is fairly long. @@ -6437,7 +6586,7 @@ the beginning and end of a @emph{line}. As a result, something like @samp{RS = "^[[:upper:]]"} can only match at the beginning of a file. This is because @command{gawk} views the input file as one long string that happens to contain newline characters. -It is thus best to avoid anchor characters in the value of @code{RS}. +It is thus best to avoid anchor metacharacters in the value of @code{RS}. @end quotation @cindex differences in @command{awk} and @command{gawk}, @code{RS}/@code{RT} variables @@ -6461,7 +6610,7 @@ a value that you know doesn't occur in the input file. This is hard to do in a general way, such that a program always works for arbitrary input files. -You might think that for text files, the @value{NUL} character, which +You might think that for text files, the @sc{nul} character, which consists of a character with all bits equal to zero, is a good value to use for @code{RS} in this case: @@ -6470,30 +6619,30 @@ BEGIN @{ RS = "\0" @} # whole file becomes one record? @end example @cindex differences in @command{awk} and @command{gawk}, strings, storing -@command{gawk} in fact accepts this, and uses the @value{NUL} +@command{gawk} in fact accepts this, and uses the @sc{nul} character for the record separator. This works for certain special files, such as @file{/proc/environ} on -GNU/Linux systems, where the @value{NUL} character is in fact the record separator. +GNU/Linux systems, where the @sc{nul} character is in fact the record separator. However, this usage is @emph{not} portable to most other @command{awk} implementations. @cindex dark corner, strings, storing Almost all other @command{awk} implementations@footnote{At least that we know about.} store strings internally as C-style strings. C strings use the -@value{NUL} character as the string terminator. In effect, this means that +@sc{nul} character as the string terminator. In effect, this means that @samp{RS = "\0"} is the same as @samp{RS = ""}. @value{DARKCORNER} -It happens that recent versions of @command{mawk} can use the @value{NUL} +It happens that recent versions of @command{mawk} can use the @sc{nul} character as a record separator. However, this is a special case: -@command{mawk} does not allow embedded @value{NUL} characters in strings. +@command{mawk} does not allow embedded @sc{nul} characters in strings. (This may change in a future version of @command{mawk}.) @cindex records, treating files as @cindex treating files, as single records -@xref{Readfile Function}, for an interesting way to read -whole files. If you are using @command{gawk}, see @ref{Extension Sample -Readfile}, for another option. +@DBXREF{Readfile Function} for an interesting way to read +whole files. If you are using @command{gawk}, see @DBREF{Extension Sample +Readfile} for another option. @docbook </sidebar> @@ -6512,7 +6661,7 @@ a value that you know doesn't occur in the input file. This is hard to do in a general way, such that a program always works for arbitrary input files. -You might think that for text files, the @value{NUL} character, which +You might think that for text files, the @sc{nul} character, which consists of a character with all bits equal to zero, is a good value to use for @code{RS} in this case: @@ -6521,34 +6670,32 @@ BEGIN @{ RS = "\0" @} # whole file becomes one record? @end example @cindex differences in @command{awk} and @command{gawk}, strings, storing -@command{gawk} in fact accepts this, and uses the @value{NUL} +@command{gawk} in fact accepts this, and uses the @sc{nul} character for the record separator. This works for certain special files, such as @file{/proc/environ} on -GNU/Linux systems, where the @value{NUL} character is in fact the record separator. +GNU/Linux systems, where the @sc{nul} character is in fact the record separator. However, this usage is @emph{not} portable to most other @command{awk} implementations. @cindex dark corner, strings, storing Almost all other @command{awk} implementations@footnote{At least that we know about.} store strings internally as C-style strings. C strings use the -@value{NUL} character as the string terminator. In effect, this means that +@sc{nul} character as the string terminator. In effect, this means that @samp{RS = "\0"} is the same as @samp{RS = ""}. @value{DARKCORNER} -It happens that recent versions of @command{mawk} can use the @value{NUL} +It happens that recent versions of @command{mawk} can use the @sc{nul} character as a record separator. However, this is a special case: -@command{mawk} does not allow embedded @value{NUL} characters in strings. +@command{mawk} does not allow embedded @sc{nul} characters in strings. (This may change in a future version of @command{mawk}.) @cindex records, treating files as @cindex treating files, as single records -@xref{Readfile Function}, for an interesting way to read -whole files. If you are using @command{gawk}, see @ref{Extension Sample -Readfile}, for another option. +@DBXREF{Readfile Function} for an interesting way to read +whole files. If you are using @command{gawk}, see @DBREF{Extension Sample +Readfile} for another option. @end cartouche @end ifnotdocbook -@c ENDOFRANGE inspl -@c ENDOFRANGE recspl @node Fields @section Examining Fields @@ -6556,7 +6703,6 @@ Readfile}, for another option. @cindex examining fields @cindex fields @cindex accessing fields -@c STARTOFRANGE fiex @cindex fields, examining @cindex POSIX @command{awk}, field separators and @cindex field separators, POSIX and @@ -6567,9 +6713,9 @@ called @dfn{fields}. By default, fields are separated by @dfn{whitespace}, like words in a line. Whitespace in @command{awk} means any string of one or more spaces, TABs, or newlines;@footnote{In POSIX @command{awk}, newlines are not -considered whitespace for separating fields.} other characters, such as -formfeed, vertical tab, etc., that are -considered whitespace by other languages, are @emph{not} considered +considered whitespace for separating fields.} other characters +that are considered whitespace by other languages +(such as formfeed, vertical tab, etc.) are @emph{not} considered whitespace by @command{awk}. The purpose of fields is to make it more convenient for you to refer to @@ -6581,12 +6727,12 @@ simple @command{awk} programs so powerful. @cindex @code{$} (dollar sign), @code{$} field operator @cindex dollar sign (@code{$}), @code{$} field operator @cindex field operators@comma{} dollar sign as -You use a dollar-sign (@samp{$}) +You use a dollar sign (@samp{$}) to refer to a field in an @command{awk} program, followed by the number of the field you want. Thus, @code{$1} refers to the first field, @code{$2} to the second, and so on. -(Unlike the Unix shells, the field numbers are not limited to single digits. -@code{$127} is the one hundred twenty-seventh field in the record.) +(Unlike in the Unix shells, the field numbers are not limited to single digits. +@code{$127} is the 127th field in the record.) For example, suppose the following is a line of input: @example @@ -6611,7 +6757,7 @@ If you try to reference a field beyond the last one (such as @code{$8} when the record has only seven fields), you get the empty string. (If used in a numeric operation, you get zero.) -The use of @code{$0}, which looks like a reference to the ``zero-th'' field, is +The use of @code{$0}, which looks like a reference to the ``zeroth'' field, is a special case: it represents the whole input record. Use it when you are not interested in specific fields. Here are some more examples: @@ -6637,7 +6783,6 @@ $ @kbd{awk '/li/ @{ print $1, $NF @}' mail-list} @print{} Julie F @print{} Samuel A @end example -@c ENDOFRANGE fiex @node Nonconstant Fields @section Nonconstant Field Numbers @@ -6656,7 +6801,7 @@ awk '@{ print $NR @}' @noindent Recall that @code{NR} is the number of records read so far: one in the -first record, two in the second, etc. So this example prints the first +first record, two in the second, and so on. So this example prints the first field of the first record, the second field of the second record, and so on. For the twentieth record, field number 20 is printed; most likely, the record has fewer than 20 fields, so this prints a blank line. @@ -6667,13 +6812,13 @@ awk '@{ print $(2*2) @}' mail-list @end example @command{awk} evaluates the expression @samp{(2*2)} and uses -its value as the number of the field to print. The @samp{*} sign +its value as the number of the field to print. The @samp{*} represents multiplication, so the expression @samp{2*2} evaluates to four. The parentheses are used so that the multiplication is done before the @samp{$} operation; they are necessary whenever there is a binary operator@footnote{A @dfn{binary operator}, such as @samp{*} for multiplication, is one that takes two operands. The distinction -is required, since @command{awk} also has unary (one-operand) +is required because @command{awk} also has unary (one-operand) and ternary (three-operand) operators.} in the field-number expression. This example, then, prints the type of relationship (the fourth field) for every line of the file @@ -6698,7 +6843,6 @@ evaluating @code{NF} and using its value as a field number. @node Changing Fields @section Changing the Contents of a Field -@c STARTOFRANGE ficon @cindex fields, changing contents of The contents of a field, as seen by @command{awk}, can be changed within an @command{awk} program; this changes what @command{awk} perceives as the @@ -6747,7 +6891,7 @@ $ @kbd{awk '@{ $2 = $2 - 10; print $0 @}' inventory-shipped} @dots{} @end example -It is also possible to also assign contents to fields that are out +It is also possible to assign contents to fields that are out of range. For example: @example @@ -6798,9 +6942,9 @@ else @noindent should print @samp{everything is normal}, because @code{NF+1} is certain -to be out of range. (@xref{If Statement}, +to be out of range. (@DBXREF{If Statement} for more information about @command{awk}'s @code{if-else} statements. -@xref{Typing and Comparison}, +@DBXREF{Typing and Comparison} for more information about the @samp{!=} operator.) It is important to note that making an assignment to an existing field @@ -6840,8 +6984,8 @@ after the new value of @code{NF} and recomputes @code{$0}. Here is an example: @example -$ echo a b c d e f | awk '@{ print "NF =", NF; -> NF = 3; print $0 @}' +$ @kbd{echo a b c d e f | awk '@{ print "NF =", NF;} +> @kbd{ NF = 3; print $0 @}'} @print{} NF = 6 @print{} a b c @end example @@ -6854,7 +6998,7 @@ rebuild @code{$0} when @code{NF} is decremented. Finally, there are times when it is convenient to force @command{awk} to rebuild the entire record, using the current -value of the fields and @code{OFS}. To do this, use the +values of the fields and @code{OFS}. To do this, use the seemingly innocuous assignment: @example @@ -6883,14 +7027,14 @@ such as @code{sub()} and @code{gsub()} It is important to remember that @code{$0} is the @emph{full} record, exactly as it was read from the input. This includes any leading or trailing whitespace, and the exact whitespace (or other -characters) that separate the fields. +characters) that separates the fields. It is a common error to try to change the field separators in a record simply by setting @code{FS} and @code{OFS}, and then expecting a plain @samp{print} or @samp{print $0} to print the modified record. -But this does not work, since nothing was done to change the record +But this does not work, because nothing was done to change the record itself. Instead, you must force the record to be rebuilt, typically with a statement such as @samp{$1 = $1}, as described earlier. @@ -6908,20 +7052,19 @@ with a statement such as @samp{$1 = $1}, as described earlier. It is important to remember that @code{$0} is the @emph{full} record, exactly as it was read from the input. This includes any leading or trailing whitespace, and the exact whitespace (or other -characters) that separate the fields. +characters) that separates the fields. It is a common error to try to change the field separators in a record simply by setting @code{FS} and @code{OFS}, and then expecting a plain @samp{print} or @samp{print $0} to print the modified record. -But this does not work, since nothing was done to change the record +But this does not work, because nothing was done to change the record itself. Instead, you must force the record to be rebuilt, typically with a statement such as @samp{$1 = $1}, as described earlier. @end cartouche @end ifnotdocbook -@c ENDOFRANGE ficon @node Field Separators @section Specifying How Fields Are Separated @@ -6937,9 +7080,7 @@ with a statement such as @samp{$1 = $1}, as described earlier. @cindex @code{FS} variable @cindex fields, separating -@c STARTOFRANGE fisepr @cindex field separators -@c STARTOFRANGE fisepg @cindex fields, separating The @dfn{field separator}, which is either a single character or a regular expression, controls the way @command{awk} splits an input record into fields. @@ -6968,7 +7109,7 @@ the Unix Bourne shell, @command{sh}, or Bash). @cindex @code{FS} variable, changing value of The value of @code{FS} can be changed in the @command{awk} program with the assignment operator, @samp{=} (@pxref{Assignment Ops}). -Often the right time to do this is at the beginning of execution +Often, the right time to do this is at the beginning of execution before any input has been processed, so that the very first record is read with the proper separator. To do this, use the special @code{BEGIN} pattern @@ -7005,7 +7146,7 @@ John Q. Smith, LXIX, 29 Oak St., Walamazoo, MI 42139 @end example @noindent -The same program would extract @samp{@bullet{}LXIX}, instead of +The same program would extract @samp{@bullet{}LXIX} instead of @samp{@bullet{}29@bullet{}Oak@bullet{}St.}. If you were expecting the program to print the address, you would be surprised. The moral is to choose your data layout and @@ -7039,9 +7180,7 @@ rules. @node Regexp Field Splitting @subsection Using Regular Expressions to Separate Fields -@c STARTOFRANGE regexpfs @cindex regular expressions, as field separators -@c STARTOFRANGE fsregexp @cindex field separators, regular expressions as The previous @value{SUBSECTION} discussed the use of single characters or simple strings as the @@ -7124,7 +7263,7 @@ statement prints the new @code{$0}. @cindex dark corner, @code{^}, in @code{FS} There is an additional subtlety to be aware of when using regular expressions for field splitting. -It is not well-specified in the POSIX standard, or anywhere else, what @samp{^} +It is not well specified in the POSIX standard, or anywhere else, what @samp{^} means when splitting fields. Does the @samp{^} match only at the beginning of the entire record? Or is each field separator a new string? It turns out that different @command{awk} versions answer this question differently, and you @@ -7145,8 +7284,6 @@ $ @kbd{echo 'xxAA xxBxx C' |} @print{} -->xxBxx<-- @print{} -->C<-- @end example -@c ENDOFRANGE regexpfs -@c ENDOFRANGE fsregexp @node Single Character Fields @subsection Making Each Character a Separate Field @@ -7270,7 +7407,7 @@ choosing your field and record separators. @cindex Unix @command{awk}, password files@comma{} field separators and Perhaps the most common use of a single character as the field separator occurs when processing the Unix system password file. On many Unix -systems, each user has a separate entry in the system password file, one +systems, each user has a separate entry in the system password file, with one line per user. The information in these lines is separated by colons. The first field is the user's login name and the second is the user's encrypted or shadow password. (A shadow password is indicated by the @@ -7290,11 +7427,11 @@ awk -F: '$5 == ""' /etc/passwd @end example @node Full Line Fields -@subsection Making The Full Line Be A Single Field +@subsection Making the Full Line Be a Single Field Occasionally, it's useful to treat the whole input line as a single field. This can be done easily and portably simply by -setting @code{FS} to @code{"\n"} (a newline).@footnote{Thanks to +setting @code{FS} to @code{"\n"} (a newline):@footnote{Thanks to Andrew Schorr for this tip.} @example @@ -7304,42 +7441,6 @@ awk -F'\n' '@var{program}' @var{files @dots{}} @noindent When you do this, @code{$1} is the same as @code{$0}. -@node Field Splitting Summary -@subsection Field-Splitting Summary - -It is important to remember that when you assign a string constant -as the value of @code{FS}, it undergoes normal @command{awk} string -processing. For example, with Unix @command{awk} and @command{gawk}, -the assignment @samp{FS = "\.."} assigns the character string @code{".."} -to @code{FS} (the backslash is stripped). This creates a regexp meaning -``fields are separated by occurrences of any two characters.'' -If instead you want fields to be separated by a literal period followed -by any single character, use @samp{FS = "\\.."}. - -The following list summarizes how fields are split, based on the value -of @code{FS} (@samp{==} means ``is equal to''): - -@table @code -@item FS == " " -Fields are separated by runs of whitespace. Leading and trailing -whitespace are ignored. This is the default. - -@item FS == @var{any other single character} -Fields are separated by each occurrence of the character. Multiple -successive occurrences delimit empty fields, as do leading and -trailing occurrences. -The character can even be a regexp metacharacter; it does not need -to be escaped. - -@item FS == @var{regexp} -Fields are separated by occurrences of characters that match @var{regexp}. -Leading and trailing matches of @var{regexp} delimit empty fields. - -@item FS == "" -Each individual character in the record becomes a separate field. -(This is a common extension; it is not specified by the POSIX standard.) -@end table - @cindex sidebar, Changing @code{FS} Does Not Affect the Fields @ifdocbook @docbook @@ -7352,7 +7453,7 @@ Each individual character in the record becomes a separate field. According to the POSIX standard, @command{awk} is supposed to behave as if each record is split into fields at the time it is read. In particular, this means that if you change the value of @code{FS} -after a record is read, the value of the fields (i.e., how they were split) +after a record is read, the values of the fields (i.e., how they were split) should reflect the old value of @code{FS}, not the new one. @cindex dark corner, field separators @@ -7365,10 +7466,7 @@ using the @emph{current} value of @code{FS}! @value{DARKCORNER} This behavior can be difficult to diagnose. The following example illustrates the difference -between the two methods. -(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' -Its behavior is also defined by the POSIX standard.} -command prints just the first line of @file{/etc/passwd}.) +between the two methods: @example sed 1q /etc/passwd | awk '@{ FS = ":" ; print $1 @}' @@ -7386,9 +7484,13 @@ on an incorrect implementation of @command{awk}, while @command{gawk} prints the full first line of the file, something like: @example -root:nSijPlPhZZwgE:0:0:Root:/: +root:x:0:0:Root:/: @end example +(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' +Its behavior is also defined by the POSIX standard.} +command prints just the first line of @file{/etc/passwd}.) + @docbook </sidebar> @end docbook @@ -7405,7 +7507,7 @@ root:nSijPlPhZZwgE:0:0:Root:/: According to the POSIX standard, @command{awk} is supposed to behave as if each record is split into fields at the time it is read. In particular, this means that if you change the value of @code{FS} -after a record is read, the value of the fields (i.e., how they were split) +after a record is read, the values of the fields (i.e., how they were split) should reflect the old value of @code{FS}, not the new one. @cindex dark corner, field separators @@ -7418,10 +7520,7 @@ using the @emph{current} value of @code{FS}! @value{DARKCORNER} This behavior can be difficult to diagnose. The following example illustrates the difference -between the two methods. -(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' -Its behavior is also defined by the POSIX standard.} -command prints just the first line of @file{/etc/passwd}.) +between the two methods: @example sed 1q /etc/passwd | awk '@{ FS = ":" ; print $1 @}' @@ -7439,11 +7538,51 @@ on an incorrect implementation of @command{awk}, while @command{gawk} prints the full first line of the file, something like: @example -root:nSijPlPhZZwgE:0:0:Root:/: +root:x:0:0:Root:/: @end example + +(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' +Its behavior is also defined by the POSIX standard.} +command prints just the first line of @file{/etc/passwd}.) @end cartouche @end ifnotdocbook +@node Field Splitting Summary +@subsection Field-Splitting Summary + +It is important to remember that when you assign a string constant +as the value of @code{FS}, it undergoes normal @command{awk} string +processing. For example, with Unix @command{awk} and @command{gawk}, +the assignment @samp{FS = "\.."} assigns the character string @code{".."} +to @code{FS} (the backslash is stripped). This creates a regexp meaning +``fields are separated by occurrences of any two characters.'' +If instead you want fields to be separated by a literal period followed +by any single character, use @samp{FS = "\\.."}. + +The following list summarizes how fields are split, based on the value +of @code{FS} (@samp{==} means ``is equal to''): + +@table @code +@item FS == " " +Fields are separated by runs of whitespace. Leading and trailing +whitespace are ignored. This is the default. + +@item FS == @var{any other single character} +Fields are separated by each occurrence of the character. Multiple +successive occurrences delimit empty fields, as do leading and +trailing occurrences. +The character can even be a regexp metacharacter; it does not need +to be escaped. + +@item FS == @var{regexp} +Fields are separated by occurrences of characters that match @var{regexp}. +Leading and trailing matches of @var{regexp} delimit empty fields. + +@item FS == "" +Each individual character in the record becomes a separate field. +(This is a common extension; it is not specified by the POSIX standard.) +@end table + @cindex sidebar, @code{FS} and @code{IGNORECASE} @ifdocbook @docbook @@ -7467,7 +7606,7 @@ print $1 @noindent The output is @samp{aCa}. If you really want to split fields on an alphabetic character while ignoring case, use a regexp that will -do it for you. E.g., @samp{FS = "[c]"}. In this case, @code{IGNORECASE} +do it for you (e.g., @samp{FS = "[c]"}). In this case, @code{IGNORECASE} will take effect. @docbook @@ -7497,31 +7636,29 @@ print $1 @noindent The output is @samp{aCa}. If you really want to split fields on an alphabetic character while ignoring case, use a regexp that will -do it for you. E.g., @samp{FS = "[c]"}. In this case, @code{IGNORECASE} +do it for you (e.g., @samp{FS = "[c]"}). In this case, @code{IGNORECASE} will take effect. @end cartouche @end ifnotdocbook -@c ENDOFRANGE fisepr -@c ENDOFRANGE fisepg @node Constant Size @section Reading Fixed-Width Data -@quotation NOTE +@cindex data, fixed-width +@cindex fixed-width data +@cindex advanced features, fixed-width data + +@c O'Reilly doesn't like it as a note the first thing in the section. This @value{SECTION} discusses an advanced feature of @command{gawk}. If you are a novice @command{awk} user, you might want to skip it on the first reading. -@end quotation -@cindex data, fixed-width -@cindex fixed-width data -@cindex advanced features, fixed-width data -@command{gawk} provides a facility for dealing with -fixed-width fields with no distinctive field separator. For example, -data of this nature arises in the input for old Fortran programs where -numbers are run together, or in the output of programs that did not -anticipate the use of their output as input for other programs. +@command{gawk} provides a facility for dealing with fixed-width fields +with no distinctive field separator. For example, data of this nature +arises in the input for old Fortran programs where numbers are run +together, or in the output of programs that did not anticipate the use +of their output as input for other programs. An example of the latter is a table where all the columns are lined up by the use of a variable number of spaces and @emph{empty fields are just @@ -7541,7 +7678,7 @@ variable @code{FIELDWIDTHS}. Each number specifies the width of the field, @emph{including} columns between fields. If you want to ignore the columns between fields, you can specify the width as a separate field that is subsequently ignored. -It is a fatal error to supply a field width that is not a positive number. +It is a fatal error to supply a field width that has a negative value. The following data is the output of the Unix @command{w} utility. It is useful to illustrate the use of @code{FIELDWIDTHS}: @@ -7560,15 +7697,10 @@ dave ttyq4 26Jun9115days 46 46 wnewmail @end group @end example -The following program takes the above input, converts the idle time to +The following program takes this input, converts the idle time to number of seconds, and prints out the first two fields and the calculated idle time: -@quotation NOTE -This program uses a number of @command{awk} features that -haven't been introduced yet. -@end quotation - @example BEGIN @{ FIELDWIDTHS = "9 6 10 6 7 7 35" @} NR > 2 @{ @@ -7587,6 +7719,11 @@ NR > 2 @{ @} @end example +@quotation NOTE +The preceding program uses a number of @command{awk} features that +haven't been introduced yet. +@end quotation + Running the program on the data produces the following results: @example @@ -7618,7 +7755,7 @@ In order to tell which kind of field splitting is in effect, use @code{PROCINFO["FS"]} (@pxref{Auto-set}). The value is @code{"FS"} if regular field splitting is being used, -or it is @code{"FIELDWIDTHS"} if fixed-width field splitting is being used: +or @code{"FIELDWIDTHS"} if fixed-width field splitting is being used: @example if (PROCINFO["FS"] == "FS") @@ -7632,17 +7769,16 @@ else This information is useful when writing a function that needs to temporarily change @code{FS} or @code{FIELDWIDTHS}, read some records, and then restore the original settings -(@pxref{Passwd Functions}, +(@DBPXREF{Passwd Functions} for an example of such a function). @node Splitting By Content -@section Defining Fields By Content +@section Defining Fields by Content -@quotation NOTE +@c O'Reilly doesn't like it as a note the first thing in the section. This @value{SECTION} discusses an advanced feature of @command{gawk}. If you are a novice @command{awk} user, you might want to skip it on the first reading. -@end quotation @cindex advanced features, specifying field content Normally, when using @code{FS}, @command{gawk} defines the fields as the @@ -7653,14 +7789,16 @@ However, there are times when you really want to define the fields by what they are, and not by what they are not. The most notorious such case -is so-called @dfn{comma separated value} (CSV) data. Many spreadsheet programs, +is so-called @dfn{comma-separated values} (CSV) data. Many spreadsheet programs, for example, can export their data into text files, where each record is -terminated with a newline, and fields are separated by commas. If only -commas separated the data, there wouldn't be an issue. The problem comes when -one of the fields contains an @emph{embedded} comma. While there is no -formal standard specification for CSV data@footnote{At least, we don't know of one.}, -in such cases, most programs embed the field in double quotes. So we might -have data like this: +terminated with a newline, and fields are separated by commas. If +commas only separated the data, there wouldn't be an issue. The problem comes when +one of the fields contains an @emph{embedded} comma. +In such cases, most programs embed the field in double quotes.@footnote{The +CSV format lacked a formal standard definition for many years. +@uref{http://www.ietf.org/rfc/rfc4180.txt, RFC 4180} +standardizes the most common practices.} +So, we might have data like this: @example @c file eg/misc/addresses.csv @@ -7674,7 +7812,7 @@ The @code{FPAT} variable offers a solution for cases like this. The value of @code{FPAT} should be a string that provides a regular expression. This regular expression describes the contents of each field. -In the case of CSV data as presented above, each field is either ``anything that +In the case of CSV data as presented here, each field is either ``anything that is not a comma,'' or ``a double quote, anything that is not a double quote, and a closing double quote.'' If written as a regular expression constant (@pxref{Regexp}), @@ -7739,15 +7877,15 @@ will be @code{"FPAT"} if content-based field splitting is being used. @quotation NOTE Some programs export CSV data that contains embedded newlines between the double quotes. @command{gawk} provides no way to deal with this. -Since there is no formal specification for CSV data, there isn't much +Even though a formal specification for CSV data exists, there isn't much more to be done; the @code{FPAT} mechanism provides an elegant solution for the majority -of cases, and the @command{gawk} developers are satisfied with that. +of cases, and the @command{gawk} developers are satisfied with that. @end quotation As written, the regexp used for @code{FPAT} requires that each field -have a least one character. A straightforward modification -(changing changed the first @samp{+} to @samp{*}) allows fields to be empty: +contain at least one character. A straightforward modification +(changing the first @samp{+} to @samp{*}) allows fields to be empty: @example FPAT = "([^,]*)|(\"[^\"]+\")" @@ -7757,20 +7895,17 @@ Finally, the @code{patsplit()} function makes the same functionality available for splitting regular strings (@pxref{String Functions}). To recap, @command{gawk} provides three independent methods -to split input records into fields. @command{gawk} uses whichever -mechanism was last chosen based on which of the three -variables---@code{FS}, @code{FIELDWIDTHS}, and @code{FPAT}---was +to split input records into fields. +The mechanism used is based on which of the three +variables---@code{FS}, @code{FIELDWIDTHS}, or @code{FPAT}---was last assigned to. @node Multiple Line @section Multiple-Line Records @cindex multiple-line records -@c STARTOFRANGE recm @cindex records, multiline -@c STARTOFRANGE imr @cindex input, multiline records -@c STARTOFRANGE frm @cindex files, reading, multiline records @cindex input, files, See input files In some databases, a single line cannot conveniently hold all the @@ -7805,7 +7940,7 @@ at the end of the record and one or more blank lines after the record. In addition, a regular expression always matches the longest possible sequence when there is a choice (@pxref{Leftmost Longest}). -So the next record doesn't start until +So, the next record doesn't start until the first nonblank line that follows---no matter how many blank lines appear in a row, they are considered one record separator. @@ -7820,10 +7955,10 @@ In the second case, this special processing is not done. @cindex field separator, in multiline records @cindex @code{FS}, in multiline records Now that the input is separated into records, the second step is to -separate the fields in the record. One way to do this is to divide each +separate the fields in the records. One way to do this is to divide each of the lines into fields in the normal manner. This happens by default as the result of a special feature. When @code{RS} is set to the empty -string, @emph{and} @code{FS} is set to a single character, +string @emph{and} @code{FS} is set to a single character, the newline character @emph{always} acts as a field separator. This is in addition to whatever field separations result from @code{FS}.@footnote{When @code{FS} is the null string (@code{""}) @@ -7838,7 +7973,7 @@ want the newline character to separate fields, because there is no way to prevent it. However, you can work around this by using the @code{split()} function to break up the record manually (@pxref{String Functions}). -If you have a single character field separator, you can work around +If you have a single-character field separator, you can work around the special feature in a different way, by making @code{FS} into a regexp for that single character. For example, if the field separator is a percent character, instead of @@ -7846,10 +7981,10 @@ separator is a percent character, instead of Another way to separate fields is to put each field on a separate line: to do this, just set the -variable @code{FS} to the string @code{"\n"}. (This single -character separator matches a single newline.) +variable @code{FS} to the string @code{"\n"}. +(This single-character separator matches a single newline.) A practical example of a @value{DF} organized this way might be a mailing -list, where each entry is separated by blank lines. Consider a mailing +list, where blank lines separate the entries. Consider a mailing list in a file named @file{addresses}, which looks like this: @example @@ -7896,7 +8031,7 @@ $ @kbd{awk -f addrs.awk addresses} @dots{} @end example -@xref{Labels Program}, for a more realistic program that deals with +@DBXREF{Labels Program} for a more realistic program dealing with address lists. The following list summarizes how records are split, based on the value of @ifinfo @@ -7937,20 +8072,15 @@ If not in compatibility mode (@pxref{Options}), @command{gawk} sets @code{RT} to the input text that matched the value specified by @code{RS}. But if the input file ended without any text that matches @code{RS}, then @command{gawk} sets @code{RT} to the null string. -@c ENDOFRANGE recm -@c ENDOFRANGE imr -@c ENDOFRANGE frm @node Getline @section Explicit Input with @code{getline} -@c STARTOFRANGE getl @cindex @code{getline} command, explicit input with -@c STARTOFRANGE inex @cindex input, explicit So far we have been getting our input data from @command{awk}'s main input stream---either the standard input (usually your keyboard, sometimes -the output from another program) or from the +the output from another program) or the files specified on the command line. The @command{awk} language has a special built-in command called @code{getline} that can be used to read input under your explicit control. @@ -7986,12 +8116,19 @@ a record, such as a file that cannot be opened, then @code{getline} returns @minus{}1. In this case, @command{gawk} sets the variable @code{ERRNO} to a string describing the error that occurred. +If @code{ERRNO} indicates that the I/O operation may be +retried, and @code{PROCINFO["@var{input}", "RETRY"]} is set, +then @code{getline} returns @minus{}2 +instead of @minus{}1, and further calls to @code{getline} +may be attemped. @DBXREF{Retrying Input} for further information about +this feature. + In the following examples, @var{command} stands for a string value that represents a shell command. @quotation NOTE When @option{--sandbox} is specified (@pxref{Options}), -reading lines from files, pipes and coprocesses is disabled. +reading lines from files, pipes, and coprocesses is disabled. @end quotation @menu @@ -8134,7 +8271,7 @@ free @end example The @code{getline} command used in this way sets only the variables -@code{NR}, @code{FNR} and @code{RT} (and of course, @var{var}). +@code{NR}, @code{FNR}, and @code{RT} (and, of course, @var{var}). The record is not split into fields, so the values of the fields (including @code{$0}) and the value of @code{NF} do not change. @@ -8149,7 +8286,7 @@ the value of @code{NF} do not change. @cindex left angle bracket (@code{<}), @code{<} operator (I/O) @cindex operators, input/output Use @samp{getline < @var{file}} to read the next record from @var{file}. -Here @var{file} is a string-valued expression that +Here, @var{file} is a string-valued expression that specifies the @value{FN}. @samp{< @var{file}} is called a @dfn{redirection} because it directs input to come from a different place. For example, the following @@ -8188,7 +8325,7 @@ you want your program to be portable to all @command{awk} implementations. Use @samp{getline @var{var} < @var{file}} to read input from the file -@var{file}, and put it in the variable @var{var}. As above, @var{file} +@var{file}, and put it in the variable @var{var}. As earlier, @var{file} is a string-valued expression that specifies the file from which to read. In this version of @code{getline}, none of the predefined variables are @@ -8224,7 +8361,7 @@ One deficiency of this program is that it does not process nested @code{@@include} statements (i.e., @code{@@include} statements in included files) the way a true macro preprocessor would. -@xref{Igawk Program}, for a program +@DBXREF{Igawk Program} for a program that does handle nested @code{@@include} statements. @node Getline/Pipe @@ -8327,7 +8464,7 @@ of a construct like @samp{@w{"echo "} "date" | getline}. Most versions, including the current version, treat it at as @samp{@w{("echo "} "date") | getline}. (This is also how BWK @command{awk} behaves.) -Some versions changed and treated it as +Some versions instead treat it as @samp{@w{"echo "} ("date" | getline)}. (This is how @command{mawk} behaves.) In short, @emph{always} use explicit parentheses, and then you won't @@ -8375,7 +8512,7 @@ program to be portable to other @command{awk} implementations. @cindex operators, input/output @cindex differences in @command{awk} and @command{gawk}, input/output operators -Input into @code{getline} from a pipe is a one-way operation. +Reading input into @code{getline} from a pipe is a one-way operation. The command that is started with @samp{@var{command} | getline} only sends data @emph{to} your @command{awk} program. @@ -8385,7 +8522,7 @@ for processing and then read the results back. communications are possible. This is done with the @samp{|&} operator. Typically, you write data to the coprocess first and then -read results back, as shown in the following: +read the results back, as shown in the following: @example print "@var{some query}" |& "db_server" @@ -8468,17 +8605,23 @@ also @pxref{Auto-set}.) @item Using @code{FILENAME} with @code{getline} (@samp{getline < FILENAME}) -is likely to be a source for +is likely to be a source of confusion. @command{awk} opens a separate input stream from the current input file. However, by not using a variable, @code{$0} -and @code{NR} are still updated. If you're doing this, it's +and @code{NF} are still updated. If you're doing this, it's probably by accident, and you should reconsider what it is you're trying to accomplish. @item -@DBREF{Getline Summary} presents a table summarizing the +@ifdocbook +The next section +@end ifdocbook +@ifnotdocbook +@ref{Getline Summary}, +@end ifnotdocbook +presents a table summarizing the @code{getline} variants and which variables they can affect. -It is worth noting that those variants which do not use redirection +It is worth noting that those variants that do not use redirection can cause @code{FILENAME} to be updated if they cause @command{awk} to start reading a new input file. @@ -8487,7 +8630,7 @@ can cause @code{FILENAME} to be updated if they cause If the variable being assigned is an expression with side effects, different versions of @command{awk} behave differently upon encountering end-of-file. Some versions don't evaluate the expression; many versions -(including @command{gawk}) do. Here is an example, due to Duncan Moore: +(including @command{gawk}) do. Here is an example, courtesy of Duncan Moore: @ignore Date: Sun, 01 Apr 2012 11:49:33 +0100 @@ -8504,7 +8647,7 @@ BEGIN @{ @noindent Here, the side effect is the @samp{++c}. Is @code{c} incremented if -end of file is encountered, before the element in @code{a} is assigned? +end-of-file is encountered before the element in @code{a} is assigned? @command{gawk} treats @code{getline} like a function call, and evaluates the expression @samp{a[++c]} before attempting to read from @file{f}. @@ -8523,7 +8666,7 @@ and whether the variant is standard or a @command{gawk} extension. Note: for each variant, @command{gawk} sets the @code{RT} predefined variable. @float Table,table-getline-variants -@caption{@code{getline} Variants and What They Set} +@caption{@code{getline} variants and what they set} @multitable @columnfractions .33 .38 .27 @headitem Variant @tab Effect @tab @command{awk} / @command{gawk} @item @code{getline} @tab Sets @code{$0}, @code{NF}, @code{FNR}, @code{NR}, and @code{RT} @tab @command{awk} @@ -8536,12 +8679,9 @@ Note: for each variant, @command{gawk} sets the @code{RT} predefined variable. @item @var{command} @code{|& getline} @var{var} @tab Sets @var{var} and @code{RT} @tab @command{gawk} @end multitable @end float -@c ENDOFRANGE getl -@c ENDOFRANGE inex -@c ENDOFRANGE infir @node Read Timeout -@section Reading Input With A Timeout +@section Reading Input with a Timeout @cindex timeout, reading input @cindex differences in @command{awk} and @command{gawk}, read timeouts @@ -8549,8 +8689,8 @@ This @value{SECTION} describes a feature that is specific to @command{gawk}. You may specify a timeout in milliseconds for reading input from the keyboard, a pipe, or two-way communication, including TCP/IP sockets. This can be done -on a per input, command or connection basis, by setting a special element -in the @code{PROCINFO} array (@pxref{Auto-set}): +on a per-input, per-command, or per-connection basis, by setting a special +element in the @code{PROCINFO} array (@pxref{Auto-set}): @example PROCINFO["input_name", "READ_TIMEOUT"] = @var{timeout in milliseconds} @@ -8581,7 +8721,7 @@ while ((getline < "/dev/stdin") > 0) @end example @command{gawk} terminates the read operation if input does not -arrive after waiting for the timeout period, returns failure +arrive after waiting for the timeout period, returns failure, and sets @code{ERRNO} to an appropriate string value. A negative or zero value for the timeout is the same as specifying no timeout at all. @@ -8591,7 +8731,7 @@ loop that reads input records and matches them against patterns, like so: @example -$ @kbd{ gawk 'BEGIN @{ PROCINFO["-", "READ_TIMEOUT"] = 5000 @}} +$ @kbd{gawk 'BEGIN @{ PROCINFO["-", "READ_TIMEOUT"] = 5000 @}} > @kbd{@{ print "You entered: " $0 @}'} @kbd{gawk} @print{} You entered: gawk @@ -8614,7 +8754,7 @@ for the input to arrive: PROCINFO[Service, "READ_TIMEOUT"] = 1000 while ((Service |& getline) > 0) @{ print $0 - PROCINFO[S, "READ_TIMEOUT"] -= 100 + PROCINFO[Service, "READ_TIMEOUT"] -= 100 @} @end example @@ -8623,21 +8763,22 @@ You should not assume that the read operation will block exactly after the tenth record has been printed. It is possible that @command{gawk} will read and buffer more than one record's worth of data the first time. Because of this, changing the value -of timeout like in the above example is not very useful. +of timeout like in the preceding example is not very useful. @end quotation -If the @code{PROCINFO} element is not present and the environment -variable @env{GAWK_READ_TIMEOUT} exists, +If the @code{PROCINFO} element is not present and the +@env{GAWK_READ_TIMEOUT} environment variable exists, @command{gawk} uses its value to initialize the timeout value. The exclusive use of the environment variable to specify timeout has the disadvantage of not being able to control it -on a per command or connection basis. +on a per-command or per-connection basis. @command{gawk} considers a timeout event to be an error even though the attempt to read from the underlying device may succeed in a later attempt. This is a limitation, and it also means that you cannot use this to multiplex input from -two or more sources. +two or more sources. @DBXREF{Retrying Input} for a way to enable +later I/O attempts to succeed. Assigning a timeout value prevents read operations from blocking indefinitely. But bear in mind that there are other ways @@ -8647,8 +8788,38 @@ a connection before it can start reading any data, or the attempt to open a FIFO special file for reading can block indefinitely until some other process opens it for writing. +@node Retrying Input +@section Retrying Reads After Certain Input Errors +@cindex retrying input + +@cindex differences in @command{awk} and @command{gawk}, retrying input +This @value{SECTION} describes a feature that is specific to @command{gawk}. + +When @command{gawk} encounters an error while reading input, by +default @code{getline} returns @minus{}1, and subsequent attempts to +read from that file result in an end-of-file indication. However, you +may optionally instruct @command{gawk} to allow I/O to be retried when +certain errors are encountered by setting a special element in +the @code{PROCINFO} array (@pxref{Auto-set}): + +@example +PROCINFO["@var{input_name}", "RETRY"] = 1 +@end example + +When this element exists, @command{gawk} checks the value of the system +(C language) +@code{errno} variable when an I/O error occurs. If @code{errno} indicates +a subsequent I/O attempt may succeed, @code{getline} instead returns +@minus{}2 and +further calls to @code{getline} may succeed. This applies to the @code{errno} +values @code{EAGAIN}, @code{EWOULDBLOCK}, @code{EINTR}, or @code{ETIMEDOUT}. + +This feature is useful in conjunction with +@code{PROCINFO["@var{input_name}", "READ_TIMEOUT"]} or situations where a file +descriptor has been configured to behave in a non-blocking fashion. + @node Command-line directories -@section Directories On The Command Line +@section Directories on the Command Line @cindex differences in @command{awk} and @command{gawk}, command-line directories @cindex directories, command-line @cindex command line, directories on @@ -8663,14 +8834,14 @@ command line, but otherwise ignores it. This makes it easier to use shell wildcards with your @command{awk} program: @example -$ @kbd{gawk -f whizprog.awk *} @ii{Directories could kill this progam} +$ @kbd{gawk -f whizprog.awk *} @ii{Directories could kill this program} @end example If either of the @option{--posix} or @option{--traditional} options is given, then @command{gawk} reverts to treating a directory on the command line as a fatal error. -@xref{Extension Sample Readdir}, for a way to treat directories +@DBXREF{Extension Sample Readdir} for a way to treat directories as usable data from an @command{awk} program. @node Input Summary @@ -8682,7 +8853,7 @@ Input is split into records based on the value of @code{RS}. The possibilities are as follows: @multitable @columnfractions .25 .35 .40 -@headitem Value of @code{RS} @tab Records are split on @tab @command{awk} / @command{gawk} +@headitem Value of @code{RS} @tab Records are split on @dots{} @tab @command{awk} / @command{gawk} @item Any single character @tab That character @tab @command{awk} @item The empty string (@code{""}) @tab Runs of two or more newlines @tab @command{awk} @item A regexp @tab Text that matches the regexp @tab @command{gawk} @@ -8697,7 +8868,7 @@ The possibilities are as follows: @item After splitting the input into records, @command{awk} further splits -the record into individual fields, named @code{$1}, @code{$2} and so +the records into individual fields, named @code{$1}, @code{$2}, and so on. @code{$0} is the whole record, and @code{NF} indicates how many fields there are. The default way to split fields is between whitespace characters. @@ -8711,14 +8882,14 @@ greater than @code{NF} creates the field and rebuilds the record, using thing. Decrementing @code{NF} throws away fields and rebuilds the record. @item -Field splitting is more complicated than record splitting. +Field splitting is more complicated than record splitting: -@multitable @columnfractions .40 .45 .15 +@multitable @columnfractions .40 .40 .20 @headitem Field separator value @tab Fields are split @dots{} @tab @command{awk} / @command{gawk} @item @code{FS == " "} @tab On runs of whitespace @tab @command{awk} @item @code{FS == @var{any single character}} @tab On that character @tab @command{awk} @item @code{FS == @var{regexp}} @tab On text matching the regexp @tab @command{awk} -@item @code{FS == ""} @tab Each individual character is a separate field @tab @command{gawk} +@item @code{FS == ""} @tab Such that each individual character is a separate field @tab @command{gawk} @item @code{FIELDWIDTHS == @var{list of columns}} @tab Based on character position @tab @command{gawk} @item @code{FPAT == @var{regexp}} @tab On the text surrounding text matching the regexp @tab @command{gawk} @end multitable @@ -8735,11 +8906,11 @@ This can also be done using command-line variable assignment. Use @code{PROCINFO["FS"]} to see how fields are being split. @item -Use @code{getline} in its various forms to read additional records, +Use @code{getline} in its various forms to read additional records from the default input stream, from a file, or from a pipe or coprocess. @item -Use @code{PROCINFO[@var{file}, "READ_TIMEOUT"]} to cause reads to timeout +Use @code{PROCINFO[@var{file}, "READ_TIMEOUT"]} to cause reads to time out for @var{file}. @item @@ -8773,7 +8944,6 @@ That can be fixed by making one simple change. What is it? @node Printing @chapter Printing Output -@c STARTOFRANGE prnt @cindex printing @cindex output, printing, See printing One of the most common programming actions is to @dfn{print}, or output, @@ -8784,12 +8954,11 @@ The @code{print} statement is not limited when computing @emph{which} values to print. However, with two exceptions, you cannot specify @emph{how} to print them---how many columns, whether to use exponential notation or not, and so on. -(For the exceptions, @pxref{Output Separators}, and +(For the exceptions, @DBPXREF{Output Separators} and @ref{OFMT}.) For printing with specifications, you need the @code{printf} statement (@pxref{Printf}). -@c STARTOFRANGE prnts @cindex @code{print} statement @cindex @code{printf} statement Besides basic and formatted printing, this @value{CHAPTER} @@ -8810,6 +8979,7 @@ and discusses the @code{close()} built-in function. @command{gawk} allows access to inherited file descriptors. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. @end menu @@ -8850,7 +9020,7 @@ space is printed between any two items. Note that the @code{print} statement is a statement and not an expression---you can't use it in the pattern part of a -@var{pattern}-@var{action} statement, for example. +pattern--action statement, for example. @node Print Examples @section @code{print} Statement Examples @@ -8969,7 +9139,6 @@ You can continue either a @code{print} or @code{printf} statement simply by putting a newline after any comma (@pxref{Statements/Lines}). @end quotation -@c ENDOFRANGE prnts @node Output Separators @section Output Separators @@ -8981,14 +9150,14 @@ separated by single spaces. However, this doesn't need to be the case; a single space is simply the default. Any string of characters may be used as the @dfn{output field separator} by setting the predefined variable @code{OFS}. The initial value of this variable -is the string @w{@code{" "}}---that is, a single space. +is the string @w{@code{" "}} (i.e., a single space). -The output from an entire @code{print} statement is called an -@dfn{output record}. Each @code{print} statement outputs one output -record, and then outputs a string called the @dfn{output record separator} -(or @code{ORS}). The initial -value of @code{ORS} is the string @code{"\n"}; i.e., a newline -character. Thus, each @code{print} statement normally makes a separate line. +The output from an entire @code{print} statement is called an @dfn{output +record}. Each @code{print} statement outputs one output record, and +then outputs a string called the @dfn{output record separator} (or +@code{ORS}). The initial value of @code{ORS} is the string @code{"\n"} +(i.e., a newline character). Thus, each @code{print} statement normally +makes a separate line. @cindex output, records @cindex output record separator, See @code{ORS} variable @@ -9011,27 +9180,27 @@ newline: $ @kbd{awk 'BEGIN @{ OFS = ";"; ORS = "\n\n" @}} > @kbd{@{ print $1, $2 @}' mail-list} @print{} Amelia;555-5553 -@print{} +@print{} @print{} Anthony;555-3412 -@print{} +@print{} @print{} Becky;555-7685 -@print{} +@print{} @print{} Bill;555-1675 -@print{} +@print{} @print{} Broderick;555-0542 -@print{} +@print{} @print{} Camilla;555-2912 -@print{} +@print{} @print{} Fabius;555-1234 -@print{} +@print{} @print{} Julie;555-6699 -@print{} +@print{} @print{} Martin;555-6480 -@print{} +@print{} @print{} Samuel;555-3430 -@print{} +@print{} @print{} Jean-Paul;555-2127 -@print{} +@print{} @end example If the value of @code{ORS} does not contain a newline, the program's output @@ -9042,7 +9211,7 @@ runs together on a single line. @cindex numeric, output format @cindex formats@comma{} numeric output When printing numeric values with the @code{print} statement, -@command{awk} internally converts the number to a string of characters +@command{awk} internally converts each number to a string of characters and prints that string. @command{awk} uses the @code{sprintf()} function to do this conversion (@pxref{String Functions}). @@ -9082,7 +9251,6 @@ if @code{OFMT} contains anything but a floating-point conversion specification. @node Printf @section Using @code{printf} Statements for Fancier Printing -@c STARTOFRANGE printfs @cindex @code{printf} statement @cindex output, formatted @cindex formatting output @@ -9112,9 +9280,9 @@ printf @var{format}, @var{item1}, @var{item2}, @dots{} @end example @noindent -As print @code{print}, the entire list of arguments may optionally be +As for @code{print}, the entire list of arguments may optionally be enclosed in parentheses. Here too, the parentheses are necessary if any -of the item expressions use the @samp{>} relational operator; otherwise, +of the item expressions uses the @samp{>} relational operator; otherwise, it can be confused with an output redirection (@pxref{Redirection}). @cindex format specifiers @@ -9145,7 +9313,7 @@ $ @kbd{awk 'BEGIN @{} @end example @noindent -Here, neither the @samp{+} nor the @samp{OUCH} appear in +Here, neither the @samp{+} nor the @samp{OUCH!} appears in the output message. @node Control Letters @@ -9189,11 +9357,11 @@ a single byte (0--255). @item @code{%d}, @code{%i} Print a decimal integer. The two control letters are equivalent. -(The @code{%i} specification is for compatibility with ISO C.) +(The @samp{%i} specification is for compatibility with ISO C.) @item @code{%e}, @code{%E} -Print a number in scientific (exponential) notation; -for example: +Print a number in scientific (exponential) notation. +For example: @example printf "%4.3e\n", 1950 @@ -9204,7 +9372,7 @@ prints @samp{1.950e+03}, with a total of four significant figures, three of which follow the decimal point. (The @samp{4.3} represents two modifiers, discussed in the next @value{SUBSECTION}.) -@code{%E} uses @samp{E} instead of @samp{e} in the output. +@samp{%E} uses @samp{E} instead of @samp{e} in the output. @item @code{%f} Print a number in floating-point notation. @@ -9220,26 +9388,26 @@ which follow the decimal point. (The @samp{4.3} represents two modifiers, discussed in the next @value{SUBSECTION}.) -On systems supporting IEEE 754 floating point format, values +On systems supporting IEEE 754 floating-point format, values representing negative infinity are formatted as @samp{-inf} or @samp{-infinity}, and positive infinity as -@samp{inf} and @samp{infinity}. +@samp{inf} or @samp{infinity}. The special ``not a number'' value formats as @samp{-nan} or @samp{nan} (@pxref{Math Definitions}). @item @code{%F} -Like @code{%f} but the infinity and ``not a number'' values are spelled +Like @samp{%f}, but the infinity and ``not a number'' values are spelled using uppercase letters. -The @code{%F} format is a POSIX extension to ISO C; not all systems -support it. On those that don't, @command{gawk} uses @code{%f} instead. +The @samp{%F} format is a POSIX extension to ISO C; not all systems +support it. On those that don't, @command{gawk} uses @samp{%f} instead. @item @code{%g}, @code{%G} Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters; if the result is printed in -scientific notation, @code{%G} uses @samp{E} instead of @samp{e}. +scientific notation, @samp{%G} uses @samp{E} instead of @samp{e}. @item @code{%o} Print an unsigned octal integer @@ -9251,11 +9419,11 @@ Print a string. @item @code{%u} Print an unsigned decimal integer. (This format is of marginal use, because all numbers in @command{awk} -are floating-point; it is provided primarily for compatibility with C.) +are floating point; it is provided primarily for compatibility with C.) @item @code{%x}, @code{%X} Print an unsigned hexadecimal integer; -@code{%X} uses the letters @samp{A} through @samp{F} +@samp{%X} uses the letters @samp{A} through @samp{F} instead of @samp{a} through @samp{f} (@pxref{Nondecimal-numbers}). @@ -9270,7 +9438,7 @@ argument and it ignores any modifiers. @quotation NOTE When using the integer format-control letters for values that are outside the range of the widest C integer type, @command{gawk} switches to -the @code{%g} format specifier. If @option{--lint} is provided on the +the @samp{%g} format specifier. If @option{--lint} is provided on the command line (@pxref{Options}), @command{gawk} warns about this. Other versions of @command{awk} may print invalid values or do something else entirely. @@ -9280,7 +9448,6 @@ values or do something else entirely. @node Format Modifiers @subsection Modifiers for @code{printf} Formats -@c STARTOFRANGE pfm @cindex @code{printf} statement, modifiers @cindex modifiers@comma{} in format specifiers A format specification can also include @dfn{modifiers} that can control @@ -9291,12 +9458,12 @@ represent spaces in the output. Here are the possible modifiers, in the order in which they may appear: -@table @code +@table @asis @cindex differences in @command{awk} and @command{gawk}, @code{print}/@code{printf} statements @cindex @code{printf} statement, positional specifiers @c the code{} does NOT start a secondary @cindex positional specifiers, @code{printf} statement -@item @var{N}$ +@item @code{@var{N}$} An integer constant followed by a @samp{$} is a @dfn{positional specifier}. Normally, format specifications are applied to arguments in the order given in the format string. With a positional specifier, the format @@ -9319,7 +9486,7 @@ messages at runtime. which describes how and why to use positional specifiers. For now, we ignore them. -@item - +@item @code{-} (Minus) The minus sign, used before the width modifier (see later on in this list), says to left-justify @@ -9337,31 +9504,31 @@ prints @samp{foo@bullet{}}. For numeric conversions, prefix positive values with a space and negative values with a minus sign. -@item + +@item @code{+} The plus sign, used before the width modifier (see later on in this list), says to always supply a sign for numeric conversions, even if the data to format is positive. The @samp{+} overrides the space modifier. -@item # -Use an ``alternate form'' for certain control letters. -For @code{%o}, supply a leading zero. -For @code{%x} and @code{%X}, supply a leading @code{0x} or @samp{0X} for +@item @code{#} +Use an ``alternative form'' for certain control letters. +For @samp{%o}, supply a leading zero. +For @samp{%x} and @samp{%X}, supply a leading @samp{0x} or @samp{0X} for a nonzero result. -For @code{%e}, @code{%E}, @code{%f}, and @code{%F}, the result always +For @samp{%e}, @samp{%E}, @samp{%f}, and @samp{%F}, the result always contains a decimal point. -For @code{%g} and @code{%G}, trailing zeros are not removed from the result. +For @samp{%g} and @samp{%G}, trailing zeros are not removed from the result. -@item 0 +@item @code{0} A leading @samp{0} (zero) acts as a flag indicating that output should be padded with zeros instead of spaces. This applies only to the numeric output formats. This flag only has an effect when the field width is wider than the value to print. -@item ' +@item @code{'} A single quote or apostrophe character is a POSIX extension to ISO C. -It indicates that the integer part of a floating point value, or the +It indicates that the integer part of a floating-point value, or the entire part of an integer decimal value, should have a thousands-separator character in it. This only works in locales that support such characters. For example: @@ -9412,7 +9579,7 @@ prints @samp{foobar}. Preceding the @var{width} with a minus sign causes the output to be padded with spaces on the right, instead of on the left. -@item .@var{prec} +@item @code{.@var{prec}} A period followed by an integer constant specifies the precision to use when printing. The meaning of the precision varies by control letter: @@ -9442,7 +9609,7 @@ prints @samp{foob}. @end table The C library @code{printf}'s dynamic @var{width} and @var{prec} -capability (for example, @code{"%*.*s"}) is supported. Instead of +capability (e.g., @code{"%*.*s"}) is supported. Instead of supplying explicit @var{width} and/or @var{prec} values in the format string, they are passed in the argument list. For example: @@ -9475,7 +9642,7 @@ printf "%" w "." p "s\n", s @end example @noindent -This is not particularly easy to read but it does work. +This is not particularly easy to read, but it does work. @c @cindex lint checks @cindex troubleshooting, fatal errors, @code{printf} format strings @@ -9486,7 +9653,6 @@ format strings. These are not valid in @command{awk}. Most @command{awk} implementations silently ignore them. If @option{--lint} is provided on the command line (@pxref{Options}), @command{gawk} warns about their use. If @option{--posix} is supplied, their use is a fatal error. -@c ENDOFRANGE pfm @node Printf Examples @subsection Examples Using @code{printf} @@ -9522,7 +9688,7 @@ $ @kbd{awk '@{ printf "%-10s %s\n", $1, $2 @}' mail-list} @end example In this case, the phone numbers had to be printed as strings because -the numbers are separated by a dash. Printing the phone numbers as +the numbers are separated by dashes. Printing the phone numbers as numbers would have produced just the first three digits: @samp{555}. This would have been pretty confusing. @@ -9542,7 +9708,7 @@ awk 'BEGIN @{ print "Name Number" @{ printf "%-10s %s\n", $1, $2 @}' mail-list @end example -The above example mixes @code{print} and @code{printf} statements in +The preceding example mixes @code{print} and @code{printf} statements in the same program. Using just @code{printf} statements can produce the same results: @@ -9567,14 +9733,11 @@ awk 'BEGIN @{ format = "%-10s %s\n" @{ printf format, $1, $2 @}' mail-list @end example -@c ENDOFRANGE printfs @node Redirection @section Redirecting Output of @code{print} and @code{printf} -@c STARTOFRANGE outre @cindex output redirection -@c STARTOFRANGE reout @cindex redirection of output @cindex @option{--sandbox} option, output redirection with @code{print}, @code{printf} So far, the output from @code{print} and @code{printf} has gone @@ -9585,7 +9748,7 @@ This is called @dfn{redirection}. @quotation NOTE When @option{--sandbox} is specified (@pxref{Options}), -redirecting output to files, pipes and coprocesses is disabled. +redirecting output to files, pipes, and coprocesses is disabled. @end quotation A redirection appears after the @code{print} or @code{printf} statement. @@ -9638,7 +9801,7 @@ Each output file contains one name or number per line. @cindex @code{>} (right angle bracket), @code{>>} operator (I/O) @cindex right angle bracket (@code{>}), @code{>>} operator (I/O) @item print @var{items} >> @var{output-file} -This redirection prints the items into the pre-existing output file +This redirection prints the items into the preexisting output file named @var{output-file}. The difference between this and the single-@samp{>} redirection is that the old contents (if any) of @var{output-file} are not erased. Instead, the @command{awk} output is @@ -9677,7 +9840,7 @@ The unsorted list is written with an ordinary redirection, while the sorted list is written by piping through the @command{sort} utility. The next example uses redirection to mail a message to the mailing -list @samp{bug-system}. This might be useful when trouble is encountered +list @code{bug-system}. This might be useful when trouble is encountered in an @command{awk} script run periodically for system maintenance: @example @@ -9689,7 +9852,7 @@ close(report) The @code{close()} function is called here because it's a good idea to close the pipe as soon as all the intended output has been sent to it. -@xref{Close Files And Pipes}, +@DBXREF{Close Files And Pipes} for more information. This example also illustrates the use of a variable to represent @@ -9708,15 +9871,23 @@ This redirection prints the items to the input of @var{command}. The difference between this and the single-@samp{|} redirection is that the output from @var{command} can be read with @code{getline}. -Thus @var{command} is a @dfn{coprocess}, which works together with, -but subsidiary to, the @command{awk} program. +Thus, @var{command} is a @dfn{coprocess}, which works together with +but is subsidiary to the @command{awk} program. This feature is a @command{gawk} extension, and is not available in POSIX @command{awk}. +@ifnotdocbook @xref{Getline/Coprocess}, for a brief discussion. @xref{Two-way I/O}, for a more complete discussion. +@end ifnotdocbook +@ifdocbook +@DBXREF{Getline/Coprocess} +for a brief discussion and +@DBREF{Two-way I/O} +for a more complete discussion. +@end ifdocbook @end table Redirecting output using @samp{>}, @samp{>>}, @samp{|}, or @samp{|&} @@ -9739,9 +9910,9 @@ print "Avoid improbability generators" >> "guide.txt" @noindent This is indeed how redirections must be used from the shell. But in @command{awk}, it isn't necessary. In this kind of case, a program should -use @samp{>} for all the @code{print} statements, since the output file +use @samp{>} for all the @code{print} statements, because the output file is only opened once. (It happens that if you mix @samp{>} and @samp{>>} -that output is produced in the expected order. However, mixing the operators +output is produced in the expected order. However, mixing the operators for the same file is definitely poor style, and is confusing to readers of your program.) @@ -9793,7 +9964,7 @@ The program builds up a list of command lines, using the @command{mv} utility to rename the files. It then sends the list to the shell for execution. -@xref{Shell Quoting}, for a function that can help in generating +@DBXREF{Shell Quoting} for a function that can help in generating command lines to be fed to the shell. @docbook @@ -9828,15 +9999,13 @@ The program builds up a list of command lines, using the @command{mv} utility to rename the files. It then sends the list to the shell for execution. -@xref{Shell Quoting}, for a function that can help in generating +@DBXREF{Shell Quoting} for a function that can help in generating command lines to be fed to the shell. @end cartouche @end ifnotdocbook -@c ENDOFRANGE outre -@c ENDOFRANGE reout @node Special FD -@section Special Files for Standard Pre-Opened Data Streams +@section Special Files for Standard Preopened Data Streams @cindex standard input @cindex input, standard @cindex standard output @@ -9849,7 +10018,7 @@ command lines to be fed to the shell. Running programs conventionally have three input and output streams already available to them for reading and writing. These are known as the @dfn{standard input}, @dfn{standard output}, and @dfn{standard -error output}. These open streams (and any other open file or pipe) +error output}. These open streams (and any other open files or pipes) are often referred to by the technical term @dfn{file descriptors}. These streams are, by default, connected to your keyboard and screen, but @@ -9887,14 +10056,14 @@ that is connected to your keyboard and screen. It represents the ``terminal,''@footnote{The ``tty'' in @file{/dev/tty} stands for ``Teletype,'' a serial terminal.} which on modern systems is a keyboard and screen, not a serial console.) -This generally has the same effect but not always: although the +This generally has the same effect, but not always: although the standard error stream is usually the screen, it can be redirected; when that happens, writing to the screen is not correct. In fact, if @command{awk} is run from a background job, it may not have a terminal at all. Then opening @file{/dev/tty} fails. -@command{gawk}, BWK @command{awk} and @command{mawk} provide +@command{gawk}, BWK @command{awk}, and @command{mawk} provide special @value{FN}s for accessing the three standard streams. If the @value{FN} matches one of these special names when @command{gawk} (or one of the others) redirects input or output, then it directly uses @@ -9932,21 +10101,20 @@ print "Serious error detected!" > "/dev/stderr" @cindex troubleshooting, quotes with file names Note the use of quotes around the @value{FN}. -Like any other redirection, the value must be a string. +Like with any other redirection, the value must be a string. It is a common error to omit the quotes, which leads to confusing results. @command{gawk} does not treat these @value{FN}s as special when -in POSIX compatibility mode. However, since BWK @command{awk} +in POSIX-compatibility mode. However, because BWK @command{awk} supports them, @command{gawk} does support them even when invoked with the @option{--traditional} option (@pxref{Options}). @node Special Files @section Special @value{FFN}s in @command{gawk} -@c STARTOFRANGE gfn @cindex @command{gawk}, file names in -Besides access to standard input, stanard output, and standard error, +Besides access to standard input, standard output, and standard error, @command{gawk} provides access to any open file descriptor. Additionally, there are special @value{FN}s reserved for TCP/IP networking. @@ -9959,7 +10127,7 @@ TCP/IP networking. @end menu @node Other Inherited Files -@subsection Accessing Other Open Files With @command{gawk} +@subsection Accessing Other Open Files with @command{gawk} Besides the @code{/dev/stdin}, @code{/dev/stdout}, and @code{/dev/stderr} special @value{FN}s mentioned earlier, @command{gawk} provides syntax @@ -9995,7 +10163,7 @@ This is done using a special @value{FN} of the form: @file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}} @end example -The @var{net-type} is one of @samp{inet}, @samp{inet4} or @samp{inet6}. +The @var{net-type} is one of @samp{inet}, @samp{inet4}, or @samp{inet6}. The @var{protocol} is one of @samp{tcp} or @samp{udp}, and the other fields represent the other essential pieces of information for making a networking connection. @@ -10016,7 +10184,7 @@ special @value{FN}s that @command{gawk} provides: @cindex compatibility mode (@command{gawk}), file names @cindex file names, in compatibility mode @item -Recognition of the @value{FN}s for the three standard pre-opened +Recognition of the @value{FN}s for the three standard preopened files is disabled only in POSIX mode. @item @@ -10029,23 +10197,18 @@ compatibility mode (either @option{--traditional} or @option{--posix}; interprets these special @value{FN}s. For example, using @samp{/dev/fd/4} for output actually writes on file descriptor 4, and not on a new -file descriptor that is @code{dup()}'ed from file descriptor 4. Most of +file descriptor that is @code{dup()}ed from file descriptor 4. Most of the time this does not matter; however, it is important to @emph{not} close any of the files related to file descriptors 0, 1, and 2. Doing so results in unpredictable behavior. @end itemize -@c ENDOFRANGE gfn @node Close Files And Pipes @section Closing Input and Output Redirections @cindex files, output, See output files -@c STARTOFRANGE ifc @cindex input files, closing -@c STARTOFRANGE ofc @cindex output, files@comma{} closing -@c STARTOFRANGE pc @cindex pipe, closing -@c STARTOFRANGE cc @cindex coprocesses, closing @cindex @code{getline} command, coprocesses@comma{} using from @@ -10184,7 +10347,7 @@ is not closed and released until @code{close()} is called or @command{awk} exits. @code{close()} silently does nothing if given an argument that -does not represent a file, pipe or coprocess that was opened with +does not represent a file, pipe, or coprocess that was opened with a redirection. In such a case, it returns a negative value, indicating an error. In addition, @command{gawk} sets @code{ERRNO} to a string indicating the error. @@ -10223,9 +10386,10 @@ which describes it in more detail and gives an example. @cindex Unix @command{awk}, @code{close()} function and In many older versions of Unix @command{awk}, the @code{close()} function -is actually a statement. It is a syntax error to try and use the return -value from @code{close()}: +is actually a statement. @value{DARKCORNER} +It is a syntax error to try and use the return +value from @code{close()}: @example command = "@dots{}" @@ -10255,9 +10419,9 @@ This value is zero if the close succeeds, or @minus{}1 if it fails. The POSIX standard is very vague; it says that @code{close()} -returns zero on success and nonzero otherwise. In general, +returns zero on success and a nonzero value otherwise. In general, different implementations vary in what they report when closing -pipes; thus the return value cannot be used portably. +pipes; thus, the return value cannot be used portably. @value{DARKCORNER} In POSIX mode (@pxref{Options}), @command{gawk} just returns zero when closing a pipe. @@ -10279,9 +10443,10 @@ when closing a pipe. @cindex Unix @command{awk}, @code{close()} function and In many older versions of Unix @command{awk}, the @code{close()} function -is actually a statement. It is a syntax error to try and use the return -value from @code{close()}: +is actually a statement. @value{DARKCORNER} +It is a syntax error to try and use the return +value from @code{close()}: @example command = "@dots{}" @@ -10311,19 +10476,80 @@ This value is zero if the close succeeds, or @minus{}1 if it fails. The POSIX standard is very vague; it says that @code{close()} -returns zero on success and nonzero otherwise. In general, +returns zero on success and a nonzero value otherwise. In general, different implementations vary in what they report when closing -pipes; thus the return value cannot be used portably. +pipes; thus, the return value cannot be used portably. @value{DARKCORNER} In POSIX mode (@pxref{Options}), @command{gawk} just returns zero when closing a pipe. @end cartouche @end ifnotdocbook -@c ENDOFRANGE ifc -@c ENDOFRANGE ofc -@c ENDOFRANGE pc -@c ENDOFRANGE cc + +@node Nonfatal +@section Enabling Nonfatal Output + +This @value{SECTION} describes a @command{gawk}-specific feature. + +In standard @command{awk}, output with @code{print} or @code{printf} +to a nonexistent file, or some other I/O error (such as filling up the +disk) is a fatal error. + +@example +$ @kbd{gawk 'BEGIN @{ print "hi" > "/no/such/file" @}'} +@error{} gawk: cmd. line:1: fatal: can't redirect to `/no/such/file' (No such file or directory) +@end example + +@command{gawk} makes it possible to detect that an error has +occurred, allowing you to possibly recover from the error, or +at least print an error message of your choosing before exiting. +You can do this in one of two ways: + +@itemize @bullet +@item +For all output files, by assigning any value to @code{PROCINFO["NONFATAL"]}. + +@item +On a per-file basis, by assigning any value to +@code{PROCINFO[@var{filename}, "NONFATAL"]}. +Here, @var{filename} is the name of the file to which +you wish output to be nonfatal. +@end itemize + +Once you have enabled nonfatal output, you must check @code{ERRNO} +after every relevant @code{print} or @code{printf} statement to +see if something went wrong. It is also a good idea to initialize +@code{ERRNO} to zero before attempting the output. For example: + +@example +$ @kbd{gawk '} +> @kbd{BEGIN @{} +> @kbd{ PROCINFO["NONFATAL"] = 1} +> @kbd{ ERRNO = 0} +> @kbd{ print "hi" > "/no/such/file"} +> @kbd{ if (ERRNO) @{} +> @kbd{ print("Output failed:", ERRNO) > "/dev/stderr"} +> @kbd{ exit 1} +> @kbd{ @}} +> @kbd{@}'} +@error{} Output failed: No such file or directory +@end example + +Here, @command{gawk} did not produce a fatal error; instead +it let the @command{awk} program code detect the problem and handle it. + +This mechanism works also for standard output and standard error. +For standard output, you may use @code{PROCINFO["-", "NONFATAL"]} +or @code{PROCINFO["/dev/stdout", "NONFATAL"]}. For standard error, use +@code{PROCINFO["/dev/stderr", "NONFATAL"]}. + +When attempting to open a TCP/IP socket (@pxref{TCP/IP Networking}), +@command{gawk} tries multiple times. The @env{GAWK_SOCK_RETRIES} +environment variable (@pxref{Other Environment Variables}) allows you to +override @command{gawk}'s builtin default number of attempts. However, +once nonfatal I/O is enabled for a given socket, @command{gawk} only +retries once, relying on @command{awk}-level code to notice that there +was a problem. @node Output Summary @section Summary @@ -10337,22 +10563,28 @@ for numeric values for the @code{print} statement. @item The @code{printf} statement provides finer-grained control over output, -with format control letters for different data types and various flags -that modify the behavior of the format control letters. +with format-control letters for different data types and various flags +that modify the behavior of the format-control letters. @item Output from both @code{print} and @code{printf} may be redirected to files, pipes, and coprocesses. @item -@command{gawk} provides special file names for access to standard input, -output and error, and for network communications. +@command{gawk} provides special @value{FN}s for access to standard input, +output, and error, and for network communications. @item -Use @code{close()} to close open file, pipe and coprocess redirections. +Use @code{close()} to close open file, pipe, and coprocess redirections. For coprocesses, it is possible to close only one direction of the communications. +@item +Normally errors with @code{print} or @code{printf} are fatal. +@command{gawk} lets you make output errors be nonfatal either for +all files or on a per-file basis. You must then check for errors +after every relevant output statement. + @end itemize @c EXCLUDE START @@ -10387,11 +10619,9 @@ BEGIN @{ print "Serious error detected!" > /dev/stderr @} @end enumerate @c EXCLUDE END -@c ENDOFRANGE prnt @node Expressions @chapter Expressions -@c STARTOFRANGE exps @cindex expressions Expressions are the basic building blocks of @command{awk} patterns @@ -10402,7 +10632,7 @@ can assign a new value to a variable or a field by using an assignment operator. An expression can serve as a pattern or action statement on its own. Most other kinds of statements contain one or more expressions that specify the data on which to -operate. As in other languages, expressions in @command{awk} include +operate. As in other languages, expressions in @command{awk} can include variables, array references, constants, and function calls, as well as combinations of these with various operators. @@ -10417,11 +10647,11 @@ combinations of these with various operators. @end menu @node Values -@section Constants, Variables and Conversions +@section Constants, Variables, and Conversions Expressions are built up from values and the operations performed upon them. This @value{SECTION} describes the elementary objects -which provide the values used in expressions. +that provide the values used in expressions. @menu * Constants:: String, numeric and regexp constants. @@ -10434,7 +10664,6 @@ which provide the values used in expressions. @node Constants @subsection Constant Expressions -@c STARTOFRANGE cnst @cindex constants, types of The simplest type of expression is the @dfn{constant}, which always has @@ -10443,7 +10672,7 @@ string, and regular expression. Each is used in the appropriate context when you need a data value that isn't going to change. Numeric constants can -have different forms, but are stored identically internally. +have different forms, but are internally stored in an identical manner. @menu * Scalar Constants:: Numeric and string constants. @@ -10459,7 +10688,7 @@ have different forms, but are stored identically internally. A @dfn{numeric constant} stands for a number. This number can be an integer, a decimal fraction, or a number in scientific (exponential) notation.@footnote{The internal representation of all numbers, -including integers, uses double precision floating-point numbers. +including integers, uses double-precision floating-point numbers. On most modern systems, these are in IEEE 754 standard format. @xref{Arbitrary Precision Arithmetic}, for much more information.} Here are some examples of numeric constants that all @@ -10472,8 +10701,8 @@ have the same value: @end example @cindex string constants -A string constant consists of a sequence of characters enclosed in -double-quotation marks. For example: +A @dfn{string constant} consists of a sequence of characters enclosed in +double quotation marks. For example: @example "parrot" @@ -10484,7 +10713,7 @@ double-quotation marks. For example: @cindex strings, length limitations represents the string whose contents are @samp{parrot}. Strings in @command{gawk} can be of any length, and they can contain any of the possible -eight-bit ASCII characters including ASCII @value{NUL} (character code zero). +eight-bit ASCII characters, including ASCII @sc{nul} (character code zero). Other @command{awk} implementations may have difficulty with some character codes. @@ -10495,19 +10724,19 @@ implementations may have difficulty with some character codes. @cindex numbers, octal @cindex numbers, hexadecimal -In @command{awk}, all numbers are in decimal; i.e., base 10. Many other +In @command{awk}, all numbers are in decimal (i.e., base 10). Many other programming languages allow you to specify numbers in other bases, often octal (base 8) and hexadecimal (base 16). -In octal, the numbers go 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, etc. -Just as @samp{11}, in decimal, is 1 times 10 plus 1, so -@samp{11}, in octal, is 1 times 8, plus 1. This equals 9 in decimal. -In hexadecimal, there are 16 digits. Since the everyday decimal +In octal, the numbers go 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, and so on. +Just as @samp{11} in decimal is 1 times 10 plus 1, so +@samp{11} in octal is 1 times 8 plus 1. This equals 9 in decimal. +In hexadecimal, there are 16 digits. Because the everyday decimal number system only has ten digits (@samp{0}--@samp{9}), the letters @samp{a} through @samp{f} are used to represent the rest. (Case in the letters is usually irrelevant; hexadecimal @samp{a} and @samp{A} have the same value.) -Thus, @samp{11}, in -hexadecimal, is 1 times 16 plus 1, which equals 17 in decimal. +Thus, @samp{11} in +hexadecimal is 1 times 16 plus 1, which equals 17 in decimal. Just by looking at plain @samp{11}, you can't tell what base it's in. So, in C, C++, and other languages derived from C, @@ -10518,13 +10747,13 @@ and hexadecimal numbers start with a leading @samp{0x} or @samp{0X}: @table @code @item 11 -Decimal value 11. +Decimal value 11 @item 011 -Octal 11, decimal value 9. +Octal 11, decimal value 9 @item 0x11 -Hexadecimal 11, decimal value 17. +Hexadecimal 11, decimal value 17 @end table This example shows the difference: @@ -10552,12 +10781,13 @@ you can use the @code{strtonum()} function (@pxref{String Functions}) to convert the data into a number. Most of the time, you will want to use octal or hexadecimal constants -when working with the built-in bit manipulation functions; -see @ref{Bitwise Functions}, +when working with the built-in bit-manipulation functions; +see @DBREF{Bitwise Functions} for more information. -Unlike some early C implementations, @samp{8} and @samp{9} are not valid -in octal constants; e.g., @command{gawk} treats @samp{018} as decimal 18: +Unlike in some early C implementations, @samp{8} and @samp{9} are not +valid in octal constants. For example, @command{gawk} treats @samp{018} +as decimal 18: @example $ @kbd{gawk 'BEGIN @{ print "021 is", 021 ; print 018 @}'} @@ -10619,19 +10849,17 @@ $ @kbd{gawk 'BEGIN @{ printf "0x11 is <%s>\n", 0x11 @}'} @node Regexp Constants @subsubsection Regular Expression Constants -@c STARTOFRANGE rec @cindex regexp constants @cindex @code{~} (tilde), @code{~} operator @cindex tilde (@code{~}), @code{~} operator @cindex @code{!} (exclamation point), @code{!~} operator @cindex exclamation point (@code{!}), @code{!~} operator -A regexp constant is a regular expression description enclosed in +A @dfn{regexp constant} is a regular expression description enclosed in slashes, such as @code{@w{/^beginning and end$/}}. Most regexps used in @command{awk} programs are constant, but the @samp{~} and @samp{!~} matching operators can also match computed or dynamic regexps (which are typically just ordinary strings or variables that contain a regexp, -but could be a more complex expression). -@c ENDOFRANGE cnst +but could be more complex expressions). @node Using Constant Regexps @subsection Using Regular Expression Constants @@ -10643,7 +10871,7 @@ matched. However, regexp constants (such as @code{/foo/}) may be used like simple expressions. When a regexp constant appears by itself, it has the same meaning as if it appeared -in a pattern, i.e., @samp{($0 ~ /foo/)} +in a pattern (i.e., @samp{($0 ~ /foo/)}). @value{DARKCORNER} @xref{Expression Patterns}. This means that the following two code segments: @@ -10711,7 +10939,7 @@ the third argument of @code{split()} to be a regexp constant, but some older implementations do not. @value{DARKCORNER} Because some built-in functions accept regexp constants as arguments, -it can be confusing when attempting to use regexp constants as arguments +confusion can arise when attempting to use regexp constants as arguments to user-defined functions (@pxref{User-defined}). For example: @example @@ -10737,19 +10965,18 @@ function mysub(pat, repl, str, global) In this example, the programmer wants to pass a regexp constant to the user-defined function @code{mysub()}, which in turn passes it on to either @code{sub()} or @code{gsub()}. However, what really happens is that -the @code{pat} parameter is either one or zero, depending upon whether +the @code{pat} parameter is assigned a value of either one or zero, depending upon whether or not @code{$0} matches @code{/hi/}. @command{gawk} issues a warning when it sees a regexp constant used as -a parameter to a user-defined function, since passing a truth value in +a parameter to a user-defined function, because passing a truth value in this way is probably not what was intended. -@c ENDOFRANGE rec @node Variables @subsection Variables @cindex variables, user-defined @cindex user-defined, variables -Variables are ways of storing values at one point in your program for +@dfn{Variables} are ways of storing values at one point in your program for use later in another part of your program. They can be manipulated entirely within the program text, and they can also be assigned values on the @command{awk} command line. @@ -10777,18 +11004,18 @@ are distinct variables. A variable name is a valid expression by itself; it represents the variable's current value. Variables are given new values with @dfn{assignment operators}, @dfn{increment operators}, and -@dfn{decrement operators}. -@xref{Assignment Ops}. +@dfn{decrement operators} +(@pxref{Assignment Ops}). In addition, the @code{sub()} and @code{gsub()} functions can -change a variable's value, and the @code{match()}, @code{split()} +change a variable's value, and the @code{match()}, @code{split()}, and @code{patsplit()} functions can change the contents of their -array parameters. @xref{String Functions}. +array parameters (@pxref{String Functions}). @cindex variables, built-in @cindex variables, initializing A few variables have special built-in meanings, such as @code{FS} (the -field separator), and @code{NF} (the number of fields in the current input -record). @xref{Built-in Variables}, for a list of the predefined variables. +field separator) and @code{NF} (the number of fields in the current input +record). @DBXREF{Built-in Variables} for a list of the predefined variables. These predefined variables can be used and assigned just like all other variables, but their values are also used or changed automatically by @command{awk}. All predefined variables' names are entirely uppercase. @@ -10829,7 +11056,7 @@ as in the following: the variable is set at the very beginning, even before the @code{BEGIN} rules execute. The @option{-v} option and its assignment must precede all the @value{FN} arguments, as well as the program text. -(@xref{Options}, for more information about +(@DBXREF{Options} for more information about the @option{-v} option.) Otherwise, the variable assignment is performed at a time determined by its position among the input file arguments---after the processing of the @@ -10869,7 +11096,7 @@ sequences @node Conversion @subsection Conversion of Strings and Numbers -Number to string and string to number conversion are generally +Number-to-string and string-to-number conversion are generally straightforward. There can be subtleties to be aware of; this @value{SECTION} discusses this important facet of @command{awk}. @@ -10880,7 +11107,7 @@ this @value{SECTION} discusses this important facet of @command{awk}. @end menu @node Strings And Numbers -@subsubsection How @command{awk} Converts Between Strings And Numbers +@subsubsection How @command{awk} Converts Between Strings and Numbers @cindex converting, strings to numbers @cindex strings, converting @@ -10911,7 +11138,7 @@ string, concatenate that number with the empty string, @code{""}. To force a string to be converted to a number, add zero to that string. A string is converted to a number by interpreting any numeric prefix of the string as numerals: -@code{"2.5"} converts to 2.5, @code{"1e3"} converts to 1000, and @code{"25fix"} +@code{"2.5"} converts to 2.5, @code{"1e3"} converts to 1,000, and @code{"25fix"} has a numeric value of 25. Strings that can't be interpreted as valid numbers convert to zero. @@ -10951,10 +11178,10 @@ b = a "" @code{b} has the value @code{"12"}, not @code{"12.00"}. @value{DARKCORNER} -@cindex sidebar, Pre-POSIX @command{awk} Used @code{OFMT} For String Conversion +@cindex sidebar, Pre-POSIX @command{awk} Used @code{OFMT} for String Conversion @ifdocbook @docbook -<sidebar><title>Pre-POSIX @command{awk} Used @code{OFMT} For String Conversion</title> +<sidebar><title>Pre-POSIX @command{awk} Used @code{OFMT} for String Conversion</title> @end docbook @cindex POSIX @command{awk}, @code{OFMT} variable and @@ -10968,7 +11195,7 @@ specifies the output format to use when printing numbers with @code{print}. conversion from the semantics of printing. Both @code{CONVFMT} and @code{OFMT} have the same default value: @code{"%.6g"}. In the vast majority of cases, old @command{awk} programs do not change their behavior. -@xref{Print}, for more information on the @code{print} statement. +@DBXREF{Print} for more information on the @code{print} statement. @docbook </sidebar> @@ -10977,7 +11204,7 @@ of cases, old @command{awk} programs do not change their behavior. @ifnotdocbook @cartouche -@center @b{Pre-POSIX @command{awk} Used @code{OFMT} For String Conversion} +@center @b{Pre-POSIX @command{awk} Used @code{OFMT} for String Conversion} @cindex POSIX @command{awk}, @code{OFMT} variable and @@ -10991,7 +11218,7 @@ specifies the output format to use when printing numbers with @code{print}. conversion from the semantics of printing. Both @code{CONVFMT} and @code{OFMT} have the same default value: @code{"%.6g"}. In the vast majority of cases, old @command{awk} programs do not change their behavior. -@xref{Print}, for more information on the @code{print} statement. +@DBXREF{Print} for more information on the @code{print} statement. @end cartouche @end ifnotdocbook @@ -11014,7 +11241,7 @@ The POSIX standard says that @command{awk} always uses the period as the decimal point when reading the @command{awk} program source code, and for command-line variable assignments (@pxref{Other Arguments}). However, when interpreting input data, for @code{print} and @code{printf} output, -and for number to string conversion, the local decimal point character +and for number-to-string conversion, the local decimal point character is used. @value{DARKCORNER} In all cases, numbers in source code and in input data cannot have a thousands separator. Here are some examples indicating the difference in behavior, on a GNU/Linux system: @@ -11039,12 +11266,12 @@ as the full number including the fractional part, 4.321. Some earlier versions of @command{gawk} fully complied with this aspect of the standard. However, many users in non-English locales complained -about this behavior, since their data used a period as the decimal +about this behavior, because their data used a period as the decimal point, so the default behavior was restored to use a period as the decimal point character. You can use the @option{--use-lc-numeric} option (@pxref{Options}) to force @command{gawk} to use the locale's decimal point character. (@command{gawk} also uses the locale's decimal -point character when in POSIX mode, either via @option{--posix}, or the +point character when in POSIX mode, either via @option{--posix} or the @env{POSIXLY_CORRECT} environment variable, as shown previously.) @ref{table-locale-affects} describes the cases in which the locale's decimal @@ -11052,7 +11279,7 @@ point character is used and when a period is used. Some of these features have not been described yet. @float Table,table-locale-affects -@caption{Locale Decimal Point versus A Period} +@caption{Locale decimal point versus a period} @multitable @columnfractions .15 .20 .45 @headitem Feature @tab Default @tab @option{--posix} or @option{--use-lc-numeric} @item @code{%'g} @tab Use locale @tab Use locale @@ -11062,15 +11289,15 @@ features have not been described yet. @end multitable @end float -Finally, modern day formal standards and IEEE standard floating point +Finally, modern-day formal standards and the IEEE standard floating-point representation can have an unusual but important effect on the way @command{gawk} converts some special string values to numbers. The details are presented in @ref{POSIX Floating Point Problems}. @node All Operators -@section Operators: Doing Something With Values +@section Operators: Doing Something with Values -This @value{SECTION} introduces the @dfn{operators} which make use +This @value{SECTION} introduces the @dfn{operators} that make use of the values provided by constants and variables. @menu @@ -11147,7 +11374,7 @@ Multiplication. Division; because all numbers in @command{awk} are floating-point numbers, the result is @emph{not} rounded to an integer---@samp{3 / 4} has the value 0.75. (It is a common mistake, especially for C programmers, -to forget that @emph{all} numbers in @command{awk} are floating-point, +to forget that @emph{all} numbers in @command{awk} are floating point, and that division of integer-looking constants produces a real number, not an integer.) @@ -11232,7 +11459,7 @@ $ @kbd{awk '@{ print "Field number one:" $1 @}' mail-list} @cindex troubleshooting, string concatenation Because string concatenation does not have an explicit operator, it is -often necessary to insure that it happens at the right time by using +often necessary to ensure that it happens at the right time by using parentheses to enclose the items to concatenate. For example, you might expect that the following code fragment concatenates @code{file} and @code{name}: @@ -11248,7 +11475,7 @@ print "something meaningful" > file name @noindent This produces a syntax error with some versions of Unix @command{awk}.@footnote{It happens that BWK -@command{awk}, @command{gawk} and @command{mawk} all ``get it right,'' +@command{awk}, @command{gawk}, and @command{mawk} all ``get it right,'' but you should not rely on this.} It is necessary to use the following: @@ -11337,11 +11564,8 @@ you're never quite sure what you'll get. @node Assignment Ops @subsection Assignment Expressions -@c STARTOFRANGE asop @cindex assignment operators -@c STARTOFRANGE opas @cindex operators, assignment -@c STARTOFRANGE exas @cindex expressions, assignment @cindex @code{=} (equals sign), @code{=} operator @cindex equals sign (@code{=}), @code{=} operator @@ -11494,7 +11718,14 @@ The indices of @code{bar} are practically guaranteed to be different, because @code{rand()} returns different values each time it is called. (Arrays and the @code{rand()} function haven't been covered yet. @xref{Arrays}, -and see @ref{Numeric Functions}, for more information). +and +@ifnotdocbook +@DBPXREF{Numeric Functions} +@end ifnotdocbook +@ifdocbook +@DBREF{Numeric Functions} +@end ifdocbook +for more information.) This example illustrates an important fact about assignment operators: the lefthand expression is only evaluated @emph{once}. @@ -11527,7 +11758,7 @@ to a number. @cindex @code{*} (asterisk), @code{**=} operator @cindex asterisk (@code{*}), @code{**=} operator @float Table,table-assign-ops -@caption{Arithmetic Assignment Operators} +@caption{Arithmetic assignment operators} @multitable @columnfractions .30 .70 @headitem Operator @tab Effect @item @var{lvalue} @code{+=} @var{increment} @tab Add @var{increment} to the value of @var{lvalue}. @@ -11539,7 +11770,7 @@ to a number. @cindex extensions, common@comma{} @code{**=} operator @cindex @command{awk} language, POSIX version @cindex POSIX @command{awk} -@item @var{lvalue} @code{^=} @var{power} @tab +@item @var{lvalue} @code{^=} @var{power} @tab Raise @var{lvalue} to the power @var{power}. @item @var{lvalue} @code{**=} @var{power} @tab Raise @var{lvalue} to the power @var{power}. @value{COMMONEXT} @end multitable @end float @@ -11576,7 +11807,7 @@ This is most notable in some commercial @command{awk} versions. For example: @example -$ awk /==/ /dev/null +$ @kbd{awk /==/ /dev/null} @error{} awk: syntax error at source line 1 @error{} context is @error{} >>> /= <<< @@ -11622,7 +11853,7 @@ This is most notable in some commercial @command{awk} versions. For example: @example -$ awk /==/ /dev/null +$ @kbd{awk /==/ /dev/null} @error{} awk: syntax error at source line 1 @error{} context is @error{} >>> /= <<< @@ -11640,16 +11871,11 @@ awk '/[=]=/' /dev/null and @command{mawk} also do not. @end cartouche @end ifnotdocbook -@c ENDOFRANGE exas -@c ENDOFRANGE opas -@c ENDOFRANGE asop @node Increment Ops @subsection Increment and Decrement Operators -@c STARTOFRANGE inop @cindex increment operators -@c STARTOFRANGE opde @cindex operators, decrement/increment @dfn{Increment} and @dfn{decrement operators} increase or decrease the value of a variable by one. An assignment operator can do the same thing, so @@ -11673,13 +11899,14 @@ has the value four, but it changes the value of @code{foo} to five. In other words, the operator returns the old value of the variable, but with the side effect of incrementing it. +@c FIXME: Use @sup here for superscript The post-increment @samp{foo++} is nearly the same as writing @samp{(foo += 1) - 1}. It is not perfectly equivalent because all numbers in -@command{awk} are floating-point---in floating-point, @samp{foo + 1 - 1} does +@command{awk} are floating point---in floating point, @samp{foo + 1 - 1} does not necessarily equal @code{foo}. But the difference is minute as long as you stick to numbers that are fairly small (less than @iftex -@math{10^12}). +@math{10^{12}}). @end iftex @ifnottex @ifnotdocbook @@ -11697,7 +11924,6 @@ just like variables. (Use @samp{$(i++)} when you want to do a field reference and a variable increment at the same time. The parentheses are necessary because of the precedence of the field reference operator @samp{$}.) -@c STARTOFRANGE deop @cindex decrement operators The decrement operator @samp{--} works just like @samp{++}, except that it subtracts one instead of adding it. As with @samp{++}, it can be used before @@ -11742,8 +11968,8 @@ like @samp{@var{lvalue}++}, but instead of adding, it subtracts.) @cindex evaluation order @cindex Marx, Groucho @quotation -@i{Doctor, doctor! It hurts when I do this!@* -So don't do that!} +@i{Doctor, it hurts when I do this!@* +Then don't do that!} @author Groucho Marx @end quotation @@ -11767,7 +11993,7 @@ print b @cindex side effects In other words, when do the various side effects prescribed by the postfix operators (@samp{b++}) take effect? -When side effects happen is @dfn{implementation defined}. +When side effects happen is @dfn{implementation-defined}. In other words, it is up to the particular version of @command{awk}. The result for the first example may be 12 or 13, and for the second, it may be 22 or 23. @@ -11794,8 +12020,8 @@ You should avoid such things in your own programs. @cindex evaluation order @cindex Marx, Groucho @quotation -@i{Doctor, doctor! It hurts when I do this!@* -So don't do that!} +@i{Doctor, it hurts when I do this!@* +Then don't do that!} @author Groucho Marx @end quotation @@ -11819,7 +12045,7 @@ print b @cindex side effects In other words, when do the various side effects prescribed by the postfix operators (@samp{b++}) take effect? -When side effects happen is @dfn{implementation defined}. +When side effects happen is @dfn{implementation-defined}. In other words, it is up to the particular version of @command{awk}. The result for the first example may be 12 or 13, and for the second, it may be 22 or 23. @@ -11831,14 +12057,11 @@ You should avoid such things in your own programs. @c in the mirror in the morning. @end cartouche @end ifnotdocbook -@c ENDOFRANGE inop -@c ENDOFRANGE opde -@c ENDOFRANGE deop @node Truth Values and Conditions @section Truth Values and Conditions -In certain contexts, expression values also serve as ``truth values;'' i.e., +In certain contexts, expression values also serve as ``truth values''; i.e., they determine what should happen next as the program runs. This @value{SECTION} describes how @command{awk} defines ``true'' and ``false'' and how values are compared. @@ -11895,22 +12118,21 @@ the string constant @code{"0"} is actually true, because it is non-null. @subsection Variable Typing and Comparison Expressions @quotation @i{The Guide is definitive. Reality is frequently inaccurate.} -@author The Hitchhiker's Guide to the Galaxy +@author Douglas Adams, @cite{The Hitchhiker's Guide to the Galaxy} @end quotation +@c 2/2015: Antonio Colombo points out that this is really from +@c The Restaurant at the End of the Universe. But I'm going to +@c leave it alone. -@c STARTOFRANGE comex @cindex comparison expressions -@c STARTOFRANGE excom @cindex expressions, comparison @cindex expressions, matching, See comparison expressions @cindex matching, expressions, See comparison expressions @cindex relational operators, See comparison operators @cindex operators, relational, See operators@comma{} comparison -@c STARTOFRANGE varting @cindex variable typing -@c STARTOFRANGE vartypc @cindex variables, types of, comparison expressions and -Unlike other programming languages, @command{awk} variables do not have a +Unlike in other programming languages, in @command{awk} variables do not have a fixed type. Instead, they can be either a number or a string, depending upon the value that is assigned to them. We look now at how variables are typed, and how @command{awk} @@ -11923,7 +12145,7 @@ compares variables. @end menu @node Variable Typing -@subsubsection String Type Versus Numeric Type +@subsubsection String Type versus Numeric Type @cindex numeric, strings @cindex strings, numeric @@ -11939,20 +12161,20 @@ Variable typing follows these rules: @itemize @value{BULLET} @item -A numeric constant or the result of a numeric operation has the @var{numeric} +A numeric constant or the result of a numeric operation has the @dfn{numeric} attribute. @item -A string constant or the result of a string operation has the @var{string} +A string constant or the result of a string operation has the @dfn{string} attribute. @item Fields, @code{getline} input, @code{FILENAME}, @code{ARGV} elements, @code{ENVIRON} elements, and the elements of an array created by -@code{match()}, @code{split()} and @code{patsplit()} that are numeric -strings have the @var{strnum} attribute. Otherwise, they have -the @var{string} attribute. Uninitialized variables also have the -@var{strnum} attribute. +@code{match()}, @code{split()}, and @code{patsplit()} that are numeric +strings have the @dfn{strnum} attribute. Otherwise, they have +the @dfn{string} attribute. Uninitialized variables also have the +@dfn{strnum} attribute. @item Attributes propagate across assignments but are not changed by @@ -12096,13 +12318,13 @@ constant, then a string comparison is performed. Otherwise, a numeric comparison is performed. This point bears additional emphasis: All user input is made of characters, -and so is first and foremost of @var{string} type; input strings -that look numeric are additionally given the @var{strnum} attribute. +and so is first and foremost of string type; input strings +that look numeric are additionally given the strnum attribute. Thus, the six-character input string @w{@samp{ +3.14}} receives the -@var{strnum} attribute. In contrast, the eight characters +strnum attribute. In contrast, the eight characters @w{@code{" +3.14"}} appearing in program text comprise a string constant. The following examples print @samp{1} when the comparison between -the two different constants is true, @samp{0} otherwise: +the two different constants is true, and @samp{0} otherwise: @c 22.9.2014: Tested with mawk and BWK awk, got same results. @example @@ -12150,18 +12372,18 @@ operators}, which are a superset of those in C. @cindex exclamation point (@code{!}), @code{!~} operator @cindex @code{in} operator @float Table,table-relational-ops -@caption{Relational Operators} +@caption{Relational operators} @multitable @columnfractions .25 .75 @headitem Expression @tab Result -@item @var{x} @code{<} @var{y} @tab True if @var{x} is less than @var{y}. -@item @var{x} @code{<=} @var{y} @tab True if @var{x} is less than or equal to @var{y}. -@item @var{x} @code{>} @var{y} @tab True if @var{x} is greater than @var{y}. -@item @var{x} @code{>=} @var{y} @tab True if @var{x} is greater than or equal to @var{y}. -@item @var{x} @code{==} @var{y} @tab True if @var{x} is equal to @var{y}. -@item @var{x} @code{!=} @var{y} @tab True if @var{x} is not equal to @var{y}. -@item @var{x} @code{~} @var{y} @tab True if the string @var{x} matches the regexp denoted by @var{y}. -@item @var{x} @code{!~} @var{y} @tab True if the string @var{x} does not match the regexp denoted by @var{y}. -@item @var{subscript} @code{in} @var{array} @tab True if the array @var{array} has an element with the subscript @var{subscript}. +@item @var{x} @code{<} @var{y} @tab True if @var{x} is less than @var{y} +@item @var{x} @code{<=} @var{y} @tab True if @var{x} is less than or equal to @var{y} +@item @var{x} @code{>} @var{y} @tab True if @var{x} is greater than @var{y} +@item @var{x} @code{>=} @var{y} @tab True if @var{x} is greater than or equal to @var{y} +@item @var{x} @code{==} @var{y} @tab True if @var{x} is equal to @var{y} +@item @var{x} @code{!=} @var{y} @tab True if @var{x} is not equal to @var{y} +@item @var{x} @code{~} @var{y} @tab True if the string @var{x} matches the regexp denoted by @var{y} +@item @var{x} @code{!~} @var{y} @tab True if the string @var{x} does not match the regexp denoted by @var{y} +@item @var{subscript} @code{in} @var{array} @tab True if the array @var{array} has an element with the subscript @var{subscript} @end multitable @end float @@ -12199,24 +12421,24 @@ The following list of expressions illustrates the kinds of comparisons @table @code @item 1.5 <= 2.0 -numeric comparison (true) +Numeric comparison (true) @item "abc" >= "xyz" -string comparison (false) +String comparison (false) @item 1.5 != " +2" -string comparison (true) +String comparison (true) @item "1e2" < "3" -string comparison (true) +String comparison (true) @item a = 2; b = "2" @itemx a == b -string comparison (true) +String comparison (true) @item a = 2; b = " +2" @itemx a == b -string comparison (false) +String comparison (false) @end table In this example: @@ -12232,7 +12454,7 @@ $ @kbd{echo 1e2 3 | awk '@{ print ($1 < $2) ? "true" : "false" @}'} @noindent the result is @samp{false} because both @code{$1} and @code{$2} are user input. They are numeric strings---therefore both have -the @var{strnum} attribute, dictating a numeric comparison. +the strnum attribute, dictating a numeric comparison. The purpose of the comparison rules and the use of numeric strings is to attempt to produce the behavior that is ``least surprising,'' while still ``doing the right thing.'' @@ -12269,7 +12491,7 @@ dynamic regexp (@pxref{Regexp Usage}; also @cindex @command{awk}, regexp constants and @cindex regexp constants A constant regular -expression in slashes by itself is also an expression. The regexp +expression in slashes by itself is also an expression. @code{/@var{regexp}/} is an abbreviation for the following comparison expression: @example @@ -12283,7 +12505,7 @@ One special place where @code{/foo/} is @emph{not} an abbreviation for where this is discussed in more detail. @node POSIX String Comparison -@subsubsection String Comparison With POSIX Rules +@subsubsection String Comparison with POSIX Rules The POSIX standard says that string comparison is performed based on the locale's @dfn{collating order}. This is the order in which @@ -12291,7 +12513,7 @@ characters sort, as defined by the locale (for more discussion, @pxref{Locales}). This order is usually very different from the results obtained when doing straight character-by-character comparison.@footnote{Technically, string comparison is supposed -to behave the same way as if the strings are compared with the C +to behave the same way as if the strings were compared with the C @code{strcoll()} function.} Because this behavior differs considerably from existing practice, @@ -12308,19 +12530,13 @@ $ @kbd{gawk --posix 'BEGIN @{ printf("ABC < abc = %s\n",} @print{} ABC < abc = FALSE @end example -@c ENDOFRANGE comex -@c ENDOFRANGE excom -@c ENDOFRANGE vartypc -@c ENDOFRANGE varting @node Boolean Ops @subsection Boolean Expressions @cindex and Boolean-logic operator @cindex or Boolean-logic operator @cindex not Boolean-logic operator -@c STARTOFRANGE exbo @cindex expressions, Boolean -@c STARTOFRANGE boex @cindex Boolean expressions @cindex operators, Boolean, See Boolean expressions @cindex Boolean operators, See Boolean expressions @@ -12404,7 +12620,7 @@ BEGIN @{ if (! ("HOME" in ENVIRON)) @cindex vertical bar (@code{|}), @code{||} operator The @samp{&&} and @samp{||} operators are called @dfn{short-circuit} operators because of the way they work. Evaluation of the full expression -is ``short-circuited'' if the result can be determined part way through +is ``short-circuited'' if the result can be determined partway through its evaluation. @cindex line continuations @@ -12466,8 +12682,6 @@ next record, and start processing the rules over again at the top. The reason it's there is to avoid printing the bracketing @samp{START} and @samp{END} lines. @end quotation -@c ENDOFRANGE exbo -@c ENDOFRANGE boex @node Conditional Exp @subsection Conditional Expressions @@ -12478,8 +12692,8 @@ The reason it's there is to avoid printing the bracketing A @dfn{conditional expression} is a special kind of expression that has three operands. It allows you to use one expression's value to select one of two other expressions. -The conditional expression is the same as in the C language, -as shown here: +The conditional expression in @command{awk} is the same as in the C +language, as shown here: @example @var{selector} ? @var{if-true-exp} : @var{if-false-exp} @@ -12488,8 +12702,8 @@ as shown here: @noindent There are three subexpressions. The first, @var{selector}, is always computed first. If it is ``true'' (not zero or not null), then -@var{if-true-exp} is computed next and its value becomes the value of -the whole expression. Otherwise, @var{if-false-exp} is computed next +@var{if-true-exp} is computed next, and its value becomes the value of +the whole expression. Otherwise, @var{if-false-exp} is computed next, and its value becomes the value of the whole expression. For example, the following expression produces the absolute value of @code{x}: @@ -12537,15 +12751,15 @@ ask for it by name at any point in the program. For example, the function @code{sqrt()} computes the square root of a number. @cindex functions, built-in -A fixed set of functions are @dfn{built-in}, which means they are +A fixed set of functions are @dfn{built in}, which means they are available in every @command{awk} program. The @code{sqrt()} function is one -of these. @xref{Built-in}, for a list of built-in +of these. @DBXREF{Built-in} for a list of built-in functions and their descriptions. In addition, you can define functions for use in your program. -@xref{User-defined}, +@DBXREF{User-defined} for instructions on how to do this. Finally, @command{gawk} lets you write functions in C or C++ -that may be called from your program: see @ref{Dynamic Extensions}. +that may be called from your program (@pxref{Dynamic Extensions}). @cindex arguments, in function calls The way to use a function is with a @dfn{function call} expression, @@ -12564,7 +12778,7 @@ rand() @ii{no arguments} @cindex troubleshooting, function call syntax @quotation CAUTION -Do not put any space between the function name and the open-parenthesis! +Do not put any space between the function name and the opening parenthesis! A user-defined function name looks just like the name of a variable---a space would make the expression look like concatenation of a variable with an expression inside parentheses. @@ -12585,7 +12799,7 @@ Some of the built-in functions have one or more optional arguments. If those arguments are not supplied, the functions use a reasonable default value. -@xref{Built-in}, for full details. If arguments +@DBXREF{Built-in} for full details. If arguments are omitted in calls to user-defined functions, then those arguments are treated as local variables. Such local variables act like the empty string if referenced where a string value is required, @@ -12646,9 +12860,7 @@ $ @kbd{awk -f matchit.awk} @node Precedence @section Operator Precedence (How Operators Nest) -@c STARTOFRANGE prec @cindex precedence -@c STARTOFRANGE oppr @cindex operators, precedence @dfn{Operator precedence} determines how operators are grouped when @@ -12713,7 +12925,7 @@ Increment, decrement. @cindex @code{*} (asterisk), @code{**} operator @cindex asterisk (@code{*}), @code{**} operator @item @code{^ **} -Exponentiation. These operators group right-to-left. +Exponentiation. These operators group right to left. @cindex @code{+} (plus sign), @code{+} operator @cindex plus sign (@code{+}), @code{+} operator @@ -12740,7 +12952,7 @@ Multiplication, division, remainder. @item @code{+ -} Addition, subtraction. -@item String Concatenation +@item String concatenation There is no special symbol for concatenation. The operands are simply written side by side (@pxref{Concatenation}). @@ -12779,7 +12991,7 @@ statements belong to the statement level, not to expressions. The redirection does not produce an expression that could be the operand of another operator. As a result, it does not make sense to use a redirection operator near another operator of lower precedence without -parentheses. Such combinations (for example, @samp{print foo > a ? b : c}), +parentheses. Such combinations (e.g., @samp{print foo > a ? b : c}) result in syntax errors. The correct way to write this statement is @samp{print foo > (a ? b : c)}. @@ -12797,17 +13009,17 @@ Array membership. @cindex @code{&} (ampersand), @code{&&} operator @cindex ampersand (@code{&}), @code{&&} operator @item @code{&&} -Logical ``and''. +Logical ``and.'' @cindex @code{|} (vertical bar), @code{||} operator @cindex vertical bar (@code{|}), @code{||} operator @item @code{||} -Logical ``or''. +Logical ``or.'' @cindex @code{?} (question mark), @code{?:} operator @cindex question mark (@code{?}), @code{?:} operator @item @code{?:} -Conditional. This operator groups right-to-left. +Conditional. This operator groups right to left. @cindex @code{+} (plus sign), @code{+=} operator @cindex plus sign (@code{+}), @code{+=} operator @@ -12824,7 +13036,7 @@ Conditional. This operator groups right-to-left. @cindex @code{^} (caret), @code{^=} operator @cindex caret (@code{^}), @code{^=} operator @item @code{= += -= *= /= %= ^= **=} -Assignment. These operators group right-to-left. +Assignment. These operators group right to left. @end table @cindex POSIX @command{awk}, @code{**} operator and @@ -12833,11 +13045,9 @@ Assignment. These operators group right-to-left. The @samp{|&}, @samp{**}, and @samp{**=} operators are not specified by POSIX. For maximum portability, do not use them. @end quotation -@c ENDOFRANGE prec -@c ENDOFRANGE oppr @node Locales -@section Where You Are Makes A Difference +@section Where You Are Makes a Difference @cindex locale, definition of Modern systems support the notion of @dfn{locales}: a way to tell the @@ -12857,8 +13067,8 @@ character}, to find the record terminator. Locales can affect how dates and times are formatted (@pxref{Time Functions}). For example, a common way to abbreviate the date September -4, 2015 in the United States is ``9/4/15.'' In many countries in -Europe, however, it is abbreviated ``4.9.15.'' Thus, the @code{%x} +4, 2015, in the United States is ``9/4/15.'' In many countries in +Europe, however, it is abbreviated ``4.9.15.'' Thus, the @samp{%x} specification in a @code{"US"} locale might produce @samp{9/4/15}, while in a @code{"EUROPE"} locale, it might produce @samp{4.9.15}. @@ -12876,13 +13086,13 @@ in @ref{Conversion}. @itemize @value{BULLET} @item Expressions are the basic elements of computation in programs. They are -built from constants, variables, function calls and combinations of the +built from constants, variables, function calls, and combinations of the various kinds of values with operators. @item @command{awk} supplies three kinds of constants: numeric, string, and regexp. @command{gawk} lets you specify numeric constants in octal -and hexadecimal (bases 8 and 16) in addition to decimal (base 10). +and hexadecimal (bases 8 and 16) as well as decimal (base 10). In certain contexts, a standalone regexp constant such as @code{/foo/} has the same meaning as @samp{$0 ~ /foo/}. @@ -12900,8 +13110,8 @@ Locales can influence the conversions. @item @command{awk} provides the usual arithmetic operators (addition, subtraction, multiplication, division, modulus), and unary plus and minus. -It also provides comparison operators, boolean operators, array membership -testing, and regexp +It also provides comparison operators, Boolean operators, an array membership +testing operator, and regexp matching operators. String concatenation is accomplished by placing two expressions next to each other; there is no explicit operator. The three-operand @samp{?:} operator provides an ``if-else'' test within @@ -12912,7 +13122,7 @@ Assignment operators provide convenient shorthands for common arithmetic operations. @item -In @command{awk}, a value is considered to be true if it is non-zero +In @command{awk}, a value is considered to be true if it is nonzero @emph{or} non-null. Otherwise, the value is false. @item @@ -12921,11 +13131,11 @@ lifetime. The type determines how it behaves in comparisons (string or numeric). @item -Function calls return a value which may be used as part of a larger +Function calls return a value that may be used as part of a larger expression. Expressions used to pass parameter values are fully evaluated before the function is called. @command{awk} provides -built-in and user-defined functions; this is described later on in this -@value{DOCUMENT}. +built-in and user-defined functions; this is described in +@ref{Functions}. @item Operator precedence specifies the order in which operations are performed, @@ -12938,11 +13148,9 @@ program, and occasionally the format for data read as input. @end itemize -@c ENDOFRANGE exps @node Patterns and Actions @chapter Patterns, Actions, and Variables -@c STARTOFRANGE pat @cindex patterns As you have already seen, each @command{awk} statement consists of @@ -12950,7 +13158,7 @@ a pattern with an associated action. This @value{CHAPTER} describes how you build patterns and actions, what kinds of things you can do within actions, and @command{awk}'s predefined variables. -The pattern-action rules and the statements available for use +The pattern--action rules and the statements available for use within actions form the core of @command{awk} programming. In a sense, everything covered up to here has been the foundation @@ -13086,8 +13294,8 @@ $ @kbd{awk '$1 ~ /li/ @{ print $2 @}' mail-list} @cindex regexp constants, as patterns @cindex patterns, regexp constants as A regexp constant as a pattern is also a special case of an expression -pattern. The expression @samp{/li/} has the value one if @samp{li} -appears in the current input record. Thus, as a pattern, @samp{/li/} +pattern. The expression @code{/li/} has the value one if @samp{li} +appears in the current input record. Thus, as a pattern, @code{/li/} matches any record containing @samp{li}. @cindex Boolean expressions, as patterns @@ -13138,11 +13346,11 @@ The subexpressions of a Boolean operator in a pattern can be constant regular expressions, comparisons, or any other @command{awk} expressions. Range patterns are not expressions, so they cannot appear inside Boolean patterns. Likewise, the special patterns @code{BEGIN}, @code{END}, -@code{BEGINFILE} and @code{ENDFILE}, +@code{BEGINFILE}, and @code{ENDFILE}, which never match any input record, are not expressions and cannot appear inside Boolean patterns. -The precedence of the different operators which can appear in +The precedence of the different operators that can appear in patterns is described in @ref{Precedence}. @node Ranges @@ -13168,7 +13376,7 @@ prints every record in @file{myfile} between @samp{on}/@samp{off} pairs, inclusi A range pattern starts out by matching @var{begpat} against every input record. When a record matches @var{begpat}, the range pattern is -@dfn{turned on} and the range pattern matches this record as well. As long as +@dfn{turned on}, and the range pattern matches this record as well. As long as the range pattern stays turned on, it automatically matches every input record read. The range pattern also matches @var{endpat} against every input record; when this succeeds, the range pattern is @dfn{turned off} again @@ -13239,9 +13447,7 @@ a range pattern. @value{DARKCORNER} @node BEGIN/END @subsection The @code{BEGIN} and @code{END} Special Patterns -@c STARTOFRANGE beg @cindex @code{BEGIN} pattern -@c STARTOFRANGE end @cindex @code{END} pattern All the patterns described so far are for matching input records. The @code{BEGIN} and @code{END} special patterns are different. @@ -13249,7 +13455,7 @@ They supply startup and cleanup actions for @command{awk} programs. @code{BEGIN} and @code{END} rules must have actions; there is no default action for these rules because there is no current record when they run. @code{BEGIN} and @code{END} rules are often referred to as -``@code{BEGIN} and @code{END} blocks'' by long-time @command{awk} +``@code{BEGIN} and @code{END} blocks'' by longtime @command{awk} programmers. @menu @@ -13280,7 +13486,7 @@ $ @kbd{awk '} This program finds the number of records in the input file @file{mail-list} that contain the string @samp{li}. The @code{BEGIN} rule prints a title for the report. There is no need to use the @code{BEGIN} rule to -initialize the counter @code{n} to zero, since @command{awk} does this +initialize the counter @code{n} to zero, as @command{awk} does this automatically (@pxref{Variables}). The second rule increments the variable @code{n} every time a record containing the pattern @samp{li} is read. The @code{END} rule @@ -13308,13 +13514,13 @@ The order in which library functions are named on the command line controls the order in which their @code{BEGIN} and @code{END} rules are executed. Therefore, you have to be careful when writing such rules in library files so that the order in which they are executed doesn't matter. -@xref{Options}, for more information on +@DBXREF{Options} for more information on using library functions. @xref{Library Functions}, for a number of useful library functions. If an @command{awk} program has only @code{BEGIN} rules and no -other rules, then the program exits after the @code{BEGIN} rule is +other rules, then the program exits after the @code{BEGIN} rules are run.@footnote{The original version of @command{awk} kept reading and ignoring input until the end of the file was seen.} However, if an @code{END} rule exists, then the input is read, even if there are @@ -13342,7 +13548,7 @@ Another way is simply to assign a value to @code{$0}. @cindex @code{print} statement, @code{BEGIN}/@code{END} patterns and @cindex @code{BEGIN} pattern, @code{print} statement and @cindex @code{END} pattern, @code{print} statement and -The second point is similar to the first but from the other direction. +The second point is similar to the first, but from the other direction. Traditionally, due largely to implementation issues, @code{$0} and @code{NF} were @emph{undefined} inside an @code{END} rule. The POSIX standard specifies that @code{NF} is available in an @code{END} @@ -13357,11 +13563,11 @@ of Unix @command{awk} do not. The third point follows from the first two. The meaning of @samp{print} inside a @code{BEGIN} or @code{END} rule is the same as always: @samp{print $0}. If @code{$0} is the null string, then this prints an -empty record. Many long time @command{awk} programmers use an unadorned +empty record. Many longtime @command{awk} programmers use an unadorned @samp{print} in @code{BEGIN} and @code{END} rules, to mean @samp{@w{print ""}}, relying on @code{$0} being null. Although one might generally get away with this in @code{BEGIN} rules, it is a very bad idea in @code{END} rules, -at least in @command{gawk}. It is also poor style, since if an empty +at least in @command{gawk}. It is also poor style, because if an empty line is needed in the output, the program should print one explicitly. @cindex @code{next} statement, @code{BEGIN}/@code{END} patterns and @@ -13371,11 +13577,14 @@ line is needed in the output, the program should print one explicitly. Finally, the @code{next} and @code{nextfile} statements are not allowed in a @code{BEGIN} rule, because the implicit read-a-record-and-match-against-the-rules loop has not started yet. Similarly, those statements -are not valid in an @code{END} rule, since all the input has been read. -(@xref{Next Statement}, and see -@ref{Nextfile Statement}.) -@c ENDOFRANGE beg -@c ENDOFRANGE end +are not valid in an @code{END} rule, because all the input has been read. +(@DBXREF{Next Statement} and +@ifnotdocbook +@DBPXREF{Nextfile Statement}.) +@end ifnotdocbook +@ifdocbook +@DBREF{Nextfile Statement}.) +@end ifdocbook @node BEGINFILE/ENDFILE @subsection The @code{BEGINFILE} and @code{ENDFILE} Special Patterns @@ -13428,7 +13637,7 @@ fatal error. @item If you have written extensions that modify the record handling (by -inserting an ``input parser,'' @pxref{Input Parsers}), you can invoke +inserting an ``input parser''; @pxref{Input Parsers}), you can invoke them at this point, before @command{gawk} has started processing the file. (This is a @emph{very} advanced feature, currently used only by the @uref{http://gawkextlib.sourceforge.net, @code{gawkextlib} project}.) @@ -13439,8 +13648,8 @@ the last record in an input file. For the last input file, it will be called before any @code{END} rules. The @code{ENDFILE} rule is executed even for empty input files. -Normally, when an error occurs when reading input in the normal input -processing loop, the error is fatal. However, if an @code{ENDFILE} +Normally, when an error occurs when reading input in the normal +input-processing loop, the error is fatal. However, if an @code{ENDFILE} rule is present, the error becomes non-fatal, and instead @code{ERRNO} is set. This makes it possible to catch and process I/O errors at the level of the @command{awk} program. @@ -13449,7 +13658,7 @@ level of the @command{awk} program. The @code{next} statement (@pxref{Next Statement}) is not allowed inside either a @code{BEGINFILE} or an @code{ENDFILE} rule. The @code{nextfile} statement is allowed only inside a -@code{BEGINFILE} rule, but not inside an @code{ENDFILE} rule. +@code{BEGINFILE} rule, not inside an @code{ENDFILE} rule. @cindex @code{getline} statement, @code{BEGINFILE}/@code{ENDFILE} patterns and The @code{getline} statement (@pxref{Getline}) is restricted inside @@ -13496,7 +13705,6 @@ awk '@{ print $1 @}' mail-list @noindent prints the first field of every record. -@c ENDOFRANGE pat @node Using Shell Variables @section Using Shell Variables in Programs @@ -13530,7 +13738,7 @@ The first part is double-quoted, which allows substitution of the @code{pattern} shell variable inside the quotes. The second part is single-quoted. -Variable substitution via quoting works, but can be potentially +Variable substitution via quoting works, but can potentially be messy. It requires a good understanding of the shell's quoting rules (@pxref{Quoting}), and it's often difficult to correctly @@ -13557,7 +13765,7 @@ The assignment @samp{-v pat="$pattern"} still requires double quotes, in case there is whitespace in the value of @code{$pattern}. The @command{awk} variable @code{pat} could be named @code{pattern} too, but that would be more confusing. Using a variable also -provides more flexibility, since the variable can be used anywhere inside +provides more flexibility, as the variable can be used anywhere inside the program---for printing, as an array subscript, or for any other use---without requiring the quoting tricks at every point in the program. @@ -13630,7 +13838,7 @@ is used in order to put several statements together in the body of an Use the @code{getline} command (@pxref{Getline}). Also supplied in @command{awk} are the @code{next} -statement (@pxref{Next Statement}), +statement (@pxref{Next Statement}) and the @code{nextfile} statement (@pxref{Nextfile Statement}). @@ -13645,11 +13853,8 @@ For deleting array elements. @node Statements @section Control Statements in Actions -@c STARTOFRANGE csta @cindex control statements -@c STARTOFRANGE acs @cindex statements, control, in actions -@c STARTOFRANGE accs @cindex actions, control statements in @dfn{Control statements}, such as @code{if}, @code{while}, and so on, @@ -13718,7 +13923,7 @@ else print "x is odd" @end example -In this example, if the expression @samp{x % 2 == 0} is true (that is, +In this example, if the expression @samp{x % 2 == 0} is true (i.e., if the value of @code{x} is evenly divisible by two), then the first @code{print} statement is executed; otherwise, the second @code{print} statement is executed. @@ -13792,13 +13997,13 @@ The body of this loop is a compound statement enclosed in braces, containing two statements. The loop works in the following manner: first, the value of @code{i} is set to one. Then, the @code{while} statement tests whether @code{i} is less than or equal to -three. This is true when @code{i} equals one, so the @code{i}-th +three. This is true when @code{i} equals one, so the @code{i}th field is printed. Then the @samp{i++} increments the value of @code{i} and the loop repeats. The loop terminates when @code{i} reaches four. A newline is not required between the condition and the -body; however using one makes the program clearer unless the body is a -compound statement or else is very simple. The newline after the open-brace +body; however, using one makes the program clearer unless the body is a +compound statement or else is very simple. The newline after the open brace that begins the compound statement is not required either, but the program is harder to read without it. @@ -13828,9 +14033,9 @@ while (@var{condition}) @end example @noindent -This statement does not execute @var{body} even once if the @var{condition} -is false to begin with. -The following is an example of a @code{do} statement: +This statement does not execute the @var{body} even once if the +@var{condition} is false to begin with. The following is an example of +a @code{do} statement: @example @{ @@ -13844,7 +14049,7 @@ The following is an example of a @code{do} statement: @noindent This program prints each input record 10 times. However, it isn't a very -realistic example, since in this case an ordinary @code{while} would do +realistic example, because in this case an ordinary @code{while} would do just as well. This situation reflects actual experience; only occasionally is there a real use for a @code{do} statement. @@ -13897,7 +14102,7 @@ their assignments as separate statements preceding the @code{for} loop.) The same is true of the @var{increment} part. Incrementing additional variables requires separate statements at the end of the loop. The C compound expression, using C's comma operator, is useful in -this context but it is not supported in @command{awk}. +this context, but it is not supported in @command{awk}. Most often, @var{increment} is an increment expression, as in the previous example. But this is not required; it can be any expression @@ -13941,7 +14146,7 @@ very common in loops. It can be easier to think of this counting as part of looping rather than as something to do inside the loop. @cindex @code{in} operator -There is an alternate version of the @code{for} loop, for iterating over +There is an alternative version of the @code{for} loop, for iterating over all the indices of an array: @example @@ -13950,7 +14155,7 @@ for (i in array) @end example @noindent -@xref{Scanning an Array}, +@DBXREF{Scanning an Array} for more information on this version of the @code{for} loop. @node Switch Statement @@ -13970,7 +14175,7 @@ are checked for a match in the order they are defined. If no suitable Each @code{case} contains a single constant, be it numeric, string, or regexp. The @code{switch} expression is evaluated, and then each -@code{case}'s constant is compared against the result in turn. The type of constant +@code{case}'s constant is compared against the result in turn. The type of constant determines the comparison: numeric or string do the usual comparisons. A regexp constant does a regular expression match against the string value of the original expression. The general form of the @code{switch} @@ -13988,7 +14193,7 @@ default: Control flow in the @code{switch} statement works as it does in C. Once a match to a given case is made, the case statement bodies execute until a @code{break}, -@code{continue}, @code{next}, @code{nextfile} or @code{exit} is encountered, +@code{continue}, @code{next}, @code{nextfile}, or @code{exit} is encountered, or the end of the @code{switch} statement itself. For example: @example @@ -14017,9 +14222,9 @@ while ((c = getopt(ARGC, ARGV, "aksx")) != -1) @{ @} @end example -Note that if none of the statements specified above halt execution +Note that if none of the statements specified here halt execution of a matched @code{case} statement, execution falls through to the -next @code{case} until execution halts. In the above example, the +next @code{case} until execution halts. In this example, the @code{case} for @code{"?"} falls through to the @code{default} case, which is to call a function named @code{usage()}. (The @code{getopt()} function being called here is @@ -14040,12 +14245,12 @@ numbers: # find smallest divisor of num @{ num = $1 - for (div = 2; div * div <= num; div++) @{ - if (num % div == 0) + for (divisor = 2; divisor * divisor <= num; divisor++) @{ + if (num % divisor == 0) break @} - if (num % div == 0) - printf "Smallest divisor of %d is %d\n", num, div + if (num % divisor == 0) + printf "Smallest divisor of %d is %d\n", num, divisor else printf "%d is prime\n", num @} @@ -14066,12 +14271,12 @@ an @code{if}: # find smallest divisor of num @{ num = $1 - for (div = 2; ; div++) @{ - if (num % div == 0) @{ - printf "Smallest divisor of %d is %d\n", num, div + for (divisor = 2; ; divisor++) @{ + if (num % divisor == 0) @{ + printf "Smallest divisor of %d is %d\n", num, divisor break @} - if (div * div > num) @{ + if (divisor * divisor > num) @{ printf "%d is prime\n", num break @} @@ -14146,7 +14351,7 @@ BEGIN @{ @end example @noindent -This program loops forever once @code{x} reaches 5, since +This program loops forever once @code{x} reaches 5, because the increment (@samp{x++}) is never reached. @c @cindex @code{continue}, outside of loops @@ -14162,7 +14367,12 @@ body of a loop. Historical versions of @command{awk} treated a @code{continue} statement outside a loop the same way they treated a @code{break} statement outside a loop: as if it were a @code{next} statement +@ifset FOR_PRINT +(discussed in the following section). +@end ifset +@ifclear FOR_PRINT (@pxref{Next Statement}). +@end ifclear @value{DARKCORNER} Recent versions of BWK @command{awk} no longer work this way, nor does @command{gawk}. @@ -14207,7 +14417,7 @@ Because of the @code{next} statement, the program's subsequent rules won't see the bad record. The error message is redirected to the standard error output stream, as error messages should be. -For more detail see +For more detail, see @ref{Special Files}. If the @code{next} statement causes the end of the input to be reached, @@ -14273,7 +14483,7 @@ rule to skip over a file that would otherwise cause @command{gawk} to exit with a fatal error. In this case, @code{ENDFILE} rules are not executed. @xref{BEGINFILE/ENDFILE}. -While one might think that @samp{close(FILENAME)} would accomplish +Although it might seem that @samp{close(FILENAME)} would accomplish the same as @code{nextfile}, this isn't true. @code{close()} is reserved for closing files, pipes, and coprocesses that are opened with redirections. It is not related to the main processing that @@ -14281,7 +14491,7 @@ opened with redirections. It is not related to the main processing that @quotation NOTE For many years, @code{nextfile} was a -common extension. In September, 2012, it was accepted for +common extension. In September 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=607, the Austin Group website}. @end quotation @@ -14290,7 +14500,7 @@ See @uref{http://austingroupbugs.net/view.php?id=607, the Austin Group website}. @cindex @code{nextfile} statement, user-defined functions and @cindex Brian Kernighan's @command{awk} @cindex @command{mawk} utility -The current version of BWK @command{awk}, and @command{mawk} +The current version of BWK @command{awk} and @command{mawk} also support @code{nextfile}. However, they don't allow the @code{nextfile} statement inside function bodies (@pxref{User-defined}). @command{gawk} does; a @code{nextfile} inside a function body reads the @@ -14328,9 +14538,9 @@ any @code{ENDFILE} rules; they do not execute. In such a case, if you don't want the @code{END} rule to do its job, set a variable -to nonzero before the @code{exit} statement and check that variable in +to a nonzero value before the @code{exit} statement and check that variable in the @code{END} rule. -@xref{Assert Function}, +@DBXREF{Assert Function} for an example that does this. @cindex dark corner, @code{exit} statement @@ -14341,7 +14551,7 @@ In the case where an argument is supplied to a first @code{exit} statement, and then @code{exit} is called a second time from an @code{END} rule with no argument, @command{awk} uses the previously supplied exit value. @value{DARKCORNER} -@xref{Exit Status}, for more information. +@DBXREF{Exit Status} for more information. @cindex programming conventions, @code{exit} statement For example, suppose an error condition occurs that is difficult or @@ -14367,15 +14577,10 @@ Negative values, and values of 127 or greater, may not produce consistent results across different operating systems. @end quotation -@c ENDOFRANGE csta -@c ENDOFRANGE acs -@c ENDOFRANGE accs @node Built-in Variables @section Predefined Variables -@c STARTOFRANGE bvar @cindex predefined variables -@c STARTOFRANGE varb @cindex variables, predefined Most @command{awk} variables are available to use for your own @@ -14402,9 +14607,7 @@ their areas of activity. @node User-modified @subsection Built-in Variables That Control @command{awk} -@c STARTOFRANGE bvaru @cindex predefined variables, user-modifiable -@c STARTOFRANGE nmbv @cindex user-modifiable variables The following is an alphabetical list of variables that you can change to @@ -14432,7 +14635,7 @@ respectively, should use binary I/O. A string value of @code{"rw"} or @code{"wr"} indicates that all files should use binary I/O. Any other string value is treated the same as @code{"rw"}, but causes @command{gawk} to generate a warning message. @code{BINMODE} is described in more -detail in @ref{PC Using}. @command{mawk} (@pxref{Other Versions}), +detail in @ref{PC Using}. @command{mawk} (@pxref{Other Versions}) also supports this variable, but only using numeric values. @cindex @code{CONVFMT} variable @@ -14440,7 +14643,7 @@ also supports this variable, but only using numeric values. @cindex numbers, converting, to strings @cindex strings, converting, numbers to @item @code{CONVFMT} -This string controls conversion of numbers to +A string that controls the conversion of numbers to strings (@pxref{Conversion}). It works by being passed, in effect, as the first argument to the @code{sprintf()} function @@ -14458,7 +14661,7 @@ A space-separated list of columns that tells @command{gawk} how to split input with fixed columnar boundaries. Assigning a value to @code{FIELDWIDTHS} overrides the use of @code{FS} and @code{FPAT} for field splitting. -@xref{Constant Size}, for more information. +@DBXREF{Constant Size} for more information. @cindex @command{gawk}, @code{FPAT} variable in @cindex @code{FPAT} variable @@ -14470,7 +14673,7 @@ A regular expression (as a string) that tells @command{gawk} to create the fields based on text that matches the regular expression. Assigning a value to @code{FPAT} overrides the use of @code{FS} and @code{FIELDWIDTHS} for field splitting. -@xref{Splitting By Content}, for more information. +@DBXREF{Splitting By Content} for more information. @cindex @code{FS} variable @cindex separators, field @@ -14515,12 +14718,13 @@ is to simply say @samp{FS = FS}, perhaps with an explanatory comment. @cindex regular expressions, case sensitivity @item IGNORECASE # If @code{IGNORECASE} is nonzero or non-null, then all string comparisons -and all regular expression matching are case independent. Thus, regexp -matching with @samp{~} and @samp{!~}, as well as the @code{gensub()}, -@code{gsub()}, @code{index()}, @code{match()}, @code{patsplit()}, -@code{split()}, and @code{sub()} -functions, record termination with @code{RS}, and field splitting with -@code{FS} and @code{FPAT}, all ignore case when doing their particular regexp operations. +and all regular expression matching are case-independent. +This applies to +regexp matching with @samp{~} and @samp{!~}, +the @code{gensub()}, @code{gsub()}, @code{index()}, @code{match()}, +@code{patsplit()}, @code{split()}, and @code{sub()} functions, +record termination with @code{RS}, and field splitting with +@code{FS} and @code{FPAT}. However, the value of @code{IGNORECASE} does @emph{not} affect array subscripting and it does not affect field splitting when using a single-character field separator. @@ -14541,7 +14745,7 @@ Any other true value prints nonfatal warnings. Assigning a false value to @code{LINT} turns off the lint warnings. This variable is a @command{gawk} extension. It is not special -in other @command{awk} implementations. Unlike the other special variables, +in other @command{awk} implementations. Unlike with the other special variables, changing @code{LINT} does affect the production of lint warnings, even if @command{gawk} is in compatibility mode. Much as the @option{--lint} and @option{--traditional} options independently @@ -14553,7 +14757,7 @@ of @command{awk} being executed. @cindex numbers, converting, to strings @cindex strings, converting, numbers to @item OFMT -Controls conversion of numbers to +A string that controls conversion of numbers to strings (@pxref{Conversion}) for printing with the @code{print} statement. It works by being passed as the first argument to the @code{sprintf()} function @@ -14568,7 +14772,7 @@ strings in general expressions; this is now done by @code{CONVFMT}. @cindex separators, field @cindex field separators @item OFS -This is the output field separator (@pxref{Output Separators}). It is +The output field separator (@pxref{Output Separators}). It is output between the fields printed by a @code{print} statement. Its default value is @w{@code{" "}}, a string consisting of a single space. @@ -14580,13 +14784,13 @@ character. (@xref{Output Separators}.) @cindex @code{PREC} variable @item PREC # -The working precision of arbitrary precision floating-point numbers, +The working precision of arbitrary-precision floating-point numbers, 53 bits by default (@pxref{Setting precision}). @cindex @code{ROUNDMODE} variable @item ROUNDMODE # -The rounding mode to use for arbitrary precision arithmetic on -numbers, by default @code{"N"} (@samp{roundTiesToEven} in +The rounding mode to use for arbitrary-precision arithmetic on +numbers, by default @code{"N"} (@code{roundTiesToEven} in the IEEE 754 standard; @pxref{Setting the rounding mode}). @cindex @code{RS} variable @@ -14615,7 +14819,7 @@ just the first character of @code{RS}'s value is used. @item @code{SUBSEP} The subscript separator. It has the default value of @code{"\034"} and is used to separate the parts of the indices of a -multidimensional array. Thus, the expression @code{@w{foo["A", "B"]}} +multidimensional array. Thus, the expression @samp{@w{foo["A", "B"]}} really accesses @code{foo["A\034B"]} (@pxref{Multidimensional}). @@ -14627,21 +14831,15 @@ really accesses @code{foo["A\034B"]} Used for internationalization of programs at the @command{awk} level. It sets the default text domain for specially marked string constants in the source text, as well as for the -@code{dcgettext()}, @code{dcngettext()} and @code{bindtextdomain()} functions +@code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()} functions (@pxref{Internationalization}). The default value of @code{TEXTDOMAIN} is @code{"messages"}. @end table -@c ENDOFRANGE bvar -@c ENDOFRANGE varb -@c ENDOFRANGE bvaru -@c ENDOFRANGE nmbv @node Auto-set @subsection Built-in Variables That Convey Information -@c STARTOFRANGE bvconi @cindex predefined variables, conveying information -@c STARTOFRANGE vbconi @cindex variables, predefined conveying information The following is an alphabetical list of variables that @command{awk} sets automatically on certain occasions in order to provide @@ -14650,7 +14848,7 @@ information to your program. The variables that are specific to @command{gawk} are marked with a pound sign (@samp{#}). These variables are @command{gawk} extensions. In other @command{awk} implementations or if @command{gawk} is in compatibility -mode (@pxref{Options}), they are not special. +mode (@pxref{Options}), they are not special: @c @asis for docbook @table @asis @@ -14691,7 +14889,7 @@ method of accessing command-line arguments. The value of @code{ARGV[0]} can vary from system to system. Also, you should note that the program text is @emph{not} included in @code{ARGV}, nor are any of @command{awk}'s command-line options. -@xref{ARGC and ARGV}, for information +@DBXREF{ARGC and ARGV} for information about how @command{awk} uses these variables. @value{DARKCORNER} @@ -14733,12 +14931,23 @@ that it creates. You should therefore be especially careful if you modify @code{ENVIRON["PATH"]"}, which is the search path for finding executable programs. +This can also affect the running @command{gawk} program, since some of the +built-in functions may pay attention to certain environment variables. +The most notable instance of this is @code{mktime()} (@pxref{Time +Functions}), which pays attention the value of the @env{TZ} environment +variable on many systems. + Some operating systems may not have environment variables. On such systems, the @code{ENVIRON} array is empty (except for @w{@code{ENVIRON["AWKPATH"]}} and @w{@code{ENVIRON["AWKLIBPATH"]}}; -@pxref{AWKPATH Variable}, and +@DBPXREF{AWKPATH Variable} and +@ifdocbook +@DBREF{AWKLIBPATH Variable}). +@end ifdocbook +@ifnotdocbook @pxref{AWKLIBPATH Variable}). +@end ifnotdocbook @cindex @command{gawk}, @code{ERRNO} variable in @cindex @code{ERRNO} variable @@ -14760,6 +14969,11 @@ value to be meaningful when an I/O operation returns a failure value, such as @code{getline} returning @minus{}1. You are, of course, free to clear it yourself before doing an I/O operation. +If the value of @code{ERRNO} corresponds to a system error in the C +@code{errno} variable, then @code{PROCINFO["errno"]} will be set to the value +of @code{errno}. For non-system errors, @code{PROCINFO["errno"]} will +be zero. + @cindex @code{FILENAME} variable @cindex dark corner, @code{FILENAME} variable @item @code{FILENAME} @@ -14767,7 +14981,7 @@ The name of the current input file. When no @value{DF}s are listed on the command line, @command{awk} reads from the standard input and @code{FILENAME} is set to @code{"-"}. @code{FILENAME} changes each time a new file is read (@pxref{Reading Files}). Inside a @code{BEGIN} -rule, the value of @code{FILENAME} is @code{""}, since there are no input +rule, the value of @code{FILENAME} is @code{""}, because there are no input files being processed yet.@footnote{Some early implementations of Unix @command{awk} initialized @code{FILENAME} to @code{"-"}, even if there were @value{DF}s to be processed. This behavior was incorrect and should @@ -14786,12 +15000,12 @@ input file. @item @code{NF} The number of fields in the current input record. @code{NF} is set each time a new record is read, when a new field is -created or when @code{$0} changes (@pxref{Fields}). +created, or when @code{$0} changes (@pxref{Fields}). Unlike most of the variables described in this @value{SUBSECTION}, assigning a value to @code{NF} has the potential to affect @command{awk}'s internal workings. In particular, assignments -to @code{NF} can be used to create or remove fields from the +to @code{NF} can be used to create fields in or remove fields from the current record. @xref{Changing Fields}. @cindex @code{FUNCTAB} array @@ -14799,7 +15013,7 @@ current record. @xref{Changing Fields}. @cindex differences in @command{awk} and @command{gawk}, @code{FUNCTAB} variable @item @code{FUNCTAB #} An array whose indices and corresponding values are the names of all -the built-in, user-defined and extension functions in the program. +the built-in, user-defined, and extension functions in the program. @quotation NOTE Attempting to use the @code{delete} statement with the @code{FUNCTAB} @@ -14828,6 +15042,10 @@ are guaranteed to be available: @item PROCINFO["egid"] The value of the @code{getegid()} system call. +@item PROCINFO["errno"] +The value of the C @code{errno} variable when @code{ERRNO} is set to +the associated error message. + @item PROCINFO["euid"] @cindex effective user ID of @command{gawk} user The value of the @code{geteuid()} system call. @@ -14841,7 +15059,7 @@ or @code{"FPAT"} if field matching with @code{FPAT} is in effect. @item PROCINFO["identifiers"] @cindex program identifiers A subarray, indexed by the names of all identifiers used in the text of -the AWK program. An @dfn{identifier} is simply the name of a variable +the @command{awk} program. An @dfn{identifier} is simply the name of a variable (be it scalar or array), built-in function, user-defined function, or extension function. For each identifier, the value of the element is one of the following: @@ -14861,7 +15079,7 @@ The identifier is an extension function loaded via The identifier is a scalar. @item "untyped" -The identifier is untyped (could be used as a scalar or array, +The identifier is untyped (could be used as a scalar or an array; @command{gawk} doesn't know yet). @item "user" @@ -14878,7 +15096,7 @@ while the program runs. The value of the @code{getgid()} system call. @item PROCINFO["pgrpid"] -@cindex process group idIDof @command{gawk} process +@cindex process group ID of @command{gawk} process The process group ID of the current process. @item PROCINFO["pid"] @@ -14893,7 +15111,7 @@ The parent process ID of the current process. If this element exists in @code{PROCINFO}, its value controls the order in which array indices will be processed by @samp{for (@var{indx} in @var{array})} loops. -Since this is an advanced feature, we defer the +This is an advanced feature, so we defer the full description until later; see @ref{Scanning an Array}. @@ -14913,10 +15131,10 @@ The version of @command{gawk}. The following additional elements in the array are available to provide information about the MPFR and GMP libraries -if your version of @command{gawk} supports arbitrary precision arithmetic -(@pxref{Arbitrary Precision Arithmetic}): +if your version of @command{gawk} supports arbitrary-precision arithmetic +(@pxref{Arbitrary Precision Arithmetic}): -@table @code +@table @code @cindex version of GNU MPFR library @item PROCINFO["mpfr_version"] The version of the GNU MPFR library. @@ -14930,7 +15148,7 @@ The version of the GNU MP library. The maximum precision supported by MPFR. @item PROCINFO["prec_min"] -@cindex minimum precision supported by MPFR library +@cindex minimum precision required by MPFR library The minimum precision required by MPFR. @end table @@ -14964,7 +15182,12 @@ The @code{PROCINFO} array has the following additional uses: @item It may be used to provide a timeout when reading from any open input file, pipe, or coprocess. -@xref{Read Timeout}, for more information. +@DBXREF{Read Timeout} for more information. + +@item +It may be used to indicate that input may be retried when it fails due to +certain errors. +@DBXREF{Retrying Input} for more information. @item It may be used to cause coprocesses to communicate over pseudo-ttys @@ -14982,7 +15205,7 @@ is the length of the matched string, or @minus{}1 if no match is found. @cindex @code{RSTART} variable @item @code{RSTART} -The start-index in characters of the substring that is matched by the +The start index in characters of the substring that is matched by the @code{match()} function (@pxref{String Functions}). @code{RSTART} is set by invoking the @code{match()} function. Its value @@ -15049,11 +15272,9 @@ function multiply(variable, amount) @quotation NOTE In order to avoid severe time-travel paradoxes,@footnote{Not to mention difficult implementation issues.} neither @code{FUNCTAB} nor @code{SYMTAB} -are available as elements within the @code{SYMTAB} array. +is available as an element within the @code{SYMTAB} array. @end quotation @end table -@c ENDOFRANGE bvconi -@c ENDOFRANGE vbconi @cindex sidebar, Changing @code{NR} and @code{FNR} @ifdocbook @@ -15209,8 +15430,14 @@ use the @code{delete} statement to remove elements from All of these actions are typically done in the @code{BEGIN} rule, before actual processing of the input begins. -@xref{Split Program}, and see -@ref{Tee Program}, for examples +@DBXREF{Split Program} and +@ifnotdocbook +@DBPXREF{Tee Program} +@end ifnotdocbook +@ifdocbook +@DBREF{Tee Program} +@end ifdocbook +for examples of each way of removing elements from @code{ARGV}. To actually get options into an @command{awk} program, @@ -15222,7 +15449,7 @@ awk -f myprog.awk -- -v -q file1 file2 @dots{} @end example The following fragment processes @code{ARGV} in order to examine, and -then remove, the above command-line options: +then remove, the previously mentioned command-line options: @example BEGIN @{ @@ -15258,29 +15485,36 @@ gawk -f myprog.awk -q -v file1 file2 @dots{} @noindent Because @option{-q} is not a valid @command{gawk} option, it and the following @option{-v} are passed on to the @command{awk} program. -(@xref{Getopt Function}, for an @command{awk} library function that +(@DBXREF{Getopt Function} for an @command{awk} library function that parses command-line options.) When designing your program, you should choose options that don't -conflict with @command{gawk}'s, since it will process any options +conflict with @command{gawk}'s, because it will process any options that it accepts before passing the rest of the command line on to your program. Using @samp{#!} with the @option{-E} option may help -(@pxref{Executable Scripts}, and @pxref{Options}). +(@DBPXREF{Executable Scripts} +and +@ifnotdocbook +@DBPXREF{Options}). +@end ifnotdocbook +@ifdocbook +@DBREF{Options}). +@end ifdocbook @node Pattern Action Summary @section Summary @itemize @value{BULLET} @item -Pattern-action pairs make up the basic elements of an @command{awk} +Pattern--action pairs make up the basic elements of an @command{awk} program. Patterns are either normal expressions, range expressions, -regexp constants, one of the special keywords @code{BEGIN}, @code{END}, -@code{BEGINFILE}, @code{ENDFILE}, or empty. The action executes if +or regexp constants; one of the special keywords @code{BEGIN}, @code{END}, +@code{BEGINFILE}, or @code{ENDFILE}; or empty. The action executes if the current record matches the pattern. Empty (missing) patterns match all records. @item -I/O from @code{BEGIN} and @code{END} rules have certain constraints. +I/O from @code{BEGIN} and @code{END} rules has certain constraints. This is also true, only more so, for @code{BEGINFILE} and @code{ENDFILE} rules. The latter two give you ``hooks'' into @command{gawk}'s file processing, allowing you to recover from a file that otherwise would @@ -15310,12 +15544,12 @@ iteration of a loop (or get out of a @code{switch}). @item @code{next} and @code{nextfile} let you read the next record and start -over at the top of your program, or skip to the next input file and +over at the top of your program or skip to the next input file and start over, respectively. @item The @code{exit} statement terminates your program. When executed -from an action (or function body) it transfers control to the +from an action (or function body), it transfers control to the @code{END} statements. From an @code{END} statement body, it exits immediately. You may pass an optional numeric value to be used as @command{awk}'s exit status. @@ -15333,7 +15567,6 @@ control how @command{awk} will process the provided @value{DF}s. @node Arrays @chapter Arrays in @command{awk} -@c STARTOFRANGE arrs @cindex arrays An @dfn{array} is a table of values called @dfn{elements}. The @@ -15408,7 +15641,7 @@ In most other languages, arrays must be @dfn{declared} before use, including a specification of how many elements or components they contain. In such languages, the declaration causes a contiguous block of memory to be allocated for that -many elements. Usually, an index in the array must be a positive integer. +many elements. Usually, an index in the array must be a nonnegative integer. For example, the index zero specifies the first element in the array, which is actually stored at the beginning of the block of memory. Index one specifies the second element, which is stored in memory right after the @@ -15419,19 +15652,21 @@ the declaration. indices---e.g., @samp{15 .. 27}---but the size of the array is still fixed when the array is declared.) -A contiguous array of four elements might look like the following example, -conceptually, if the element values are 8, @code{"foo"}, -@code{""}, and 30 +@c 1/2015: Do not put the numeric values into @code. Array element +@c values are no different than scalar variable values. +A contiguous array of four elements might look like @ifnotdocbook -as shown in @ref{figure-array-elements}: +@ref{figure-array-elements}, @end ifnotdocbook @ifdocbook -as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: +@inlineraw{docbook, <xref linkend="figure-array-elements"/>}, @end ifdocbook +conceptually, if the element values are eight, @code{"foo"}, +@code{""}, and 30. @ifnotdocbook @float Figure,figure-array-elements -@caption{A Contiguous Array} +@caption{A contiguous array} @ifinfo @center @image{array-elements, , , Basic Program Stages, txt} @end ifinfo @@ -15443,7 +15678,7 @@ as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: @docbook <figure id="figure-array-elements" float="0"> -<title>A Contiguous Array</title> +<title>A contiguous array</title> <mediaobject> <imageobject role="web"><imagedata fileref="array-elements.png" format="PNG"/></imageobject> </mediaobject> @@ -15452,33 +15687,33 @@ as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: @noindent Only the values are stored; the indices are implicit from the order of -the values. Here, 8 is the value at index zero, because 8 appears in the +the values. Here, eight is the value at index zero, because eight appears in the position with zero elements before it. -@c STARTOFRANGE arrin @cindex arrays, indexing -@c STARTOFRANGE inarr @cindex indexing arrays @cindex associative arrays @cindex arrays, associative Arrays in @command{awk} are different---they are @dfn{associative}. This means -that each array is a collection of pairs: an index and its corresponding +that each array is a collection of pairs---an index and its corresponding array element value: @ifnotdocbook -@example -@r{Index} 3 @r{Value} 30 -@r{Index} 1 @r{Value} "foo" -@r{Index} 0 @r{Value} 8 -@r{Index} 2 @r{Value} "" -@end example +@c extra empty column to indent it right +@multitable @columnfractions .1 .1 .1 +@headitem @tab Index @tab Value +@item @tab @code{3} @tab @code{30} +@item @tab @code{1} @tab @code{"foo"} +@item @tab @code{0} @tab @code{8} +@item @tab @code{2} @tab @code{""} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -15524,20 +15759,22 @@ at any time. For example, suppose a tenth element is added to the array whose value is @w{@code{"number ten"}}. The result is: @ifnotdocbook -@example -@r{Index} 10 @r{Value} "number ten" -@r{Index} 3 @r{Value} 30 -@r{Index} 1 @r{Value} "foo" -@r{Index} 0 @r{Value} 8 -@r{Index} 2 @r{Value} "" -@end example +@c extra empty column to indent it right +@multitable @columnfractions .1 .1 .2 +@headitem @tab Index @tab Value +@item @tab @code{10} @tab @code{"number ten"} +@item @tab @code{3} @tab @code{30} +@item @tab @code{1} @tab @code{"foo"} +@item @tab @code{0} @tab @code{8} +@item @tab @code{2} @tab @code{""} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -15584,24 +15821,25 @@ Now the array is @dfn{sparse}, which just means some indices are missing. It has elements 0--3 and 10, but doesn't have elements 4, 5, 6, 7, 8, or 9. Another consequence of associative arrays is that the indices don't -have to be positive integers. Any number, or even a string, can be +have to be nonnegative integers. Any number, or even a string, can be an index. For example, the following is an array that translates words from English to French: @ifnotdocbook -@example -@r{Index} "dog" @r{Value} "chien" -@r{Index} "cat" @r{Value} "chat" -@r{Index} "one" @r{Value} "un" -@r{Index} 1 @r{Value} "un" -@end example +@multitable @columnfractions .1 .1 .1 +@headitem @tab Index @tab Value +@item @tab @code{"dog"} @tab @code{"chien"} +@item @tab @code{"cat"} @tab @code{"chat"} +@item @tab @code{"one"} @tab @code{"un"} +@item @tab @code{1} @tab @code{"un"} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -15643,7 +15881,7 @@ numbers and strings as indices. There are some subtleties to how numbers work when used as array subscripts; this is discussed in more detail in @ref{Numeric Array Subscripts}.) -Here, the number @code{1} isn't double-quoted, since @command{awk} +Here, the number @code{1} isn't double-quoted, because @command{awk} automatically converts it to a string. @cindex @command{gawk}, @code{IGNORECASE} variable in @@ -15660,8 +15898,6 @@ that array's indices are consecutive integers starting at one. @command{awk}'s arrays are efficient---the time to access an element is independent of the number of elements in the array. -@c ENDOFRANGE arrin -@c ENDOFRANGE inarr @node Reference to Elements @subsection Referring to an Array Element @@ -15670,7 +15906,7 @@ is independent of the number of elements in the array. @cindex elements of arrays The principal way to use an array is to refer to one of its elements. -An array reference is an expression as follows: +An @dfn{array reference} is an expression as follows: @example @var{array}[@var{index-expression}] @@ -15680,8 +15916,11 @@ An array reference is an expression as follows: Here, @var{array} is the name of an array. The expression @var{index-expression} is the index of the desired element of the array. +@c 1/2015: Having the 4.3 in @samp is a little iffy. It's essentially +@c an expression though, so leave be. It's to early in the discussion +@c to mention that it's really a string. The value of the array reference is the current value of that array -element. For example, @code{foo[4.3]} is an expression for the element +element. For example, @code{foo[4.3]} is an expression referencing the element of array @code{foo} at index @samp{4.3}. @cindex arrays, unassigned elements @@ -15728,7 +15967,7 @@ This expression tests whether the particular index @var{indx} exists, without the side effect of creating that element if it is not present. The expression has the value one (true) if @code{@var{array}[@var{indx}]} exists and zero (false) if it does not exist. -(We use @var{indx} here, since @samp{index} is the name of a built-in +(We use @var{indx} here, because @samp{index} is the name of a built-in function.) For example, this statement tests whether the array @code{frequencies} contains the index @samp{2}: @@ -15773,7 +16012,7 @@ assign to that element of the array. The following program takes a list of lines, each beginning with a line number, and prints them out in order of line number. The line numbers -are not in order when they are first read---instead they +are not in order when they are first read---instead, they are scrambled. This program sorts the lines by making an array using the line numbers as subscripts. The program then prints out the lines in sorted order of their numbers. It is a very simple program and gets @@ -15845,7 +16084,7 @@ END @{ In programs that use arrays, it is often necessary to use a loop that executes once for each element of an array. In other languages, where -arrays are contiguous and indices are limited to positive integers, +arrays are contiguous and indices are limited to nonnegative integers, this is easy: all the valid indices can be found by counting from the lowest index up to the highest. This technique won't do the job in @command{awk}, because any number or string can be an array index. @@ -15867,11 +16106,11 @@ program has previously used, with the variable @var{var} set to that index. The following program uses this form of the @code{for} statement. The first rule scans the input records and notes which words appear (at least once) in the input, by storing a one into the array @code{used} with -the word as index. The second rule scans the elements of @code{used} to +the word as the index. The second rule scans the elements of @code{used} to find all the distinct words that appear in the input. It prints each word that is more than 10 characters long and also prints the number of such words. -@xref{String Functions}, +@DBXREF{String Functions} for more information on the built-in function @code{length()}. @example @@ -15894,7 +16133,7 @@ END @{ @end example @noindent -@xref{Word Sorting}, +@DBXREF{Word Sorting} for a more detailed example of this type. @cindex arrays, elements, order of access by @code{in} operator @@ -15949,7 +16188,7 @@ $ @kbd{nawk -f loopcheck.awk} @end example @node Controlling Scanning -@subsection Using Predefined Array Scanning Orders With @command{gawk} +@subsection Using Predefined Array Scanning Orders with @command{gawk} This @value{SUBSECTION} describes a feature that is specific to @command{gawk}. @@ -15964,7 +16203,7 @@ and will vary from one version of @command{awk} to the next. Often, though, you may wish to do something simple, such as ``traverse the array by comparing the indices in ascending order,'' or ``traverse the array by comparing the values in descending order.'' -@command{gawk} provides two mechanisms which give you this control. +@command{gawk} provides two mechanisms that give you this control: @itemize @value{BULLET} @item @@ -15974,7 +16213,7 @@ We describe this now. @item Set @code{PROCINFO["sorted_in"]} to the name of a user-defined function to use for comparison of array elements. This advanced feature -is described later, in @ref{Array Sorting}. +is described later in @ref{Array Sorting}. @end itemize @cindex @code{PROCINFO}, values of @code{sorted_in} @@ -15992,7 +16231,7 @@ the index is @code{"10"} rather than numeric 10.) @item "@@ind_num_asc" Order by indices in ascending order but force them to be treated as numbers in the process. -Any index with a non-numeric value will end up positioned as if it were zero. +Any index with a non-numeric value will end up positioned as if it were zero. @item "@@val_type_asc" Order by element values in ascending order (rather than by indices). @@ -16004,11 +16243,11 @@ which in turn come before all subarrays. @pxref{Arrays of Arrays}.) @item "@@val_str_asc" -Order by element values in ascending order (rather than by indices). Scalar values are +Order by element values in ascending order (rather than by indices). Scalar values are compared as strings. Subarrays, if present, come out last. @item "@@val_num_asc" -Order by element values in ascending order (rather than by indices). Scalar values are +Order by element values in ascending order (rather than by indices). Scalar values are compared as numbers. Subarrays, if present, come out last. When numeric values are equal, the string values are used to provide an ordering: this guarantees consistent results across different @@ -16021,21 +16260,26 @@ across different environments.} which @command{gawk} uses internally to perform the sorting. @item "@@ind_str_desc" -String indices ordered from high to low. +Like @code{"@@ind_str_asc"}, but the +string indices are ordered from high to low. @item "@@ind_num_desc" -Numeric indices ordered from high to low. +Like @code{"@@ind_num_asc"}, but the +numeric indices are ordered from high to low. @item "@@val_type_desc" -Element values, based on type, ordered from high to low. +Like @code{"@@val_type_asc"}, but the +element values, based on type, are ordered from high to low. Subarrays, if present, come out first. @item "@@val_str_desc" -Element values, treated as strings, ordered from high to low. +Like @code{"@@val_str_asc"}, but the +element values, treated as strings, are ordered from high to low. Subarrays, if present, come out first. @item "@@val_num_desc" -Element values, treated as numbers, ordered from high to low. +Like @code{"@@val_num_asc"}, but the +element values, treated as numbers, are ordered from high to low. Subarrays, if present, come out first. @end table @@ -16069,11 +16313,11 @@ $ @kbd{gawk '} When sorting an array by element values, if a value happens to be a subarray then it is considered to be greater than any string or numeric value, regardless of what the subarray itself contains, -and all subarrays are treated as being equal to each other. Their +and all subarrays are treated as being equal to each other. Their order relative to each other is determined by their index strings. Here are some additional things to bear in mind about sorted -array traversal. +array traversal: @itemize @value{BULLET} @item @@ -16093,7 +16337,7 @@ if (save_sorted) @end example @item -As mentioned, the default array traversal order is represented by +As already mentioned, the default array traversal order is represented by @code{"@@unsorted"}. You can also get the default behavior by assigning the null string to @code{PROCINFO["sorted_in"]} or by just deleting the @code{"sorted_in"} element from the @code{PROCINFO} array with @@ -16138,7 +16382,7 @@ The program then changes the value of @code{CONVFMT}. The test @samp{(xyz in data)} generates a new string value from @code{xyz}---this time @code{"12.15"}---because the value of @code{CONVFMT} only allows two significant digits. This test fails, -since @code{"12.15"} is different from @code{"12.153"}. +because @code{"12.15"} is different from @code{"12.153"}. @cindex converting integer array subscripts @cindex integer array indices @@ -16156,19 +16400,19 @@ for (i = 1; i <= maxsub; i++) The ``integer values always convert to strings as integers'' rule has an additional consequence for array indexing. Octal and hexadecimal constants +@ifnotdocbook (@pxref{Nondecimal-numbers}) +@end ifnotdocbook +@ifdocbook +(covered in @ref{Nondecimal-numbers}) +@end ifdocbook are converted internally into numbers, and their original form -is forgotten. -This means, for example, that -@code{array[17]}, -@code{array[021]}, -and -@code{array[0x11]} -all refer to the same element! +is forgotten. This means, for example, that @code{array[17]}, +@code{array[021]}, and @code{array[0x11]} all refer to the same element! As with many things in @command{awk}, the majority of the time things work as you would expect them to. But it is useful to have a precise -knowledge of the actual rules since they can sometimes have a subtle +knowledge of the actual rules, as they can sometimes have a subtle effect on your programs. @node Uninitialized Subscripts @@ -16258,7 +16502,7 @@ for (i in frequencies) @noindent This example removes all the elements from the array @code{frequencies}. Once an element is deleted, a subsequent @code{for} statement to scan the array -does not report that element and the @code{in} operator to check for +does not report that element and using the @code{in} operator to check for the presence of that element returns zero (i.e., false): @example @@ -16311,7 +16555,7 @@ by a number of other implementations. @cindex Brian Kernighan's @command{awk} @quotation NOTE For many years, using @code{delete} without a subscript was a common -extension. In September, 2012, it was accepted for inclusion into the +extension. In September 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=544, the Austin Group website}. @end quotation @@ -16353,7 +16597,7 @@ a = 3 @cindex subscripts in arrays, multidimensional @cindex arrays, multidimensional -A multidimensional array is an array in which an element is identified +A @dfn{multidimensional array} is an array in which an element is identified by a sequence of indices instead of a single index. For example, a two-dimensional array requires two indices. The usual way (in many languages, including @command{awk}) to refer to an element of a @@ -16395,7 +16639,7 @@ stored as @samp{foo["a@@b@@c"]}. @cindex @code{in} operator, index existence in multidimensional arrays To test whether a particular index sequence exists in a multidimensional array, use the same operator (@code{in}) that is -used for single dimensional arrays. Write the whole sequence of indices +used for single-dimensional arrays. Write the whole sequence of indices in parentheses, separated by commas, as the left operand: @example @@ -16518,8 +16762,8 @@ a[1][2] = 2 This simulates a true two-dimensional array. Each subarray element can contain another subarray as a value, which in turn can hold other arrays as well. In this way, you can create arrays of three or more dimensions. -The indices can be any @command{awk} expression, including scalars -separated by commas (that is, a regular @command{awk} simulated +The indices can be any @command{awk} expressions, including scalars +separated by commas (i.e., a regular @command{awk} simulated multidimensional subscript). So the following is valid in @command{gawk}: @@ -16530,15 +16774,15 @@ a[1][3][1, "name"] = "barney" Each subarray and the main array can be of different length. In fact, the elements of an array or its subarray do not all have to have the same type. This means that the main array and any of its subarrays can be -non-rectangular, or jagged in structure. You can assign a scalar value to +nonrectangular, or jagged in structure. You can assign a scalar value to the index @code{4} of the main array @code{a}, even though @code{a[1]} is itself an array and not a scalar: @example a[4] = "An element in a jagged array" @end example - -The terms @dfn{dimension}, @dfn{row} and @dfn{column} are + +The terms @dfn{dimension}, @dfn{row}, and @dfn{column} are meaningless when applied to such an array, but we will use ``dimension'' henceforth to imply the maximum number of indices needed to refer to an existing element. The @@ -16554,7 +16798,8 @@ a[4][5][6][7] = "An element in a four-dimensional array" @noindent This removes the scalar value from index @code{4} and then inserts a -subarray of subarray of subarray containing a scalar. You can also +three-level nested subarray +containing a scalar. You can also delete an entire subarray or subarray of subarrays: @example @@ -16565,7 +16810,7 @@ a[4][5] = "An element in subarray a[4]" But recall that you can not delete the main array @code{a} and then use it as a scalar. -The built-in functions which take array arguments can also be used +The built-in functions that take array arguments can also be used with subarrays. For example, the following code fragment uses @code{length()} (@pxref{String Functions}) to determine the number of elements in the main array @code{a} and @@ -16594,14 +16839,14 @@ The @samp{for (item in array)} statement (@pxref{Scanning an Array}) can be nested to scan all the elements of an array of arrays if it is rectangular in structure. In order to print the contents (scalar values) of a two-dimensional array of arrays -(i.e., in which each first-level element is itself an -array, not necessarily of the same length) +(i.e., in which each first-level element is itself an +array, not necessarily of the same length), you could use the following code: @example for (i in array) for (j in array[i]) - print array[i][j] + print array[i][j] @end example The @code{isarray()} function (@pxref{Type Functions}) @@ -16611,7 +16856,7 @@ lets you test if an array element is itself an array: for (i in array) @{ if (isarray(array[i]) @{ for (j in array[i]) @{ - print array[i][j] + print array[i][j] @} @} else @@ -16621,7 +16866,7 @@ for (i in array) @{ If the structure of a jagged array of arrays is known in advance, you can often devise workarounds using control statements. For example, -the following code prints the elements of our main array @code{a}: +the following code prints the elements of our main array @code{a}: @example for (i in a) @{ @@ -16631,13 +16876,13 @@ for (i in a) @{ print a[i][j][k] @} else print a[i][j] - @} + @} @} @end example @noindent -@xref{Walking Arrays}, for a user-defined function that ``walks'' an -arbitrarily-dimensioned array of arrays. +@DBXREF{Walking Arrays} for a user-defined function that ``walks'' an +arbitrarily dimensioned array of arrays. Recall that a reference to an uninitialized array element yields a value of @code{""}, the null string. This has one important implication when you @@ -16687,16 +16932,17 @@ special predefined values to @code{PROCINFO["sorted_in"]}. @item Use @samp{delete @var{array}[@var{indx}]} to delete an individual element. -You may also use @samp{delete @var{array}} to delete all of the elements -in the array. This latter feature has been a common extension for many +To delete all of the elements in an array, +use @samp{delete @var{array}}. +This latter feature has been a common extension for many years and is now standard, but may not be supported by all commercial versions of @command{awk}. @item Standard @command{awk} simulates multidimensional arrays by separating -subscript values with a comma. The values are concatenated into a +subscript values with commas. The values are concatenated into a single string, separated by the value of @code{SUBSEP}. The fact -that such a subscript was created in this way is not retained; thus +that such a subscript was created in this way is not retained; thus, changing @code{SUBSEP} may have unexpected consequences. You can use @samp{(@var{sub1}, @var{sub2}, @dots{}) in @var{array}} to see if such a multidimensional subscript exists in @var{array}. @@ -16705,7 +16951,7 @@ a multidimensional subscript exists in @var{array}. @command{gawk} provides true arrays of arrays. You use a separate set of square brackets for each dimension in such an array: @code{data[row][col]}, for example. Array elements may thus be either -scalar values (number or string) or another array. +scalar values (number or string) or other arrays. @item Use the @code{isarray()} built-in function to determine if an array @@ -16713,25 +16959,26 @@ element is itself a subarray. @end itemize -@c ENDOFRANGE arrs @node Functions @chapter Functions -@c STARTOFRANGE funcbi @cindex functions, built-in -@c STARTOFRANGE bifunc @cindex built-in functions This @value{CHAPTER} describes @command{awk}'s built-in functions, which fall into three categories: numeric, string, and I/O. @command{gawk} provides additional groups of functions to work with values that represent time, do -bit manipulation, sort arrays, and internationalize and localize programs. +bit manipulation, sort arrays, +provide type information, and internationalize and localize programs. Besides the built-in functions, @command{awk} has provisions for writing new functions that the rest of a program can use. The second half of this @value{CHAPTER} describes these @dfn{user-defined} functions. +Finally, we explore indirect function calls, a @command{gawk}-specific +extension that lets you determine at runtime what function is to +be called. @menu * Built-in:: Summarizes the built-in functions. @@ -16774,7 +17021,7 @@ is a call to the function @code{atan2()} and has two arguments. @cindex programming conventions, functions, calling @cindex whitespace, functions@comma{} calling Whitespace is ignored between the built-in function name and the -open parenthesis, but nonetheless it is good practice to avoid using whitespace +opening parenthesis, but nonetheless it is good practice to avoid using whitespace there. User-defined functions do not permit whitespace in this way, and it is easier to avoid mistakes by following a simple convention that always works---no whitespace after a function name. @@ -16811,11 +17058,11 @@ right to left. For example: @example i = 5 -j = atan2(i++, i *= 2) +j = atan2(++i, i *= 2) @end example If the order of evaluation is left to right, then @code{i} first becomes -6, and then 12, and @code{atan2()} is called with the two arguments 6 +six, and then 12, and @code{atan2()} is called with the two arguments six and 12. But if the order of evaluation is right to left, @code{i} first becomes 10, then 11, and @code{atan2()} is called with the two arguments 11 and 10. @@ -16842,23 +17089,6 @@ You can use @samp{pi = atan2(0, -1)} to retrieve the value of @cindex cosine Return the cosine of @var{x}, with @var{x} in radians. -@item @code{div(@var{numerator}, @var{denominator}, @var{result})} -@cindexawkfunc{div} -@cindex div -Perform integer division, similar to the standard C function of the -same name. First, truncate @code{numerator} and @code{denominator} -towards zero, creating integer values. Clear the @code{result} -array, and then set @code{result["quotient"]} to the result of -@samp{numerator / denominator}, truncated towards zero to an integer, -and set @code{result["remainder"]} to the result of @samp{numerator % -denominator}, truncated towards zero to an integer. This function is -primarily intended for use with arbitrary length integers; it avoids -creating MPFR arbitrary precision floating-point values (@pxref{Arbitrary -Precision Integers}). - -This function is a @code{gawk} extension. It is not available in -compatibility mode (@pxref{Options}). - @item @code{exp(@var{x})} @cindexawkfunc{exp} @cindex exponent @@ -16871,10 +17101,26 @@ depends on your machine's floating-point representation. @cindex round to nearest integer Return the nearest integer to @var{x}, located between @var{x} and zero and truncated toward zero. - For example, @code{int(3)} is 3, @code{int(3.9)} is 3, @code{int(-3.9)} is @minus{}3, and @code{int(-3)} is @minus{}3 as well. +@item @code{intdiv(@var{numerator}, @var{denominator}, @var{result})} +@cindexawkfunc{intdiv} +@cindex intdiv +Perform integer division, similar to the standard C function of the +same name. First, truncate @code{numerator} and @code{denominator} +towards zero, creating integer values. Clear the @code{result} +array, and then set @code{result["quotient"]} to the result of +@samp{numerator / denominator}, truncated towards zero to an integer, +and set @code{result["remainder"]} to the result of @samp{numerator % +denominator}, truncated towards zero to an integer. This function is +primarily intended for use with arbitrary length integers; it avoids +creating MPFR arbitrary precision floating-point values (@pxref{Arbitrary +Precision Integers}). + +This function is a @code{gawk} extension. It is not available in +compatibility mode (@pxref{Options}). + @item @code{log(@var{x})} @cindexawkfunc{log} @cindex logarithm @@ -16897,7 +17143,7 @@ In fact, @command{gawk} uses the BSD @code{random()} function, which is considerably better than @code{rand()}, to produce random numbers.} Often random integers are needed instead. Following is a user-defined function -that can be used to obtain a random non-negative integer less than @var{n}: +that can be used to obtain a random nonnegative integer less than @var{n}: @example function randint(n) @@ -16907,8 +17153,8 @@ function randint(n) @end example @noindent -The multiplication produces a random number greater than zero and less -than @code{n}. Using @code{int()}, this result is made into +The multiplication produces a random number greater than or equal to +zero and less than @code{n}. Using @code{int()}, this result is made into an integer between zero and @code{n} @minus{} 1, inclusive. The following example uses a similar function to produce random integers @@ -16960,8 +17206,8 @@ for generating random numbers to the value @var{x}. Each seed value leads to a particular sequence of random numbers.@footnote{Computer-generated random numbers really are not truly -random. They are technically known as ``pseudorandom.'' This means -that while the numbers in a sequence appear to be random, you can in +random. They are technically known as @dfn{pseudorandom}. This means +that although the numbers in a sequence appear to be random, you can in fact generate the same sequence of random numbers over and over again.} Thus, if the seed is set to the same value a second time, the same sequence of random numbers is produced again. @@ -16992,7 +17238,7 @@ implementations. The functions in this @value{SECTION} look at or change the text of one or more strings. -@code{gawk} understands locales (@pxref{Locales}), and does all +@command{gawk} understands locales (@pxref{Locales}) and does all string processing in terms of @emph{characters}, not @emph{bytes}. This distinction is particularly important to understand for locales where one character may be represented by multiple bytes. Thus, for @@ -17011,7 +17257,7 @@ doing index calculations, particularly if you are used to C. In the following list, optional parameters are enclosed in square brackets@w{ ([ ]).} Several functions perform string substitution; the full discussion is provided in the description of the @code{sub()} function, which comes -towards the end since the list is presented alphabetically. +toward the end, because the list is presented alphabetically. Those functions that are specific to @command{gawk} are marked with a pound sign (@samp{#}). They are not available in compatibility mode @@ -17037,10 +17283,10 @@ These two functions are similar in behavior, so they are described together. @quotation NOTE -The following description ignores the third argument, @var{how}, since it +The following description ignores the third argument, @var{how}, as it requires understanding features that we have not discussed yet. Thus, the discussion here is a deliberate simplification. (We do provide all -the details later on: @xref{Array Sorting Functions}, for the full story.) +the details later on; see @DBREF{Array Sorting Functions} for the full story.) @end quotation Both functions return the number of elements in the array @var{source}. @@ -17081,7 +17327,7 @@ a[2] = "de" a[3] = "sac" @end example -The @code{asorti()} function works similarly to @code{asort()}, however, +The @code{asorti()} function works similarly to @code{asort()}; however, the @emph{indices} are sorted, instead of the values. Thus, in the previous example, starting with the same initial set of indices and values in @code{a}, calling @samp{asorti(a)} would yield: @@ -17192,10 +17438,11 @@ $ @kbd{awk 'BEGIN @{ print index("peanut", "an") @}'} @noindent If @var{find} is not found, @code{index()} returns zero. +@cindex dark corner, regexp as second argument to @code{index()} With BWK @command{awk} and @command{gawk}, it is a fatal error to use a regexp constant for @var{find}. Other implementations allow it, simply treating the regexp -constant as an expression meaning @samp{$0 ~ /regexp/}. @value{DARKCORNER}. +constant as an expression meaning @samp{$0 ~ /regexp/}. @value{DARKCORNER} @item @code{length(}[@var{string}]@code{)} @cindexawkfunc{length} @@ -17278,7 +17525,7 @@ If @option{--posix} is supplied, using an array argument is a fatal error @cindex string, regular expression match @cindex match regexp in string Search @var{string} for the -longest, leftmost substring matched by the regular expression, +longest, leftmost substring matched by the regular expression @var{regexp} and return the character position (index) at which that substring begins (one, if it starts at the beginning of @var{string}). If no match is found, return zero. @@ -17286,11 +17533,11 @@ at which that substring begins (one, if it starts at the beginning of The @var{regexp} argument may be either a regexp constant (@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}). In the latter case, the string is treated as a regexp to be matched. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between the two forms, and the implications for writing your program correctly. -The order of the first two arguments is backwards from most other string +The order of the first two arguments is the opposite of most other string functions that work with regular expressions, such as @code{sub()} and @code{gsub()}. It might help to remember that for @code{match()}, the order is the same as for the @samp{~} operator: @@ -17379,7 +17626,7 @@ $ @kbd{echo foooobazbarrrrr |} @end example There may not be subscripts for the start and index for every parenthesized -subexpression, since they may not all have matched text; thus they +subexpression, because they may not all have matched text; thus, they should be tested for with the @code{in} operator (@pxref{Reference to Elements}). @@ -17426,15 +17673,15 @@ a regexp describing where to split @var{string} (much as @code{FS} can be a regexp describing where to split input records). If @var{fieldsep} is omitted, the value of @code{FS} is used. @code{split()} returns the number of elements created. -@var{seps} is a @command{gawk} extension with @code{@var{seps}[@var{i}]} +@var{seps} is a @command{gawk} extension, with @code{@var{seps}[@var{i}]} being the separator string -between @code{@var{array}[@var{i}]} and @code{@var{array}[@var{i}+1]}. +between @code{@var{array}[@var{i}]} and @code{@var{array}[@var{i}+1]}. If @var{fieldsep} is a single -space then any leading whitespace goes into @code{@var{seps}[0]} and +space, then any leading whitespace goes into @code{@var{seps}[0]} and any trailing -whitespace goes into @code{@var{seps}[@var{n}]} where @var{n} is the -return value of -@code{split()} (that is, the number of elements in @var{array}). +whitespace goes into @code{@var{seps}[@var{n}]}, where @var{n} is the +return value of +@code{split()} (i.e., the number of elements in @var{array}). The @code{split()} function splits strings into pieces in a manner similar to the way input lines are split into fields. For example: @@ -17445,7 +17692,7 @@ split("cul-de-sac", a, "-", seps) @noindent @cindex strings splitting, example -splits the string @samp{cul-de-sac} into three fields using @samp{-} as the +splits the string @code{"cul-de-sac"} into three fields using @samp{-} as the separator. It sets the contents of the array @code{a} as follows: @example @@ -17470,21 +17717,20 @@ As with input field-splitting, when the value of @var{fieldsep} is the elements of @var{array} but not in @var{seps}, and the elements are separated by runs of whitespace. -Also as with input field-splitting, if @var{fieldsep} is the null string, each +Also, as with input field splitting, if @var{fieldsep} is the null string, each individual character in the string is split into its own array element. @value{COMMONEXT} Note, however, that @code{RS} has no effect on the way @code{split()} -works. Even though @samp{RS = ""} causes newline to also be an input +works. Even though @samp{RS = ""} causes the newline character to also be an input field separator, this does not affect how @code{split()} splits strings. @cindex dark corner, @code{split()} function Modern implementations of @command{awk}, including @command{gawk}, allow -the third argument to be a regexp constant (@code{/abc/}) as well as a -string. -@value{DARKCORNER} +the third argument to be a regexp constant (@w{@code{/}@dots{}@code{/}}) +as well as a string. @value{DARKCORNER} The POSIX standard allows this as well. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between using a string constant or a regexp constant, and the implications for writing your program correctly. @@ -17535,7 +17781,7 @@ Using the @code{strtonum()} function is @emph{not} the same as adding zero to a string value; the automatic coercion of strings to numbers works only for decimal data, not for octal or hexadecimal.@footnote{Unless you use the @option{--non-decimal-data} option, which isn't recommended. -@xref{Nondecimal Data}, for more information.} +@DBXREF{Nondecimal Data} for more information.} Note also that @code{strtonum()} uses the current locale's decimal point for recognizing numbers (@pxref{Locales}). @@ -17553,7 +17799,7 @@ Return the number of substitutions made (zero or one). The @var{regexp} argument may be either a regexp constant (@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}). In the latter case, the string is treated as a regexp to be matched. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between the two forms, and the implications for writing your program correctly. @@ -17619,7 +17865,7 @@ an @samp{&}: @cindex @code{sub()} function, arguments of @cindex @code{gsub()} function, arguments of As mentioned, the third argument to @code{sub()} must -be a variable, field or array element. +be a variable, field, or array element. Some versions of @command{awk} allow the third argument to be an expression that is not an lvalue. In such a case, @code{sub()} still searches for the pattern and returns zero or one, but the result of @@ -17772,7 +18018,7 @@ Although this makes a certain amount of sense, it can be surprising. @node Gory Details -@subsubsection More About @samp{\} and @samp{&} with @code{sub()}, @code{gsub()}, and @code{gensub()} +@subsubsection More about @samp{\} and @samp{&} with @code{sub()}, @code{gsub()}, and @code{gensub()} @cindex escape processing, @code{gsub()}/@code{gensub()}/@code{sub()} functions @cindex @code{sub()} function, escape processing @@ -17811,15 +18057,15 @@ example, @code{"a\qb"} is treated as @code{"aqb"}. At the runtime level, the various functions handle sequences of @samp{\} and @samp{&} differently. The situation is (sadly) somewhat complex. -Historically, the @code{sub()} and @code{gsub()} functions treated the two -character sequence @samp{\&} specially; this sequence was replaced in +Historically, the @code{sub()} and @code{gsub()} functions treated the +two-character sequence @samp{\&} specially; this sequence was replaced in the generated text with a single @samp{&}. Any other @samp{\} within the @var{replacement} string that did not precede an @samp{&} was passed through unchanged. This is illustrated in @ref{table-sub-escapes}. @c Thank to Karl Berry for help with the TeX stuff. @float Table,table-sub-escapes -@caption{Historical Escape Sequence Processing for @code{sub()} and @code{gsub()}} +@caption{Historical escape sequence processing for @code{sub()} and @code{gsub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17870,7 +18116,7 @@ _bigskip} @end float @noindent -This table shows both the lexical-level processing, where +This table shows the lexical-level processing, where an odd number of backslashes becomes an even number at the runtime level, as well as the runtime processing done by @code{sub()}. (For the sake of simplicity, the rest of the following tables only show the @@ -17891,7 +18137,7 @@ This is shown in @ref{table-sub-proposed}. @float Table,table-sub-proposed -@caption{GNU @command{awk} Rules For @code{sub()} And Backslash} +@caption{@command{gawk} rules for @code{sub()} and backslash} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17936,7 +18182,7 @@ _bigskip} @end float In a nutshell, at the runtime level, there are now three special sequences -of characters (@samp{\\\&}, @samp{\\&} and @samp{\&}) whereas historically +of characters (@samp{\\\&}, @samp{\\&}, and @samp{\&}) whereas historically there was only one. However, as in the historical case, any @samp{\} that is not part of one of these three sequences is not special and appears in the output literally. @@ -17954,7 +18200,7 @@ by anything else is not special; the @samp{\} is placed straight into the output These rules are presented in @ref{table-posix-sub}. @float Table,table-posix-sub -@caption{POSIX Rules For @code{sub()} And @code{gsub()}} +@caption{POSIX rules for @code{sub()} and @code{gsub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -18002,13 +18248,13 @@ The only case where the difference is noticeable is the last one: @samp{\\\\} is seen as @samp{\\} and produces @samp{\} instead of @samp{\\}. Starting with @value{PVERSION} 3.1.4, @command{gawk} followed the POSIX rules -when @option{--posix} is specified (@pxref{Options}). Otherwise, -it continued to follow the proposed rules, since +when @option{--posix} was specified (@pxref{Options}). Otherwise, +it continued to follow the proposed rules, as that had been its behavior for many years. When @value{PVERSION} 4.0.0 was released, the @command{gawk} maintainer made the POSIX rules the default, breaking well over a decade's worth -of backwards compatibility.@footnote{This was rather naive of him, despite +of backward compatibility.@footnote{This was rather naive of him, despite there being a note in this section indicating that the next major version would move to the POSIX rules.} Needless to say, this was a bad idea, and as of @value{PVERSION} 4.0.1, @command{gawk} resumed its historical @@ -18023,7 +18269,7 @@ appears in the generated text and the @samp{\} does not, as shown in @ref{table-gensub-escapes}. @float Table,table-gensub-escapes -@caption{Escape Sequence Processing For @code{gensub()}} +@caption{Escape sequence processing for @code{gensub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -18070,7 +18316,7 @@ _bigskip} @end ifnottex @end float -Because of the complexity of the lexical and runtime level processing +Because of the complexity of the lexical- and runtime-level processing and the special cases for @code{sub()} and @code{gsub()}, we recommend the use of @command{gawk} and @code{gensub()} when you have to do substitutions. @@ -18090,12 +18336,13 @@ Optional parameters are enclosed in square brackets ([ ]): Close the file @var{filename} for input or output. Alternatively, the argument may be a shell command that was used for creating a coprocess, or for redirecting to or from a pipe; then the coprocess or pipe is closed. -@xref{Close Files And Pipes}, +@DBXREF{Close Files And Pipes} for more information. When closing a coprocess, it is occasionally useful to first close one end of the two-way pipe and then to close the other. This is done by providing a second argument to @code{close()}. This second argument +(@var{how}) should be one of the two string values @code{"to"} or @code{"from"}, indicating which end of the pipe to close. Case in the string does not matter. @@ -18114,25 +18361,25 @@ a pipe or coprocess. @cindex buffers, flushing @cindex output, buffering -Many utility programs @dfn{buffer} their output; i.e., they save information +Many utility programs @dfn{buffer} their output (i.e., they save information to write to a disk file or the screen in memory until there is enough -for it to be worthwhile to send the data to the output device. +for it to be worthwhile to send the data to the output device). This is often more efficient than writing every little bit of information as soon as it is ready. However, sometimes -it is necessary to force a program to @dfn{flush} its buffers; that is, -write the information to its destination, even if a buffer is not full. +it is necessary to force a program to @dfn{flush} its buffers (i.e., +write the information to its destination, even if a buffer is not full). This is the purpose of the @code{fflush()} function---@command{gawk} also -buffers its output and the @code{fflush()} function forces +buffers its output, and the @code{fflush()} function forces @command{gawk} to flush its buffers. @cindex extensions, common@comma{} @code{fflush()} function @cindex Brian Kernighan's @command{awk} Brian Kernighan added @code{fflush()} to his @command{awk} in April -of 1992. For two decades, it was a common extension. In December, +1992. For two decades, it was a common extension. In December 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=634, the Austin Group website}. -POSIX standardizes @code{fflush()} as follows: If there +POSIX standardizes @code{fflush()} as follows: if there is no argument, or if the argument is the null string (@w{@code{""}}), then @command{awk} flushes the buffers for @emph{all} open output files and pipes. @@ -18143,7 +18390,7 @@ would flush only the standard output if there was no argument, and flush all output files and pipes if the argument was the null string. This was changed in order to be compatible with Brian Kernighan's @command{awk}, in the hope that standardizing this -feature in POSIX would then be easier (which indeed helped). +feature in POSIX would then be easier (which indeed proved to be the case). With @command{gawk}, you can use @samp{fflush("/dev/stdout")} if you wish to flush @@ -18154,7 +18401,7 @@ only the standard output. @c @cindex warnings, automatic @cindex troubleshooting, @code{fflush()} function @code{fflush()} returns zero if the buffer is successfully flushed; -otherwise, it returns non-zero. (@command{gawk} returns @minus{}1.) +otherwise, it returns a nonzero value. (@command{gawk} returns @minus{}1.) In the case where all buffers are flushed, the return value is zero only if all buffers were flushed successfully. Otherwise, it is @minus{}1, and @command{gawk} warns about the problem @var{filename}. @@ -18164,53 +18411,6 @@ a file or pipe that was opened for reading (such as with @code{getline}), or if @var{filename} is not an open file, pipe, or coprocess. In such a case, @code{fflush()} returns @minus{}1, as well. -@item @code{system(@var{command})} -@cindexawkfunc{system} -@cindex invoke shell command -@cindex interacting with other programs -Execute the operating-system -command @var{command} and then return to the @command{awk} program. -Return @var{command}'s exit status. - -For example, if the following fragment of code is put in your @command{awk} -program: - -@example -END @{ - system("date | mail -s 'awk run done' root") -@} -@end example - -@noindent -the system administrator is sent mail when the @command{awk} program -finishes processing input and begins its end-of-input processing. - -Note that redirecting @code{print} or @code{printf} into a pipe is often -enough to accomplish your task. If you need to run many commands, it -is more efficient to simply print them down a pipeline to the shell: - -@example -while (@var{more stuff to do}) - print @var{command} | "/bin/sh" -close("/bin/sh") -@end example - -@noindent -@cindex troubleshooting, @code{system()} function -@cindex @option{--sandbox} option, disabling @code{system()} function -However, if your @command{awk} -program is interactive, @code{system()} is useful for running large -self-contained programs, such as a shell or an editor. -Some operating systems cannot implement the @code{system()} function. -@code{system()} causes a fatal error if it is not supported. - -@quotation NOTE -When @option{--sandbox} is specified, the @code{system()} function is disabled -(@pxref{Options}). -@end quotation - -@end table - @cindex sidebar, Interactive Versus Noninteractive Buffering @ifdocbook @docbook @@ -18219,16 +18419,16 @@ When @option{--sandbox} is specified, the @code{system()} function is disabled @cindex buffering, interactive vs.@: noninteractive -As a side point, buffering issues can be even more confusing, depending -upon whether your program is @dfn{interactive}, i.e., communicating -with a user sitting at a keyboard.@footnote{A program is interactive +As a side point, buffering issues can be even more confusing if +your program is @dfn{interactive} (i.e., communicating +with a user sitting at a keyboard).@footnote{A program is interactive if the standard output is connected to a terminal device. On modern systems, this means your keyboard and screen.} @c Thanks to Walter.Mecky@dresdnerbank.de for this example, and for @c motivating me to write this section. -Interactive programs generally @dfn{line buffer} their output; i.e., they -write out every line. Noninteractive programs wait until they have +Interactive programs generally @dfn{line buffer} their output (i.e., they +write out every line). Noninteractive programs wait until they have a full buffer, which may be many lines of output. Here is an example of the difference: @@ -18270,16 +18470,16 @@ it is all buffered and sent down the pipe to @command{cat} in one shot. @cindex buffering, interactive vs.@: noninteractive -As a side point, buffering issues can be even more confusing, depending -upon whether your program is @dfn{interactive}, i.e., communicating -with a user sitting at a keyboard.@footnote{A program is interactive +As a side point, buffering issues can be even more confusing if +your program is @dfn{interactive} (i.e., communicating +with a user sitting at a keyboard).@footnote{A program is interactive if the standard output is connected to a terminal device. On modern systems, this means your keyboard and screen.} @c Thanks to Walter.Mecky@dresdnerbank.de for this example, and for @c motivating me to write this section. -Interactive programs generally @dfn{line buffer} their output; i.e., they -write out every line. Noninteractive programs wait until they have +Interactive programs generally @dfn{line buffer} their output (i.e., they +write out every line). Noninteractive programs wait until they have a full buffer, which may be many lines of output. Here is an example of the difference: @@ -18311,6 +18511,53 @@ it is all buffered and sent down the pipe to @command{cat} in one shot. @end cartouche @end ifnotdocbook +@item @code{system(@var{command})} +@cindexawkfunc{system} +@cindex invoke shell command +@cindex interacting with other programs +Execute the operating system +command @var{command} and then return to the @command{awk} program. +Return @var{command}'s exit status. + +For example, if the following fragment of code is put in your @command{awk} +program: + +@example +END @{ + system("date | mail -s 'awk run done' root") +@} +@end example + +@noindent +the system administrator is sent mail when the @command{awk} program +finishes processing input and begins its end-of-input processing. + +Note that redirecting @code{print} or @code{printf} into a pipe is often +enough to accomplish your task. If you need to run many commands, it +is more efficient to simply print them down a pipeline to the shell: + +@example +while (@var{more stuff to do}) + print @var{command} | "/bin/sh" +close("/bin/sh") +@end example + +@noindent +@cindex troubleshooting, @code{system()} function +@cindex @option{--sandbox} option, disabling @code{system()} function +However, if your @command{awk} +program is interactive, @code{system()} is useful for running large +self-contained programs, such as a shell or an editor. +Some operating systems cannot implement the @code{system()} function. +@code{system()} causes a fatal error if it is not supported. + +@quotation NOTE +When @option{--sandbox} is specified, the @code{system()} function is disabled +(@pxref{Options}). +@end quotation + +@end table + @cindex sidebar, Controlling Output Buffering with @code{system()} @ifdocbook @docbook @@ -18334,7 +18581,7 @@ system("") # flush output @command{gawk} treats this use of the @code{system()} function as a special case and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with @command{gawk}, this -idiom is not only useful, it is also efficient. While this method should work +idiom is not only useful, it is also efficient. Although this method should work with other @command{awk} implementations, it does not necessarily avoid starting an unnecessary shell. (Other implementations may only flush the buffer associated with the standard output and not necessarily @@ -18399,7 +18646,7 @@ system("") # flush output @command{gawk} treats this use of the @code{system()} function as a special case and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with @command{gawk}, this -idiom is not only useful, it is also efficient. While this method should work +idiom is not only useful, it is also efficient. Although this method should work with other @command{awk} implementations, it does not necessarily avoid starting an unnecessary shell. (Other implementations may only flush the buffer associated with the standard output and not necessarily @@ -18443,18 +18690,14 @@ you would see the latter (undesirable) output. @subsection Time Functions @cindex time functions -@c STARTOFRANGE tst @cindex timestamps -@c STARTOFRANGE logftst @cindex log files, timestamps in -@c STARTOFRANGE filogtst @cindex files, log@comma{} timestamps in -@c STARTOFRANGE gawtst @cindex @command{gawk}, timestamps @cindex POSIX @command{awk}, timestamps and -@code{awk} programs are commonly used to process log files +@command{awk} programs are commonly used to process log files containing timestamp information, indicating when a -particular log record was written. Many programs log their timestamp +particular log record was written. Many programs log their timestamps in the form returned by the @code{time()} system call, which is the number of seconds since a particular epoch. On POSIX-compliant systems, it is the number of seconds since @@ -18481,6 +18724,7 @@ which is sufficient to represent times through 2038-01-19 03:14:07 UTC. Many systems support a wider range of timestamps, including negative timestamps that represent times before the epoch. +@c FIXME: Use @sup here for superscript @cindex @command{date} utility, GNU @cindex time, retrieving @@ -18515,7 +18759,7 @@ The values of these numbers need not be within the ranges specified; for example, an hour of @minus{}1 means 1 hour before midnight. The origin-zero Gregorian calendar is assumed, with year 0 preceding year 1 and year @minus{}1 preceding year 0. -The time is assumed to be in the local timezone. +The time is assumed to be in the local time zone. If the daylight-savings flag is positive, the time is assumed to be daylight savings time; if zero, the time is assumed to be standard time; and if negative (the default), @code{mktime()} attempts to determine @@ -18527,7 +18771,6 @@ is out of range, @code{mktime()} returns @minus{}1. @cindex @command{gawk}, @code{PROCINFO} array in @cindex @code{PROCINFO} array @item @code{strftime(}[@var{format} [@code{,} @var{timestamp} [@code{,} @var{utc-flag}] ] ]@code{)} -@c STARTOFRANGE strf @cindexgawkfunc{strftime} @cindex format time string Format the time specified by @var{timestamp} @@ -18539,14 +18782,14 @@ Mean Time). Otherwise, the value is formatted for the local time zone. The @var{timestamp} is in the same format as the value returned by the @code{systime()} function. If no @var{timestamp} argument is supplied, @command{gawk} uses the current time of day as the timestamp. -If no @var{format} argument is supplied, @code{strftime()} uses +Without a @var{format} argument, @code{strftime()} uses the value of @code{PROCINFO["strftime"]} as the format string (@pxref{Built-in Variables}). The default string value is @code{@w{"%a %b %e %H:%M:%S %Z %Y"}}. This format string produces output that is equivalent to that of the @command{date} utility. You can assign a new value to @code{PROCINFO["strftime"]} to -change the default format; see below for the various format directives. +change the default format; see the following list for the various format directives. @item @code{systime()} @cindexgawkfunc{systime} @@ -18623,16 +18866,16 @@ This is the ISO 8601 date format. @item %g The year modulo 100 of the ISO 8601 week number, as a decimal number (00--99). -For example, January 1, 2012 is in week 53 of 2011. Thus, the year +For example, January 1, 2012, is in week 53 of 2011. Thus, the year of its ISO 8601 week number is 2011, even though its year is 2012. -Similarly, December 31, 2012 is in week 1 of 2013. Thus, the year +Similarly, December 31, 2012, is in week 1 of 2013. Thus, the year of its ISO week number is 2013, even though its year is 2012. @item %G The full year of the ISO week number, as a decimal number. @item %h -Equivalent to @code{%b}. +Equivalent to @samp{%b}. @item %H The hour (24-hour clock) as a decimal number (00--23). @@ -18676,12 +18919,12 @@ Equivalent to specifying @samp{%H:%M:%S}. The weekday as a decimal number (1--7). Monday is day one. @item %U -The week number of the year (the first Sunday as the first day of week one) +The week number of the year (with the first Sunday as the first day of week one) as a decimal number (00--53). @c @cindex ISO 8601 @item %V -The week number of the year (the first Monday as the first +The week number of the year (with the first Monday as the first day of week one) as a decimal number (01--53). The method for determining the week number is as specified by ISO 8601. (To wit: if the week containing January 1 has four or more days in the @@ -18692,7 +18935,7 @@ and the next week is week one.) The weekday as a decimal number (0--6). Sunday is day zero. @item %W -The week number of the year (the first Monday as the first day of week one) +The week number of the year (with the first Monday as the first day of week one) as a decimal number (00--53). @item %x @@ -18701,7 +18944,7 @@ The locale's ``appropriate'' date representation. @item %X The locale's ``appropriate'' time representation. -(This is @code{%T} in the @code{"C"} locale.) +(This is @samp{%T} in the @code{"C"} locale.) @item %y The year modulo 100 as a decimal number (00--99). @@ -18712,8 +18955,8 @@ The full year as a decimal number (e.g., 2015). @c @cindex RFC 822 @c @cindex RFC 1036 @item %z -The timezone offset in a +HHMM format (e.g., the format necessary to -produce RFC 822/RFC 1036 date headers). +The time zone offset in a @samp{+@var{HHMM}} format (e.g., the format +necessary to produce RFC 822/RFC 1036 date headers). @item %Z The time zone name or abbreviation; no characters if @@ -18721,8 +18964,8 @@ no time zone is determinable. @item %Ec %EC %Ex %EX %Ey %EY %Od %Oe %OH @itemx %OI %Om %OM %OS %Ou %OU %OV %Ow %OW %Oy -``Alternate representations'' for the specifications -that use only the second letter (@code{%c}, @code{%C}, +``Alternative representations'' for the specifications +that use only the second letter (@samp{%c}, @samp{%C}, and so on).@footnote{If you don't understand any of this, don't worry about it; these facilities are meant to make it easier to ``internationalize'' programs. @@ -18734,7 +18977,7 @@ Other internationalization features are described in A literal @samp{%}. @end table -If a conversion specifier is not one of the above, the behavior is +If a conversion specifier is not one of those just listed, the behavior is undefined.@footnote{This is because ISO C leaves the behavior of the C version of @code{strftime()} undefined and @command{gawk} uses the system's version of @code{strftime()} if it's there. @@ -18761,11 +19004,11 @@ Single-digit numbers are padded with a space. @ignore @item %N The ``Emperor/Era'' name. -Equivalent to @code{%C}. +Equivalent to @samp{%C}. @item %o The ``Emperor/Era'' year. -Equivalent to @code{%y}. +Equivalent to @samp{%y}. @end ignore @item %s @@ -18776,9 +19019,8 @@ The time as a decimal timestamp in seconds since the epoch. The date in VMS format (e.g., @samp{20-JUN-1991}). @end ignore @end table -@c ENDOFRANGE strf -Additionally, the alternate representations are recognized but their +Additionally, the alternative representations are recognized but their normal representations are used. @cindex @code{date} utility, POSIX @@ -18792,7 +19034,7 @@ interprets the current time according to the format specifiers in the string. For example: @example -$ date '+Today is %A, %B %d, %Y.' +$ @kbd{date '+Today is %A, %B %d, %Y.'} @print{} Today is Monday, September 22, 2014. @end example @@ -18827,23 +19069,14 @@ gawk 'BEGIN @{ exit exitval @}' "$@@" @end example -@c ENDOFRANGE tst -@c ENDOFRANGE logftst -@c ENDOFRANGE filogtst -@c ENDOFRANGE gawtst @node Bitwise Functions @subsection Bit-Manipulation Functions @cindex bit-manipulation functions -@c STARTOFRANGE bit @cindex bitwise, operations -@c STARTOFRANGE and @cindex AND bitwise operation -@c STARTOFRANGE oro @cindex OR bitwise operation -@c STARTOFRANGE xor @cindex XOR bitwise operation -@c STARTOFRANGE opbit @cindex operations, bitwise @quotation @i{I can explain it for you, but I can't understand it for you.} @@ -18856,12 +19089,14 @@ each successive pair of bits in the operands. Three common operations are bitwise AND, OR, and XOR. The operations are described in @ref{table-bitwise-ops}. +@c 11/2014: Postprocessing turns the docbook informaltable +@c into a table. Hurray for scripting! @float Table,table-bitwise-ops -@caption{Bitwise Operations} +@caption{Bitwise operations} @ifnottex @ifnotdocbook @display - Bit Operator + Bit operator | AND | OR | XOR |---+---+---+---+---+--- Operands | 0 | 1 | 0 | 1 | 0 | 1 @@ -18919,7 +19154,7 @@ Operands | 0 | 1 | 0 | 1 | 0 | 1 <tbody> <row> <entry colsep="0"></entry> -<entry spanname="optitle"><emphasis role="bold">Bit Operator</emphasis></entry> +<entry spanname="optitle"><emphasis role="bold">Bit operator</emphasis></entry> </row> <row rowsep="1"> @@ -18983,10 +19218,9 @@ of a given value. Finally, two other common operations are to shift the bits left or right. For example, if you have a bit string @samp{10111001} and you shift it right by three bits, you end up with @samp{00010111}.@footnote{This example -shows that 0's come in on the left side. For @command{gawk}, this is +shows that zeros come in on the left side. For @command{gawk}, this is always true, but in some languages, it's possible to have the left side -fill with 1's.} -@c Purposely decided to use 0's and 1's here. 2/2001. +fill with ones.} If you start over again with @samp{10111001} and shift it left by three bits, you end up with @samp{11001000}. The following list describes @command{gawk}'s built-in functions that implement the bitwise operations. @@ -19025,7 +19259,7 @@ Return the value of @var{val}, shifted right by @var{count} bits. Return the bitwise XOR of the arguments. There must be at least two. @end table -For all of these functions, first the double precision floating-point value is +For all of these functions, first the double-precision floating-point value is converted to the widest C unsigned integer type, then the bitwise operation is performed. If the result cannot be represented exactly as a C @code{double}, leading nonzero bits are removed one by one until it can be represented @@ -19040,7 +19274,7 @@ that illustrates the use of these functions: @example @group @c file eg/lib/bits2str.awk -# bits2str --- turn a byte into readable 1's and 0's +# bits2str --- turn a byte into readable ones and zeros function bits2str(bits, data, mask) @{ @@ -19114,17 +19348,18 @@ $ @kbd{gawk -f testbits.awk} @cindex converting, numbers to strings @cindex number as string of bits The @code{bits2str()} function turns a binary number into a string. -The number @code{1} represents a binary value where the rightmost bit -is set to 1. Using this mask, +Initializing @code{mask} to one creates +a binary value where the rightmost bit +is set to one. Using this mask, the function repeatedly checks the rightmost bit. ANDing the mask with the value indicates whether the -rightmost bit is 1 or not. If so, a @code{"1"} is concatenated onto the front +rightmost bit is one or not. If so, a @code{"1"} is concatenated onto the front of the string. Otherwise, a @code{"0"} is added. The value is then shifted right by one bit and the loop continues -until there are no more 1 bits. +until there are no more one bits. -If the initial value is zero it returns a simple @code{"0"}. +If the initial value is zero, it returns a simple @code{"0"}. Otherwise, at the end, it pads the value with zeros to represent multiples of 8-bit quantities. This is typical in modern computers. @@ -19133,11 +19368,6 @@ decimal and octal values for the same numbers (@pxref{Nondecimal-numbers}), and then demonstrates the results of the @code{compl()}, @code{lshift()}, and @code{rshift()} functions. -@c ENDOFRANGE bit -@c ENDOFRANGE and -@c ENDOFRANGE oro -@c ENDOFRANGE xor -@c ENDOFRANGE opbit @node Type Functions @subsection Getting Type Information @@ -19151,7 +19381,7 @@ that traverses every element of an array of arrays @cindexgawkfunc{isarray} @cindex scalar or array @item isarray(@var{x}) -Return a true value if @var{x} is an array. Otherwise return false. +Return a true value if @var{x} is an array. Otherwise, return false. @end table @code{isarray()} is meant for use in two circumstances. The first is when @@ -19161,8 +19391,8 @@ an array or not. The second is inside the body of a user-defined function array or not. @quotation NOTE -Using @code{isarray()} at the global level to test -variables makes no sense. Since you are the one writing the program, you +Using @code{isarray()} at the global level to test +variables makes no sense. Because you are the one writing the program, you are supposed to know if your variables are arrays or not. And in fact, due to the way @command{gawk} works, if you pass the name of a variable that has not been previously used to @code{isarray()}, @command{gawk} @@ -19212,25 +19442,21 @@ The default value for @var{category} is @code{"LC_MESSAGES"}. Return the plural form used for @var{number} of the translation of @var{string1} and @var{string2} in text domain @var{domain} for locale category @var{category}. @var{string1} is the -English singular variant of a message, and @var{string2} the English plural +English singular variant of a message, and @var{string2} is the English plural variant of the same message. The default value for @var{domain} is the current value of @code{TEXTDOMAIN}. The default value for @var{category} is @code{"LC_MESSAGES"}. @end table -@c ENDOFRANGE funcbi -@c ENDOFRANGE bifunc @node User-defined @section User-Defined Functions -@c STARTOFRANGE udfunc @cindex user-defined functions -@c STARTOFRANGE funcud @cindex functions, user-defined Complicated @command{awk} programs can often be simplified by defining your own functions. User-defined functions can be called just like built-in ones (@pxref{Function Calls}), but it is up to you to define -them, i.e., to tell @command{awk} what they should do. +them (i.e., to tell @command{awk} what they should do). @menu * Definition Syntax:: How to write definitions and what they mean. @@ -19245,12 +19471,11 @@ them, i.e., to tell @command{awk} what they should do. @subsection Function Definition Syntax @quotation -@i{It's entirely fair to say that the @command{awk} syntax for local +@i{It's entirely fair to say that the awk syntax for local variable definitions is appallingly awful.} @author Brian Kernighan @end quotation -@c STARTOFRANGE fdef @cindex functions, defining Definitions of functions can appear anywhere between the rules of an @command{awk} program. Thus, the general form of an @command{awk} program is @@ -19288,14 +19513,23 @@ the call. A function cannot have two parameters with the same name, nor may it have a parameter with the same name as the function itself. -In addition, according to the POSIX standard, function parameters + +@quotation CAUTION +According to the POSIX standard, function parameters cannot have the same name as one of the special predefined variables -(@pxref{Built-in Variables}). Not all versions of @command{awk} enforce -this restriction. +(@pxref{Built-in Variables}), nor may a function parameter have the +same name as another function. + +Not all versions of @command{awk} enforce +these restrictions. +@command{gawk} always enforces the first restriction. +With @option{--posix} (@pxref{Options}), +it also enforces the second restriction. +@end quotation Local variables act like the empty string if referenced where a string value is required, and like zero if referenced where a numeric value -is required. This is the same as regular variables that have never been +is required. This is the same as the behavior of regular variables that have never been assigned a value. (There is more to understand about local variables; @pxref{Dynamic Typing}.) @@ -19329,7 +19563,7 @@ During execution of the function body, the arguments and local variable values hide, or @dfn{shadow}, any variables of the same names used in the rest of the program. The shadowed variables are not accessible in the function definition, because there is no way to name them while their -names have been taken away for the local variables. All other variables +names have been taken away for the arguments and local variables. All other variables used in the @command{awk} program can be referenced or set normally in the function's body. @@ -19369,13 +19603,13 @@ func foo() @{ a = sqrt($1) ; print a @} @end example @noindent -Instead it defines a rule that, for each record, concatenates the value +Instead, it defines a rule that, for each record, concatenates the value of the variable @samp{func} with the return value of the function @samp{foo}. If the resulting string is non-null, the action is executed. This is probably not what is desired. (@command{awk} accepts this input as syntactically valid, because functions may be used before they are defined in @command{awk} programs.@footnote{This program won't actually run, -since @code{foo()} is undefined.}) +because @code{foo()} is undefined.}) @cindex portability, functions@comma{} defining To ensure that your @command{awk} programs are portable, always use the @@ -19396,7 +19630,7 @@ function myprint(num) @end example @noindent -To illustrate, here is an @command{awk} rule that uses our @code{myprint} +To illustrate, here is an @command{awk} rule that uses our @code{myprint()} function: @example @@ -19437,16 +19671,16 @@ in an array and start over with a new list of elements (@pxref{Delete}). Instead of having to repeat this loop everywhere that you need to clear out -an array, your program can just call @code{delarray}. +an array, your program can just call @code{delarray()}. (This guarantees portability. The use of @samp{delete @var{array}} to delete the contents of an entire array is a relatively recent@footnote{Late in 2012.} addition to the POSIX standard.) The following is an example of a recursive function. It takes a string -as an input parameter and returns the string in backwards order. +as an input parameter and returns the string in reverse order. Recursive functions must always have a test that stops the recursion. In this case, the recursion terminates when the input string is -already empty. +already empty: @c 8/2014: Thanks to Mike Brennan for the improved formulation @cindex @code{rev()} user-defined function @@ -19494,15 +19728,13 @@ function ctime(ts, format) @end example You might think that @code{ctime()} could use @code{PROCINFO["strftime"]} -for its format string. That would be a mistake, since @code{ctime()} is +for its format string. That would be a mistake, because @code{ctime()} is supposed to return the time formatted in a standard fashion, and user-level code could have changed @code{PROCINFO["strftime"]}. -@c ENDOFRANGE fdef @node Function Caveats @subsection Calling User-Defined Functions -@c STARTOFRANGE fudc @cindex functions, user-defined, calling @dfn{Calling a function} means causing the function to run and do its job. A function call is an expression and its value is the value returned by @@ -19515,7 +19747,7 @@ the function. @end menu @node Calling A Function -@subsubsection Writing A Function Call +@subsubsection Writing a Function Call A function call consists of the function name followed by the arguments in parentheses. @command{awk} expressions are what you write in the @@ -19530,7 +19762,7 @@ foo(x y, "lose", 4 * z) @quotation CAUTION Whitespace characters (spaces and TABs) are not allowed -between the function name and the open-parenthesis of the argument list. +between the function name and the opening parenthesis of the argument list. If you write whitespace by mistake, @command{awk} might think that you mean to concatenate a variable with an expression in parentheses. However, it notices that you used a function name and not a variable name, and reports @@ -19542,7 +19774,7 @@ an error. @cindex local variables, in a function @cindex variables, local to a function -Unlike many languages, +Unlike in many languages, there is no way to make a variable local to a @code{@{} @dots{} @code{@}} block in @command{awk}, but you can make a variable local to a function. It is good practice to do so whenever a variable is needed only in that @@ -19551,7 +19783,7 @@ function. To make a variable local to a function, simply declare the variable as an argument after the actual function arguments (@pxref{Definition Syntax}). -Look at the following example where variable +Look at the following example, where variable @code{i} is a global variable used by both functions @code{foo()} and @code{bar()}: @@ -19570,7 +19802,7 @@ function foo(j) print "foo's i=" i @} -BEGIN @{ +BEGIN @{ i = 10 print "top's i=" i foo(0) @@ -19592,14 +19824,14 @@ foo's i=3 top's i=3 @end example -If you want @code{i} to be local to both @code{foo()} and @code{bar()} do as -follows (the extra-space before @code{i} is a coding convention to +If you want @code{i} to be local to both @code{foo()} and @code{bar()}, do as +follows (the extra space before @code{i} is a coding convention to indicate that @code{i} is a local variable, not an argument): @example function bar( i) @{ - for (i = 0; i < 3; i++) + for (i = 0; i < 3; i++) print "bar's i=" i @} @@ -19611,10 +19843,10 @@ function foo(j, i) print "foo's i=" i @} -BEGIN @{ +BEGIN @{ i = 10 print "top's i=" i - foo(0) + foo(0) print "top's i=" i @} @end example @@ -19673,21 +19905,16 @@ At level 2, index 2 is found in a @end example @node Pass By Value/Reference -@subsubsection Passing Function Arguments By Value Or By Reference +@subsubsection Passing Function Arguments by Value Or by Reference In @command{awk}, when you declare a function, there is no way to declare explicitly whether the arguments are passed @dfn{by value} or @dfn{by reference}. -Instead the passing convention is determined at runtime when -the function is called according to the following rule: - -@itemize -@item -If the argument is an array variable, then it is passed by reference, -@item -Otherwise the argument is passed by value. -@end itemize +Instead, the passing convention is determined at runtime when +the function is called, according to the following rule: +if the argument is an array variable, then it is passed by reference. +Otherwise, the argument is passed by value. @cindex call by value Passing an argument by value means that when a function is called, it @@ -19762,7 +19989,7 @@ prints @samp{a[1] = 1, a[2] = two, a[3] = 3}, because @cindex undefined functions @cindex functions, undefined Some @command{awk} implementations allow you to call a function that -has not been defined. They only report a problem at runtime when the +has not been defined. They only report a problem at runtime, when the program actually tries to call the function. For example: @example @@ -19790,10 +20017,15 @@ If @option{--lint} is specified Some @command{awk} implementations generate a runtime error if you use either the @code{next} statement or the @code{nextfile} statement -(@pxref{Next Statement}, also @pxref{Nextfile Statement}) +(@pxref{Next Statement}, and +@ifdocbook +@ref{Nextfile Statement}) +@end ifdocbook +@ifnotdocbook +@pxref{Nextfile Statement}) +@end ifnotdocbook inside a user-defined function. @command{gawk} does not have this limitation. -@c ENDOFRANGE fudc @node Return Statement @subsection The @code{return} Statement @@ -19816,15 +20048,15 @@ makes the returned value undefined, and therefore, unpredictable. In practice, though, all versions of @command{awk} simply return the null string, which acts like zero if used in a numeric context. -A @code{return} statement with no value expression is assumed at the end of -every function definition. So if control reaches the end of the function -body, then technically, the function returns an unpredictable value. +A @code{return} statement without an @var{expression} is assumed at the end of +every function definition. So, if control reaches the end of the function +body, then technically the function returns an unpredictable value. In practice, it returns the empty string. @command{awk} does @emph{not} warn you if you use the return value of such a function. Sometimes, you want to write a function for what it does, not for what it returns. Such a function corresponds to a @code{void} function -in C, C++ or Java, or to a @code{procedure} in Ada. Thus, it may be appropriate to not +in C, C++, or Java, or to a @code{procedure} in Ada. Thus, it may be appropriate to not return any value; simply bear in mind that you should not be using the return value of such a function. @@ -19846,8 +20078,8 @@ function maxelt(vec, i, ret) @noindent You call @code{maxelt()} with one argument, which is an array name. The local variables @code{i} and @code{ret} are not intended to be arguments; -while there is nothing to stop you from passing more than one argument -to @code{maxelt()}, the results would be strange. The extra space before +there is nothing to stop you from passing more than one argument +to @code{maxelt()} but the results would be strange. The extra space before @code{i} in the function parameter list indicates that @code{i} and @code{ret} are local variables. You should follow this convention when defining functions. @@ -19921,7 +20153,6 @@ does report the second error. Usually, such things aren't a big issue, but it's worth being aware of them. -@c ENDOFRANGE udfunc @node Indirect Calls @section Indirect Function Calls @@ -19944,13 +20175,15 @@ function calls, you can specify the name of the function to call as a string variable, and then call the function. Let's look at an example. Suppose you have a file with your test scores for the classes you -are taking. The first field is the class name. The following fields +are taking, and +you wish to get the sum and the average of +your test scores. +The first field is the class name. The following fields are the functions to call to process the data, up to a ``marker'' field @samp{data:}. Following the marker, to the end of the record, are the various numeric test scores. -Here is the initial file; you wish to get the sum and the average of -your test scores: +Here is the initial file: @example @c file eg/data/class_data1 @@ -19984,8 +20217,8 @@ variable as the @emph{name} of the function to call. @cindex indirect function calls, @code{@@}-notation @cindex function calls, indirect, @code{@@}-notation for The syntax is similar to that of a regular function call: an identifier -immediately followed by a left parenthesis, any arguments, and then -a closing right parenthesis, with the addition of a leading @samp{@@} +immediately followed by an opening parenthesis, any arguments, and then +a closing parenthesis, with the addition of a leading @samp{@@} character: @example @@ -19994,7 +20227,7 @@ result = @@the_func() # calls the sum() function @end example Here is a full program that processes the previously shown data, -using indirect function calls. +using indirect function calls: @example @c file eg/prog/indirectcall.awk @@ -20033,9 +20266,9 @@ function sum(first, last, ret, i) @c endfile @end example -These two functions expect to work on fields; thus the parameters +These two functions expect to work on fields; thus, the parameters @code{first} and @code{last} indicate where in the fields to start and end. -Otherwise they perform the expected computations and are not unusual. +Otherwise, they perform the expected computations and are not unusual: @example @c file eg/prog/indirectcall.awk @@ -20068,7 +20301,7 @@ saving it in @code{start}. The last part of the code loops through each function name (from @code{$2} up to the marker, @samp{data:}), calling the function named by the field. The indirect function call itself occurs as a parameter in the call to @code{printf}. -(The @code{printf} format string uses @code{%s} as the format specifier so that we +(The @code{printf} format string uses @samp{%s} as the format specifier so that we can use functions that return strings, as well as numbers. Note that the result from the indirect call is concatenated with the empty string, in order to force it to be a string value.) @@ -20080,11 +20313,11 @@ $ @kbd{gawk -f indirectcall.awk class_data1} @print{} Biology 101: @print{} sum: <352.8> @print{} average: <88.2> -@print{} +@print{} @print{} Chemistry 305: @print{} sum: <356.4> @print{} average: <89.1> -@print{} +@print{} @print{} English 401: @print{} sum: <376.1> @print{} average: <94.025> @@ -20094,8 +20327,8 @@ The ability to use indirect function calls is more powerful than you may think at first. The C and C++ languages provide ``function pointers,'' which are a mechanism for calling a function chosen at runtime. One of the most well-known uses of this ability is the C @code{qsort()} function, which sorts -an array using the famous ``quick sort'' algorithm -(see @uref{http://en.wikipedia.org/wiki/Quick_sort, the Wikipedia article} +an array using the famous ``quicksort'' algorithm +(see @uref{http://en.wikipedia.org/wiki/Quicksort, the Wikipedia article} for more information). To use this function, you supply a pointer to a comparison function. This mechanism allows you to sort arbitrary data in an arbitrary fashion. @@ -20114,11 +20347,11 @@ We can do something similar using @command{gawk}, like this: # January 2009 @c endfile - @end ignore @c file eg/lib/quicksort.awk -# quicksort --- C.A.R. Hoare's quick sort algorithm. See Wikipedia -# or almost any algorithms or computer science text + +# quicksort --- C.A.R. Hoare's quicksort algorithm. See Wikipedia +# or almost any algorithms or computer science text. @c endfile @ignore @c file eg/lib/quicksort.awk @@ -20156,7 +20389,7 @@ function quicksort_swap(data, i, j, temp) The @code{quicksort()} function receives the @code{data} array, the starting and ending indices to sort (@code{left} and @code{right}), and the name of a function that -performs a ``less than'' comparison. It then implements the quick sort algorithm. +performs a ``less than'' comparison. It then implements the quicksort algorithm. To make use of the sorting function, we return to our previous example. The first thing to do is write some comparison functions: @@ -20206,7 +20439,7 @@ function do_sort(first, last, compare, data, i, retval) retval = data[1] for (i = 2; i in data; i++) retval = retval " " data[i] - + return retval @} @c endfile @@ -20252,13 +20485,13 @@ $ @kbd{gawk -f quicksort.awk -f indirectcall.awk class_data2} @print{} average: <88.2> @print{} sort: <78.5 87.0 92.4 94.9> @print{} rsort: <94.9 92.4 87.0 78.5> -@print{} +@print{} @print{} Chemistry 305: @print{} sum: <356.4> @print{} average: <89.1> @print{} sort: <75.2 88.2 94.7 98.3> @print{} rsort: <98.3 94.7 88.2 75.2> -@print{} +@print{} @print{} English 401: @print{} sum: <376.1> @print{} average: <94.025> @@ -20267,76 +20500,29 @@ $ @kbd{gawk -f quicksort.awk -f indirectcall.awk class_data2} @end example Another example where indirect functions calls are useful can be found in -processing arrays. @DBREF{Walking Arrays} presented a simple function -for ``walking'' an array of arrays. That function simply printed the -name and value of each scalar array element. However, it is easy to -generalize that function, by passing in the name of a function to call -when walking an array. The modified function looks like this: - -@example -@c file eg/lib/processarray.awk -function process_array(arr, name, process, do_arrays, i, new_name) -@{ - for (i in arr) @{ - new_name = (name "[" i "]") - if (isarray(arr[i])) @{ - if (do_arrays) - @@process(new_name, arr[i]) - process_array(arr[i], new_name, process, do_arrays) - @} else - @@process(new_name, arr[i]) - @} -@} -@c endfile -@end example - -The arguments are as follows: - -@table @code -@item arr -The array. - -@item name -The name of the array (a string). - -@item process -The name of the function to call. - -@item do_arrays -If this is true, the function can handle elements that are subarrays. -@end table - -If subarrays are to be processed, that is done before walking them further. - -When run with the following scaffolding, the function produces the same -results as does the earlier @code{walk_array()} function: - -@example -BEGIN @{ - a[1] = 1 - a[2][1] = 21 - a[2][2] = 22 - a[3] = 3 - a[4][1][1] = 411 - a[4][2] = 42 - - process_array(a, "a", "do_print", 0) -@} - -function do_print(name, element) -@{ - printf "%s = %s\n", name, element -@} -@end example +processing arrays. This is described in @ref{Walking Arrays}. Remember that you must supply a leading @samp{@@} in front of an indirect function call. Starting with @value{PVERSION} 4.1.2 of @command{gawk}, indirect function calls may also be used with built-in functions and with extension functions -(@pxref{Dynamic Extensions}). The only thing you cannot do is pass a regular -expression constant to a built-in function through an indirect function -call.@footnote{This may change in a future version; recheck the documentation that -comes with your version of @command{gawk} to see if it has.} +(@pxref{Dynamic Extensions}). There are some limitations when calling +built-in functions indirectly, as follows. + +@itemize @value{BULLET} +@item +You cannot pass a regular expression constant to a built-in function +through an indirect function call.@footnote{This may change in a future +version; recheck the documentation that comes with your version of +@command{gawk} to see if it has.} This applies to the @code{sub()}, +@code{gsub()}, @code{gensub()}, @code{match()}, @code{split()} and +@code{patsplit()} functions. + +@item +If calling @code{sub()} or @code{gsub()}, you may only pass two arguments, +since those functions are unusual in that they update their third argument. +This means that @code{$0} will be updated. +@end itemize @command{gawk} does its best to make indirect function calls efficient. For example, in the following case: @@ -20347,7 +20533,7 @@ for (i = 1; i <= n; i++) @end example @noindent -@code{gawk} looks up the actual function to call only once. +@command{gawk} looks up the actual function to call only once. @node Functions Summary @section Summary @@ -20361,7 +20547,7 @@ functions. POSIX @command{awk} provides three kinds of built-in functions: numeric, string, and I/O. @command{gawk} provides functions that sort arrays, work with values representing time, do bit manipulation, determine variable -type (array vs.@: scalar), and internationalize and localize programs. +type (array versus scalar), and internationalize and localize programs. @command{gawk} also provides several extensions to some of standard functions, typically in the form of additional arguments. @@ -20414,10 +20600,9 @@ program. This is equivalent to function pointers in C and C++. @end itemize -@c ENDOFRANGE funcud @ifnotinfo -@part @value{PART2}Problem Solving With @command{awk} +@part @value{PART2}Problem Solving with @command{awk} @end ifnotinfo @ifdocbook @@ -20427,27 +20612,24 @@ It contains the following chapters: @itemize @value{BULLET} @item -@ref{Library Functions}. +@ref{Library Functions} @item -@ref{Sample Programs}. +@ref{Sample Programs} @end itemize @end ifdocbook @node Library Functions @chapter A Library of @command{awk} Functions -@c STARTOFRANGE libf @cindex libraries of @command{awk} functions -@c STARTOFRANGE flib @cindex functions, library -@c STARTOFRANGE fudlib @cindex functions, user-defined, library of @DBREF{User-defined} describes how to write your own @command{awk} functions. Writing functions is important, because it allows you to encapsulate algorithms and program tasks in a single place. It simplifies programming, making program development more -manageable, and making programs more readable. +manageable and making programs more readable. @cindex Kernighan, Brian @cindex Plauger, P.J.@: @@ -20491,9 +20673,9 @@ and would like to contribute them to the @command{awk} user community, see @cindex portability, example programs The programs in this @value{CHAPTER} and in @ref{Sample Programs}, -freely use features that are @command{gawk}-specific. +freely use @command{gawk}-specific features. Rewriting these programs for different implementations of @command{awk} -is pretty straightforward. +is pretty straightforward: @itemize @value{BULLET} @item @@ -20563,7 +20745,7 @@ Library functions often need to have global variables that they can use to preserve state information between calls to the function---for example, @code{getopt()}'s variable @code{_opti} (@pxref{Getopt Function}). -Such variables are called @dfn{private}, since the only functions that need to +Such variables are called @dfn{private}, as the only functions that need to use them are the ones in the library. When writing a library function, you should try to choose names for your @@ -20576,7 +20758,7 @@ often use variable names like these for their own purposes. The example programs shown in this @value{CHAPTER} all start the names of their private variables with an underscore (@samp{_}). Users generally don't use leading underscores in their variable names, so this convention immediately -decreases the chances that the variable name will be accidentally shared +decreases the chances that the variable names will be accidentally shared with the user's program. @cindex @code{_} (underscore), in names of private variables @@ -20585,17 +20767,17 @@ In addition, several of the library functions use a prefix that helps indicate what function or set of functions use the variables---for example, @code{_pw_byname()} in the user database routines (@pxref{Passwd Functions}). -This convention is recommended, since it even further decreases the +This convention is recommended, as it even further decreases the chance of inadvertent conflict among variable names. Note that this convention is used equally well for variable names and for private -function names.@footnote{While all the library routines could have +function names.@footnote{Although all the library routines could have been rewritten to use this convention, this was not done, in order to show how our own @command{awk} programming style has evolved and to provide some basis for this discussion.} As a final note on variable naming, if a function makes global variables -available for use by a main program, it is a good convention to start that -variable's name with a capital letter---for +available for use by a main program, it is a good convention to start those +variables' names with a capital letter---for example, @code{getopt()}'s @code{Opterr} and @code{Optind} variables (@pxref{Getopt Function}). The leading capital letter indicates that it is global, while the fact that @@ -20606,7 +20788,7 @@ not one of @command{awk}'s predefined variables, such as @code{FS}. It is also important that @emph{all} variables in library functions that do not need to save state are, in fact, declared local.@footnote{@command{gawk}'s @option{--dump-variables} command-line -option is useful for verifying this.} If this is not done, the variable +option is useful for verifying this.} If this is not done, the variables could accidentally be used in the user's program, leading to bugs that are very difficult to track down: @@ -20661,7 +20843,7 @@ programming use. @end menu @node Strtonum Function -@subsection Converting Strings To Numbers +@subsection Converting Strings to Numbers The @code{strtonum()} function (@pxref{String Functions}) is a @command{gawk} extension. The following function @@ -20729,7 +20911,7 @@ function mystrtonum(str, ret, n, i, k, c) # a[6] = "1.e3" # a[7] = "1.32" # a[8] = "1.32E2" -# +# # for (i = 1; i in a; i++) # print a[i], strtonum(a[i]), mystrtonum(a[i]) # @} @@ -20743,7 +20925,7 @@ string. It sets @code{k} to the index in @code{"1234567"} of the current octal digit. The return value will either be the same number as the digit, or zero if the character is not there, which will be true for a @samp{0}. -This is safe, since the regexp test in the @code{if} ensures that +This is safe, because the regexp test in the @code{if} ensures that only octal values are converted. Similar logic applies to the code that checks for and converts a @@ -20763,13 +20945,9 @@ be tested with @command{gawk} and the results compared to the built-in @node Assert Function @subsection Assertions -@c STARTOFRANGE asse @cindex assertions -@c STARTOFRANGE assef @cindex @code{assert()} function (C library) -@c STARTOFRANGE libfass @cindex libraries of @command{awk} functions, assertions -@c STARTOFRANGE flibass @cindex functions, library, assertions @cindex @command{awk} programs, lengthy, assertions When writing large programs, it is often useful to know @@ -20808,7 +20986,7 @@ Following is the function: @example @c file eg/lib/assert.awk -# assert --- assert that a condition is true. Otherwise exit. +# assert --- assert that a condition is true. Otherwise, exit. @c endfile @ignore @@ -20844,7 +21022,7 @@ is false, it prints a message to standard error, using the @code{string} parameter to describe the failed condition. It then sets the variable @code{_assert_exit} to one and executes the @code{exit} statement. The @code{exit} statement jumps to the @code{END} rule. If the @code{END} -rules finds @code{_assert_exit} to be true, it exits immediately. +rule finds @code{_assert_exit} to be true, it exits immediately. The purpose of the test in the @code{END} rule is to keep any other @code{END} rules from running. When an assertion fails, the @@ -20885,10 +21063,6 @@ most likely causing the program to hang as it waits for input. There is a simple workaround to this: make sure that such a @code{BEGIN} rule always ends with an @code{exit} statement. -@c ENDOFRANGE asse -@c ENDOFRANGE assef -@c ENDOFRANGE flibass -@c ENDOFRANGE libfass @node Round Function @subsection Rounding Numbers @@ -21090,7 +21264,7 @@ is always 1. This means that on those systems, characters have numeric values from 128 to 255. Finally, large mainframe systems use the EBCDIC character set, which uses all 256 values. -While there are other character sets in use on some older systems, +There are other character sets in use on some older systems, but they are not really worth worrying about: @example @@ -21140,11 +21314,11 @@ all the strings in an array into one long string. The following function, the application programs (@pxref{Sample Programs}). -Good function design is important; this function needs to be general but it +Good function design is important; this function needs to be general, but it should also have a reasonable default behavior. It is called with an array as well as the beginning and ending indices of the elements in the array to be merged. This assumes that the array indices are numeric---a reasonable -assumption since the array was likely created with @code{split()} +assumption, as the array was likely created with @code{split()} (@pxref{String Functions}): @cindex @code{join()} user-defined function @@ -21197,7 +21371,7 @@ more difficult than they really need to be.} The @code{systime()} and @code{strftime()} functions described in @DBREF{Time Functions} provide the minimum functionality necessary for dealing with the time of day -in human readable form. While @code{strftime()} is extensive, the control +in human-readable form. Although @code{strftime()} is extensive, the control formats are not necessarily easy to remember or intuitively obvious when reading a program. @@ -21288,7 +21462,7 @@ allowed the user to supply an optional timestamp value to use instead of the current time. @node Readfile Function -@subsection Reading A Whole File At Once +@subsection Reading a Whole File at Once Often, it is convenient to have the entire contents of a file available in memory as a single string. A straightforward but naive way to @@ -21345,13 +21519,13 @@ function readfile(file, tmp, save_rs) It works by setting @code{RS} to @samp{^$}, a regular expression that will never match if the file has contents. @command{gawk} reads data from -the file into @code{tmp} attempting to match @code{RS}. The match fails +the file into @code{tmp}, attempting to match @code{RS}. The match fails after each read, but fails quickly, such that @command{gawk} fills @code{tmp} with the entire contents of the file. -(@xref{Records}, for information on @code{RT} and @code{RS}.) +(@DBXREF{Records} for information on @code{RT} and @code{RS}.) In the case that @code{file} is empty, the return value is the null -string. Thus calling code may use something like: +string. Thus, calling code may use something like: @example contents = readfile("/some/path") @@ -21362,11 +21536,11 @@ if (length(contents) == 0) This tests the result to see if it is empty or not. An equivalent test would be @samp{contents == ""}. -@xref{Extension Sample Readfile}, for an extension function that +@DBXREF{Extension Sample Readfile} for an extension function that also reads an entire file into memory. @node Shell Quoting -@subsection Quoting Strings to Pass to The Shell +@subsection Quoting Strings to Pass to the Shell @c included by permission @ignore @@ -21408,7 +21582,7 @@ chmod -w file.flac Note the need for shell quoting. The function @code{shell_quote()} does it. @code{SINGLE} is the one-character string @code{"'"} and -@code{QSINGLE} is the three-character string @code{"\"'\""}. +@code{QSINGLE} is the three-character string @code{"\"'\""}: @example @c file eg/lib/shellquote.awk @@ -21446,11 +21620,8 @@ function shell_quote(s, # parameter @node Data File Management @section @value{DDF} Management -@c STARTOFRANGE dataf @cindex files, managing -@c STARTOFRANGE libfdataf @cindex libraries of @command{awk} functions, managing, data files -@c STARTOFRANGE flibdataf @cindex functions, library, managing data files This @value{SECTION} presents functions that are useful for managing command-line @value{DF}s. @@ -21468,12 +21639,12 @@ command-line @value{DF}s. @cindex files, managing, data file boundaries @cindex files, initialization and cleanup -The @code{BEGIN} and @code{END} rules are each executed exactly once at +The @code{BEGIN} and @code{END} rules are each executed exactly once, at the beginning and end of your @command{awk} program, respectively (@pxref{BEGIN/END}). We (the @command{gawk} authors) once had a user who mistakenly thought that the -@code{BEGIN} rule is executed at the beginning of each @value{DF} and the -@code{END} rule is executed at the end of each @value{DF}. +@code{BEGIN} rules were executed at the beginning of each @value{DF} and the +@code{END} rules were executed at the end of each @value{DF}. When informed that this was not the case, the user requested that we add new special @@ -21513,7 +21684,7 @@ END @{ endfile(FILENAME) @} This file must be loaded before the user's ``main'' program, so that the rule it supplies is executed first. -This rule relies on @command{awk}'s @code{FILENAME} variable that +This rule relies on @command{awk}'s @code{FILENAME} variable, which automatically changes for each new @value{DF}. The current @value{FN} is saved in a private variable, @code{_oldfilename}. If @code{FILENAME} does not equal @code{_oldfilename}, then a new @value{DF} is being processed and @@ -21529,7 +21700,7 @@ first @value{DF}. The program also supplies an @code{END} rule to do the final processing for the last file. Because this @code{END} rule comes before any @code{END} rules supplied in the ``main'' program, @code{endfile()} is called first. Once -again the value of multiple @code{BEGIN} and @code{END} rules should be clear. +again, the value of multiple @code{BEGIN} and @code{END} rules should be clear. @cindex @code{beginfile()} user-defined function @cindex @code{endfile()} user-defined function @@ -21540,7 +21711,7 @@ The following version solves the problem: @example @c file eg/lib/ftrans.awk -# ftrans.awk --- handle data file transitions +# ftrans.awk --- handle datafile transitions # # user supplies beginfile() and endfile() functions @c endfile @@ -21568,24 +21739,25 @@ END @{ endfile(_filename_) @} shows how this library function can be used and how it simplifies writing the main program. -@cindex sidebar, So Why Does @command{gawk} have @code{BEGINFILE} and @code{ENDFILE}? +@cindex sidebar, So Why Does @command{gawk} Have @code{BEGINFILE} and @code{ENDFILE}? @ifdocbook @docbook -<sidebar><title>So Why Does @command{gawk} have @code{BEGINFILE} and @code{ENDFILE}?</title> +<sidebar><title>So Why Does @command{gawk} Have @code{BEGINFILE} and @code{ENDFILE}?</title> @end docbook You are probably wondering, if @code{beginfile()} and @code{endfile()} functions can do the job, why does @command{gawk} have -@code{BEGINFILE} and @code{ENDFILE} patterns (@pxref{BEGINFILE/ENDFILE})? +@code{BEGINFILE} and @code{ENDFILE} patterns? Good question. Normally, if @command{awk} cannot open a file, this causes an immediate fatal error. In this case, there is no way for a -user-defined function to deal with the problem, since the mechanism for +user-defined function to deal with the problem, as the mechanism for calling it relies on the file being open and at the first record. Thus, the main reason for @code{BEGINFILE} is to give you a ``hook'' to catch files that cannot be processed. @code{ENDFILE} exists for symmetry, and because it provides an easy way to do per-file cleanup processing. +For more information, refer to @ref{BEGINFILE/ENDFILE}. @docbook </sidebar> @@ -21594,21 +21766,22 @@ and because it provides an easy way to do per-file cleanup processing. @ifnotdocbook @cartouche -@center @b{So Why Does @command{gawk} have @code{BEGINFILE} and @code{ENDFILE}?} +@center @b{So Why Does @command{gawk} Have @code{BEGINFILE} and @code{ENDFILE}?} You are probably wondering, if @code{beginfile()} and @code{endfile()} functions can do the job, why does @command{gawk} have -@code{BEGINFILE} and @code{ENDFILE} patterns (@pxref{BEGINFILE/ENDFILE})? +@code{BEGINFILE} and @code{ENDFILE} patterns? Good question. Normally, if @command{awk} cannot open a file, this causes an immediate fatal error. In this case, there is no way for a -user-defined function to deal with the problem, since the mechanism for +user-defined function to deal with the problem, as the mechanism for calling it relies on the file being open and at the first record. Thus, the main reason for @code{BEGINFILE} is to give you a ``hook'' to catch files that cannot be processed. @code{ENDFILE} exists for symmetry, and because it provides an easy way to do per-file cleanup processing. +For more information, refer to @ref{BEGINFILE/ENDFILE}. @end cartouche @end ifnotdocbook @@ -21616,7 +21789,7 @@ and because it provides an easy way to do per-file cleanup processing. @subsection Rereading the Current File @cindex files, reading -Another request for a new built-in function was for a @code{rewind()} +Another request for a new built-in function was for a function that would make it possible to reread the current file. The requesting user didn't want to have to use @code{getline} (@pxref{Getline}) @@ -21625,7 +21798,7 @@ inside a loop. However, as long as you are not in the @code{END} rule, it is quite easy to arrange to immediately close the current input file and then start over with it from the top. -For lack of a better name, we'll call it @code{rewind()}: +For lack of a better name, we'll call the function @code{rewind()}: @cindex @code{rewind()} user-defined function @example @@ -21663,8 +21836,8 @@ The @code{rewind()} function relies on the @code{ARGIND} variable (@pxref{Auto-set}), which is specific to @command{gawk}. It also relies on the @code{nextfile} keyword (@pxref{Nextfile Statement}). Because of this, you should not call it from an @code{ENDFILE} rule. -(This isn't necessary anyway, since as soon as an @code{ENDFILE} rule -finishes @command{gawk} goes to the next file!) +(This isn't necessary anyway, because @command{gawk} goes to the next +file as soon as an @code{ENDFILE} rule finishes!) @node File Checking @subsection Checking for Readable @value{DDF}s @@ -21712,22 +21885,22 @@ BEGIN @{ @cindex troubleshooting, @code{getline} function This works, because the @code{getline} won't be fatal. Removing the element from @code{ARGV} with @code{delete} -skips the file (since it's no longer in the list). +skips the file (because it's no longer in the list). See also @ref{ARGC and ARGV}. -The regular expression check purposely does not use character classes +Because @command{awk} variable names only allow the English letters, +the regular expression check purposely does not use character classes such as @samp{[:alpha:]} and @samp{[:alnum:]} -(@pxref{Bracket Expressions}) -since @command{awk} variable names only allow the English letters. +(@pxref{Bracket Expressions}). @node Empty Files -@subsection Checking for Zero-length Files +@subsection Checking for Zero-Length Files All known @command{awk} implementations silently skip over zero-length files. -This is a by-product of @command{awk}'s implicit +This is a by-product of @command{awk}'s implicit read-a-record-and-match-against-the-rules loop: when @command{awk} tries to read a record from an empty file, it immediately receives an -end of file indication, closes the file, and proceeds on to the next +end-of-file indication, closes the file, and proceeds on to the next command-line @value{DF}, @emph{without} executing any user-level @command{awk} program code. @@ -21792,7 +21965,7 @@ Occasionally, you might not want @command{awk} to process command-line variable assignments (@pxref{Assignment Options}). In particular, if you have a @value{FN} that contains an @samp{=} character, -@command{awk} treats the @value{FN} as an assignment, and does not process it. +@command{awk} treats the @value{FN} as an assignment and does not process it. Some users have suggested an additional command-line option for @command{gawk} to disable command-line assignments. However, some simple programming with @@ -21842,30 +22015,22 @@ The use of @code{No_command_assign} allows you to disable command-line assignments at invocation time, by giving the variable a true value. When not set, it is initially zero (i.e., false), so the command-line arguments are left alone. -@c ENDOFRANGE dataf -@c ENDOFRANGE flibdataf -@c ENDOFRANGE libfdataf @node Getopt Function @section Processing Command-Line Options -@c STARTOFRANGE libfclo @cindex libraries of @command{awk} functions, command-line options -@c STARTOFRANGE flibclo @cindex functions, library, command-line options -@c STARTOFRANGE clop @cindex command-line options, processing -@c STARTOFRANGE oclp @cindex options, command-line, processing -@c STARTOFRANGE clibf @cindex functions, library, C library @cindex arguments, processing -Most utilities on POSIX compatible systems take options on +Most utilities on POSIX-compatible systems take options on the command line that can be used to change the way a program behaves. @command{awk} is an example of such a program (@pxref{Options}). -Often, options take @dfn{arguments}; i.e., data that the program needs to -correctly obey the command-line option. For example, @command{awk}'s +Often, options take @dfn{arguments} (i.e., data that the program needs to +correctly obey the command-line option). For example, @command{awk}'s @option{-F} option requires a string to use as the field separator. The first occurrence on the command line of either @option{--} or a string that does not begin with @samp{-} ends the options. @@ -21969,7 +22134,7 @@ necessary for accessing individual characters (@pxref{String Functions}).@footnote{This function was written before @command{gawk} acquired the ability to split strings into single characters using @code{""} as the separator. -We have left it alone, since using @code{substr()} is more portable.} +We have left it alone, as using @code{substr()} is more portable.} The discussion that follows walks through the code a bit at a time: @@ -22137,9 +22302,9 @@ next element in @code{argv}. If neither condition is true, then only on the next call to @code{getopt()}. The @code{BEGIN} rule initializes both @code{Opterr} and @code{Optind} to one. -@code{Opterr} is set to one, since the default behavior is for @code{getopt()} +@code{Opterr} is set to one, because the default behavior is for @code{getopt()} to print a diagnostic message upon seeing an invalid option. @code{Optind} -is set to one, since there's no reason to look at the program name, which is +is set to one, because there's no reason to look at the program name, which is in @code{ARGV[0]}: @example @@ -22162,8 +22327,8 @@ BEGIN @{ @c endfile @end example -The rest of the @code{BEGIN} rule is a simple test program. Here is the -result of two sample runs of the test program: +The rest of the @code{BEGIN} rule is a simple test program. Here are the +results of two sample runs of the test program: @example $ @kbd{awk -f getopt.awk -v _getopt_test=1 -- -a -cbARG bax -x} @@ -22189,46 +22354,44 @@ etc., as its own options. @quotation NOTE After @code{getopt()} is through, -user level code must clear out all the elements of @code{ARGV} from 1 +user-level code must clear out all the elements of @code{ARGV} from 1 to @code{Optind}, so that @command{awk} does not try to process the command-line options as @value{FN}s. @end quotation Using @samp{#!} with the @option{-E} option may help avoid conflicts between your program's options and @command{gawk}'s options, -since @option{-E} causes @command{gawk} to abandon processing of +as @option{-E} causes @command{gawk} to abandon processing of further options -(@pxref{Executable Scripts}, and @pxref{Options}). +(@DBPXREF{Executable Scripts} and +@ifnotdocbook +@pxref{Options}). +@end ifnotdocbook +@ifdocbook +@ref{Options}). +@end ifdocbook Several of the sample programs presented in @ref{Sample Programs}, use @code{getopt()} to process their arguments. -@c ENDOFRANGE libfclo -@c ENDOFRANGE flibclo -@c ENDOFRANGE clop -@c ENDOFRANGE oclp @node Passwd Functions @section Reading the User Database -@c STARTOFRANGE libfudata @cindex libraries of @command{awk} functions, user database, reading -@c STARTOFRANGE flibudata @cindex functions, library, user database@comma{} reading -@c STARTOFRANGE udatar @cindex user database@comma{} reading -@c STARTOFRANGE dataur @cindex database, users@comma{} reading @cindex @code{PROCINFO} array The @code{PROCINFO} array (@pxref{Built-in Variables}) provides access to the current user's real and effective user and group ID -numbers, and if available, the user's supplementary group set. +numbers, and, if available, the user's supplementary group set. However, because these are numbers, they do not provide very useful information to the average user. There needs to be some way to find the user information associated with the user and group ID numbers. This @value{SECTION} presents a suite of functions for retrieving information from the -user database. @xref{Group Functions}, +user database. @DBXREF{Group Functions} for a similar suite that retrieves information from the group database. @cindex @code{getpwent()} function (C library) @@ -22243,11 +22406,11 @@ kept. Instead, it provides the @code{<pwd.h>} header file and several C language subroutines for obtaining user information. The primary function is @code{getpwent()}, for ``get password entry.'' The ``password'' comes from the original user database file, -@file{/etc/passwd}, which stores user information, along with the +@file{/etc/passwd}, which stores user information along with the encrypted passwords (hence the name). @cindex @command{pwcat} program -While an @command{awk} program could simply read @file{/etc/passwd} +Although an @command{awk} program could simply read @file{/etc/passwd} directly, this file may not contain complete information about the system's set of users.@footnote{It is often the case that password information is stored in a network database.} To be sure you are able to @@ -22342,12 +22505,12 @@ The user's encrypted password. This may not be available on some systems. @item User-ID The user's numeric user ID number. -(On some systems it's a C @code{long}, and not an @code{int}. Thus +(On some systems, it's a C @code{long}, and not an @code{int}. Thus, we cast it to @code{long} for all cases.) @item Group-ID The user's numeric group ID number. -(Similar comments about @code{long} vs.@: @code{int} apply here.) +(Similar comments about @code{long} versus @code{int} apply here.) @item Full name The user's full name, and perhaps other information associated with the @@ -22369,7 +22532,7 @@ A few lines representative of @command{pwcat}'s output are as follows: @cindex Robbins, Miriam @example $ @kbd{pwcat} -@print{} root:3Ov02d5VaUPB6:0:1:Operator:/:/bin/sh +@print{} root:x:0:1:Operator:/:/bin/sh @print{} nobody:*:65534:65534::/: @print{} daemon:*:1:1::/: @print{} sys:*:2:2::/:/bin/csh @@ -22448,7 +22611,7 @@ The function @code{_pw_init()} fills three copies of the user information into three associative arrays. The arrays are indexed by username (@code{_pw_byname}), by user ID number (@code{_pw_byuid}), and by order of occurrence (@code{_pw_bycount}). -The variable @code{_pw_inited} is used for efficiency, since @code{_pw_init()} +The variable @code{_pw_inited} is used for efficiency, as @code{_pw_init()} needs to be called only once. @cindex @code{PROCINFO} array, testing the field splitting @@ -22457,7 +22620,7 @@ Because this function uses @code{getline} to read information from @command{pwcat}, it first saves the values of @code{FS}, @code{RS}, and @code{$0}. It notes in the variable @code{using_fw} whether field splitting with @code{FIELDWIDTHS} is in effect or not. -Doing so is necessary, since these functions could be called +Doing so is necessary, as these functions could be called from anywhere within a user's program, and the user may have his or her own way of splitting records and fields. This makes it possible to restore the correct @@ -22469,7 +22632,7 @@ The code that checks for using @code{FPAT}, using @code{using_fpat} and @code{PROCINFO["FS"]}, is similar. The main part of the function uses a loop to read database lines, split -the line into fields, and then store the line into each array as necessary. +the lines into fields, and then store the lines into each array as necessary. When the loop is done, @code{@w{_pw_init()}} cleans up by closing the pipeline, setting @code{@w{_pw_inited}} to one, and restoring @code{FS} (and @code{FIELDWIDTHS} or @code{FPAT} @@ -22559,26 +22722,18 @@ In turn, calling @code{_pw_init()} is not too expensive, because the once. If you are worried about squeezing every last cycle out of your @command{awk} program, the check of @code{_pw_inited} could be moved out of @code{_pw_init()} and duplicated in all the other functions. In practice, -this is not necessary, since most @command{awk} programs are I/O-bound, +this is not necessary, as most @command{awk} programs are I/O-bound, and such a change would clutter up the code. The @command{id} program in @DBREF{Id Program} uses these functions. -@c ENDOFRANGE libfudata -@c ENDOFRANGE flibudata -@c ENDOFRANGE udatar -@c ENDOFRANGE dataur @node Group Functions @section Reading the Group Database -@c STARTOFRANGE libfgdata @cindex libraries of @command{awk} functions, group database, reading -@c STARTOFRANGE flibgdata @cindex functions, library, group database@comma{} reading -@c STARTOFRANGE gdatar @cindex group database, reading -@c STARTOFRANGE datagr @cindex database, group, reading @cindex @code{PROCINFO} array, and group membership @cindex @code{getgrent()} function (C library) @@ -22694,11 +22849,11 @@ it is usually empty or set to @samp{*}. @item Group ID Number The group's numeric group ID number; the association of name to number must be unique within the file. -(On some systems it's a C @code{long}, and not an @code{int}. Thus +(On some systems it's a C @code{long}, and not an @code{int}. Thus, we cast it to @code{long} for all cases.) @item Group Member List -A comma-separated list of user names. These users are members of the group. +A comma-separated list of usernames. These users are members of the group. Modern Unix systems allow users to be members of several groups simultaneously. If your system does, then there are elements @code{"group1"} through @code{"group@var{N}"} in @code{PROCINFO} @@ -22808,32 +22963,32 @@ The @code{@w{_gr_init()}} function first saves @code{FS}, @code{$0}, and then sets @code{FS} and @code{RS} to the correct values for scanning the group information. It also takes care to note whether @code{FIELDWIDTHS} or @code{FPAT} -is being used, and to restore the appropriate field splitting mechanism. +is being used, and to restore the appropriate field-splitting mechanism. -The group information is stored is several associative arrays. +The group information is stored in several associative arrays. The arrays are indexed by group name (@code{@w{_gr_byname}}), by group ID number (@code{@w{_gr_bygid}}), and by position in the database (@code{@w{_gr_bycount}}). -There is an additional array indexed by user name (@code{@w{_gr_groupsbyuser}}), +There is an additional array indexed by username (@code{@w{_gr_groupsbyuser}}), which is a space-separated list of groups to which each user belongs. -Unlike the user database, it is possible to have multiple records in the +Unlike in the user database, it is possible to have multiple records in the database for the same group. This is common when a group has a large number of members. A pair of such entries might look like the following: @example -tvpeople:*:101:johny,jay,arsenio +tvpeople:*:101:johnny,jay,arsenio tvpeople:*:101:david,conan,tom,joan @end example For this reason, @code{_gr_init()} looks to see if a group name or -group ID number is already seen. If it is, then the user names are -simply concatenated onto the previous list of users.@footnote{There is actually a +group ID number is already seen. If so, the usernames are +simply concatenated onto the previous list of users.@footnote{There is a subtle problem with the code just presented. Suppose that the first time there were no names. This code adds the names with a leading comma. It also doesn't check that there is a @code{$4}.} Finally, @code{_gr_init()} closes the pipeline to @command{grcat}, restores -@code{FS} (and @code{FIELDWIDTHS} or @code{FPAT} if necessary), @code{RS}, and @code{$0}, +@code{FS} (and @code{FIELDWIDTHS} or @code{FPAT}, if necessary), @code{RS}, and @code{$0}, initializes @code{_gr_count} to zero (it is used later), and makes @code{_gr_inited} nonzero. @@ -22872,7 +23027,7 @@ function getgrgid(gid) @cindex @code{getgruser()} function (C library) The @code{getgruser()} function does not have a C counterpart. It takes a -user name and returns the list of groups that have the user as a member: +username and returns the list of groups that have the user as a member: @cindex @code{getgruser()} function, user-defined @example @@ -22901,7 +23056,6 @@ function getgrent() @} @c endfile @end example -@c ENDOFRANGE clibf @cindex @code{endgrent()} function (C library) The @code{endgrent()} function resets @code{_gr_count} to zero so that @code{getgrent()} can @@ -22934,12 +23088,12 @@ uses these functions. @DBREF{Arrays of Arrays} described how @command{gawk} provides arrays of arrays. In particular, any element of -an array may be either a scalar, or another array. The +an array may be either a scalar or another array. The @code{isarray()} function (@pxref{Type Functions}) lets you distinguish an array from a scalar. The following function, @code{walk_array()}, recursively traverses -an array, printing each element's indices and value. +an array, printing the element indices and values. You call it with the array and a string representing the name of the array: @@ -22990,10 +23144,66 @@ $ @kbd{gawk -f walk_array.awk} @print{} a[4][2] = 42 @end example -@c ENDOFRANGE libfgdata -@c ENDOFRANGE flibgdata -@c ENDOFRANGE gdatar -@c ENDOFRANGE libf +The function just presented simply prints the +name and value of each scalar array element. However, it is easy to +generalize it, by passing in the name of a function to call +when walking an array. The modified function looks like this: + +@example +@c file eg/lib/processarray.awk +function process_array(arr, name, process, do_arrays, i, new_name) +@{ + for (i in arr) @{ + new_name = (name "[" i "]") + if (isarray(arr[i])) @{ + if (do_arrays) + @@process(new_name, arr[i]) + process_array(arr[i], new_name, process, do_arrays) + @} else + @@process(new_name, arr[i]) + @} +@} +@c endfile +@end example + +The arguments are as follows: + +@table @code +@item arr +The array. + +@item name +The name of the array (a string). + +@item process +The name of the function to call. + +@item do_arrays +If this is true, the function can handle elements that are subarrays. +@end table + +If subarrays are to be processed, that is done before walking them further. + +When run with the following scaffolding, the function produces the same +results as does the earlier version of @code{walk_array()}: + +@example +BEGIN @{ + a[1] = 1 + a[2][1] = 21 + a[2][2] = 22 + a[3] = 3 + a[4][1][1] = 411 + a[4][2] = 42 + + process_array(a, "a", "do_print", 0) +@} + +function do_print(name, element) +@{ + printf "%s = %s\n", name, element +@} +@end example @node Library Functions Summary @section Summary @@ -23015,24 +23225,24 @@ The functions presented here fit into the following categories: @c nested list @table @asis @item General problems -Number to string conversion, assertions, rounding, random number +Number-to-string conversion, testing assertions, rounding, random number generation, converting characters to numbers, joining strings, getting easily usable time-of-day information, and reading a whole file in -one shot. +one shot @item Managing @value{DF}s Noting @value{DF} boundaries, rereading the current file, checking for readable files, checking for zero-length files, and treating assignments -as @value{FN}s. +as @value{FN}s @item Processing command-line options -An @command{awk} version of the standard C @code{getopt()} function. +An @command{awk} version of the standard C @code{getopt()} function @item Reading the user and group databases -Two sets of routines that parallel the C library versions. +Two sets of routines that parallel the C library versions @item Traversing arrays of arrays -A simple function to traverse an array of arrays to any depth. +Two functions that traverse an array of arrays to any depth @end table @c end nested list @@ -23075,7 +23285,7 @@ ARGIND != Argind @{ @} END @{ if (ARGIND < ARGC - 1) - ARGIND = ARGC - 1 + ARGIND = ARGC - 1 if (ARGIND > Argind) for (Argind++; Argind <= ARGIND; Argind++) zerofile(ARGV[Argind], Argind) @@ -23107,13 +23317,9 @@ output identical to that of the original version. @end enumerate @c EXCLUDE END -@c ENDOFRANGE flib -@c ENDOFRANGE fudlib -@c ENDOFRANGE datagr @node Sample Programs @chapter Practical @command{awk} Programs -@c STARTOFRANGE awkpex @cindex @command{awk} programs, examples of @c FULLXREF ON @@ -23131,10 +23337,10 @@ in this @value{CHAPTER}. The second presents @command{awk} versions of several common POSIX utilities. These are programs that you are hopefully already familiar with, -and therefore, whose problems are understood. +and therefore whose problems are understood. By reimplementing these programs in @command{awk}, you can focus on the @command{awk}-related aspects of solving -the programming problem. +the programming problems. The third is a grab bag of interesting programs. These solve a number of different data-manipulation and management @@ -23183,7 +23389,6 @@ cut.awk -- -c1-8 myfiles > results @node Clones @section Reinventing Wheels for Fun and Profit -@c STARTOFRANGE posimawk @cindex POSIX, programs@comma{} implementing in @command{awk} This @value{SECTION} presents a number of POSIX utilities implemented in @@ -23195,7 +23400,7 @@ It should be noted that these programs are not necessarily intended to replace the installed versions on your system. Nor may all of these programs be fully compliant with the most recent POSIX standard. This is not a problem; their -purpose is to illustrate @command{awk} language programming for ``real world'' +purpose is to illustrate @command{awk} language programming for ``real-world'' tasks. The programs are presented in alphabetical order. @@ -23211,14 +23416,11 @@ The programs are presented in alphabetical order. @end menu @node Cut Program -@subsection Cutting out Fields and Columns +@subsection Cutting Out Fields and Columns @cindex @command{cut} utility -@c STARTOFRANGE cut @cindex @command{cut} utility -@c STARTOFRANGE ficut @cindex fields, cutting -@c STARTOFRANGE colcut @cindex columns, cutting The @command{cut} utility selects, or ``cuts,'' characters or fields from its standard input and sends them to its standard output. @@ -23227,7 +23429,7 @@ but you may supply a command-line option to change the field @dfn{delimiter} (i.e., the field-separator character). @command{cut}'s definition of fields is less general than @command{awk}'s. -A common use of @command{cut} might be to pull out just the login name of +A common use of @command{cut} might be to pull out just the login names of logged-on users from the output of @command{who}. For example, the following pipeline generates a sorted, unique list of the logged-on users: @@ -23488,7 +23690,7 @@ function set_charlist( field, i, j, f, g, n, m, t, @c endfile @end example -Next is the rule that actually processes the data. If the @option{-s} option +Next is the rule that processes the data. If the @option{-s} option is given, then @code{suppress} is true. The first @code{if} statement makes sure that the input record does have the field separator. If @command{cut} is processing fields, @code{suppress} is true, and the field @@ -23520,27 +23722,20 @@ written out between the fields: @end example This version of @command{cut} relies on @command{gawk}'s @code{FIELDWIDTHS} -variable to do the character-based cutting. While it is possible in +variable to do the character-based cutting. It is possible in other @command{awk} implementations to use @code{substr()} -(@pxref{String Functions}), +(@pxref{String Functions}), but it is also extremely painful. The @code{FIELDWIDTHS} variable supplies an elegant solution to the problem of picking the input line apart by characters. -@c ENDOFRANGE cut -@c ENDOFRANGE ficut -@c ENDOFRANGE colcut @node Egrep Program @subsection Searching for Regular Expressions in Files -@c STARTOFRANGE regexps @cindex regular expressions, searching for -@c STARTOFRANGE sfregexp @cindex searching, files for regular expressions -@c STARTOFRANGE fsregexp @cindex files, searching for regular expressions -@c STARTOFRANGE egrep @cindex @command{egrep} utility The @command{egrep} utility searches files for patterns. It uses regular expressions that are almost identical to those available in @command{awk} @@ -23667,7 +23862,7 @@ matched lines in the output: @c endfile @end example -The last two lines are commented out, since they are not needed in +The last two lines are commented out, as they are not needed in @command{gawk}. They should be uncommented if you have to use another version of @command{awk}. @@ -23677,7 +23872,7 @@ into lowercase if the @option{-i} option is specified.@footnote{It also introduces a subtle bug; if a match happens, we output the translated line, not the original.} The rule is -commented out since it is not necessary with @command{gawk}: +commented out as it is not necessary with @command{gawk}: @example @c file eg/prog/egrep.awk @@ -23743,7 +23938,7 @@ successful or unsuccessful match. If the line does not match, the @code{next} statement just moves on to the next record. A number of additional tests are made, but they are only done if we -are not counting lines. First, if the user only wants exit status +are not counting lines. First, if the user only wants the exit status (@code{no_print} is true), then it is enough to know that @emph{one} line in this file matched, and we can skip on to the next file with @code{nextfile}. Similarly, if we are only printing @value{FN}s, we can @@ -23784,7 +23979,7 @@ if necessary: @end example The @code{END} rule takes care of producing the correct exit status. If -there are no matches, the exit status is one; otherwise it is zero: +there are no matches, the exit status is one; otherwise, it is zero: @example @c file eg/prog/egrep.awk @@ -23808,17 +24003,12 @@ function usage() @c endfile @end example -@c ENDOFRANGE regexps -@c ENDOFRANGE sfregexp -@c ENDOFRANGE fsregexp -@c ENDOFRANGE egrep @node Id Program -@subsection Printing out User Information +@subsection Printing Out User Information @cindex printing, user information @cindex users, information about, printing -@c STARTOFRANGE id @cindex @command{id} utility The @command{id} utility lists a user's real and effective user ID numbers, real and effective group ID numbers, and the user's group set, if any. @@ -23841,7 +24031,8 @@ Here is a simple version of @command{id} written in @command{awk}. It uses the user database library functions (@pxref{Passwd Functions}) and the group database library functions -(@pxref{Group Functions}): +(@pxref{Group Functions}) +from @ref{Library Functions}. The program is fairly straightforward. All the work is done in the @code{BEGIN} rule. The user and group ID numbers are obtained from @@ -23929,7 +24120,7 @@ function pr_first_field(str, a) The test in the @code{for} loop is worth noting. Any supplementary groups in the @code{PROCINFO} array have the indices @code{"group1"} through @code{"group@var{N}"} for some -@var{N}, i.e., the total number of supplementary groups. +@var{N} (i.e., the total number of supplementary groups). However, we don't know in advance how many of these groups there are. @@ -23947,16 +24138,13 @@ code that is used repeatedly, making the whole program shorter and cleaner. In particular, moving the check for the empty string into this function saves several lines of code. -@c ENDOFRANGE id @node Split Program @subsection Splitting a Large File into Pieces @c FIXME: One day, update to current POSIX version of split -@c STARTOFRANGE filspl @cindex files, splitting -@c STARTOFRANGE split @cindex @code{split} utility The @command{split} program splits large text files into smaller pieces. Usage is as follows:@footnote{This is the traditional usage. The @@ -23969,10 +24157,10 @@ aims to demonstrate.} By default, the output files are named @file{xaa}, @file{xab}, and so on. Each file has -1000 lines in it, with the likely exception of the last file. To change the +1,000 lines in it, with the likely exception of the last file. To change the number of lines in each file, supply a number on the command line -preceded with a minus; e.g., @samp{-500} for files with 500 lines in them -instead of 1000. To change the name of the output files to something like +preceded with a minus sign (e.g., @samp{-500} for files with 500 lines in them +instead of 1,000). To change the names of the output files to something like @file{myfileaa}, @file{myfileab}, and so on, supply an additional argument that specifies the @value{FN} prefix. @@ -24020,7 +24208,7 @@ BEGIN @{ @} # test argv in case reading from stdin instead of file if (i in ARGV) - i++ # skip data file name + i++ # skip datafile name if (i in ARGV) @{ outfile = ARGV[i] ARGV[i] = "" @@ -24091,15 +24279,12 @@ You might want to consider how to eliminate the use of way as to solve the EBCDIC issue as well. @end ifset -@c ENDOFRANGE filspl -@c ENDOFRANGE split @node Tee Program @subsection Duplicating Output into Multiple Files @cindex files, multiple@comma{} duplicating output into @cindex output, duplicating into files -@c STARTOFRANGE tee @cindex @code{tee} utility The @code{tee} program is known as a ``pipe fitting.'' @code{tee} copies its standard input to its standard output and also duplicates it to the @@ -24114,8 +24299,8 @@ truncating them and starting over. The @code{BEGIN} rule first makes a copy of all the command-line arguments into an array named @code{copy}. -@code{ARGV[0]} is not copied, since it is not needed. -@code{tee} cannot use @code{ARGV} directly, since @command{awk} attempts to +@code{ARGV[0]} is not needed, so it is not copied. +@code{tee} cannot use @code{ARGV} directly, because @command{awk} attempts to process each @value{FN} in @code{ARGV} as input data. @cindex flag variables @@ -24164,7 +24349,7 @@ BEGIN @{ @c endfile @end example -The following single rule does all the work. Since there is no pattern, it is +The following single rule does all the work. Because there is no pattern, it is executed for each line of input. The body of the rule simply prints the line into each file on the command line, and then to the standard output: @@ -24195,7 +24380,7 @@ for (i in copy) @end example @noindent -This is more concise but it is also less efficient. The @samp{if} is +This is more concise, but it is also less efficient. The @samp{if} is tested for each record and for each output file. By duplicating the loop body, the @samp{if} is only tested once for each input record. If there are @var{N} input records and @var{M} output files, the first method only @@ -24212,18 +24397,14 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE tee @node Uniq Program @subsection Printing Nonduplicated Lines of Text @c FIXME: One day, update to current POSIX version of uniq -@c STARTOFRANGE prunt @cindex printing, unduplicated lines of text -@c STARTOFRANGE tpul @cindex text@comma{} printing, unduplicated lines of -@c STARTOFRANGE uniq @cindex @command{uniq} utility The @command{uniq} utility reads sorted lines of data on its standard input, and by default removes duplicate lines. In other words, it only @@ -24415,10 +24596,10 @@ The second rule does the work. The variable @code{equal} is one or zero, depending upon the results of @code{are_equal()}'s comparison. If @command{uniq} is counting repeated lines, and the lines are equal, then it increments the @code{count} variable. Otherwise, it prints the line and resets @code{count}, -since the two lines are not equal. +because the two lines are not equal. If @command{uniq} is not counting, and if the lines are equal, @code{count} is incremented. -Nothing is printed, since the point is to remove duplicates. +Nothing is printed, as the point is to remove duplicates. Otherwise, if @command{uniq} is counting repeated lines and more than one line is seen, or if @command{uniq} is counting nonrepeated lines and only one line is seen, then the line is printed, and @code{count} @@ -24487,31 +24668,22 @@ Brian Kernighan suggests that ``an alternative approach to state machines is to just read the input into an array, then use indexing. It's almost always easier code, and for most inputs where you would use this, just -as fast.'' Consider how to rewrite the logic to follow this +as fast.'' Consider how to rewrite the logic to follow this suggestion. @end ifset -@c ENDOFRANGE prunt -@c ENDOFRANGE tpul -@c ENDOFRANGE uniq @node Wc Program @subsection Counting Things @c FIXME: One day, update to current POSIX version of wc -@c STARTOFRANGE count @cindex counting -@c STARTOFRANGE infco @cindex input files, counting elements in -@c STARTOFRANGE woco @cindex words, counting -@c STARTOFRANGE chco @cindex characters, counting -@c STARTOFRANGE lico @cindex lines, counting -@c STARTOFRANGE wc @cindex @command{wc} utility The @command{wc} (word count) utility counts lines, words, and characters in one or more input files. Its usage is as follows: @@ -24539,7 +24711,7 @@ Count only characters. @end table Implementing @command{wc} in @command{awk} is particularly elegant, -since @command{awk} does a lot of the work for us; it splits lines into +because @command{awk} does a lot of the work for us; it splits lines into words (i.e., fields) and counts them, it counts lines (i.e., records), and it can easily tell us how long a line is. @@ -24644,7 +24816,7 @@ function endfile(file) @end example There is one rule that is executed for each line. It adds the length of -the record, plus one, to @code{chars}.@footnote{Since @command{gawk} +the record, plus one, to @code{chars}.@footnote{Because @command{gawk} understands multibyte locales, this code counts characters, not bytes.} Adding one plus the record length is needed because the newline character separating records (the value @@ -24681,13 +24853,6 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE count -@c ENDOFRANGE infco -@c ENDOFRANGE lico -@c ENDOFRANGE woco -@c ENDOFRANGE chco -@c ENDOFRANGE wc -@c ENDOFRANGE posimawk @node Miscellaneous Programs @section A Grab Bag of @command{awk} Programs @@ -24818,9 +24983,7 @@ Aharon Robbins <arnold@skeeve.com> wrote: @author Erik Quanstrom @end quotation -@c STARTOFRANGE tialarm @cindex time, alarm clock example program -@c STARTOFRANGE alaex @cindex alarm clock example program The following program is a simple ``alarm clock'' program. You give it a time of day and an optional message. At the specified time, @@ -24836,7 +24999,7 @@ checking and setting of defaults: the delay, the count, and the message to print. If the user supplied a message without the ASCII BEL character (known as the ``alert'' character, @code{"\a"}), then it is added to the message. (On many systems, printing the ASCII BEL generates an -audible alert. Thus when the alarm goes off, the system calls attention +audible alert. Thus, when the alarm goes off, the system calls attention to itself in case the user is not looking at the computer.) Just for a change, this program uses a @code{switch} statement (@pxref{Switch Statement}), but the processing could be done with a series of @@ -24972,15 +25135,11 @@ seconds are necessary: @} @c endfile @end example -@c ENDOFRANGE tialarm -@c ENDOFRANGE alaex @node Translate Program @subsection Transliterating Characters -@c STARTOFRANGE chtra @cindex characters, transliterating -@c STARTOFRANGE tr @cindex @command{tr} utility The system @command{tr} utility transliterates characters. For example, it is often used to map uppercase letters into lowercase for further processing: @@ -24992,8 +25151,8 @@ often used to map uppercase letters into lowercase for further processing: @command{tr} requires two lists of characters.@footnote{On some older systems, including Solaris, the system version of @command{tr} may require that the lists be written as range expressions enclosed in square brackets -(@samp{[a-z]}) and quoted, to prevent the shell from attempting a file -name expansion. This is not a feature.} When processing the input, the +(@samp{[a-z]}) and quoted, to prevent the shell from attempting a +@value{FN} expansion. This is not a feature.} When processing the input, the first character in the first list is replaced with the first character in the second list, the second character in the first list is replaced with the second character in the second list, and so on. If there are @@ -25009,7 +25168,7 @@ to @command{gawk}. @c at least theoretically The following program was written to prove that character transliteration could be done with a user-level -function. This program is not as complete as the system @command{tr} utility +function. This program is not as complete as the system @command{tr} utility, but it does most of the job. The @command{translate} program was written long before @command{gawk} @@ -25021,13 +25180,13 @@ takes three arguments: @table @code @item from -A list of characters from which to translate. +A list of characters from which to translate @item to -A list of characters to which to translate. +A list of characters to which to translate @item target -The string on which to do the translation. +The string on which to do the translation @end table Associative arrays make the translation part fairly easy. @code{t_ar} holds @@ -25036,7 +25195,7 @@ loop goes through @code{from}, one character at a time. For each character in @code{from}, if the character appears in @code{target}, it is replaced with the corresponding @code{to} character. -The @code{translate()} function calls @code{stranslate()} using @code{$0} +The @code{translate()} function calls @code{stranslate()}, using @code{$0} as the target. The main program sets two global variables, @code{FROM} and @code{TO}, from the command line, and then changes @code{ARGV} so that @command{awk} reads from the standard input. @@ -25058,7 +25217,7 @@ Finally, the processing rule simply calls @code{translate()} for each record: @c endfile @end ignore @c file eg/prog/translate.awk -# Bugs: does not handle things like: tr A-Z a-z, it has +# Bugs: does not handle things like tr A-Z a-z; it has # to be spelled out. However, if `to' is shorter than `from', # the last character in `to' is used for the rest of `from'. @@ -25108,9 +25267,9 @@ BEGIN @{ @c endfile @end example -While it is possible to do character transliteration in a user-level -function, it is not necessarily efficient, and we (the @command{gawk} -authors) started to consider adding a built-in function. However, +It is possible to do character transliteration in a user-level +function, but it is not necessarily efficient, and we (the @command{gawk} +developers) started to consider adding a built-in function. However, shortly after writing this program, we learned that Brian Kernighan had added the @code{toupper()} and @code{tolower()} functions to his @command{awk} (@pxref{String Functions}). These functions handle the @@ -25128,17 +25287,13 @@ such as @samp{a-z}, as allowed by the @command{tr} utility. Look at the code for @file{cut.awk} (@pxref{Cut Program}) for inspiration. -@c ENDOFRANGE chtra -@c ENDOFRANGE tr @node Labels Program @subsection Printing Mailing Labels -@c STARTOFRANGE prml @cindex printing, mailing labels -@c STARTOFRANGE mlprint @cindex mailing labels@comma{} printing -Here is a ``real world''@footnote{``Real world'' is defined as +Here is a ``real-world''@footnote{``Real world'' is defined as ``a program actually used to get something done.''} program. This script reads lists of names and @@ -25147,14 +25302,14 @@ on it, two across and 10 down. The addresses are guaranteed to be no more than five lines of data. Each address is separated from the next by a blank line. -The basic idea is to read 20 labels worth of data. Each line of each label +The basic idea is to read 20 labels' worth of data. Each line of each label is stored in the @code{line} array. The single rule takes care of filling the @code{line} array and printing the page when 20 labels have been read. The @code{BEGIN} rule simply sets @code{RS} to the empty string, so that @command{awk} splits records at blank lines (@pxref{Records}). -It sets @code{MAXLINES} to 100, since 100 is the maximum number +It sets @code{MAXLINES} to 100, because 100 is the maximum number of lines on the page @iftex (@math{20 @cdot 5 = 100}). @@ -25170,12 +25325,12 @@ of lines on the page Most of the work is done in the @code{printpage()} function. The label lines are stored sequentially in the @code{line} array. But they -have to print horizontally; @code{line[1]} next to @code{line[6]}, +have to print horizontally: @code{line[1]} next to @code{line[6]}, @code{line[2]} next to @code{line[7]}, and so on. Two loops accomplish this. The outer loop, controlled by @code{i}, steps through every 10 lines of data; this is each row of labels. The inner loop, controlled by @code{j}, goes through the lines within the row. -As @code{j} goes from 0 to 4, @samp{i+j} is the @code{j}-th line in +As @code{j} goes from 0 to 4, @samp{i+j} is the @code{j}th line in the row, and @samp{i+j+5} is the entry next to it. The output ends up looking something like this: @@ -25200,7 +25355,6 @@ that there are two blank lines at the top and two blank lines at the bottom. The @code{END} rule arranges to flush the final page of labels; there may not have been an even multiple of 20 labels in the data: -@c STARTOFRANGE labels @cindex @code{labels.awk} program @example @c file eg/prog/labels.awk @@ -25265,14 +25419,10 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE prml -@c ENDOFRANGE mlprint -@c ENDOFRANGE labels @node Word Sorting @subsection Generating Word-Usage Counts -@c STARTOFRANGE worus @cindex words, usage counts@comma{} generating When working with large amounts of text, it can be interesting to know @@ -25298,8 +25448,8 @@ END @{ @} @end example -The program relies on @command{awk}'s default field splitting -mechanism to break each line up into ``words,'' and uses an +The program relies on @command{awk}'s default field-splitting +mechanism to break each line up into ``words'' and uses an associative array named @code{freq}, indexed by each word, to count the number of times the word occurs. In the @code{END} rule, it prints the counts. @@ -25311,9 +25461,9 @@ useful on real text files: @item The @command{awk} language considers upper- and lowercase characters to be distinct. Therefore, ``bartender'' and ``Bartender'' are not treated -as the same word. This is undesirable, since in normal text, words -are capitalized if they begin sentences, and a frequency analyzer should not -be sensitive to capitalization. +as the same word. This is undesirable, because words are capitalized +if they begin sentences in normal text, and a frequency analyzer should +not be sensitive to capitalization. @item Words are detected using the @command{awk} convention that fields are @@ -25334,7 +25484,6 @@ to remove punctuation characters. Finally, we solve the third problem by using the system @command{sort} utility to process the output of the @command{awk} script. Here is the new version of the program: -@c STARTOFRANGE wordfreq @cindex @code{wordfreq.awk} program @example @c file eg/prog/wordfreq.awk @@ -25355,8 +25504,8 @@ END @{ @} @end example -The regexp @samp{/[^[:alnum:]_[:blank:]]/} might have been written -@samp{/[[:punct:]]/}, but then underscores would also be removed, +The regexp @code{/[^[:alnum:]_[:blank:]]/} might have been written +@code{/[[:punct:]]/}, but then underscores would also be removed, and we want to keep them. Assuming we have saved this program in a file named @file{wordfreq.awk}, @@ -25399,16 +25548,13 @@ This way of sorting must be used on systems that do not have true pipes at the command-line (or batch-file) level. See the general operating system documentation for more information on how to use the @command{sort} program. -@c ENDOFRANGE worus -@c ENDOFRANGE wordfreq @node History Sorting @subsection Removing Duplicates from Unsorted Text -@c STARTOFRANGE lidu @cindex lines, duplicate@comma{} removing The @command{uniq} program -(@pxref{Uniq Program}), +(@pxref{Uniq Program}) removes duplicate lines from @emph{sorted} data. Suppose, however, you need to remove duplicate lines from a @value{DF} but @@ -25430,7 +25576,6 @@ Each element of @code{lines} is a unique command, and the indices of The @code{END} rule simply prints out the lines, in order: @cindex Rakitzis, Byron -@c STARTOFRANGE histsort @cindex @code{histsort.awk} program @example @c file eg/prog/histsort.awk @@ -25473,15 +25618,11 @@ print data[lines[i]], lines[i] @noindent This works because @code{data[$0]} is incremented each time a line is seen. -@c ENDOFRANGE lidu -@c ENDOFRANGE histsort @node Extract Program @subsection Extracting Programs from Texinfo Source Files -@c STARTOFRANGE texse @cindex Texinfo, extracting programs from source files -@c STARTOFRANGE fitex @cindex files, Texinfo@comma{} extracting programs from @ifnotinfo Both this chapter and the previous chapter @@ -25494,13 +25635,13 @@ The nodes and @ref{Sample Programs}, are the top level nodes for a large number of @command{awk} programs. @end ifinfo -If you want to experiment with these programs, it is tedious to have to type +If you want to experiment with these programs, it is tedious to type them in by hand. Here we present a program that can extract parts of a Texinfo input file into separate files. @cindex Texinfo This @value{DOCUMENT} is written in @uref{http://www.gnu.org/software/texinfo/, Texinfo}, -the GNU project's document formatting language. +the GNU Project's document formatting language. A single Texinfo source file can be used to produce both printed documentation, with @TeX{}, and online documentation. @ifnotinfo @@ -25559,7 +25700,7 @@ The Texinfo file looks something like this: @example @dots{} -This program has a @@code@{BEGIN@} rule, +This program has a @@code@{BEGIN@} rule that prints a nice message: @@example @@ -25572,7 +25713,7 @@ It also prints some final advice: @@example @@c file examples/messages.awk -END @@@{ print "Always avoid bored archeologists!" @@@} +END @@@{ print "Always avoid bored archaeologists!" @@@} @@c end file @@end example @dots{} @@ -25585,11 +25726,10 @@ The first rule handles calling @code{system()}, checking that a command is given (@code{NF} is at least three) and also checking that the command exits with a zero exit status, signifying OK: -@c STARTOFRANGE extract @cindex @code{extract.awk} program @example @c file eg/prog/extract.awk -# extract.awk --- extract files and run programs from texinfo files +# extract.awk --- extract files and run programs from Texinfo files @c endfile @ignore @c file eg/prog/extract.awk @@ -25630,12 +25770,12 @@ The second rule handles moving data into files. It verifies that a @value{FN} is given in the directive. If the file named is not the current file, then the current file is closed. Keeping the current file open until a new file is encountered allows the use of the @samp{>} -redirection for printing the contents, keeping open file management +redirection for printing the contents, keeping open-file management simple. The @code{for} loop does the work. It reads lines using @code{getline} (@pxref{Getline}). -For an unexpected end of file, it calls the @code{@w{unexpected_eof()}} +For an unexpected end-of-file, it calls the @code{@w{unexpected_eof()}} function. If the line is an ``endfile'' line, then it breaks out of the loop. If the line is an @samp{@@group} or @samp{@@end group} line, then it @@ -25731,20 +25871,17 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE texse -@c ENDOFRANGE fitex -@c ENDOFRANGE extract @node Simple Sed @subsection A Simple Stream Editor @cindex @command{sed} utility @cindex stream editors -The @command{sed} utility is a stream editor, a program that reads a +The @command{sed} utility is a @dfn{stream editor}, a program that reads a stream of data, makes changes to it, and passes it on. It is often used to make global changes to a large file or to a stream of data generated by a pipeline of commands. -While @command{sed} is a complicated program in its own right, its most common +Although @command{sed} is a complicated program in its own right, its most common use is to perform global substitutions in the middle of a pipeline: @example @@ -25753,7 +25890,7 @@ use is to perform global substitutions in the middle of a pipeline: Here, @samp{s/old/new/g} tells @command{sed} to look for the regexp @samp{old} on each input line and globally replace it with the text -@samp{new}, i.e., all the occurrences on a line. This is similar to +@samp{new} (i.e., all the occurrences on a line). This is similar to @command{awk}'s @code{gsub()} function (@pxref{String Functions}). @@ -25763,7 +25900,6 @@ additional arguments are treated as @value{DF} names to process. If none are provided, the standard input is used: @cindex Brennan, Michael -@c STARTOFRANGE awksed @cindex @command{awksed.awk} program @c @cindex simple stream editor @c @cindex stream editor, simple @@ -25837,17 +25973,14 @@ not treated as @value{FN}s (@pxref{ARGC and ARGV}). The @code{usage()} function prints an error message and exits. -Finally, the single rule handles the printing scheme outlined above, +Finally, the single rule handles the printing scheme outlined earlier, using @code{print} or @code{printf} as appropriate, depending upon the value of @code{RT}. -@c ENDOFRANGE awksed @node Igawk Program @subsection An Easy Way to Use Library Functions -@c STARTOFRANGE libfex @cindex libraries of @command{awk} functions, example program for using -@c STARTOFRANGE flibex @cindex functions, library, example program for using In @ref{Include Files}, we saw how @command{gawk} provides a built-in file-inclusion capability. However, this is a @command{gawk} extension. @@ -25881,15 +26014,15 @@ BEGIN @{ The following program, @file{igawk.sh}, provides this service. It simulates @command{gawk}'s searching of the @env{AWKPATH} variable -and also allows @dfn{nested} includes; i.e., a file that is included -with @code{@@include} can contain further @code{@@include} statements. +and also allows @dfn{nested} includes (i.e., a file that is included +with @code{@@include} can contain further @code{@@include} statements). @command{igawk} makes an effort to only include files once, so that nested includes don't accidentally include a library function twice. @command{igawk} should behave just like @command{gawk} externally. This means it should accept all of @command{gawk}'s command-line arguments, including the ability to have multiple source files specified via -@option{-f}, and the ability to mix command-line and library source files. +@option{-f} and the ability to mix command-line and library source files. The program is written using the POSIX Shell (@command{sh}) command language.@footnote{Fully explaining the @command{sh} language is beyond @@ -25912,10 +26045,10 @@ Literal text, provided with @option{-e} or @option{--source}. This text is just appended directly. @item -Source @value{FN}s, provided with @option{-f}. We use a neat trick and append -@samp{@@include @var{filename}} to the shell variable's contents. Since the file-inclusion -program works the way @command{gawk} does, this gets the text -of the file included into the program at the correct point. +Source @value{FN}s, provided with @option{-f}. We use a neat trick and +append @samp{@@include @var{filename}} to the shell variable's contents. +Because the file-inclusion program works the way @command{gawk} does, this +gets the text of the file included in the program at the correct point. @end enumerate @item @@ -25928,7 +26061,7 @@ Run the expanded program with @command{gawk} and any other original command-line arguments that the user supplied (such as the @value{DF} names). @end enumerate -This program uses shell variables extensively: for storing command-line arguments, +This program uses shell variables extensively: for storing command-line arguments and the text of the @command{awk} program that will expand the user's program, for the user's original program, and for the expanded program. Doing so removes some potential problems that might arise were we to use temporary files instead, @@ -25986,7 +26119,6 @@ program. The program is as follows: -@c STARTOFRANGE igawk @cindex @code{igawk.sh} program @example @c file eg/prog/igawk.sh @@ -26214,9 +26346,10 @@ EOF @c endfile @end example -The shell construct @samp{@var{command} << @var{marker}} is called a @dfn{here document}. -Everything in the shell script up to the @var{marker} is fed to @var{command} as input. -The shell processes the contents of the here document for variable and command substitution +The shell construct @samp{@var{command} << @var{marker}} is called +a @dfn{here document}. Everything in the shell script up to the +@var{marker} is fed to @var{command} as input. The shell processes +the contents of the here document for variable and command substitution (and possibly other things as well, depending upon the shell). The shell construct @samp{$(@dots{})} is called @dfn{command substitution}. @@ -26231,34 +26364,21 @@ It's done in these steps: @enumerate @item Run @command{gawk} with the @code{@@include}-processing program (the -value of the @code{expand_prog} shell variable) on standard input. +value of the @code{expand_prog} shell variable) reading standard input. @item -Standard input is the contents of the user's program, from the shell variable @code{program}. -Its contents are fed to @command{gawk} via a here document. +Standard input is the contents of the user's program, +from the shell variable @code{program}. +Feed its contents to @command{gawk} via a here document. @item -The results of this processing are saved in the shell variable @code{processed_program} by using command substitution. +Save the results of this processing in the shell variable +@code{processed_program} by using command substitution. @end enumerate The last step is to call @command{gawk} with the expanded program, along with the original -options and command-line arguments that the user supplied. - -@c this causes more problems than it solves, so leave it out. -@ignore -The special file @file{/dev/null} is passed as a @value{DF} to @command{gawk} -to handle an interesting case. Suppose that the user's program only has -a @code{BEGIN} rule and there are no @value{DF}s to read. -The program should exit without reading any @value{DF}s. -However, suppose that an included library file defines an @code{END} -rule of its own. In this case, @command{gawk} will hang, reading standard -input. In order to avoid this, @file{/dev/null} is explicitly added to the -command line. Reading from @file{/dev/null} always returns an immediate -end of file indication. - -@c Hmm. Add /dev/null if $# is 0? Still messes up ARGV. Sigh. -@end ignore +options and command-line arguments that the user supplied: @example @c file eg/prog/igawk.sh @@ -26308,13 +26428,9 @@ features to a program; they can often be layered on top.@footnote{@command{gawk} does @code{@@include} processing itself in order to support the use of @command{awk} programs as Web CGI scripts.} -@c ENDOFRANGE libfex -@c ENDOFRANGE flibex -@c ENDOFRANGE awkpex -@c ENDOFRANGE igawk @node Anagram Program -@subsection Finding Anagrams From A Dictionary +@subsection Finding Anagrams from a Dictionary @cindex anagrams, finding An interesting programming challenge is to @@ -26323,24 +26439,23 @@ word list (such as @file{/usr/share/dict/words} on many GNU/Linux systems). One word is an anagram of another if both words contain the same letters -(for example, ``babbling'' and ``blabbing''). +(e.g., ``babbling'' and ``blabbing''). -Column 2, Problem C of Jon Bentley's @cite{Programming Pearls}, second -edition, presents an elegant algorithm. The idea is to give words that +Column 2, Problem C, of Jon Bentley's @cite{Programming Pearls}, Second +Edition, presents an elegant algorithm. The idea is to give words that are anagrams a common signature, sort all the words together by their -signature, and then print them. Dr.@: Bentley observes that taking the -letters in each word and sorting them produces that common signature. +signatures, and then print them. Dr.@: Bentley observes that taking the +letters in each word and sorting them produces those common signatures. The following program uses arrays of arrays to bring together words with the same signature and array sorting to print the words -in sorted order. +in sorted order: -@c STARTOFRANGE anagram @cindex @code{anagram.awk} program @example @c file eg/prog/anagram.awk -# anagram.awk --- An implementation of the anagram finding algorithm -# from Jon Bentley's "Programming Pearls", 2nd edition. +# anagram.awk --- An implementation of the anagram-finding algorithm +# from Jon Bentley's "Programming Pearls," 2nd edition. # Addison Wesley, 2000, ISBN 0-201-65788-0. # Column 2, Problem C, section 2.8, pp 18-20. @c endfile @@ -26388,7 +26503,7 @@ sorts the letters, and then joins them back together: @example @c file eg/prog/anagram.awk -# word2key --- split word apart into letters, sort, joining back together +# word2key --- split word apart into letters, sort, and join back together function word2key(word, a, i, n, result) @{ @@ -26405,7 +26520,7 @@ function word2key(word, a, i, n, result) Finally, the @code{END} rule traverses the array and prints out the anagram lists. It sends the output -to the system @command{sort} command, since otherwise +to the system @command{sort} command because otherwise the anagrams would appear in arbitrary order: @example @@ -26433,21 +26548,20 @@ Here is some partial output when the program is run: @example $ @kbd{gawk -f anagram.awk /usr/share/dict/words | grep '^b'} @dots{} -babbled blabbed -babbler blabber brabble -babblers blabbers brabbles -babbling blabbing -babbly blabby -babel bable -babels beslab -babery yabber +babbled blabbed +babbler blabber brabble +babblers blabbers brabbles +babbling blabbing +babbly blabby +babel bable +babels beslab +babery yabber @dots{} @end example -@c ENDOFRANGE anagram @node Signature Program -@subsection And Now For Something Completely Different +@subsection And Now for Something Completely Different @cindex signature program @cindex Brini, Davide @@ -26487,28 +26601,28 @@ Subject: The GNU Awk User's Guide, Section 13.3.11 From: "Chris Johansen" <johansen@main.nc.us> Message-ID: <op.v0iw6wlv7finx3@asusodin.thrudvang.lan> -Arnold, you don't know me, but we have a tenuous connection. My wife is +Arnold, you don't know me, but we have a tenuous connection. My wife is Barbara A. Field, FAIA, GIT '65 (B. Arch.). -I have had a couple of paper copies of "Effective Awk Programming" for -years, and now I'm going through a Kindle version of "The GNU Awk User's -Guide" again. When I got to section 13.3.11, I reformatted and lightly +I have had a couple of paper copies of "Effective Awk Programming" for +years, and now I'm going through a Kindle version of "The GNU Awk User's +Guide" again. When I got to section 13.3.11, I reformatted and lightly commented Davide Brin's signature script to understand its workings. -It occurs to me that this might have pedagogical value as an example -(although imperfect) of the value of whitespace and comments, and a -starting point for that discussion. It certainly helped _me_ understand -what's going on. You are welcome to it, as-is or modified (subject to +It occurs to me that this might have pedagogical value as an example +(although imperfect) of the value of whitespace and comments, and a +starting point for that discussion. It certainly helped _me_ understand +what's going on. You are welcome to it, as-is or modified (subject to Davide's constraints, of course, which I think I have met). -If I were to include it in a future edition, I would put it at some -distance from section 13.3.11, say, as a note or an appendix, so as not to +If I were to include it in a future edition, I would put it at some +distance from section 13.3.11, say, as a note or an appendix, so as not to be a "spoiler" to the puzzle. Best regards, --- +-- Chris Johansen {johansen at main dot nc dot us} - . . . collapsing the probability wave function, sending ripples of + . . . collapsing the probability wave function, sending ripples of certainty through the space-time continuum. @@ -26517,7 +26631,7 @@ certainty through the space-time continuum. # From "13.3.11 And Now For Something Completely Different" # http://www.gnu.org/software/gawk/manual/html_node/Signature-Program.html#Signature-Program -# Copyright © 2008 Davide Brini +# Copyright © 2008 Davide Brini # Copying and distribution of the code published in this page, with # or without modification, are permitted in any medium without @@ -26584,12 +26698,13 @@ characters. The ability to use @code{split()} with the empty string as the separator can considerably simplify such tasks. @item -The library functions from @ref{Library Functions}, proved their -usefulness for a number of real (if small) programs. +The examples here demonstrate the usefulness of the library +functions from @DBREF{Library Functions} +for a number of real (if small) programs. @item Besides reinventing POSIX wheels, other programs solved a selection of -interesting problems, such as finding duplicates words in text, printing +interesting problems, such as finding duplicate words in text, printing mailing labels, and finding anagrams. @end itemize @@ -26636,7 +26751,7 @@ Brian Kernighan suggests that ``an alternative approach to state machines is to just read the input into an array, then use indexing. It's almost always easier code, and for most inputs where you would use this, just -as fast.'' Rewrite the logic to follow this +as fast.'' Rewrite the logic to follow this suggestion. @@ -26737,7 +26852,7 @@ the use of the external @command{sort} utility. @c EXCLUDE END @ifnotinfo -@part @value{PART3}Moving Beyond Standard @command{awk} With @command{gawk} +@part @value{PART3}Moving Beyond Standard @command{awk} with @command{gawk} @end ifnotinfo @ifdocbook @@ -26746,27 +26861,25 @@ It contains the following chapters: @itemize @value{BULLET} @item -@ref{Advanced Features}. +@ref{Advanced Features} @item -@ref{Internationalization}. +@ref{Internationalization} @item -@ref{Debugger}. +@ref{Debugger} @item -@ref{Arbitrary Precision Arithmetic}. +@ref{Arbitrary Precision Arithmetic} @item -@ref{Dynamic Extensions}. +@ref{Dynamic Extensions} @end itemize @end ifdocbook @node Advanced Features @chapter Advanced Features of @command{gawk} -@c STARTOFRANGE gawadv @cindex @command{gawk}, features, advanced -@c STARTOFRANGE advgaw @cindex advanced features, @command{gawk} @ignore Contributed by: Peter Langston <pud!psl@bellcore.bellcore.com> @@ -26787,18 +26900,18 @@ a violent psychopath who knows where you live.} This @value{CHAPTER} discusses advanced features in @command{gawk}. It's a bit of a ``grab bag'' of items that are otherwise unrelated to each other. -First, a command-line option allows @command{gawk} to recognize +First, we look at a command-line option that allows @command{gawk} to recognize nondecimal numbers in input data, not just in @command{awk} programs. Then, @command{gawk}'s special features for sorting arrays are presented. Next, two-way I/O, discussed briefly in earlier parts of this @value{DOCUMENT}, is described in full detail, along with the basics -of TCP/IP networking. Finally, @command{gawk} +of TCP/IP networking. Finally, we see how @command{gawk} can @dfn{profile} an @command{awk} program, making it possible to tune it for performance. @c FULLXREF ON -A number of advanced features require separate @value{CHAPTER}s of their +Additional advanced features are discussed in separate @value{CHAPTER}s of their own: @itemize @value{BULLET} @@ -26892,7 +27005,8 @@ This option may disappear in a future version of @command{gawk}. @node Array Sorting @section Controlling Array Traversal and Array Sorting -@command{gawk} lets you control the order in which a @samp{for (i in array)} +@command{gawk} lets you control the order in which a +@samp{for (@var{indx} in @var{array})} loop traverses an array. In addition, two built-in functions, @code{asort()} and @code{asorti()}, @@ -26908,7 +27022,7 @@ to order the elements during sorting. @node Controlling Array Traversal @subsection Controlling Array Traversal -By default, the order in which a @samp{for (i in array)} loop +By default, the order in which a @samp{for (@var{indx} in @var{array})} loop scans an array is not defined; it is generally based upon the internal implementation of arrays inside @command{awk}. @@ -26917,7 +27031,7 @@ in a particular order that you, the programmer, choose. @command{gawk} lets you do this. @DBREF{Controlling Scanning} describes how you can assign special, -pre-defined values to @code{PROCINFO["sorted_in"]} in order to +predefined values to @code{PROCINFO["sorted_in"]} in order to control the order in which @command{gawk} traverses an array during a @code{for} loop. @@ -26937,23 +27051,23 @@ function comp_func(i1, v1, i2, v2) @} @end example -Here, @var{i1} and @var{i2} are the indices, and @var{v1} and @var{v2} +Here, @code{i1} and @code{i2} are the indices, and @code{v1} and @code{v2} are the corresponding values of the two elements being compared. -Either @var{v1} or @var{v2}, or both, can be arrays if the array being +Either @code{v1} or @code{v2}, or both, can be arrays if the array being traversed contains subarrays as values. -(@xref{Arrays of Arrays}, for more information about subarrays.) +(@DBXREF{Arrays of Arrays} for more information about subarrays.) The three possible return values are interpreted as follows: @table @code @item comp_func(i1, v1, i2, v2) < 0 -Index @var{i1} comes before index @var{i2} during loop traversal. +Index @code{i1} comes before index @code{i2} during loop traversal. @item comp_func(i1, v1, i2, v2) == 0 -Indices @var{i1} and @var{i2} -come together but the relative order with respect to each other is undefined. +Indices @code{i1} and @code{i2} +come together, but the relative order with respect to each other is undefined. @item comp_func(i1, v1, i2, v2) > 0 -Index @var{i1} comes after index @var{i2} during loop traversal. +Index @code{i1} comes after index @code{i2} during loop traversal. @end table Our first comparison function can be used to scan an array in @@ -26984,7 +27098,7 @@ function cmp_str_val(i1, v1, i2, v2) The third comparison function makes all numbers, and numeric strings without -any leading or trailing spaces, come out first during loop traversal: +any leading or trailing spaces, come out first during loop traversal: @example function cmp_num_str_val(i1, v1, i2, v2, n1, n2) @@ -26992,10 +27106,10 @@ function cmp_num_str_val(i1, v1, i2, v2, n1, n2) # numbers before string value comparison, ascending order n1 = v1 + 0 n2 = v2 + 0 - if (n1 == v1) + if (n1 == v1) return (n2 == v2) ? (n1 - n2) : -1 else if (n2 == v2) - return 1 + return 1 return (v1 < v2) ? -1 : (v1 != v2) @} @end example @@ -27010,7 +27124,7 @@ BEGIN @{ data[10] = "one" data[100] = 100 data[20] = "two" - + f[1] = "cmp_num_idx" f[2] = "cmp_str_val" f[3] = "cmp_num_str_val" @@ -27034,14 +27148,14 @@ $ @kbd{gawk -f compdemo.awk} @print{} data[10] = one @print{} data[20] = two @print{} data[100] = 100 -@print{} +@print{} @print{} Sort function: cmp_str_val @ii{Sort by element values as strings} @print{} data[one] = 10 @print{} data[100] = 100 @ii{String 100 is less than string 20} @print{} data[two] = 20 @print{} data[10] = one @print{} data[20] = two -@print{} +@print{} @print{} Sort function: cmp_num_str_val @ii{Sort all numeric values before all strings} @print{} data[one] = 10 @print{} data[two] = 20 @@ -27052,7 +27166,7 @@ $ @kbd{gawk -f compdemo.awk} Consider sorting the entries of a GNU/Linux system password file according to login name. The following program sorts records -by a specific field position and can be used for this purpose: +by a specific field position and can be used for this purpose: @example # passwd-sort.awk --- simple program to sort by field position @@ -27098,7 +27212,7 @@ $ @kbd{gawk -v POS=1 -F: -f sort.awk /etc/passwd} The comparison should normally always return the same value when given a specific pair of array elements as its arguments. If inconsistent -results are returned then the order is undefined. This behavior can be +results are returned, then the order is undefined. This behavior can be exploited to introduce random order into otherwise seemingly ordered data: @@ -27110,11 +27224,11 @@ function cmp_randomize(i1, v1, i2, v2) @} @end example -As mentioned above, the order of the indices is arbitrary if two +As already mentioned, the order of the indices is arbitrary if two elements compare equal. This is usually not a problem, but letting the tied elements come out in arbitrary order can be an issue, especially when comparing item values. The partial ordering of the equal elements -may change the next time the array is traversed, if other elements are added or +may change the next time the array is traversed, if other elements are added to or removed from the array. One way to resolve ties when comparing elements with otherwise equal values is to include the indices in the comparison rules. Note that doing this may make the loop traversal less efficient, @@ -27151,21 +27265,21 @@ When string comparisons are made during a sort, either for element values where one or both aren't numbers, or for element indices handled as strings, the value of @code{IGNORECASE} (@pxref{Built-in Variables}) controls whether -the comparisons treat corresponding uppercase and lowercase letters as +the comparisons treat corresponding upper- and lowercase letters as equivalent or distinct. -Another point to keep in mind is that in the case of subarrays +Another point to keep in mind is that in the case of subarrays, the element values can themselves be arrays; a production comparison function should use the @code{isarray()} function -(@pxref{Type Functions}), +(@pxref{Type Functions}) to check for this, and choose a defined sorting order for subarrays. All sorting based on @code{PROCINFO["sorted_in"]} is disabled in POSIX mode, -since the @code{PROCINFO} array is not special in that case. +because the @code{PROCINFO} array is not special in that case. As a side note, sorting the array indices before traversing -the array has been reported to add 15% to 20% overhead to the +the array has been reported to add a 15% to 20% overhead to the execution time of @command{awk} programs. For this reason, sorted array traversal is not the default. @@ -27183,8 +27297,8 @@ sorted array traversal is not the default. @cindex @code{asorti()} function (@command{gawk}), arrays@comma{} sorting @cindex sort function, arrays, sorting In most @command{awk} implementations, sorting an array requires writing -a @code{sort()} function. While this can be educational for exploring -different sorting algorithms, usually that's not the point of the program. +a @code{sort()} function. This can be educational for exploring +different sorting algorithms, but usually that's not the point of the program. @command{gawk} provides the built-in @code{asort()} and @code{asorti()} functions (@pxref{String Functions}) for sorting arrays. For example: @@ -27224,7 +27338,7 @@ However, the @code{source} array is not affected. Often, what's needed is to sort on the values of the @emph{indices} instead of the values of the elements. To do that, use the @code{asorti()} function. The interface and behavior are identical to -that of @code{asort()}, except that the index values are used for sorting, +that of @code{asort()}, except that the index values are used for sorting and become the values of the result array: @example @@ -27259,8 +27373,8 @@ it chooses}, taking into account just the indices, just the values, or both. This is extremely powerful. Once the array is sorted, @code{asort()} takes the @emph{values} in -their final order, and uses them to fill in the result array, whereas -@code{asorti()} takes the @emph{indices} in their final order, and uses +their final order and uses them to fill in the result array, whereas +@code{asorti()} takes the @emph{indices} in their final order and uses them to fill in the result array. @cindex reference counting, sorting arrays @@ -27280,8 +27394,8 @@ Because @code{IGNORECASE} affects string comparisons, the value of @code{IGNORECASE} also affects sorting for both @code{asort()} and @code{asorti()}. Note also that the locale's sorting order does @emph{not} come into play; comparisons are based on character values only.@footnote{This -is true because locale-based comparison occurs only when in POSIX -compatibility mode, and since @code{asort()} and @code{asorti()} are +is true because locale-based comparison occurs only when in +POSIX-compatibility mode, and because @code{asort()} and @code{asorti()} are @command{gawk} extensions, they are not available in that case.} @node Two-way I/O @@ -27357,7 +27471,7 @@ remain more difficult to use than two-way pipes.} @c 8/2014 @cindex @command{csh} utility, @code{|&} operator, comparison with However, with @command{gawk}, it is possible to open a @emph{two-way} pipe to another process. The second process is -termed a @dfn{coprocess}, since it runs in parallel with @command{gawk}. +termed a @dfn{coprocess}, as it runs in parallel with @command{gawk}. The two-way connection is created using the @samp{|&} operator (borrowed from the Korn shell, @command{ksh}):@footnote{This is very different from the same operator in the C shell and in Bash.} @@ -27462,7 +27576,7 @@ like so: @example command = "sort -nr" # command, save in convenience variable PROCINFO[command, "pty"] = 1 # update PROCINFO -print @dots{} |& command # start two-way pipe +print @dots{} |& command # start two-way pipe @dots{} @end example @@ -27476,7 +27590,6 @@ using regular pipes. @section Using @command{gawk} for Network Programming @cindex advanced features, network programming @cindex networks, programming -@c STARTOFRANGE tcpip @cindex TCP/IP @cindex @code{/inet/@dots{}} special files (@command{gawk}) @cindex files, @code{/inet/@dots{}} (@command{gawk}) @@ -27517,7 +27630,7 @@ You can think of this as just a @emph{very long} two-way pipeline to a coprocess. The way @command{gawk} decides that you want to use TCP/IP networking is by recognizing special @value{FN}s that begin with one of @samp{/inet/}, -@samp{/inet4/} or @samp{/inet6/}. +@samp{/inet4/}, or @samp{/inet6/}. The full syntax of the special @value{FN} is @file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}}. @@ -27546,7 +27659,7 @@ or @samp{http}, in which case @command{gawk} attempts to determine the predefined port number using the C @code{getaddrinfo()} function. @item remote-host -The IP address or fully-qualified domain name of the Internet +The IP address or fully qualified domain name of the Internet host to which you want to connect. @item remote-port @@ -27558,7 +27671,7 @@ service name. @cindex @command{gawk}, @code{ERRNO} variable in @cindex @code{ERRNO} variable @quotation NOTE -Failure in opening a two-way socket will result in a non-fatal error +Failure in opening a two-way socket will result in a nonfatal error being returned to the calling code. The value of @code{ERRNO} indicates the error (@pxref{Auto-set}). @end quotation @@ -27575,31 +27688,28 @@ BEGIN @{ @end example This program reads the current date and time from the local system's -TCP @samp{daytime} server. +TCP @code{daytime} server. It then prints the results and closes the connection. Because this topic is extensive, the use of @command{gawk} for TCP/IP programming is documented separately. @ifinfo See -@inforef{Top, , General Introduction, gawkinet, TCP/IP Internetworking with @command{gawk}}, +@inforef{Top, , General Introduction, gawkinet, @value{GAWKINETTITLE}}, @end ifinfo @ifnotinfo See @uref{http://www.gnu.org/software/gawk/manual/gawkinet/, -@cite{TCP/IP Internetworking with @command{gawk}}}, +@cite{@value{GAWKINETTITLE}}}, which comes as part of the @command{gawk} distribution, @end ifnotinfo for a much more complete introduction and discussion, as well as extensive examples. -@c ENDOFRANGE tcpip @node Profiling @section Profiling Your @command{awk} Programs -@c STARTOFRANGE awkp @cindex @command{awk} programs, profiling -@c STARTOFRANGE proawk @cindex profiling @command{awk} programs @cindex @code{awkprof.out} file @cindex files, @code{awkprof.out} @@ -27620,12 +27730,12 @@ gawk --profile=myprog.prof -f myprog.awk data1 data2 @end example @noindent -In the above example, @command{gawk} places the profile in +In the preceding example, @command{gawk} places the profile in @file{myprog.prof} instead of in @file{awkprof.out}. -Here is a sample session showing a simple @command{awk} program, its input data, and the -results from running @command{gawk} with the @option{--profile} option. -First, the @command{awk} program: +Here is a sample session showing a simple @command{awk} program, +its input data, and the results from running @command{gawk} with the +@option{--profile} option. First, the @command{awk} program: @example BEGIN @{ print "First BEGIN rule" @} @@ -27666,9 +27776,9 @@ junk @end example Here is the @file{awkprof.out} that results from running the -@command{gawk} profiler on this program and data. (This example also +@command{gawk} profiler on this program and data (this example also illustrates that @command{awk} programmers sometimes get up very early -in the morning to work.) +in the morning to work): @cindex @code{BEGIN} pattern, and profiling @cindex @code{END} pattern, and profiling @@ -27728,8 +27838,8 @@ They are as follows: @item The program is printed in the order @code{BEGIN} rules, @code{BEGINFILE} rules, -pattern/action rules, -@code{ENDFILE} rules, @code{END} rules and functions, listed +pattern--action rules, +@code{ENDFILE} rules, @code{END} rules, and functions, listed alphabetically. Multiple @code{BEGIN} and @code{END} rules retain their separate identities, as do @@ -27737,7 +27847,7 @@ multiple @code{BEGINFILE} and @code{ENDFILE} rules. @cindex patterns, counts, in a profile @item -Pattern-action rules have two counts. +Pattern--action rules have two counts. The first count, to the left of the rule, shows how many times the rule's pattern was @emph{tested}. The second count, to the right of the rule's opening left brace @@ -27783,7 +27893,7 @@ the body of an @code{if}, @code{else}, or loop is only a single statement. @item Parentheses are used only where needed, as indicated by the structure of the program and the precedence rules. -For example, @samp{(3 + 5) * 4} means add three plus five, then multiply +For example, @samp{(3 + 5) * 4} means add three and five, then multiply the total by four. However, @samp{3 + 5 * 4} has no parentheses, and means @samp{3 + (5 * 4)}. @@ -27804,13 +27914,13 @@ the target of a redirection isn't a scalar, it gets parenthesized. @command{gawk} supplies leading comments in front of the @code{BEGIN} and @code{END} rules, the @code{BEGINFILE} and @code{ENDFILE} rules, -the pattern/action rules, and the functions. +the pattern--action rules, and the functions. @end itemize The profiled version of your program may not look exactly like what you typed when you wrote it. This is because @command{gawk} creates the -profiled version by ``pretty printing'' its internal representation of +profiled version by ``pretty-printing'' its internal representation of the program. The advantage to this is that @command{gawk} can produce a standard representation. Also, things such as: @@ -27893,16 +28003,16 @@ If you use the @code{HUP} signal instead of the @code{USR1} signal, @cindex @code{SIGQUIT} signal (MS-Windows) @cindex signals, @code{QUIT}/@code{SIGQUIT} (MS-Windows) When @command{gawk} runs on MS-Windows systems, it uses the -@code{INT} and @code{QUIT} signals for producing the profile and, in +@code{INT} and @code{QUIT} signals for producing the profile, and in the case of the @code{INT} signal, @command{gawk} exits. This is because these systems don't support the @command{kill} command, so the only signals you can deliver to a program are those generated by the keyboard. The @code{INT} signal is generated by the -@kbd{Ctrl-@key{C}} or @kbd{Ctrl-@key{BREAK}} key, while the -@code{QUIT} signal is generated by the @kbd{Ctrl-@key{\}} key. +@kbd{Ctrl-c} or @kbd{Ctrl-BREAK} key, while the +@code{QUIT} signal is generated by the @kbd{Ctrl-\} key. Finally, @command{gawk} also accepts another option, @option{--pretty-print}. -When called this way, @command{gawk} ``pretty prints'' the program into +When called this way, @command{gawk} ``pretty-prints'' the program into @file{awkprof.out}, without any execution counts. @quotation NOTE @@ -27926,9 +28036,6 @@ that the profiling output does. This makes it easy to pretty-print your code once development is completed, and then use the result as the final version of your program. -@c ENDOFRANGE awkp -@c ENDOFRANGE proawk - @node Advanced Features Summary @section Summary @@ -27959,7 +28066,7 @@ optionally, close off one side of the two-way communications. @item By using special @value{FN}s with the @samp{|&} operator, you can open a -TCP/IP (or UDP/IP) connection to remote hosts in the Internet. @command{gawk} +TCP/IP (or UDP/IP) connection to remote hosts on the Internet. @command{gawk} supports both IPv4 and IPv6. @item @@ -27969,13 +28076,11 @@ you tune them more easily. Sending the @code{USR1} signal while profiling cause @command{gawk} to dump the profile and keep going, including a function call stack. @item -You can also just ``pretty print'' the program. This currently also runs +You can also just ``pretty-print'' the program. This currently also runs the program, but that will change in the next major release. @end itemize -@c ENDOFRANGE advgaw -@c ENDOFRANGE gawadv @node Internationalization @chapter Internationalization with @command{gawk} @@ -27988,7 +28093,6 @@ countries, they were able to sell more systems. As a result, internationalization and localization of programs and software systems became a common practice. -@c STARTOFRANGE inloc @cindex internationalization, localization @cindex @command{gawk}, internationalization and, See internationalization @cindex internationalization, localization, @command{gawk} and @@ -28021,7 +28125,7 @@ a requirement. @cindex localization @dfn{Internationalization} means writing (or modifying) a program once, in such a way that it can use multiple languages without requiring -further source-code changes. +further source code changes. @dfn{Localization} means providing the data necessary for an internationalized program to work in a particular language. Most typically, these terms refer to features such as the language @@ -28033,11 +28137,10 @@ monetary values are printed and read. @section GNU @command{gettext} @cindex internationalizing a program -@c STARTOFRANGE gettex @cindex @command{gettext} library @command{gawk} uses GNU @command{gettext} to provide its internationalization features. -The facilities in GNU @command{gettext} focus on messages; strings printed +The facilities in GNU @command{gettext} focus on messages: strings printed by a program, either directly or via formatting with @code{printf} or @code{sprintf()}.@footnote{For some operating systems, the @command{gawk} port doesn't support GNU @command{gettext}. @@ -28058,8 +28161,7 @@ following steps, in this order: @enumerate @item -The programmer goes -through the source for all of @command{guide}'s components +The programmer reviews the source for all of @command{guide}'s components and marks each string that is a candidate for translation. For example, @code{"`-F': option required"} is a good candidate for translation. A table with strings of option names is not (e.g., @command{gawk}'s @@ -28086,7 +28188,6 @@ lookup of the translations. @cindex @code{.po} files @cindex files, @code{.po} -@c STARTOFRANGE portobfi @cindex portable object files @cindex files, portable object @item @@ -28098,7 +28199,6 @@ For example, there might be a @file{fr.po} for a French translation. @cindex @code{.gmo} files @cindex files, @code{.gmo} @cindex message object files -@c STARTOFRANGE portmsgfi @cindex files, message object @item Each language's @file{.po} file is converted into a binary @@ -28179,8 +28279,8 @@ if necessary. (It is almost never necessary to supply a different category.) @cindex sorting characters in different languages @cindex @code{LC_COLLATE} locale category @item LC_COLLATE -Text-collation information; i.e., how different characters -and/or groups of characters sort in a given language. +Text-collation information (i.e., how different characters +and/or groups of characters sort in a given language). @cindex @code{LC_CTYPE} locale category @item LC_CTYPE @@ -28226,14 +28326,12 @@ before or after the day in a date, local month abbreviations, and so on. @item LC_ALL All of the above. (Not too useful in the context of @command{gettext}.) @end table -@c ENDOFRANGE gettex @node Programmer i18n @section Internationalizing @command{awk} Programs -@c STARTOFRANGE inap @cindex @command{awk} programs, internationalizing -@command{gawk} provides the following variables and functions for +@command{gawk} provides the following variables for internationalization: @table @code @@ -28249,7 +28347,12 @@ value is @code{"messages"}. String constants marked with a leading underscore are candidates for translation at runtime. String constants without a leading underscore are not translated. +@end table +@command{gawk} provides the following functions for +internationalization: + +@table @code @cindexgawkfunc{dcgettext} @item @code{dcgettext(@var{string}} [@code{,} @var{domain} [@code{,} @var{category}]]@code{)} Return the translation of @var{string} in @@ -28306,15 +28409,7 @@ If @var{directory} is the null string (@code{""}), then given @var{domain}. @end table -To use these facilities in your @command{awk} program, follow the steps -outlined in -@ifnotinfo -the previous @value{SECTION}, -@end ifnotinfo -@ifinfo -@ref{Explaining gettext}, -@end ifinfo -like so: +To use these facilities in your @command{awk} program, follow these steps: @enumerate @cindex @code{BEGIN} pattern, @code{TEXTDOMAIN} variable and @@ -28399,7 +28494,7 @@ BEGIN @{ @end enumerate -@xref{I18N Example}, +@DBXREF{I18N Example} for an example program showing the steps to create and use translations from @command{awk}. @@ -28460,11 +28555,9 @@ second argument to @code{dcngettext()}.@footnote{The You should distribute the generated @file{.pot} file with your @command{awk} program; translators will eventually use it to provide you translations that you can also then distribute. -@xref{I18N Example}, +@DBXREF{I18N Example} for the full list of steps to go through to create and test translations for @command{guide}. -@c ENDOFRANGE portobfi -@c ENDOFRANGE portmsgfi @node Printf Ordering @subsection Rearranging @code{printf} Arguments @@ -28588,7 +28681,7 @@ change: @cindex @code{TEXTDOMAIN} variable, portability and @item Assignments to @code{TEXTDOMAIN} won't have any effect, -since @code{TEXTDOMAIN} is not special in other @command{awk} implementations. +because @code{TEXTDOMAIN} is not special in other @command{awk} implementations. @item Non-GNU versions of @command{awk} treat marked strings @@ -28599,7 +28692,7 @@ the null string (@code{""}) as its value, leaving the original string constant a the result. @item -By defining ``dummy'' functions to replace @code{dcgettext()}, @code{dcngettext()} +By defining ``dummy'' functions to replace @code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()}, the @command{awk} program can be made to run, but all the messages are output in the original language. For example: @@ -28636,11 +28729,10 @@ enough arguments are supplied in the function call. Many versions of underlying C library version of @code{sprintf()}, but only one format and argument at a time. What happens if a positional specification is used is anybody's guess. -However, since the positional specifications are primarily for use in -@emph{translated} format strings, and since non-GNU @command{awk}s never +However, because the positional specifications are primarily for use in +@emph{translated} format strings, and because non-GNU @command{awk}s never retrieve the translated string, this should not be a problem in practice. @end itemize -@c ENDOFRANGE inap @node I18N Example @section A Simple Internationalization Example @@ -28700,7 +28792,7 @@ called ``Hippy.'' Ah, well.} @example @group -$ cp guide.pot guide-mellow.po +$ @kbd{cp guide.pot guide-mellow.po} @var{Add translations to} guide-mellow.po @dots{} @end group @end example @@ -28726,7 +28818,7 @@ msgstr "Like, the scoop is" The next step is to make the directory to hold the binary message object file and then to create the @file{guide.mo} file. We pretend that our file is to be used in the @code{en_US.UTF-8} locale, -since we have to use a locale name known to the C @command{gettext} routines. +because we have to use a locale name known to the C @command{gettext} routines. The directory layout shown here is standard for GNU @command{gettext} on GNU/Linux systems. Other versions of @command{gettext} may use a different layout: @@ -28763,7 +28855,7 @@ $ @kbd{gawk -f guide.awk} @print{} Pardon me, Zaphod who? @end example -If the three replacement functions for @code{dcgettext()}, @code{dcngettext()} +If the three replacement functions for @code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()} (@pxref{I18N Portability}) are in a file named @file{libintl.awk}, @@ -28784,15 +28876,15 @@ using the GNU @command{gettext} package. (GNU @command{gettext} is described in complete detail in @ifinfo -@inforef{Top, , GNU @command{gettext} utilities, gettext, GNU gettext tools}.) +@inforef{Top, , GNU @command{gettext} utilities, gettext, GNU @command{gettext} utilities}.) @end ifinfo @ifnotinfo @uref{http://www.gnu.org/software/gettext/manual/, -@cite{GNU gettext tools}}.) +@cite{GNU @command{gettext} utilities}}.) @end ifnotinfo As of this writing, the latest version of GNU @command{gettext} is -@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.2.tar.gz, -@value{PVERSION} 0.19.2}. +@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.4.tar.gz, +@value{PVERSION} 0.19.4}. If a translation of @command{gawk}'s messages exists, then @command{gawk} produces usage messages, warnings, @@ -28804,7 +28896,7 @@ and fatal errors in the local language. @itemize @value{BULLET} @item Internationalization means writing a program such that it can use multiple -languages without requiring source-code changes. Localization means +languages without requiring source code changes. Localization means providing the data necessary for an internationalized program to work in a particular language. @@ -28821,9 +28913,9 @@ file, and the @file{.po} files are compiled into @file{.gmo} files for use at runtime. @item -You can use position specifications with @code{sprintf()} and +You can use positional specifications with @code{sprintf()} and @code{printf} to rearrange the placement of argument values in formatted -strings and output. This is useful for the translations of format +strings and output. This is useful for the translation of format control strings. @item @@ -28836,7 +28928,6 @@ a number of translations for its messages. @end itemize -@c ENDOFRANGE inloc @node Debugger @chapter Debugging @command{awk} Programs @@ -28865,7 +28956,7 @@ how to use @command{gawk} for debugging your program is easy. @end menu @node Debugging -@section Introduction to The @command{gawk} Debugger +@section Introduction to the @command{gawk} Debugger This @value{SECTION} introduces debugging in general and begins the discussion of debugging in @command{gawk}. @@ -28880,11 +28971,10 @@ the discussion of debugging in @command{gawk}. @subsection Debugging in General (If you have used debuggers in other languages, you may want to skip -ahead to the next section on the specific features of the @command{gawk} -debugger.) +ahead to @ref{Awk Debugging}.) -Of course, a debugging program cannot remove bugs for you, since it has -no way of knowing what you or your users consider a ``bug'' and what is a +Of course, a debugging program cannot remove bugs for you, because it has +no way of knowing what you or your users consider a ``bug'' versus a ``feature.'' (Sometimes, we humans have a hard time with this ourselves.) In that case, what can you expect from such a tool? The answer to that depends on the language being debugged, but in general, you can expect at @@ -28905,7 +28995,7 @@ having to change your source files. @item The chance to see the values of data in the program at any point in execution, and also to change that data on the fly, to see how that -affects what happens afterwards. (This often includes the ability +affects what happens afterward. (This often includes the ability to look at internal data structures besides the variables you actually defined in your code.) @@ -28925,11 +29015,11 @@ functional program that you or someone else wrote). Before diving in to the details, we need to introduce several important concepts that apply to just about all debuggers. The following list defines terms used throughout the rest of -this @value{CHAPTER}. +this @value{CHAPTER}: @table @dfn @cindex stack frame -@item Stack Frame +@item Stack frame Programs generally call functions during the course of their execution. One function can call another, or a function can call itself (recursion). You can view the chain of called functions (main program calls A, which @@ -28964,7 +29054,7 @@ as many breakpoints as you like. A watchpoint is similar to a breakpoint. The difference is that breakpoints are oriented around the code: stop when a certain point in the code is reached. A watchpoint, however, specifies that program execution -should stop when a @emph{data value} is changed. This is useful, since +should stop when a @emph{data value} is changed. This is useful, as sometimes it happens that a variable receives an erroneous value, and it's hard to track down where this happens just by looking at the code. By using a watchpoint, you can stop whenever a variable is assigned to, @@ -28972,26 +29062,26 @@ and usually find the errant code quite quickly. @end table @node Awk Debugging -@subsection Awk Debugging +@subsection @command{awk} Debugging Debugging an @command{awk} program has some specific aspects that are -not shared with other programming languages. +not shared with programs written in other languages. First of all, the fact that @command{awk} programs usually take input -line-by-line from a file or files and operate on those lines using specific +line by line from a file or files and operate on those lines using specific rules makes it especially useful to organize viewing the execution of the program in terms of these rules. As we will see, each @command{awk} rule is treated almost like a function call, with its own specific block of instructions. -In addition, since @command{awk} is by design a very concise language, +In addition, because @command{awk} is by design a very concise language, it is easy to lose sight of everything that is going on ``inside'' each line of @command{awk} code. The debugger provides the opportunity to look at the individual primitive instructions carried out by the higher-level @command{awk} commands. @node Sample Debugging Session -@section Sample Debugging Session +@section Sample @command{gawk} Debugging Session @cindex sample debugging session In order to illustrate the use of @command{gawk} as a debugger, let's look at a sample @@ -29010,8 +29100,8 @@ as our example. @cindex debugger, how to start Starting the debugger is almost exactly like running @command{gawk} normally, -except you have to pass an additional option @option{--debug}, or the -corresponding short option @option{-D}. The file(s) containing the +except you have to pass an additional option, @option{--debug}, or the +corresponding short option, @option{-D}. The file(s) containing the program and any supporting code are given on the command line as arguments to one or more @option{-f} options. (@command{gawk} is not designed to debug command-line programs, only programs contained in files.) @@ -29024,7 +29114,7 @@ $ @kbd{gawk -D -f getopt.awk -f join.awk -f uniq.awk -1 inputfile} @noindent where both @file{getopt.awk} and @file{uniq.awk} are in @env{$AWKPATH}. (Experienced users of GDB or similar debuggers should note that -this syntax is slightly different from what they are used to. +this syntax is slightly different from what you are used to. With the @command{gawk} debugger, you give the arguments for running the program in the command line to the debugger rather than as part of the @code{run} command at the debugger prompt.) @@ -29111,7 +29201,7 @@ gawk> @kbd{bt} @end example This tells us that @code{are_equal()} was called by the main program at -line 88 of @file{uniq.awk}. (This is not a big surprise, since this +line 88 of @file{uniq.awk}. (This is not a big surprise, because this is the only call to @code{are_equal()} in the program, but in more complex programs, knowing who called a function and with what parameters can be the key to finding the source of the problem.) @@ -29128,7 +29218,7 @@ gawk> @kbd{p n} @end example @noindent -In this case, @code{n} is an uninitialized local variable, since the +In this case, @code{n} is an uninitialized local variable, because the function was called without arguments (@pxref{Function Calls}). A more useful variable to display might be the current record: @@ -29139,8 +29229,8 @@ gawk> @kbd{p $0} @end example @noindent -This might be a bit puzzling at first since this is the second line of -our test input above. Let's look at @code{NR}: +This might be a bit puzzling at first, as this is the second line of +our test input. Let's look at @code{NR}: @example gawk> @kbd{p NR} @@ -29178,10 +29268,10 @@ gawk> @kbd{n} @end example This tells us that @command{gawk} is now ready to execute line 66, which -decides whether to give the lines the special ``field skipping'' treatment +decides whether to give the lines the special ``field-skipping'' treatment indicated by the @option{-1} command-line option. (Notice that we skipped -from where we were before at line 63 to here, since the condition in line 63 -@samp{if (fcount == 0 && charcount == 0)} was false.) +from where we were before, at line 63, to here, because the condition +in line 63, @samp{if (fcount == 0 && charcount == 0)}, was false.) Continuing to step, we now get to the splitting of the current and last records: @@ -29210,7 +29300,7 @@ gawk> @kbd{p n m alast aline} This is kind of disappointing, though. All we found out is that there are five elements in @code{alast}; @code{m} and @code{aline} don't have -values since we are at line 68 but haven't executed it yet. +values because we are at line 68 but haven't executed it yet. This information is useful enough (we now know that none of the words were accidentally left out), but what if we want to see inside the array? @@ -29255,7 +29345,7 @@ gawk> @kbd{n} Well, here we are at our error (sorry to spoil the suspense). What we had in mind was to join the fields starting from the second one to make -the virtual record to compare, and if the first field was numbered zero, +the virtual record to compare, and if the first field were numbered zero, this would work. Let's look at what we've got: @example @@ -29264,7 +29354,7 @@ gawk> @kbd{p cline clast} @print{} clast = "awk is a wonderful program!" @end example -Hey, those look pretty familiar! They're just our original, unaltered, +Hey, those look pretty familiar! They're just our original, unaltered input records. A little thinking (the human brain is still the best debugging tool), and we realize that we were off by one! @@ -29314,13 +29404,14 @@ Miscellaneous @end itemize Each of these are discussed in the following subsections. -In the following descriptions, commands which may be abbreviated +In the following descriptions, commands that may be abbreviated show the abbreviation on a second description line. A debugger command name may also be truncated if that partial name is unambiguous. The debugger has the built-in capability to -automatically repeat the previous command just by hitting @key{Enter}. -This works for the commands @code{list}, @code{next}, @code{nexti}, @code{step}, @code{stepi} -and @code{continue} executed without any argument. +automatically repeat the previous command just by hitting @kbd{Enter}. +This works for the commands @code{list}, @code{next}, @code{nexti}, +@code{step}, @code{stepi}, and @code{continue} executed without any +argument. @menu * Breakpoint Control:: Control of Breakpoints. @@ -29335,9 +29426,9 @@ and @code{continue} executed without any argument. @node Breakpoint Control @subsection Control of Breakpoints -As we saw above, the first thing you probably want to do in a debugging -session is to get your breakpoints set up, since otherwise your program -will just run as if it was not under the debugger. The commands for +As we saw earlier, the first thing you probably want to do in a debugging +session is to get your breakpoints set up, because your program +will otherwise just run as if it was not under the debugger. The commands for controlling breakpoints are: @table @asis @@ -29367,7 +29458,7 @@ Set a breakpoint at entry to (the first instruction of) function @var{function}. @end table -Each breakpoint is assigned a number which can be used to delete it from +Each breakpoint is assigned a number that can be used to delete it from the breakpoint list using the @code{delete} command. With a breakpoint, you may also supply a condition. This is an @@ -29408,8 +29499,8 @@ that the debugger evaluates whenever the breakpoint or watchpoint is reached. If the condition is true, then the debugger stops execution and prompts for a command. Otherwise, the debugger continues executing the program. If the condition expression is -not specified, any existing condition is removed; i.e., the breakpoint or -watchpoint is made unconditional. +not specified, any existing condition is removed (i.e., the breakpoint or +watchpoint is made unconditional). @cindex debugger commands, @code{d} (@code{delete}) @cindex debugger commands, @code{delete} @@ -29419,7 +29510,7 @@ watchpoint is made unconditional. @cindex breakpoint, delete by number @item @code{delete} [@var{n1 n2} @dots{}] [@var{n}--@var{m}] @itemx @code{d} [@var{n1 n2} @dots{}] [@var{n}--@var{m}] -Delete specified breakpoints or a range of breakpoints. Deletes +Delete specified breakpoints or a range of breakpoints. Delete all defined breakpoints if no argument is supplied. @cindex debugger commands, @code{disable} @@ -29428,7 +29519,7 @@ all defined breakpoints if no argument is supplied. @cindex breakpoint, how to disable or enable @item @code{disable} [@var{n1 n2} @dots{} | @var{n}--@var{m}] Disable specified breakpoints or a range of breakpoints. Without -any argument, disables all breakpoints. +any argument, disable all breakpoints. @cindex debugger commands, @code{e} (@code{enable}) @cindex debugger commands, @code{enable} @@ -29438,18 +29529,18 @@ any argument, disables all breakpoints. @item @code{enable} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}] @itemx @code{e} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}] Enable specified breakpoints or a range of breakpoints. Without -any argument, enables all breakpoints. -Optionally, you can specify how to enable the breakpoint: +any argument, enable all breakpoints. +Optionally, you can specify how to enable the breakpoints: @c nested table @table @code @item del -Enable the breakpoint(s) temporarily, then delete it when -the program stops at the breakpoint. +Enable the breakpoints temporarily, then delete each one when +the program stops at it. @item once -Enable the breakpoint(s) temporarily, then disable it when -the program stops at the breakpoint. +Enable the breakpoints temporarily, then disable each one when +the program stops at it. @end table @cindex debugger commands, @code{ignore} @@ -29517,7 +29608,7 @@ gawk> @item @code{continue} [@var{count}] @itemx @code{c} [@var{count}] Resume program execution. If continued from a breakpoint and @var{count} is -specified, ignores the breakpoint at that location the next @var{count} times +specified, ignore the breakpoint at that location the next @var{count} times before stopping. @cindex debugger commands, @code{finish} @@ -29550,7 +29641,7 @@ Execute one (or @var{count}) instruction(s), stepping over function calls. @item @code{return} [@var{value}] Cancel execution of a function call. If @var{value} (either a string or a number) is specified, it is used as the function's return value. If used in a -frame other than the innermost one (the currently executing function, i.e., +frame other than the innermost one (the currently executing function; i.e., frame number 0), discard all inner frames in addition to the selected one, and the caller of that frame becomes the innermost frame. @@ -29571,7 +29662,7 @@ automatic display variables, and debugger options. @item @code{step} [@var{count}] @itemx @code{s} [@var{count}] Continue execution until control reaches a different source line in the -current stack frame. @code{step} steps inside any function called within +current stack frame, stepping inside any function called within the line. If the argument @var{count} is supplied, steps that many times before stopping, unless it encounters a breakpoint or watchpoint. @@ -29616,7 +29707,7 @@ gawk> @kbd{display x} @end example @noindent -displays the assigned item number, the variable name and its current value. +This displays the assigned item number, the variable name, and its current value. If the display variable refers to a function parameter, it is silently deleted from the list as soon as the execution reaches a context where no such variable of the given name exists. @@ -29684,7 +29775,7 @@ or field. String values must be enclosed between double quotes (@code{"}@dots{}@code{"}). You can also set special @command{awk} variables, such as @code{FS}, -@code{NF}, @code{NR}, etc. +@code{NF}, @code{NR}, and so on. @cindex debugger commands, @code{w} (@code{watch}) @cindex debugger commands, @code{watch} @@ -29696,7 +29787,7 @@ You can also set special @command{awk} variables, such as @code{FS}, Add variable @var{var} (or field @code{$@var{n}}) to the watch list. The debugger then stops whenever the value of the variable or field changes. Each watched item is assigned a -number which can be used to delete it from the watch list using the +number that can be used to delete it from the watch list using the @code{unwatch} command. With a watchpoint, you may also supply a condition. This is an @@ -29724,11 +29815,11 @@ watch list. @node Execution Stack @subsection Working with the Stack -Whenever you run a program which contains any function calls, +Whenever you run a program that contains any function calls, @command{gawk} maintains a stack of all of the function calls leading up to where the program is right now. You can see how you got to where you are, and also move around in the stack to see what the state of things was in the -functions which called the one you are in. The commands for doing this are: +functions that called the one you are in. The commands for doing this are: @table @asis @cindex debugger commands, @code{bt} (@code{backtrace}) @@ -29747,7 +29838,7 @@ Print a backtrace of all function calls (stack frames), or innermost @var{count} frames if @var{count} > 0. Print the outermost @var{count} frames if @var{count} < 0. The backtrace displays the name and arguments to each function, the source @value{FN}, and the line number. -The alias @code{where} for @code{backtrace} is provided for long-time +The alias @code{where} for @code{backtrace} is provided for longtime GDB users who may be used to that command. @cindex debugger commands, @code{down} @@ -29763,8 +29854,8 @@ Then select and print the frame. @item @code{frame} [@var{n}] @itemx @code{f} [@var{n}] Select and print stack frame @var{n}. Frame 0 is the currently executing, -or @dfn{innermost}, frame (function call), frame 1 is the frame that -called the innermost one. The highest numbered frame is the one for the +or @dfn{innermost}, frame (function call); frame 1 is the frame that +called the innermost one. The highest-numbered frame is the one for the main program. The printed information consists of the frame number, function and argument names, source file, and the source line. @@ -29776,11 +29867,11 @@ Then select and print the frame. @end table @node Debugger Info -@subsection Obtaining Information about the Program and the Debugger State +@subsection Obtaining Information About the Program and the Debugger State Besides looking at the values of variables, there is often a need to get other sorts of information about the state of your program and of the -debugging environment itself. The @command{gawk} debugger has one command which +debugging environment itself. The @command{gawk} debugger has one command that provides this information, appropriately called @code{info}. @code{info} is used with one of a number of arguments that tell it exactly what you want to know: @@ -29868,12 +29959,12 @@ The available options are: @table @asis @item @code{history_size} @cindex debugger history size -The maximum number of lines to keep in the history file @file{./.gawk_history}. -The default is 100. +Set the maximum number of lines to keep in the history file +@file{./.gawk_history}. The default is 100. @item @code{listsize} @cindex debugger default list amount -The number of lines that @code{list} prints. The default is 15. +Specify the number of lines that @code{list} prints. The default is 15. @item @code{outfile} @cindex redirect @command{gawk} output, in debugger @@ -29883,7 +29974,7 @@ standard output. @item @code{prompt} @cindex debugger prompt -The debugger prompt. The default is @samp{@w{gawk> }}. +Change the debugger prompt. The default is @samp{@w{gawk> }}. @item @code{save_history} [@code{on} | @code{off}] @cindex debugger history file @@ -29894,7 +29985,7 @@ The default is @code{on}. @cindex save debugger options Save current options to file @file{./.gawkrc} upon exit. The default is @code{on}. -Options are read back in to the next session upon startup. +Options are read back into the next session upon startup. @item @code{trace} [@code{on} | @code{off}] @cindex instruction tracing, in debugger @@ -29917,7 +30008,7 @@ command in the file. Also, the list of commands may include additional @code{source} commands; however, the @command{gawk} debugger will not source the same file more than once in order to avoid infinite recursion. -In addition to, or instead of the @code{source} command, you can use +In addition to, or instead of, the @code{source} command, you can use the @option{-D @var{file}} or @option{--debug=@var{file}} command-line options to execute commands from a file non-interactively (@pxref{Options}). @@ -29926,16 +30017,16 @@ options to execute commands from a file non-interactively @node Miscellaneous Debugger Commands @subsection Miscellaneous Commands -There are a few more commands which do not fit into the +There are a few more commands that do not fit into the previous categories, as follows: @table @asis @cindex debugger commands, @code{dump} @cindex @code{dump} debugger command @item @code{dump} [@var{filename}] -Dump bytecode of the program to standard output or to the file +Dump byte code of the program to standard output or to the file named in @var{filename}. This prints a representation of the internal -instructions which @command{gawk} executes to implement the @command{awk} +instructions that @command{gawk} executes to implement the @command{awk} commands in a program. This can be very enlightening, as the following partial dump of Davide Brini's obfuscated code (@pxref{Signature Program}) demonstrates: @@ -29944,51 +30035,51 @@ partial dump of Davide Brini's obfuscated code @smallexample gawk> @kbd{dump} @print{} # BEGIN -@print{} -@print{} [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] -@print{} [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc280] Op_match : -@print{} [ 1:0xfcc1e0] Op_store_var : O -@print{} [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc320] Op_equal : -@print{} [ 1:0xfcc200] Op_store_var : o -@print{} [ 1:0xfcc380] Op_push : o -@print{} [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] -@print{} [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] -@print{} [ 1:0xfcc300] Op_assign_plus : -@print{} [ :0xfcc2c0] Op_pop : -@print{} [ 1:0xfcc400] Op_push : O -@print{} [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] -@print{} [ :0xfcc4a0] Op_no_op : -@print{} [ 1:0xfcc480] Op_push : O -@print{} [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] -@print{} [ 1:0xfcc3c0] Op_store_var : x -@print{} [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] -@print{} [ 1:0xfcc3a0] Op_postincrement : -@print{} [ 1:0xfcc4e0] Op_push : x -@print{} [ 1:0xfcc540] Op_push : o -@print{} [ 1:0xfcc500] Op_plus : -@print{} [ 1:0xfcc580] Op_push : o -@print{} [ 1:0xfcc560] Op_plus : -@print{} [ 1:0xfcc460] Op_leq : -@print{} [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] -@print{} [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] -@print{} [ :0xfcc660] Op_no_op : -@print{} [ 1:0xfcc520] Op_assign_concat : c -@print{} [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] -@print{} -@dots{} -@print{} +@print{} +@print{} [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] +@print{} [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc280] Op_match : +@print{} [ 1:0xfcc1e0] Op_store_var : O +@print{} [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc320] Op_equal : +@print{} [ 1:0xfcc200] Op_store_var : o +@print{} [ 1:0xfcc380] Op_push : o +@print{} [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] +@print{} [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] +@print{} [ 1:0xfcc300] Op_assign_plus : +@print{} [ :0xfcc2c0] Op_pop : +@print{} [ 1:0xfcc400] Op_push : O +@print{} [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] +@print{} [ :0xfcc4a0] Op_no_op : +@print{} [ 1:0xfcc480] Op_push : O +@print{} [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] +@print{} [ 1:0xfcc3c0] Op_store_var : x +@print{} [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] +@print{} [ 1:0xfcc3a0] Op_postincrement : +@print{} [ 1:0xfcc4e0] Op_push : x +@print{} [ 1:0xfcc540] Op_push : o +@print{} [ 1:0xfcc500] Op_plus : +@print{} [ 1:0xfcc580] Op_push : o +@print{} [ 1:0xfcc560] Op_plus : +@print{} [ 1:0xfcc460] Op_leq : +@print{} [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] +@print{} [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] +@print{} [ :0xfcc660] Op_no_op : +@print{} [ 1:0xfcc520] Op_assign_concat : c +@print{} [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] +@print{} +@dots{} +@print{} @print{} [ 2:0xfcc5a0] Op_K_printf : [expr_count = 17] [redir_type = ""] -@print{} [ :0xfcc140] Op_no_op : -@print{} [ :0xfcc1c0] Op_atexit : -@print{} [ :0xfcc640] Op_stop : -@print{} [ :0xfcc180] Op_no_op : -@print{} [ :0xfcd150] Op_after_beginfile : -@print{} [ :0xfcc160] Op_no_op : -@print{} [ :0xfcc1a0] Op_after_endfile : +@print{} [ :0xfcc140] Op_no_op : +@print{} [ :0xfcc1c0] Op_atexit : +@print{} [ :0xfcc640] Op_stop : +@print{} [ :0xfcc180] Op_no_op : +@print{} [ :0xfcd150] Op_after_beginfile : +@print{} [ :0xfcc160] Op_no_op : +@print{} [ :0xfcc1a0] Op_after_endfile : gawk> @end smallexample @@ -30032,7 +30123,7 @@ Print lines centered around line number @var{n} in source file @var{filename}. This command may change the current source file. @item @var{function} -Print lines centered around beginning of the +Print lines centered around the beginning of the function @var{function}. This command may change the current source file. @end table @@ -30044,16 +30135,16 @@ function @var{function}. This command may change the current source file. @item @code{quit} @itemx @code{q} Exit the debugger. Debugging is great fun, but sometimes we all have -to tend to other obligations in life, and sometimes we find the bug, -and are free to go on to the next one! As we saw above, if you are -running a program, the debugger warns you if you accidentally type +to tend to other obligations in life, and sometimes we find the bug +and are free to go on to the next one! As we saw earlier, if you are +running a program, the debugger warns you when you type @samp{q} or @samp{quit}, to make sure you really want to quit. @cindex debugger commands, @code{trace} @cindex @code{trace} debugger command @item @code{trace} [@code{on} | @code{off}] -Turn on or off a continuous printing of instructions which are about to -be executed, along with printing the @command{awk} line which they +Turn on or off continuous printing of the instructions that are about to +be executed, along with the @command{awk} lines they implement. The default is @code{off}. It is to be hoped that most of the ``opcodes'' in these instructions are @@ -30069,7 +30160,7 @@ fairly self-explanatory, and using @code{stepi} and @code{nexti} while If @command{gawk} is compiled with @uref{http://cnswww.cns.cwru.edu/php/chet/readline/readline.html, -the @code{readline} library}, you can take advantage of that library's +the GNU Readline library}, you can take advantage of that library's command completion and history expansion features. The following types of completion are available: @@ -30106,7 +30197,7 @@ and We hope you find the @command{gawk} debugger useful and enjoyable to work with, but as with any program, especially in its early releases, it still has -some limitations. A few which are worth being aware of are: +some limitations. A few that it's worth being aware of are: @itemize @value{BULLET} @item @@ -30122,13 +30213,13 @@ If you perused the dump of opcodes in @ref{Miscellaneous Debugger Commands} (or if you are already familiar with @command{gawk} internals), you will realize that much of the internal manipulation of data in @command{gawk}, as in many interpreters, is done on a stack. -@code{Op_push}, @code{Op_pop}, etc., are the ``bread and butter'' of +@code{Op_push}, @code{Op_pop}, and the like are the ``bread and butter'' of most @command{gawk} code. Unfortunately, as of now, the @command{gawk} debugger does not allow you to examine the stack's contents. That is, the intermediate results of expression evaluation are on the -stack, but cannot be printed. Rather, only variables which are defined +stack, but cannot be printed. Rather, only variables that are defined in the program can be printed. Of course, a workaround for this is to use more explicit variables at the debugging stage and then change back to obscure, perhaps more optimal code later. @@ -30136,18 +30227,18 @@ change back to obscure, perhaps more optimal code later. @item There is no way to look ``inside'' the process of compiling regular expressions to see if you got it right. As an @command{awk} -programmer, you are expected to know what @code{/[^[:alnum:][:blank:]]/} -means. +programmer, you are expected to know the meaning of +@code{/[^[:alnum:][:blank:]]/}. @item The @command{gawk} debugger is designed to be used by running a program (with all its parameters) on the command line, as described in @ref{Debugger Invocation}. -There is no way (as of now) to attach or ``break in'' to a running program. -This seems reasonable for a language which is used mainly for quickly +There is no way (as of now) to attach or ``break into'' a running program. +This seems reasonable for a language that is used mainly for quickly executing, short programs. @item -The @command{gawk} debugger only accepts source supplied with the @option{-f} option. +The @command{gawk} debugger only accepts source code supplied with the @option{-f} option. @end itemize @ignore @@ -30161,8 +30252,8 @@ be added, and of course feel free to try to add them yourself! @itemize @value{BULLET} @item Programs rarely work correctly the first time. Finding bugs -is @dfn{debugging} and a program that helps you find bugs is a -@dfn{debugger}. @command{gawk} has a built-in debugger that works very +is called debugging, and a program that helps you find bugs is a +debugger. @command{gawk} has a built-in debugger that works very similarly to the GNU Debugger, GDB. @item @@ -30182,14 +30273,14 @@ breakpoints, execution, viewing and changing data, working with the stack, getting information, and other tasks. @item -If the @code{readline} library is available when @command{gawk} is +If the GNU Readline library is available when @command{gawk} is compiled, it is used by the debugger to provide command-line history and editing. @end itemize @node Arbitrary Precision Arithmetic -@chapter Arithmetic and Arbitrary Precision Arithmetic with @command{gawk} +@chapter Arithmetic and Arbitrary-Precision Arithmetic with @command{gawk} @cindex arbitrary precision @cindex multiple precision @cindex infinite precision @@ -30199,9 +30290,9 @@ This @value{CHAPTER} introduces some basic concepts relating to how computers do arithmetic and defines some important terms. It then proceeds to describe floating-point arithmetic, which is what @command{awk} uses for all its computations, including a -discussion of arbitrary precision floating point arithmetic, which is +discussion of arbitrary-precision floating-point arithmetic, which is a feature available only in @command{gawk}. It continues on to present -arbitrary precision integers, and concludes with a description of some +arbitrary-precision integers, and concludes with a description of some points where @command{gawk} and the POSIX standard are not quite in agreement. @@ -30246,7 +30337,7 @@ paper and pencil (and/or a calculator). In theory, numbers can have an arbitrary number of digits on either side (or both sides) of the decimal point, and the results of a computation are always exact. -Some modern system can do decimal arithmetic in hardware, but usually you +Some modern systems can do decimal arithmetic in hardware, but usually you need a special software library to provide access to these instructions. There are also libraries that do decimal arithmetic entirely in software. @@ -30264,55 +30355,82 @@ The disadvantage is that their range is limited. @cindex integers, unsigned In computers, integer values come in two flavors: @dfn{signed} and @dfn{unsigned}. Signed values may be negative or positive, whereas -unsigned values are always positive (that is, greater than or equal -to zero). +unsigned values are always greater than or equal +to zero. In computer systems, integer arithmetic is exact, but the possible range of values is limited. Integer arithmetic is generally faster than -floating point arithmetic. +floating-point arithmetic. -@item Floating point arithmetic +@item Floating-point arithmetic Floating-point numbers represent what were called in school ``real'' -numbers; i.e., those that have a fractional part, such as 3.1415927. +numbers (i.e., those that have a fractional part, such as 3.1415927). The advantage to floating-point numbers is that they can represent a much larger range of values than can integers. The disadvantage is that there are numbers that they cannot represent exactly. -Modern systems support floating point arithmetic in hardware, with a +Modern systems support floating-point arithmetic in hardware, with a limited range of values. There are software libraries that allow -the use of arbitrary precision floating point calculations. +the use of arbitrary-precision floating-point calculations. -POSIX @command{awk} uses @dfn{double precision} floating-point numbers, which -can hold more digits than @dfn{single precision} floating-point numbers. -@command{gawk} has facilities for performing arbitrary precision floating -point arithmetic, which we describe in more detail shortly. +POSIX @command{awk} uses @dfn{double-precision} floating-point numbers, which +can hold more digits than @dfn{single-precision} floating-point numbers. +@command{gawk} has facilities for performing arbitrary-precision +floating-point arithmetic, which we describe in more detail shortly. @end table -Computers work with integer and floating point values of different -ranges. Integer values are usually either 32 or 64 bits in size. Single -precision floating point values occupy 32 bits, whereas double precision -floating point values occupy 64 bits. Floating point values are always +Computers work with integer and floating-point values of different +ranges. Integer values are usually either 32 or 64 bits in size. +Single-precision floating-point values occupy 32 bits, whereas double-precision +floating-point values occupy 64 bits. Floating-point values are always signed. The possible ranges of values are shown in @ref{table-numeric-ranges}. @float Table,table-numeric-ranges -@caption{Value Ranges for Different Numeric Representations} +@caption{Value ranges for different numeric representations} @multitable @columnfractions .34 .33 .33 @headitem Numeric representation @tab Minimum value @tab Maximum value @item 32-bit signed integer @tab @minus{}2,147,483,648 @tab 2,147,483,647 @item 32-bit unsigned integer @tab 0 @tab 4,294,967,295 @item 64-bit signed integer @tab @minus{}9,223,372,036,854,775,808 @tab 9,223,372,036,854,775,807 @item 64-bit unsigned integer @tab 0 @tab 18,446,744,073,709,551,615 -@item Single precision floating point (approximate) @tab @code{1.175494e-38} @tab @code{3.402823e+38} -@item Double precision floating point (approximate) @tab @code{2.225074e-308} @tab @code{1.797693e+308} +@iftex +@item Single-precision floating point (approximate) @tab @math{1.175494^{-38}} @tab @math{3.402823^{38}} +@item Double-precision floating point (approximate) @tab @math{2.225074^{-308}} @tab @math{1.797693^{308}} +@end iftex +@ifnottex +@ifnotdocbook +@item Single-precision floating point (approximate) @tab 1.175494e-38 @tab 3.402823e38 +@item Double-precision floating point (approximate) @tab 2.225074e-308 @tab 1.797693e308 +@end ifnotdocbook +@end ifnottex +@ifdocbook +@item Single-precision floating point (approximate) @tab +@c FIXME: Use @sup here for superscript +@docbook +1.175494<superscript>-38</superscript> +@end docbook +@tab +@docbook +3.402823<superscript>38</superscript> +@end docbook +@item Double-precision floating point (approximate) @tab +@docbook +2.225074<superscript>-308</superscript> +@end docbook +@tab +@docbook +1.797693<superscript>308</superscript> +@end docbook +@end ifdocbook @end multitable @end float @node Math Definitions -@section Other Stuff To Know +@section Other Stuff to Know The rest of this @value{CHAPTER} uses a number of terms. Here are some informal definitions that should help you work your way through the material -here. +here: @table @dfn @item Accuracy @@ -30333,7 +30451,7 @@ A special value representing infinity. Operations involving another number and infinity produce infinity. @item NaN -``Not A Number.''@footnote{Thanks to Michael Brennan for this description, +``Not a number.''@footnote{Thanks to Michael Brennan for this description, which we have paraphrased, and for the examples.} A special value that results from attempting a calculation that has no answer as a real number. In such a case, programs can either receive a floating-point exception, @@ -30376,8 +30494,8 @@ formula: @end display @noindent -Here, @var{prec} denotes the binary precision -(measured in bits) and @var{dps} (short for decimal places) +Here, @emph{prec} denotes the binary precision +(measured in bits) and @emph{dps} (short for decimal places) is the decimal digits. @item Rounding mode @@ -30385,7 +30503,7 @@ How numbers are rounded up or down when necessary. More details are provided later. @item Significand -A floating point value consists the significand multiplied by 10 +A floating-point value consists of the significand multiplied by 10 to the power of the exponent. For example, in @code{1.2345e67}, the significand is @code{1.2345}. @@ -30403,19 +30521,19 @@ on some of those terms. On modern systems, floating-point hardware uses the representation and operations defined by the IEEE 754 standard. Three of the standard IEEE 754 types are 32-bit single precision, -64-bit double precision and 128-bit quadruple precision. +64-bit double precision, and 128-bit quadruple precision. The standard also specifies extended precision formats to allow greater precisions and larger exponent ranges. -(@command{awk} uses only the 64-bit double precision format.) +(@command{awk} uses only the 64-bit double-precision format.) @ref{table-ieee-formats} lists the precision and exponent -field values for the basic IEEE 754 binary formats: +field values for the basic IEEE 754 binary formats. @float Table,table-ieee-formats -@caption{Basic IEEE Format Values} +@caption{Basic IEEE format values} @multitable @columnfractions .20 .20 .20 .20 .20 @headitem Name @tab Total bits @tab Precision @tab Minimum exponent @tab Maximum exponent -@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127 +@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127 @item Double @tab 64 @tab 53 @tab @minus{}1022 @tab +1023 @item Quadruple @tab 128 @tab 113 @tab @minus{}16382 @tab +16383 @end multitable @@ -30427,19 +30545,19 @@ one extra bit of significand. @end quotation @node MPFR features -@section Arbitrary Precision Arithmetic Features In @command{gawk} +@section Arbitrary-Precision Arithmetic Features in @command{gawk} -By default, @command{gawk} uses the double precision floating-point values +By default, @command{gawk} uses the double-precision floating-point values supplied by the hardware of the system it runs on. However, if it was -compiled to do so, @command{gawk} uses the @uref{http://www.mpfr.org -GNU MPFR} and @uref{http://gmplib.org, GNU MP} (GMP) libraries for arbitrary -precision arithmetic on numbers. You can see if MPFR support is available -like so: +compiled to do so, @command{gawk} uses the @uref{http://www.mpfr.org, +GNU MPFR} and @uref{http://gmplib.org, GNU MP} (GMP) libraries for +arbitrary-precision arithmetic on numbers. You can see if MPFR support +is available like so: @example $ @kbd{gawk --version} @print{} GNU Awk 4.1.2, API: 1.1 (GNU MPFR 3.1.0-p3, GNU MP 5.0.2) -@print{} Copyright (C) 1989, 1991-2014 Free Software Foundation. +@print{} Copyright (C) 1989, 1991-2015 Free Software Foundation. @dots{} @end example @@ -30462,23 +30580,23 @@ Two predefined variables, @code{PREC} and @code{ROUNDMODE}, provide control over the working precision and the rounding mode. The precision and the rounding mode are set globally for every operation to follow. -@xref{Setting precision}, and @ref{Setting the rounding mode}, +@DBXREF{Setting precision} and @DBREF{Setting the rounding mode} for more information. @node FP Math Caution -@section Floating Point Arithmetic: Caveat Emptor! +@section Floating-Point Arithmetic: Caveat Emptor! @quotation @i{Math class is tough!} @author Teen Talk Barbie, July 1992 @end quotation -This @value{SECTION} provides a high level overview of the issues +This @value{SECTION} provides a high-level overview of the issues involved when doing lots of floating-point arithmetic.@footnote{There is a very nice @uref{http://www.validlab.com/goldberg/paper.pdf, paper on floating-point arithmetic} by David Goldberg, ``What Every -Computer Scientist Should Know About Floating-point Arithmetic,'' -@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03), 5-48. This is +Computer Scientist Should Know About Floating-Point Arithmetic,'' +@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03): 5-48. This is worth reading if you are interested in the details, but it does require a background in computer science.} The discussion applies to both hardware and arbitrary-precision @@ -30499,17 +30617,17 @@ rely just on what we tell you. @end menu @node Inexactness of computations -@subsection Floating Point Arithmetic Is Not Exact +@subsection Floating-Point Arithmetic Is Not Exact Binary floating-point representations and arithmetic are inexact. Simple values like 0.1 cannot be precisely represented using binary floating-point numbers, and the limited precision of floating-point numbers means that slight changes in the order of operations or the precision of intermediate storage -can change the result. To make matters worse, with arbitrary precision -floating-point, you can set the precision before starting a computation, -but then you cannot be sure of the number of significant decimal places -in the final result. +can change the result. To make matters worse, with arbitrary-precision +floating-point arithmetic, you can set the precision before starting a +computation, but then you cannot be sure of the number of significant +decimal places in the final result. @menu * Inexact representation:: Numbers are not exactly represented. @@ -30531,7 +30649,7 @@ y = 0.425 Unlike the number in @code{y}, the number stored in @code{x} is exactly representable -in binary since it can be written as a finite sum of one or +in binary because it can be written as a finite sum of one or more fractions whose denominators are all powers of two. When @command{gawk} reads a floating-point number from program source, it automatically rounds that number to whatever @@ -30547,7 +30665,7 @@ $ @kbd{gawk 'BEGIN @{ x = 0.875; y = 0.425} Often the error is so small you do not even notice it, and if you do, you can always specify how much precision you would like in your output. -Usually this is a format string like @code{"%.15g"}, which when +Usually this is a format string like @code{"%.15g"}, which, when used in the previous example, produces an output identical to the input. @node Comparing FP Values @@ -30557,7 +30675,7 @@ Because the underlying representation can be a little bit off from the exact val comparing floating-point values to see if they are exactly equal is generally a bad idea. Here is an example where it does not work like you would expect: -@example +@example $ @kbd{gawk 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 0 @end example @@ -30566,7 +30684,7 @@ The general wisdom when comparing floating-point values is to see if they are within some small range of each other (called a @dfn{delta}, or @dfn{tolerance}). You have to decide how small a delta is important to you. Code to do -this looks something like this: +this looks something like the following: @example delta = 0.00001 # for example @@ -30586,7 +30704,7 @@ else The loss of accuracy during a single computation with floating-point numbers usually isn't enough to worry about. However, if you compute a -value which is the result of a sequence of floating point operations, +value that is the result of a sequence of floating-point operations, the error can accumulate and greatly affect the computation itself. Here is an attempt to compute the value of @value{PI} using one of its many series representations: @@ -30632,23 +30750,23 @@ $ @kbd{gawk 'BEGIN @{} @end example @node Getting Accuracy -@subsection Getting The Accuracy You Need +@subsection Getting the Accuracy You Need -Can arbitrary precision arithmetic give exact results? There are +Can arbitrary-precision arithmetic give exact results? There are no easy answers. The standard rules of algebra often do not apply when using floating-point arithmetic. Among other things, the distributive and associative laws do not hold completely, and order of operation may be important -for your computation. Rounding error, cumulative precision loss +for your computation. Rounding error, cumulative precision loss, and underflow are often troublesome. When @command{gawk} tests the expressions @samp{0.1 + 12.2} and -@samp{12.3} for equality using the machine double precision arithmetic, +@samp{12.3} for equality using the machine double-precision arithmetic, it decides that they are not equal! (@xref{Comparing FP Values}.) You can get the result you want by increasing the precision; 56 bits in this case does the job: -@example +@example $ @kbd{gawk -M -v PREC=56 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 1 @end example @@ -30657,7 +30775,7 @@ If adding more bits is good, perhaps adding even more bits of precision is better? Here is what happens if we use an even larger value of @code{PREC}: -@example +@example $ @kbd{gawk -M -v PREC=201 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 0 @end example @@ -30666,20 +30784,21 @@ This is not a bug in @command{gawk} or in the MPFR library. It is easy to forget that the finite number of bits used to store the value is often just an approximation after proper rounding. The test for equality succeeds if and only if @emph{all} bits in the two operands -are exactly the same. Since this is not necessarily true after floating-point +are exactly the same. Because this is not necessarily true after floating-point computations with a particular precision and effective rounding mode, a straight test for equality may not work. Instead, compare the two numbers to see if they are within the desirable delta of each other. In applications where 15 or fewer decimal places suffice, -hardware double precision arithmetic can be adequate, and is usually much faster. +hardware double-precision arithmetic can be adequate, and is usually much faster. But you need to keep in mind that every floating-point operation -can suffer a new rounding error with catastrophic consequences as illustrated +can suffer a new rounding error with catastrophic consequences, as illustrated by our earlier attempt to compute the value of @value{PI}. Extra precision can greatly enhance the stability and the accuracy of your computation in such cases. -Repeated addition is not necessarily equivalent to multiplication +Additionally, you should understand that +repeated addition is not necessarily equivalent to multiplication in floating-point arithmetic. In the example in @ref{Errors accumulate}: @@ -30698,9 +30817,9 @@ an arbitrarily large value for @code{PREC}. Reformulation of the problem at hand is often the correct approach in such situations. @node Try To Round -@subsection Try A Few Extra Bits of Precision and Rounding +@subsection Try a Few Extra Bits of Precision and Rounding -Instead of arbitrary precision floating-point arithmetic, +Instead of arbitrary-precision floating-point arithmetic, often all you need is an adjustment of your logic or a different order for the operations in your calculation. The stability and the accuracy of the computation of @value{PI} @@ -30712,7 +30831,7 @@ simple algebraic transformation: @end example @noindent -After making this, change the program converges to +After making this change, the program converges to @value{PI} in under 30 iterations: @example @@ -30728,7 +30847,7 @@ $ @kbd{gawk -f pi2.awk} @end example @node Setting precision -@subsection Setting The Precision +@subsection Setting the Precision @command{gawk} uses a global working precision; it does not keep track of the precision or accuracy of individual numbers. Performing an arithmetic @@ -30740,14 +30859,14 @@ shown in @ref{table-predefined-precision-strings}, to emulate an IEEE 754 binary format. @float Table,table-predefined-precision-strings -@caption{Predefined Precision Strings For @code{PREC}} +@caption{Predefined precision strings for @code{PREC}} @multitable {@code{"double"}} {12345678901234567890123456789012345} -@headitem @code{PREC} @tab IEEE 754 Binary Format -@item @code{"half"} @tab 16-bit half-precision. -@item @code{"single"} @tab Basic 32-bit single precision. -@item @code{"double"} @tab Basic 64-bit double precision. -@item @code{"quad"} @tab Basic 128-bit quadruple precision. -@item @code{"oct"} @tab 256-bit octuple precision. +@headitem @code{PREC} @tab IEEE 754 binary format +@item @code{"half"} @tab 16-bit half-precision +@item @code{"single"} @tab Basic 32-bit single precision +@item @code{"double"} @tab Basic 64-bit double precision +@item @code{"quad"} @tab Basic 128-bit quadruple precision +@item @code{"oct"} @tab 256-bit octuple precision @end multitable @end float @@ -30774,11 +30893,10 @@ than the default and cannot use a command-line assignment to @code{PREC}, you should either specify the constant as a string, or as a rational number, whenever possible. The following example illustrates the differences among various ways to print a floating-point constant: -@end quotation @example $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 0.1) @}'} -@print{} 0.1000000000000000055511151 +@print{} 0.1000000000000000055511151 $ @kbd{gawk -M -v PREC=113 'BEGIN @{ printf("%0.25f\n", 0.1) @}'} @print{} 0.1000000000000000000000000 $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'} @@ -30786,22 +30904,23 @@ $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'} $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 1/10) @}'} @print{} 0.1000000000000000000000000 @end example +@end quotation @node Setting the rounding mode -@subsection Setting The Rounding Mode +@subsection Setting the Rounding Mode The @code{ROUNDMODE} variable provides -program level control over the rounding mode. +program-level control over the rounding mode. The correspondence between @code{ROUNDMODE} and the IEEE rounding modes is shown in @ref{table-gawk-rounding-modes}. @float Table,table-gawk-rounding-modes -@caption{@command{gawk} Rounding Modes} +@caption{@command{gawk} rounding modes} @multitable @columnfractions .45 .30 .25 -@headitem Rounding Mode @tab IEEE Name @tab @code{ROUNDMODE} +@headitem Rounding mode @tab IEEE name @tab @code{ROUNDMODE} @item Round to nearest, ties to even @tab @code{roundTiesToEven} @tab @code{"N"} or @code{"n"} -@item Round toward plus Infinity @tab @code{roundTowardPositive} @tab @code{"U"} or @code{"u"} -@item Round toward negative Infinity @tab @code{roundTowardNegative} @tab @code{"D"} or @code{"d"} +@item Round toward positive infinity @tab @code{roundTowardPositive} @tab @code{"U"} or @code{"u"} +@item Round toward negative infinity @tab @code{roundTowardNegative} @tab @code{"D"} or @code{"d"} @item Round toward zero @tab @code{roundTowardZero} @tab @code{"Z"} or @code{"z"} @item Round to nearest, ties away from zero @tab @code{roundTiesToAway} @tab @code{"A"} or @code{"a"} @end multitable @@ -30811,7 +30930,7 @@ rounding modes is shown in @ref{table-gawk-rounding-modes}. selects the IEEE 754 rounding mode @code{roundTiesToEven}. In @ref{table-gawk-rounding-modes}, the value @code{"A"} selects @code{roundTiesToAway}. This is only available if your version of the -MPFR library supports it; otherwise setting @code{ROUNDMODE} to @code{"A"} +MPFR library supports it; otherwise, setting @code{ROUNDMODE} to @code{"A"} has no effect. The default mode @code{roundTiesToEven} is the most preferred, @@ -30862,8 +30981,8 @@ distributes upward and downward rounds of exact halves, which might cause any accumulating round-off error to cancel itself out. This is the default rounding mode for IEEE 754 computing functions and operators. -The other rounding modes are rarely used. Round toward positive infinity -(@code{roundTowardPositive}) and round toward negative infinity +The other rounding modes are rarely used. Rounding toward positive infinity +(@code{roundTowardPositive}) and toward negative infinity (@code{roundTowardNegative}) are often used to implement interval arithmetic, where you adjust the rounding mode to calculate upper and lower bounds for the range of output. The @code{roundTowardZero} mode can @@ -30882,14 +31001,14 @@ accumulation of round-off error, look for a significant difference in output when you change the rounding mode to be sure. @node Arbitrary Precision Integers -@section Arbitrary Precision Integer Arithmetic with @command{gawk} +@section Arbitrary-Precision Integer Arithmetic with @command{gawk} @cindex integers, arbitrary precision @cindex arbitrary precision integers When given the @option{-M} option, -@command{gawk} performs all integer arithmetic using GMP arbitrary -precision integers. Any number that looks like an integer in a source -or @value{DF} is stored as an arbitrary precision integer. The size +@command{gawk} performs all integer arithmetic using GMP arbitrary-precision +integers. Any number that looks like an integer in a source +or @value{DF} is stored as an arbitrary-precision integer. The size of the integer is limited only by the available memory. For example, the following computes @iftex @@ -30904,7 +31023,8 @@ the following computes 5<superscript>4<superscript>3<superscript>2</superscript></superscript></superscript>, @c @end docbook the result of which is beyond the -limits of ordinary hardware double precision floating point values: +limits of ordinary hardware double-precision floating-point values: +@c FIXME: Use @sup here for superscript @example $ @kbd{gawk -M 'BEGIN @{} @@ -30916,21 +31036,21 @@ $ @kbd{gawk -M 'BEGIN @{} @print{} 62060698786608744707 ... 92256259918212890625 @end example -If instead you were to compute the same value using arbitrary precision +If instead you were to compute the same value using arbitrary-precision floating-point values, the precision needed for correct output (using the formula @iftex -@math{prec = 3.322 @cdot dps}), +@math{prec = 3.322 @cdot dps}) would be @math{3.322 @cdot 183231}, @end iftex @ifnottex @ifnotdocbook -@samp{prec = 3.322 * dps}), +@samp{prec = 3.322 * dps}) would be 3.322 x 183231, @end ifnotdocbook @end ifnottex @docbook -<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis>), +<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis>) would be <emphasis>prec</emphasis> = 3.322 ⋅ 183231, @c @end docbook @@ -30961,14 +31081,14 @@ floating-point results exactly. You can either increase the precision @samp{2.0} with an integer, to perform all computations using integer arithmetic to get the correct output. -Sometimes @command{gawk} must implicitly convert an arbitrary precision -integer into an arbitrary precision floating-point value. This is +Sometimes @command{gawk} must implicitly convert an arbitrary-precision +integer into an arbitrary-precision floating-point value. This is primarily because the MPFR library does not always provide the relevant -interface to process arbitrary precision integers or mixed-mode numbers +interface to process arbitrary-precision integers or mixed-mode numbers as needed by an operation or function. In such a case, the precision is set to the minimum value necessary for exact conversion, and the working precision is not used for this purpose. If this is not what you need or -want, you can employ a subterfuge, and convert the integer to floating +want, you can employ a subterfuge and convert the integer to floating point first, like this: @example @@ -30982,7 +31102,7 @@ to begin with: gawk -M 'BEGIN @{ n = 13.0; print n % 2.0 @}' @end example -Note that for the particular example above, it is likely best +Note that for this particular example, it is likely best to just use the following: @example @@ -30994,27 +31114,30 @@ When dividing two arbitrary precision integers with either precision floating point value (unless the denominator evenly divides into the numerator). In order to do integer division or remainder with arbitrary precision integers, use the built-in -@code{div()} function (@pxref{Numeric Functions}). +@code{intdiv()} function (@pxref{Numeric Functions}). -You can simulate the @code{div()} function in standard @command{awk} +You can simulate the @code{intdiv()} function in standard @command{awk} using this user-defined function: @example -@c file eg/lib/div.awk -# div --- do integer division +@c file eg/lib/intdiv.awk +# intdiv --- do integer division @c endfile @ignore -@c file eg/lib/div.awk +@c file eg/lib/intdiv.awk # # Arnold Robbins, arnold@@skeeve.com, Public Domain # July, 2014 +# +# Name changed from div() to intdiv() +# April, 2015 @c endfile @end ignore -@c file eg/lib/div.awk -function div(numerator, denominator, result) +@c file eg/lib/intdiv.awk +function intdiv(numerator, denominator, result) @{ split("", result) @@ -31029,7 +31152,7 @@ function div(numerator, denominator, result) @end example The following example program, contributed by Katie Wasserman, -uses @code{div()} to +uses @code{intdiv()} to compute the digits of @value{PI} to as many places as you choose to set: @@ -31054,7 +31177,7 @@ BEGIN @{ for (m = digits * 4; m > 0; --m) @{ d = m * 2 + 1 x = pi * m - div(x, d, result) + intdiv(x, d, result) pi = result["quotient"] pi = pi + two @} @@ -31093,7 +31216,7 @@ When asked about the algorithm used, Katie replied: @quotation It's not that well known but it's not that obscure either. It's Euler's modification to Newton's method for calculating pi. -Take a look at lines (23) - (25) here: @uref{http://mathworld.wolfram.com/PiFormulas.htm}. +Take a look at lines (23) - (25) here: @uref{http://mathworld.wolfram.com/PiFormulas.html}. The algorithm I wrote simply expands the multiply by 2 and works from the innermost expression outwards. I used this to program HP calculators @@ -31105,7 +31228,7 @@ word sizes. See @node POSIX Floating Point Problems @section Standards Versus Existing Practice -Historically, @command{awk} has converted any non-numeric looking string +Historically, @command{awk} has converted any nonnumeric-looking string to the numeric value zero, when required. Furthermore, the original definition of the language and the original POSIX standards specified that @command{awk} only understands decimal numbers (base 10), and not octal @@ -31117,13 +31240,13 @@ should support additional features. These features are: @itemize @value{BULLET} @item -Interpretation of floating point data values specified in hexadecimal +Interpretation of floating-point data values specified in hexadecimal notation (e.g., @code{0xDEADBEEF}). (Note: data values, @emph{not} source code constants.) @item -Support for the special IEEE 754 floating point values ``Not A Number'' -(NaN), positive Infinity (``inf'') and negative Infinity (``@minus{}inf''). +Support for the special IEEE 754 floating-point values ``not a number'' +(NaN), positive infinity (``inf''), and negative infinity (``@minus{}inf''). In particular, the format for these values is as specified by the ISO 1999 C standard, which ignores case and can allow implementation-dependent additional characters after the @samp{nan} and allow either @samp{inf} or @samp{infinity}. @@ -31134,8 +31257,8 @@ practice: @itemize @value{BULLET} @item -The @command{gawk} maintainer feels that supporting hexadecimal floating -point values, in particular, is ugly, and was never intended by the +The @command{gawk} maintainer feels that supporting hexadecimal +floating-point values, in particular, is ugly, and was never intended by the original designers to be part of the language. @item @@ -31143,22 +31266,22 @@ Allowing completely alphabetic strings to have valid numeric values is also a very severe departure from historical practice. @end itemize -The second problem is that the @code{gawk} maintainer feels that this -interpretation of the standard, which requires a certain amount of +The second problem is that the @command{gawk} maintainer feels that this +interpretation of the standard, which required a certain amount of ``language lawyering'' to arrive at in the first place, was not even -intended by the standard developers. In other words, ``we see how you +intended by the standard developers. In other words, ``We see how you got where you are, but we don't think that that's where you want to be.'' -Recognizing the above issues, but attempting to provide compatibility +Recognizing these issues, but attempting to provide compatibility with the earlier versions of the standard, the 2008 POSIX standard added explicit wording to allow, but not require, -that @command{awk} support hexadecimal floating point values and -special values for ``Not A Number'' and infinity. +that @command{awk} support hexadecimal floating-point values and +special values for ``not a number'' and infinity. Although the @command{gawk} maintainer continues to feel that providing those features is inadvisable, nevertheless, on systems that support IEEE floating point, it seems -reasonable to provide @emph{some} way to support NaN and Infinity values. +reasonable to provide @emph{some} way to support NaN and infinity values. The solution implemented in @command{gawk} is as follows: @itemize @value{BULLET} @@ -31178,7 +31301,7 @@ $ @kbd{echo 0xDeadBeef | gawk --posix '@{ print $1 + 0 @}'} @end example @item -Without @option{--posix}, @command{gawk} interprets the four strings +Without @option{--posix}, @command{gawk} interprets the four string values @samp{+inf}, @samp{-inf}, @samp{+nan}, @@ -31200,7 +31323,7 @@ $ @kbd{echo 0xDeadBeef | gawk '@{ print $1 + 0 @}'} @end example @command{gawk} ignores case in the four special values. -Thus @samp{+nan} and @samp{+NaN} are the same. +Thus, @samp{+nan} and @samp{+NaN} are the same. @end itemize @node Floating point summary @@ -31209,13 +31332,13 @@ Thus @samp{+nan} and @samp{+NaN} are the same. @itemize @value{BULLET} @item Most computer arithmetic is done using either integers or floating-point -values. Standard @command{awk} uses double precision +values. Standard @command{awk} uses double-precision floating-point values. @item -In the early 1990's, Barbie mistakenly said ``Math class is tough!'' -While math isn't tough, floating-point arithmetic isn't the same -as pencil and paper math, and care must be taken: +In the early 1990s Barbie mistakenly said, ``Math class is tough!'' +Although math isn't tough, floating-point arithmetic isn't the same +as pencil-and-paper math, and care must be taken: @c nested list @itemize @value{MINUS} @@ -31247,12 +31370,12 @@ arithmetic. Use @code{PREC} to set the precision in bits, and @item With @option{-M}, @command{gawk} performs -arbitrary precision integer arithmetic using the GMP library. -This is faster and more space efficient than using MPFR for +arbitrary-precision integer arithmetic using the GMP library. +This is faster and more space-efficient than using MPFR for the same calculations. @item -There are several ``dark corners'' with respect to floating-point +There are several areas with respect to floating-point numbers where @command{gawk} disagrees with the POSIX standard. It pays to be aware of them. @@ -31260,7 +31383,7 @@ It pays to be aware of them. Overall, there is no need to be unduly suspicious about the results from floating-point arithmetic. The lesson to remember is that floating-point arithmetic is always more complex than arithmetic using pencil and -paper. In order to take advantage of the power of computer floating-point, +paper. In order to take advantage of the power of floating-point arithmetic, you need to know its limitations and work within them. For most casual use of floating-point arithmetic, you will often get the expected result if you simply round the display of your final results to the correct number @@ -31302,7 +31425,7 @@ When @option{--sandbox} is specified, extensions are disabled * Finding Extensions:: How @command{gawk} finds compiled extensions. * Extension Example:: Example C code for an extension. * Extension Samples:: The sample extensions that ship with - @code{gawk}. + @command{gawk}. * gawkextlib:: The @code{gawkextlib} project. * Extension summary:: Extension summary. * Extension Exercises:: Exercises. @@ -31321,15 +31444,15 @@ Extensions are useful because they allow you (of course) to extend @command{gawk}'s functionality. For example, they can provide access to system calls (such as @code{chdir()} to change directory) and to other C library routines that could be of use. As with most software, -``the sky is the limit;'' if you can imagine something that you might +``the sky is the limit''; if you can imagine something that you might want to do and can write in C or C++, you can write an extension to do it! -Extensions are written in C or C++, using the @dfn{Application Programming -Interface} (API) defined for this purpose by the @command{gawk} +Extensions are written in C or C++, using the @dfn{application programming +interface} (API) defined for this purpose by the @command{gawk} developers. The rest of this @value{CHAPTER} explains the facilities that the API provides and how to use them, and presents a small example extension. In addition, it documents -the sample extensions included in the @command{gawk} distribution, +the sample extensions included in the @command{gawk} distribution and describes the @code{gawkextlib} project. @ifclear FOR_PRINT @xref{Extension Design}, for a discussion of the extension mechanism @@ -31362,7 +31485,7 @@ int plugin_is_GPL_compatible; @end example @node Extension Mechanism Outline -@section At A High Level How It Works +@section How It Works at a High Level Communication between @command{gawk} and an extension is two-way. First, when an extension @@ -31377,22 +31500,22 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-load-extension"/>}. @ifnotdocbook @float Figure,figure-load-extension -@caption{Loading The Extension} +@caption{Loading the extension} @c FIXME: One day, it should not be necessary to have two cases, @c but rather just the one without the "txt" final argument. @c This applies to the other figures as well. @ifinfo -@center @image{api-figure1, , , Loading The Extension, txt} +@center @image{api-figure1, , , Loading the extension, txt} @end ifinfo @ifnotinfo -@center @image{api-figure1, , , Loading The Extension} +@center @image{api-figure1, , , Loading the extension} @end ifnotinfo @end float @end ifnotdocbook @docbook <figure id="figure-load-extension" float="0"> -<title>Loading The Extension</title> +<title>Loading the extension</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure1.png" format="PNG"/></imageobject> </mediaobject> @@ -31412,19 +31535,19 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-register-new-function @ifnotdocbook @float Figure,figure-register-new-function -@caption{Registering A New Function} +@caption{Registering a new function} @ifinfo -@center @image{api-figure2, , , Registering A New Function, txt} +@center @image{api-figure2, , , Registering a new Function, txt} @end ifinfo @ifnotinfo -@center @image{api-figure2, , , Registering A New Function} +@center @image{api-figure2, , , Registering a new Function} @end ifnotinfo @end float @end ifnotdocbook @docbook <figure id="figure-register-new-function" float="0"> -<title>Registering A New Function</title> +<title>Registering a new function</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure2.png" format="PNG"/></imageobject> </mediaobject> @@ -31445,7 +31568,7 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-call-new-function"/>} @ifnotdocbook @float Figure,figure-call-new-function -@caption{Calling The New Function} +@caption{Calling the new function} @ifinfo @center @image{api-figure3, , , Calling the new function, txt} @end ifinfo @@ -31457,7 +31580,7 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-call-new-function"/>} @docbook <figure id="figure-call-new-function" float="0"> -<title>Calling The New Function</title> +<title>Calling the new function</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure3.png" format="PNG"/></imageobject> </mediaobject> @@ -31482,7 +31605,7 @@ Some other bits and pieces: @itemize @value{BULLET} @item The API provides access to @command{gawk}'s @code{do_@var{xxx}} values, -reflecting command-line options, like @code{do_lint}, @code{do_profiling} +reflecting command-line options, like @code{do_lint}, @code{do_profiling}, and so on (@pxref{Extension API Variables}). These are informational: an extension cannot affect their values inside @command{gawk}. In addition, attempting to assign to them @@ -31493,10 +31616,9 @@ The API also provides major and minor version numbers, so that an extension can check if the @command{gawk} it is loaded with supports the facilities it was compiled with. (Version mismatches ``shouldn't'' happen, but we all know how @emph{that} goes.) -@xref{Extension Versioning}, for details. +@DBXREF{Extension Versioning} for details. @end itemize - @node Extension API Description @section API Description @cindex extension API @@ -31520,6 +31642,7 @@ This (rather large) @value{SECTION} describes the API in detail. * Symbol Table Access:: Functions for accessing global variables. * Array Manipulation:: Functions for working with arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension API Boilerplate:: Boilerplate code for using the API. @end menu @@ -31527,7 +31650,7 @@ This (rather large) @value{SECTION} describes the API in detail. @node Extension API Functions Introduction @subsection Introduction -Access to facilities within @command{gawk} are made available +Access to facilities within @command{gawk} is achieved by calling through function pointers passed into your extension. API function pointers are provided for the following kinds of operations: @@ -31538,21 +31661,24 @@ Allocating, reallocating, and releasing memory. @item Registration functions. You may register: + +@c nested list @itemize @value{MINUS} @item -extension functions, +Extension functions @item -exit callbacks, +Exit callbacks @item -a version string, +A version string @item -input parsers, +Input parsers @item -output wrappers, +Output wrappers @item -and two-way processors. +Two-way processors @end itemize -All of these are discussed in detail, later in this @value{CHAPTER}. + +All of these are discussed in detail later in this @value{CHAPTER}. @item Printing fatal, warning, and ``lint'' warning messages. @@ -31590,20 +31716,25 @@ Creating a new array Clearing an array @item -Flattening an array for easy C style looping over all its indices and elements +Flattening an array for easy C-style looping over all its indices and elements @end itemize + +@item +Accessing and manipulating redirections. + @end itemize Some points about using the API: @itemize @value{BULLET} @item -The following types and/or macros and/or functions are referenced +The following types, macros, and/or functions are referenced in @file{gawkapi.h}. For correct use, you must therefore include the corresponding standard header file @emph{before} including @file{gawkapi.h}: +@c FIXME: Make this is a float at some point. @multitable {@code{memset()}, @code{memcpy()}} {@code{<sys/types.h>}} -@headitem C Entity @tab Header File +@headitem C entity @tab Header file @item @code{EOF} @tab @code{<stdio.h>} @item Values for @code{errno} @tab @code{<errno.h>} @item @code{FILE} @tab @code{<stdio.h>} @@ -31612,7 +31743,7 @@ corresponding standard header file @emph{before} including @file{gawkapi.h}: @item @code{memset()} @tab @code{<string.h>} @item @code{size_t} @tab @code{<sys/types.h>} @item @code{struct stat} @tab @code{<sys/stat.h>} -@end multitable +@end multitable Due to portability concerns, especially to systems that are not fully standards-compliant, it is your responsibility @@ -31629,7 +31760,7 @@ Doing so, however, is poor coding practice. Although the API only uses ISO C 90 features, there is an exception; the ``constructor'' functions use the @code{inline} keyword. If your compiler does not support this keyword, you should either place -@samp{-Dinline=''} on your command line, or use the GNU Autotools and include a +@samp{-Dinline=''} on your command line or use the GNU Autotools and include a @file{config.h} file in your extensions. @item @@ -31637,21 +31768,21 @@ All pointers filled in by @command{gawk} point to memory managed by @command{gawk} and should be treated by the extension as read-only. Memory for @emph{all} strings passed into @command{gawk} from the extension @emph{must} come from calling one of -@code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}, +@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}, and is managed by @command{gawk} from then on. @item The API defines several simple @code{struct}s that map values as seen from @command{awk}. A value can be a @code{double}, a string, or an array (as in multidimensional arrays, or when creating a new array). -String values maintain both pointer and length since embedded @value{NUL} +String values maintain both pointer and length, because embedded @sc{nul} characters are allowed. @quotation NOTE By intent, strings are maintained using the current multibyte encoding (as defined by @env{LC_@var{xxx}} environment variables) and not using wide characters. This matches how @command{gawk} stores strings internally -and also how characters are likely to be input and output from files. +and also how characters are likely to be input into and output from files. @end quotation @item @@ -31669,14 +31800,14 @@ so that the extension can, e.g., print an error message @c The table there should be presented here @end itemize -While you may call the API functions by using the function pointers -directly, the interface is not so pretty. To make extension code look +You may call the API functions by using the function pointers +directly, but the interface is not so pretty. To make extension code look more like regular code, the @file{gawkapi.h} header file defines several macros that you should use in your code. This @value{SECTION} presents the macros as if they were functions. @node General Data Types -@subsection General Purpose Data Types +@subsection General-Purpose Data Types @cindex Robbins, Arnold @cindex Ramey, Chet @@ -31691,9 +31822,12 @@ can accommodate both love and hate.} @author Chet Ramey @end quotation -The extension API defines a number of simple types and structures for general -purpose use. Additional, more specialized, data structures are introduced -in subsequent @value{SECTION}s, together with the functions that use them. +The extension API defines a number of simple types and structures for +general-purpose use. Additional, more specialized, data structures are +introduced in subsequent @value{SECTION}s, together with the functions +that use them. + +The general-purpose types and structures are as follows: @table @code @item typedef void *awk_ext_id_t; @@ -31711,7 +31845,7 @@ while allowing @command{gawk} to use them as it needs to. @itemx @ @ @ @ awk_false = 0, @itemx @ @ @ @ awk_true @itemx @} awk_bool_t; -A simple boolean type. +A simple Boolean type. @item typedef struct awk_string @{ @itemx @ @ @ @ char *str;@ @ @ @ @ @ /* data */ @@ -31720,8 +31854,9 @@ A simple boolean type. This represents a mutable string. @command{gawk} owns the memory pointed to if it supplied the value. Otherwise, it takes ownership of the memory pointed to. -@strong{Such memory must come from calling one of the -@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()} functions!} +@emph{Such memory must come from calling one of the +@code{gawk_malloc()}, @code{gawk_calloc()}, or +@code{gawk_realloc()} functions!} As mentioned earlier, strings are maintained using the current multibyte encoding. @@ -31747,7 +31882,7 @@ It is used in the following @code{struct}. @itemx @ @ @ @ @ @ @ @ awk_value_cookie_t@ vc; @itemx @ @ @ @ @} u; @itemx @} awk_value_t; -An ``@command{awk} value.'' +An ``@command{awk} value.'' The @code{val_type} member indicates what kind of value the @code{union} holds, and each member is of the appropriate type. @@ -31756,17 +31891,18 @@ The @code{val_type} member indicates what kind of value the @itemx #define array_cookie@ @ @ u.a @itemx #define scalar_cookie@ @ u.scl @itemx #define value_cookie@ @ @ u.vc -These macros make accessing the fields of the @code{awk_value_t} more +Using these macros makes accessing the fields of the @code{awk_value_t} more readable. @item typedef void *awk_scalar_t; -Scalars can be represented as an opaque type. These values are obtained from -@command{gawk} and then passed back into it. This is discussed in a general fashion below, -and in more detail in @ref{Symbol table by cookie}. +Scalars can be represented as an opaque type. These values are obtained +from @command{gawk} and then passed back into it. This is discussed +in a general fashion in the text following this list, and in more detail in +@ref{Symbol table by cookie}. @item typedef void *awk_value_cookie_t; A ``value cookie'' is an opaque type representing a cached value. -This is also discussed in a general fashion below, +This is also discussed in a general fashion in the text following this list, and in more detail in @ref{Cached values}. @end table @@ -31776,9 +31912,9 @@ Scalar values in @command{awk} are either numbers or strings. The indicates what is in the @code{union}. Representing numbers is easy---the API uses a C @code{double}. Strings -require more work. Since @command{gawk} allows embedded @value{NUL} bytes +require more work. Because @command{gawk} allows embedded @sc{nul} bytes in string values, a string must be represented as a pair containing a -data-pointer and length. This is the @code{awk_string_t} type. +data pointer and length. This is the @code{awk_string_t} type. Identifiers (i.e., the names of global variables) can be associated with either scalar values or with arrays. In addition, @command{gawk} @@ -31789,14 +31925,14 @@ itself be an array. Discussion of arrays is delayed until The various macros listed earlier make it easier to use the elements of the @code{union} as if they were fields in a @code{struct}; this is a common coding practice in C. Such code is easier to write and to -read, however it remains @emph{your} responsibility to make sure that +read, but it remains @emph{your} responsibility to make sure that the @code{val_type} member correctly reflects the type of the value in -the @code{awk_value_t}. +the @code{awk_value_t} struct. Conceptually, the first three members of the @code{union} (number, string, and array) are all that is needed for working with @command{awk} values. -However, since the API provides routines for accessing and changing -the value of global scalar variables only by using the variable's name, +However, because the API provides routines for accessing and changing +the value of a global scalar variable only by using the variable's name, there is a performance penalty: @command{gawk} must find the variable each time it is accessed and changed. This turns out to be a real issue, not just a theoretical one. @@ -31814,7 +31950,9 @@ See also the entry for ``Cookie'' in the @ref{Glossary}. object for that variable, and then use the cookie for getting the variable's value or for changing the variable's value. -This is the @code{awk_scalar_t} type and @code{scalar_cookie} macro. +The @code{awk_scalar_t} type holds a scalar cookie, and the +@code{scalar_cookie} macro provides access to the value of that type +in the @code{awk_value_t} struct. Given a scalar cookie, @command{gawk} can directly retrieve or modify the value, as required, without having to find it first. @@ -31823,8 +31961,8 @@ If you know that you wish to use the same numeric or string @emph{value} for one or more variables, you can create the value once, retaining a @dfn{value cookie} for it, and then pass in that value cookie whenever you wish to set the value of a -variable. This saves both storage space within the running @command{gawk} -process as well as the time needed to create the value. +variable. This saves storage space within the running @command{gawk} +process and reduces the time needed to create the value. @node Memory Allocation Functions @subsection Memory Allocation Functions and Convenience Macros @@ -31835,7 +31973,7 @@ The API provides a number of @dfn{memory allocation} functions for allocating memory that can be passed to @command{gawk}, as well as a number of convenience macros. This @value{SUBSECTION} presents them all as function prototypes, in -the way that extension code would use them. +the way that extension code would use them: @table @code @item void *gawk_malloc(size_t size); @@ -31852,13 +31990,13 @@ be passed to @command{gawk}. @item void gawk_free(void *ptr); Call the correct version of @code{free()} to release storage that was -allocated with @code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}. +allocated with @code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. @end table The API has to provide these functions because it is possible for an extension to be compiled and linked against a different version of the C library than was used for the @command{gawk} -executable.@footnote{This is more common on MS-Windows systems, but +executable.@footnote{This is more common on MS-Windows systems, but it can happen on Unix-like systems as well.} If @command{gawk} were to use its version of @code{free()} when the memory came from an unrelated version of @code{malloc()}, unexpected behavior would @@ -31868,7 +32006,7 @@ Two convenience macros may be used for allocating storage from @code{gawk_malloc()} and @code{gawk_realloc()}. If the allocation fails, they cause @command{gawk} to exit with a fatal error message. They should be used as if they were -procedure calls that do not return a value. +procedure calls that do not return a value: @table @code @item #define emalloc(pointer, type, size, message) @dots{} @@ -31880,7 +32018,8 @@ The arguments to this macro are as follows: The pointer variable to point at the allocated storage. @item type -The type of the pointer variable, used to create a cast for the call to @code{gawk_malloc()}. +The type of the pointer variable. This is used to create a cast for +the call to @code{gawk_malloc()}. @item size The total number of bytes to be allocated. @@ -31904,7 +32043,7 @@ make_malloced_string(message, strlen(message), & result); @end example @item #define erealloc(pointer, type, size, message) @dots{} -This is like @code{emalloc()}, but it calls @code{gawk_realloc()}, +This is like @code{emalloc()}, but it calls @code{gawk_realloc()} instead of @code{gawk_malloc()}. The arguments are the same as for the @code{emalloc()} macro. @end table @@ -31915,32 +32054,32 @@ The arguments are the same as for the @code{emalloc()} macro. The API provides a number of @dfn{constructor} functions for creating string and numeric values, as well as a number of convenience macros. This @value{SUBSECTION} presents them all as function prototypes, in -the way that extension code would use them. +the way that extension code would use them: @table @code @item static inline awk_value_t * -@itemx make_const_string(const char *string, size_t length, awk_value_t *result) +@itemx make_const_string(const char *string, size_t length, awk_value_t *result); This function creates a string value in the @code{awk_value_t} variable pointed to by @code{result}. It expects @code{string} to be a C string constant (or other string data), and automatically creates a @emph{copy} of the data for storage in @code{result}. It returns @code{result}. @item static inline awk_value_t * -@itemx make_malloced_string(const char *string, size_t length, awk_value_t *result) +@itemx make_malloced_string(const char *string, size_t length, awk_value_t *result); This function creates a string value in the @code{awk_value_t} variable pointed to by @code{result}. It expects @code{string} to be a @samp{char *} -value pointing to data previously obtained from @code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}. The idea here +value pointing to data previously obtained from @code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. The idea here is that the data is passed directly to @command{gawk}, which assumes responsibility for it. It returns @code{result}. @item static inline awk_value_t * -@itemx make_null_string(awk_value_t *result) +@itemx make_null_string(awk_value_t *result); This specialized function creates a null string (the ``undefined'' value) in the @code{awk_value_t} variable pointed to by @code{result}. It returns @code{result}. @item static inline awk_value_t * -@itemx make_number(double num, awk_value_t *result) +@itemx make_number(double num, awk_value_t *result); This function simply creates a numeric value in the @code{awk_value_t} variable pointed to by @code{result}. @end table @@ -31980,7 +32119,7 @@ The fields are: @table @code @item const char *name; The name of the new function. -@command{awk} level code calls the function by this name. +@command{awk}-level code calls the function by this name. This is a regular C string. Function names must obey the rules for @command{awk} @@ -31994,8 +32133,8 @@ This is a pointer to the C function that provides the extension's functionality. The function must fill in @code{*result} with either a number or a string. @command{gawk} takes ownership of any string memory. -As mentioned earlier, string memory @strong{must} come from one of @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}. +As mentioned earlier, string memory @emph{must} come from one of +@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. The @code{num_actual_args} argument tells the C function how many actual parameters were passed from the calling @command{awk} code. @@ -32030,7 +32169,7 @@ Such functions are useful if you have general ``cleanup'' tasks that should be performed in your extension (such as closing database connections or other resource deallocations). You can register such -a function with @command{gawk} using the following function. +a function with @command{gawk} using the following function: @table @code @item void awk_atexit(void (*funcp)(void *data, int exit_status), @@ -32046,24 +32185,25 @@ The @code{exit_status} parameter is the exit status value that @command{gawk} intends to pass to the @code{exit()} system call. @item arg0 -A pointer to private data which @command{gawk} saves in order to pass to +A pointer to private data that @command{gawk} saves in order to pass to the function pointed to by @code{funcp}. @end table @end table -Exit callback functions are called in Last-In-First-Out (LIFO) order---that is, in -the reverse order in which they are registered with @command{gawk}. +Exit callback functions are called in last-in, first-out (LIFO) +order---that is, in the reverse order in which they are registered with +@command{gawk}. @node Extension Version String @subsubsection Registering An Extension Version String -You can register a version string which indicates the name and -version of your extension, with @command{gawk}, as follows: +You can register a version string that indicates the name and +version of your extension with @command{gawk}, as follows: @table @code @item void register_ext_version(const char *version); Register the string pointed to by @code{version} with @command{gawk}. -@command{gawk} does @emph{not} copy the @code{version} string, so +Note that @command{gawk} does @emph{not} copy the @code{version} string, so it should not be changed. @end table @@ -32080,7 +32220,7 @@ of @code{RS} to find the end of the record, and then uses @code{FS} Additionally, it sets the value of @code{RT} (@pxref{Built-in Variables}). If you want, you can provide your own custom input parser. An input -parser's job is to return a record to the @command{gawk} record processing +parser's job is to return a record to the @command{gawk} record-processing code, along with indicators for the value and length of the data to be used for @code{RT}, if any. @@ -32088,19 +32228,19 @@ To provide an input parser, you must first provide two functions (where @var{XXX} is a prefix name for your extension): @table @code -@item awk_bool_t @var{XXX}_can_take_file(const awk_input_buf_t *iobuf) +@item awk_bool_t @var{XXX}_can_take_file(const awk_input_buf_t *iobuf); This function examines the information available in @code{iobuf} (which we discuss shortly). Based on the information there, it decides if the input parser should be used for this file. If so, it should return true. Otherwise, it should return false. It should not change any state (variable values, etc.) within @command{gawk}. -@item awk_bool_t @var{XXX}_take_control_of(awk_input_buf_t *iobuf) +@item awk_bool_t @var{XXX}_take_control_of(awk_input_buf_t *iobuf); When @command{gawk} decides to hand control of the file over to the input parser, it calls this function. This function in turn must fill -in certain fields in the @code{awk_input_buf_t} structure, and ensure +in certain fields in the @code{awk_input_buf_t} structure and ensure that certain conditions are true. It should then return true. If an -error of some kind occurs, it should not fill in any fields, and should +error of some kind occurs, it should not fill in any fields and should return false; then @command{gawk} will not use the input parser. The details are presented shortly. @end table @@ -32145,7 +32285,7 @@ appropriately. @item When your extension is loaded, register your input parser with @command{gawk} using the @code{register_input_parser()} API function -(described below). +(described next). @end enumerate An @code{awk_input_buf_t} looks like this: @@ -32175,7 +32315,7 @@ The name of the file. @item int fd; A file descriptor for the file. If @command{gawk} was able to -open the file, then @code{fd} will @emph{not} be equal to +open the file, then @code{fd} will @emph{not} be equal to @code{INVALID_HANDLE}. Otherwise, it will. @item struct stat sbuf; @@ -32189,15 +32329,15 @@ The decision can be made based upon @command{gawk} state (the value of a variable defined previously by the extension and set by @command{awk} code), the name of the file, whether or not the file descriptor is valid, the information -in the @code{struct stat}, or any combination of the above. +in the @code{struct stat}, or any combination of these factors. Once @code{@var{XXX}_can_take_file()} has returned true, and @command{gawk} has decided to use your input parser, it calls -@code{@var{XXX}_take_control_of()}. That function then fills one of +@code{@var{XXX}_take_control_of()}. That function then fills either the @code{get_record} field or the @code{read_func} field in the @code{awk_input_buf_t}. It must also ensure that @code{fd} is @emph{not} -set to @code{INVALID_HANDLE}. All of the fields that may be filled by -@code{@var{XXX}_take_control_of()} are as follows: +set to @code{INVALID_HANDLE}. The following list describes the fields that +may be filled by @code{@var{XXX}_take_control_of()}: @table @code @item void *opaque; @@ -32212,24 +32352,24 @@ is not required to use this pointer. @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ size_t *rt_len); This function pointer should point to a function that creates the input records. Said function is the core of the input parser. Its behavior -is described below. +is described in the text following this list. @item ssize_t (*read_func)(); -This function pointer should point to function that has the +This function pointer should point to a function that has the same behavior as the standard POSIX @code{read()} system call. It is an alternative to the @code{get_record} pointer. Its behavior -is also described below. +is also described in the text following this list. @item void (*close_func)(struct awk_input *iobuf); This function pointer should point to a function that does -the ``tear down.'' It should release any resources allocated by +the ``teardown.'' It should release any resources allocated by @code{@var{XXX}_take_control_of()}. It may also close the file. If it does so, it should set the @code{fd} field to @code{INVALID_HANDLE}. If @code{fd} is still not @code{INVALID_HANDLE} after the call to this function, @command{gawk} calls the regular @code{close()} system call. -Having a ``tear down'' function is optional. If your input parser does +Having a ``teardown'' function is optional. If your input parser does not need it, do not set this field. Then, @command{gawk} calls the regular @code{close()} system call on the file descriptor, so it should be valid. @@ -32240,7 +32380,7 @@ input records. The parameters are as follows: @table @code @item char **out -This is a pointer to a @code{char *} variable which is set to point +This is a pointer to a @code{char *} variable that is set to point to the record. @command{gawk} makes its own copy of the data, so the extension must manage this storage. @@ -32259,7 +32399,7 @@ If the concept of a ``record terminator'' makes sense, then @code{*rt_start} should be set to point to the data to be used for @code{RT}, and @code{*rt_len} should be set to the length of the data. Otherwise, @code{*rt_len} should be set to zero. -@code{gawk} makes its own copy of this data, so the +@command{gawk} makes its own copy of this data, so the extension must manage this storage. @end table @@ -32293,19 +32433,19 @@ set this field explicitly. You must choose one method or the other: either a function that returns a record, or one that returns raw data. In particular, if you supply a function to get a record, @command{gawk} will -call it, and never call the raw read function. +call it, and will never call the raw read function. @end quotation @command{gawk} ships with a sample extension that reads directories, -returning records for each entry in the directory (@pxref{Extension +returning records for each entry in a directory (@pxref{Extension Sample Readdir}). You may wish to use that code as a guide for writing your own input parser. When writing an input parser, you should think about (and document) how it is expected to interact with @command{awk} code. You may want -it to always be called, and take effect as appropriate (as the +it to always be called, and to take effect as appropriate (as the @code{readdir} extension does). Or you may want it to take effect -based upon the value of an @code{awk} variable, as the XML extension +based upon the value of an @command{awk} variable, as the XML extension from the @code{gawkextlib} project does (@pxref{gawkextlib}). In the latter case, code in a @code{BEGINFILE} section can look at @code{FILENAME} and @code{ERRNO} to decide whether or @@ -32356,7 +32496,7 @@ values, etc.) within @command{gawk}. The function pointed to by this field is called when @command{gawk} decides to let the output wrapper take control of the file. It should fill in appropriate members of the @code{awk_output_buf_t} structure, -as described below, and return true if successful, false otherwise. +as described next, and return true if successful, false otherwise. @item awk_const struct output_wrapper *awk_const next; This is for use by @command{gawk}; @@ -32413,7 +32553,7 @@ a pointer to any private data associated with the file. These pointers should be set to point to functions that perform the equivalent function as the @code{<stdio.h>} functions do, if appropriate. @command{gawk} uses these function pointers for all output. -@command{gawk} initializes the pointers to point to internal, ``pass through'' +@command{gawk} initializes the pointers to point to internal ``pass-through'' functions that just call the regular @code{<stdio.h>} functions, so an extension only needs to redefine those functions that are appropriate for what it does. @@ -32424,7 +32564,7 @@ upon the @code{name} and @code{mode} fields, and any additional state (such as @command{awk} variable values) that is appropriate. When @command{gawk} calls @code{@var{XXX}_take_control_of()}, that function should fill -in the other fields, as appropriate, except for @code{fp}, which it should just +in the other fields as appropriate, except for @code{fp}, which it should just use normally. You register your output wrapper with the following function: @@ -32464,14 +32604,14 @@ The fields are as follows: The name of the two-way processor. @item awk_bool_t (*can_take_two_way)(const char *name); -This function returns true if it wants to take over two-way I/O for this @value{FN}. +The function pointed to by this field should return true if it wants to take over two-way I/O for this @value{FN}. It should not change any state (variable values, etc.) within @command{gawk}. @item awk_bool_t (*take_control_of)(const char *name, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_input_buf_t *inbuf, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_output_buf_t *outbuf); -This function should fill in the @code{awk_input_buf_t} and +The function pointed to by this field should fill in the @code{awk_input_buf_t} and @code{awk_outut_buf_t} structures pointed to by @code{inbuf} and @code{outbuf}, respectively. These structures were described earlier. @@ -32499,9 +32639,9 @@ Register the two-way processor pointed to by @code{two_way_processor} with @cindex messages from extensions You can print different kinds of warning messages from your -extension, as described below. Note that for these functions, -you must pass in the extension id received from @command{gawk} -when the extension was loaded.@footnote{Because the API uses only ISO C 90 +extension, as described here. Note that for these functions, +you must pass in the extension ID received from @command{gawk} +when the extension was loaded:@footnote{Because the API uses only ISO C 90 features, it cannot make use of the ISO C 99 variadic macro feature to hide that parameter. More's the pity.} @@ -32553,12 +32693,12 @@ matches what you requested, the function returns true and fills in the @code{awk_value_t} result. Otherwise, the function returns false, and the @code{val_type} member indicates the type of the actual value. You may then -print an error message, or reissue the request for the actual +print an error message or reissue the request for the actual value type, as appropriate. This behavior is summarized in @ref{table-value-types-returned}. @float Table,table-value-types-returned -@caption{API Value Types Returned} +@caption{API value types returned} @docbook <informaltable> <tgroup cols="6"> @@ -32570,7 +32710,7 @@ value type, as appropriate. This behavior is summarized in <colspec colwidth="16.6*" colname="c6"/> <spanspec spanname="hspan" namest="c3" nameend="c6" align="center"/> <thead> - <row><entry></entry><entry spanname="hspan"><para>Type of Actual Value:</para></entry></row> + <row><entry></entry><entry spanname="hspan"><para>Type of Actual Value</para></entry></row> <row> <entry></entry> <entry></entry> @@ -32586,32 +32726,32 @@ value type, as appropriate. This behavior is summarized in <entry><para><emphasis role="bold">String</emphasis></para></entry> <entry><para>String</para></entry> <entry><para>String</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry></entry> <entry><para><emphasis role="bold">Number</emphasis></para></entry> <entry><para>Number if can be converted, else false</para></entry> <entry><para>Number</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry><para><emphasis role="bold">Type</emphasis></para></entry> <entry><para><emphasis role="bold">Array</emphasis></para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> <entry><para>Array</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> </row> <row> - <entry><para><emphasis role="bold">Requested:</emphasis></para></entry> + <entry><para><emphasis role="bold">Requested</emphasis></para></entry> <entry><para><emphasis role="bold">Scalar</emphasis></para></entry> <entry><para>Scalar</para></entry> <entry><para>Scalar</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry></entry> @@ -32623,11 +32763,11 @@ value type, as appropriate. This behavior is summarized in </row> <row> <entry></entry> - <entry><para><emphasis role="bold">Value Cookie</emphasis></para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para> - </entry><entry><para>false</para></entry> + <entry><para><emphasis role="bold">Value cookie</emphasis></para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para> + </entry><entry><para>False</para></entry> </row> </tbody> </tgroup> @@ -32637,7 +32777,7 @@ value type, as appropriate. This behavior is summarized in @ifnotplaintext @ifnotdocbook @multitable @columnfractions .50 .50 -@headitem @tab Type of Actual Value: +@headitem @tab Type of Actual Value @end multitable @c 10/2014: Thanks to Karl Berry for this bit to reduce the space: @tex @@ -32645,12 +32785,12 @@ value type, as appropriate. This behavior is summarized in @end tex @multitable @columnfractions .166 .166 .198 .15 .15 .166 @headitem @tab @tab String @tab Number @tab Array @tab Undefined -@item @tab @b{String} @tab String @tab String @tab false @tab false -@item @tab @b{Number} @tab Number if can be converted, else false @tab Number @tab false @tab false -@item @b{Type} @tab @b{Array} @tab false @tab false @tab Array @tab false -@item @b{Requested:} @tab @b{Scalar} @tab Scalar @tab Scalar @tab false @tab false +@item @tab @b{String} @tab String @tab String @tab False @tab False +@item @tab @b{Number} @tab Number if can be converted, else false @tab Number @tab False @tab False +@item @b{Type} @tab @b{Array} @tab False @tab False @tab Array @tab False +@item @b{Requested} @tab @b{Scalar} @tab Scalar @tab Scalar @tab False @tab False @item @tab @b{Undefined} @tab String @tab Number @tab Array @tab Undefined -@item @tab @b{Value Cookie} @tab false @tab false @tab false @tab false +@item @tab @b{Value cookie} @tab False @tab False @tab False @tab False @end multitable @end ifnotdocbook @end ifnotplaintext @@ -32661,21 +32801,21 @@ value type, as appropriate. This behavior is summarized in +------------+------------+-----------+-----------+ | String | Number | Array | Undefined | +-----------+-----------+------------+------------+-----------+-----------+ -| | String | String | String | false | false | +| | String | String | String | False | False | | |-----------+------------+------------+-----------+-----------+ -| | Number | Number if | Number | false | false | +| | Number | Number if | Number | False | False | | | | can be | | | | | | | converted, | | | | | | | else false | | | | | |-----------+------------+------------+-----------+-----------+ -| Type | Array | false | false | Array | false | +| Type | Array | False | False | Array | False | | Requested |-----------+------------+------------+-----------+-----------+ -| | Scalar | Scalar | Scalar | false | false | +| | Scalar | Scalar | Scalar | False | False | | |-----------+------------+------------+-----------+-----------+ | | Undefined | String | Number | Array | Undefined | | |-----------+------------+------------+-----------+-----------+ -| | Value | false | false | false | false | -| | Cookie | | | | | +| | Value | False | False | False | False | +| | cookie | | | | | +-----------+-----------+------------+------------+-----------+-----------+ @end example @end ifplaintext @@ -32692,17 +32832,17 @@ passed to your extension function. They are: @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_valtype_t wanted, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_value_t *result); Fill in the @code{awk_value_t} structure pointed to by @code{result} -with the @code{count}'th argument. Return true if the actual -type matches @code{wanted}, false otherwise. In the latter +with the @code{count}th argument. Return true if the actual +type matches @code{wanted}, and false otherwise. In the latter case, @code{result@w{->}val_type} indicates the actual type -(@pxref{table-value-types-returned}). Counts are zero based---the first +(@pxref{table-value-types-returned}). Counts are zero-based---the first argument is numbered zero, the second one, and so on. @code{wanted} indicates the type of value expected. @item awk_bool_t set_argument(size_t count, awk_array_t array); Convert a parameter that was undefined into an array; this provides -call-by-reference for arrays. Return false if @code{count} is too big, -or if the argument's type is not undefined. @xref{Array Manipulation}, +call by reference for arrays. Return false if @code{count} is too big, +or if the argument's type is not undefined. @DBXREF{Array Manipulation} for more information on creating arrays. @end table @@ -32725,8 +32865,9 @@ allows you to create and release cached values. The following routines provide the ability to access and update global @command{awk}-level variables by name. In compiler terminology, identifiers of different kinds are termed @dfn{symbols}, thus the ``sym'' -in the routines' names. The data structure which stores information +in the routines' names. The data structure that stores information about symbols is termed a @dfn{symbol table}. +The functions are as follows: @table @code @item awk_bool_t sym_lookup(const char *name, @@ -32735,14 +32876,14 @@ about symbols is termed a @dfn{symbol table}. Fill in the @code{awk_value_t} structure pointed to by @code{result} with the value of the variable named by the string @code{name}, which is a regular C string. @code{wanted} indicates the type of value expected. -Return true if the actual type matches @code{wanted}, false otherwise. +Return true if the actual type matches @code{wanted}, and false otherwise. In the latter case, @code{result->val_type} indicates the actual type (@pxref{table-value-types-returned}). @item awk_bool_t sym_update(const char *name, awk_value_t *value); Update the variable named by the string @code{name}, which is a regular C string. The variable is added to @command{gawk}'s symbol table -if it is not there. Return true if everything worked, false otherwise. +if it is not there. Return true if everything worked, and false otherwise. Changing types (scalar to array or vice versa) of an existing variable is @emph{not} allowed, nor may this routine be used to update an array. @@ -32757,7 +32898,7 @@ cannot change any of those variables. @quotation CAUTION It is possible for the lookup of @code{PROCINFO} to fail. This happens if the @command{awk} program being run does not reference @code{PROCINFO}; -in this case @command{gawk} doesn't bother to create the array and +in this case, @command{gawk} doesn't bother to create the array and populate it. @end quotation @@ -32769,7 +32910,7 @@ to a global variable or array. It is an optimization that avoids looking up variables in @command{gawk}'s symbol table every time access is needed. This was discussed earlier, in @ref{General Data Types}. -The following functions let you work with scalar cookies. +The following functions let you work with scalar cookies: @table @code @item awk_bool_t sym_lookup_scalar(awk_scalar_t cookie, @@ -32814,18 +32955,21 @@ do_magic(int nargs, awk_value_t *result) @noindent This code looks (and is) simple and straightforward. So what's the problem? -Consider what happens if @command{awk}-level code associated with your -extension calls the @code{magic()} function (implemented in C by @code{do_magic()}), -once per record, while processing hundreds of thousands or millions of records. -The @code{MAGIC_VAR} variable is looked up in the symbol table once or twice per function call! +Well, consider what happens if @command{awk}-level code associated +with your extension calls the @code{magic()} function (implemented in +C by @code{do_magic()}), once per record, while processing hundreds +of thousands or millions of records. The @code{MAGIC_VAR} variable is +looked up in the symbol table once or twice per function call! -The symbol table lookup is really pure overhead; it is considerably more efficient -to get a cookie that represents the variable, and use that to get the variable's -value and update it as needed.@footnote{The difference is measurable and quite real. Trust us.} +The symbol table lookup is really pure overhead; it is considerably +more efficient to get a cookie that represents the variable, and use +that to get the variable's value and update it as needed.@footnote{The +difference is measurable and quite real. Trust us.} -Thus, the way to use cookies is as follows. First, install your extension's variable -in @command{gawk}'s symbol table using @code{sym_update()}, as usual. Then get a -scalar cookie for the variable using @code{sym_lookup()}: +Thus, the way to use cookies is as follows. First, install +your extension's variable in @command{gawk}'s symbol table using +@code{sym_update()}, as usual. Then get a scalar cookie for the variable +using @code{sym_lookup()}: @example static awk_scalar_t magic_var_cookie; /* cookie for MAGIC_VAR */ @@ -32880,7 +33024,7 @@ and carefully check the return values from the API functions. @subsubsection Creating and Using Cached Values The routines in this section allow you to create and release -cached values. As with scalar cookies, in theory, cached values +cached values. Like scalar cookies, in theory, cached values are not necessary. You can create numbers and strings using the functions in @ref{Constructor Functions}. You can then assign those values to variables using @code{sym_update()} @@ -32888,7 +33032,7 @@ or @code{sym_update_scalar()}, as you like. However, you can understand the point of cached values if you remember that @emph{every} string value's storage @emph{must} come from @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}. +@code{gawk_calloc()}, or @code{gawk_realloc()}. If you have 20 variables, all of which have the same string value, you must create 20 identical copies of the string.@footnote{Numeric values are clearly less problematic, requiring only a C @code{double} to store.} @@ -32899,11 +33043,11 @@ is what the routines in this section let you do. The functions are as follows: @table @code @item awk_bool_t create_value(awk_value_t *value, awk_value_cookie_t *result); -Create a cached string or numeric value from @code{value} for efficient later -assignment. -Only values of type @code{AWK_NUMBER} and @code{AWK_STRING} are allowed. Any other type -is rejected. While @code{AWK_UNDEFINED} could be allowed, doing so would -result in inferior performance. +Create a cached string or numeric value from @code{value} for +efficient later assignment. Only values of type @code{AWK_NUMBER} +and @code{AWK_STRING} are allowed. Any other type is rejected. +@code{AWK_UNDEFINED} could be allowed, but doing so would result in +inferior performance. @item awk_bool_t release_value(awk_value_cookie_t vc); Release the memory associated with a value cookie obtained @@ -32924,7 +33068,7 @@ my_extension_init() size_t long_string_len; /* code from earlier */ - @dots{} + @dots{} /* @dots{} fill in long_string and long_string_len @dots{} */ make_malloced_string(long_string, long_string_len, & value); create_value(& value, & answer_cookie); /* create cookie */ @@ -32954,11 +33098,11 @@ do_magic(int nargs, awk_value_t *result) @end example @noindent -Using value cookies in this way saves considerable storage, since all of +Using value cookies in this way saves considerable storage, as all of @code{VAR1} through @code{VAR100} share the same value. You might be wondering, ``Is this sharing problematic? -What happens if @command{awk} code assigns a new value to @code{VAR1}, +What happens if @command{awk} code assigns a new value to @code{VAR1}; are all the others changed too?'' That's a great question. The answer is that no, it's not a problem. @@ -32976,7 +33120,7 @@ you should release any cached values that you created, using @subsection Array Manipulation @cindex array manipulation in extensions -The primary data structure@footnote{Okay, the only data structure.} in @command{awk} +The primary data structure@footnote{OK, the only data structure.} in @command{awk} is the associative array (@pxref{Arrays}). Extensions need to be able to manipulate @command{awk} arrays. The API provides a number of data structures for working with arrays, @@ -32997,7 +33141,7 @@ both work with and create true arrays of arrays (@pxref{General Data Types}). @node Array Data Types @subsubsection Array Data Types -The data types associated with arrays are listed below. +The data types associated with arrays are as follows: @table @code @item typedef void *awk_array_t; @@ -33062,7 +33206,7 @@ modify them. @node Array Functions @subsubsection Array Functions -The following functions relate to individual array elements. +The following functions relate to individual array elements: @table @code @item awk_bool_t get_element_count(awk_array_t a_cookie, size_t *count); @@ -33081,13 +33225,13 @@ Return false if @code{wanted} does not match the actual type or if @code{index} is not in the array (@pxref{table-value-types-returned}). The value for @code{index} can be numeric, in which case @command{gawk} -converts it to a string. Using non-integral values is possible, but +converts it to a string. Using nonintegral values is possible, but requires that you understand how such values are converted to strings -(@pxref{Conversion}); thus using integral values is safest. +(@pxref{Conversion}); thus, using integral values is safest. -As with @emph{all} strings passed into @code{gawk} from an extension, +As with @emph{all} strings passed into @command{gawk} from an extension, the string value of @code{index} must come from @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}, and +@code{gawk_calloc()}, or @code{gawk_realloc()}, and @command{gawk} releases the storage. @item awk_bool_t set_array_element(awk_array_t a_cookie, @@ -33116,7 +33260,7 @@ The following functions relate to arrays as a whole: @table @code @item awk_array_t create_array(void); Create a new array to which elements may be added. -@xref{Creating Arrays}, for a discussion of how to +@DBXREF{Creating Arrays} for a discussion of how to create a new array and add elements to it. @item awk_bool_t clear_array(awk_array_t a_cookie); @@ -33143,7 +33287,7 @@ flatten an array and work with it. @item awk_bool_t release_flattened_array(awk_array_t a_cookie, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_flat_array_t *data); When done with a flattened array, release the storage using this function. -You must pass in both the original array cookie, and the address of +You must pass in both the original array cookie and the address of the created @code{awk_flat_array_t} structure. The function returns true upon success, false otherwise. @end table @@ -33151,9 +33295,9 @@ The function returns true upon success, false otherwise. @node Flattening Arrays @subsubsection Working With All The Elements of an Array -To @dfn{flatten} an array is create a structure that +To @dfn{flatten} an array is to create a structure that represents the full array in a fashion that makes it easy -for C code to traverse the entire array. Test code +for C code to traverse the entire array. Some of the code in @file{extension/testext.c} does this, and also serves as a nice example showing how to use the APIs. @@ -33210,9 +33354,9 @@ dump_array_and_delete(int nargs, awk_value_t *result) @end example The function then proceeds in steps, as follows. First, retrieve -the name of the array, passed as the first argument. Then -retrieve the array itself. If either operation fails, print -error messages and return: +the name of the array, passed as the first argument, followed by +the array itself. If either operation fails, print an +error message and return: @example /* get argument named array as flat array and print it */ @@ -33248,7 +33392,7 @@ and print it: @end example The third step is to actually flatten the array, and then -to double check that the count in the @code{awk_flat_array_t} +to double-check that the count in the @code{awk_flat_array_t} is the same as the count just retrieved: @example @@ -33269,7 +33413,7 @@ is the same as the count just retrieved: The fourth step is to retrieve the index of the element to be deleted, which was passed as the second argument. Remember that argument counts passed to @code{get_argument()} -are zero-based, thus the second argument is numbered one: +are zero-based, and thus the second argument is numbered one: @example if (! get_argument(1, AWK_STRING, & value3)) @{ @@ -33284,7 +33428,7 @@ element values. In addition, upon finding the element with the index that is supposed to be deleted, the function sets the @code{AWK_ELEMENT_DELETE} bit in the @code{flags} field of the element. When the array is released, @command{gawk} -traverses the flattened array, and deletes any elements which +traverses the flattened array, and deletes any elements that have this flag bit set: @example @@ -33319,7 +33463,7 @@ code) once you have called @code{release_flattened_array()}: @} @end example -Finally, since everything was successful, the function sets the +Finally, because everything was successful, the function sets the return value to success, and returns: @example @@ -33354,7 +33498,7 @@ code can access them and manipulate them. There are two important points about creating arrays from extension code: -@enumerate 1 +@itemize @value{BULLET} @item You must install a new array into @command{gawk}'s symbol table immediately upon creating it. Once you have done so, @@ -33376,7 +33520,7 @@ using @code{sym_update()}, or install it as an element in a previously existing array using @code{set_array_element()}. We show example code shortly. @item -Due to gawk internals, after using @code{sym_update()} to install an array +Due to @command{gawk} internals, after using @code{sym_update()} to install an array into @command{gawk}, you have to retrieve the array cookie from the value passed in to @command{sym_update()} before doing anything else with it, like so: @@ -33396,7 +33540,7 @@ new_array = val.array_cookie; /* YOU MUST DO THIS */ If installing an array as a subarray, you must also retrieve the value of the array cookie after the call to @code{set_element()}. -@end enumerate +@end itemize The following C code is a simple test extension to create an array with two regular elements and with a subarray. The leading @code{#include} @@ -33515,7 +33659,7 @@ dl_load_func(func_table, testarray, "") @end ignore @end example -Here is sample script that loads the extension +Here is a sample script that loads the extension and then dumps the array: @example @@ -33545,9 +33689,78 @@ $ @kbd{AWKLIBPATH=$PWD ./gawk -f subarray.awk} @end example @noindent -(@xref{Finding Extensions}, for more information on the +(@DBXREF{Finding Extensions} for more information on the @env{AWKLIBPATH} environment variable.) +@node Redirection API +@subsection Accessing and Manipulating Redirections + +The following function allows extensions to access and manipulate redirections. + +@table @code +@item awk_bool_t get_file(const char *name, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ size_t name_len, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const char *filetype, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ int fd, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const awk_input_buf_t **ibufp, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const awk_output_buf_t **obufp); +Look up a file in @command{gawk}'s internal redirection table. +If @code{name} is @code{NULL} or @code{name_len} is zero, return +data for the currently open input file corresponding to @code{FILENAME}. +(This does not access the @code{filetype} argument, so that may be undefined). +If the file is not already open, attempt to open it. +The @code{filetype} argument must be zero-terminated and should be one of: + +@table @code +@item ">" +A file opened for output. + +@item ">>" +A file opened for append. + +@item "<" +A file opened for input. + +@item "|>" +A pipe opened for output. + +@item "|<" +A pipe opened for input. + +@item "|&" +A two-way coprocess. +@end table + +On error, return a @code{false} value. Otherwise, return +@code{true}, and return additional information about the redirection +in the @code{ibufp} and @code{obufp} pointers. For input +redirections, the @code{*ibufp} value should be non-@code{NULL}, +and @code{*obufp} should be @code{NULL}. For output redirections, +the @code{*obufp} value should be non-@code{NULL}, and @code{*ibufp} +should be @code{NULL}. For two-way coprocesses, both values should +be non-@code{NULL}. + +In the usual case, the extension is interested in @code{(*ibufp)->fd} +and/or @code{fileno((*obufp)->fp)}. If the file is not already +open, and the @code{fd} argument is non-negative, @command{gawk} +will use that file descriptor instead of opening the file in the +usual way. If @code{fd} is non-negative, but the file exists already, +@command{gawk} ignores @code{fd} and returns the existing file. It is +the caller's responsibility to notice that neither the @code{fd} in +the returned @code{awk_input_buf_t} nor the @code{fd} in the returned +@code{awk_output_buf_t} matches the requested value. + +Note that supplying a file descriptor is currently @emph{not} supported +for pipes. However, supplying a file descriptor should work for input, +output, append, and two-way (coprocess) sockets. If @code{filetype} +is two-way, @command{gawk} assumes that it is a socket! Note that in +the two-way case, the input and output file descriptors may differ. +To check for success, you must check whether either matches. +@end table + +It is anticipated that this API function will be used to implement I/O +multiplexing and a socket library. + @node Extension API Variables @subsection API Variables @@ -33572,10 +33785,10 @@ The API versions are available at compile time as constants: @table @code @item GAWK_API_MAJOR_VERSION -The major version of the API. +The major version of the API @item GAWK_API_MINOR_VERSION -The minor version of the API. +The minor version of the API @end table The minor version increases when new functions are added to the API. Such @@ -33593,14 +33806,14 @@ constant integers: @table @code @item api->major_version -The major version of the running @command{gawk}. +The major version of the running @command{gawk} @item api->minor_version -The minor version of the running @command{gawk}. +The minor version of the running @command{gawk} @end table It is up to the extension to decide if there are API incompatibilities. -Typically a check like this is enough: +Typically, a check like this is enough: @example if (api->major_version != GAWK_API_MAJOR_VERSION @@ -33614,7 +33827,7 @@ if (api->major_version != GAWK_API_MAJOR_VERSION @end example Such code is included in the boilerplate @code{dl_load_func()} macro -provided in @file{gawkapi.h} (discussed later, in +provided in @file{gawkapi.h} (discussed in @ref{Extension API Boilerplate}). @node Extension API Informational Variables @@ -33631,8 +33844,7 @@ whether the corresponding command-line options were enabled when This variable is true if @command{gawk} was invoked with @option{--debug} option. @item do_lint -This variable is true if @command{gawk} was invoked with @option{--lint} option -(@pxref{Options}). +This variable is true if @command{gawk} was invoked with @option{--lint} option. @item do_mpfr This variable is true if @command{gawk} was invoked with @option{--bignum} option. @@ -33657,12 +33869,12 @@ The others should not change during execution. As mentioned earlier (@pxref{Extension Mechanism Outline}), the function definitions as presented are really macros. To use these macros, your extension must provide a small amount of boilerplate code (variables and -functions) towards the top of your source file, using pre-defined names -as described below. The boilerplate needed is also provided in comments +functions) toward the top of your source file, using predefined names +as described here. The boilerplate needed is also provided in comments in the @file{gawkapi.h} header file: @example -/* Boiler plate code: */ +/* Boilerplate code: */ int plugin_is_GPL_compatible; static gawk_api_t *const api; @@ -33721,7 +33933,7 @@ to @code{NULL}, or to point to a string giving the name and version of your extension. @item static awk_ext_func_t func_table[] = @{ @dots{} @}; -This is an array of one or more @code{awk_ext_func_t} structures +This is an array of one or more @code{awk_ext_func_t} structures, as described earlier (@pxref{Extension Functions}). It can then be looped over for multiple calls to @code{add_ext_func()}. @@ -33746,7 +33958,7 @@ This macro expands to a @code{dl_load()} function that performs all the necessary initializations. @end table -The point of the all the variables and arrays is to let the +The point of all the variables and arrays is to let the @code{dl_load()} function (from the @code{dl_load_func()} macro) do all the standard work. It does the following: @@ -33781,7 +33993,7 @@ Compiled extensions have to be installed in a directory where built in the default fashion, the directory in which to find extensions is @file{/usr/local/lib/gawk}. You can also specify a search path with a list of directories to search for compiled extensions. -@xref{AWKLIBPATH Variable}, for more information. +@DBXREF{AWKLIBPATH Variable} for more information. @node Extension Example @section Example: Some File Functions @@ -33789,7 +34001,7 @@ path with a list of directories to search for compiled extensions. @quotation @i{No matter where you go, there you are.} -@author Buckaroo Bonzai +@author Buckaroo Banzai @end quotation @c It's enough to show chdir and stat, no need for fts @@ -33852,7 +34064,7 @@ the @code{stat()} fails. It fills in the following elements: @table @code @item "name" -The name of the file that was @code{stat()}'ed. +The name of the file that was @code{stat()}ed. @item "dev" @itemx "ino" @@ -33908,7 +34120,7 @@ interprocess communications). The file is a directory. @item "fifo" -The file is a named-pipe (also known as a FIFO). +The file is a named pipe (also known as a FIFO). @item "file" The file is just a regular file. @@ -33931,7 +34143,7 @@ For some other systems, @dfn{a priori} knowledge is used to provide a value. Where no value can be determined, it defaults to 512. @end table -Several additional elements may be present depending upon the operating +Several additional elements may be present, depending upon the operating system and the type of the file. You can test for them in your @command{awk} program by using the @code{in} operator (@pxref{Reference to Elements}): @@ -33961,10 +34173,10 @@ edited slightly for presentation. See @file{extension/filefuncs.c} in the @command{gawk} distribution for the complete version.} The file includes a number of standard header files, and then includes -the @file{gawkapi.h} header file which provides the API definitions. -Those are followed by the necessary variable declarations +the @file{gawkapi.h} header file, which provides the API definitions. +Those are followed by the necessary variable declarations to make use of the API macros and boilerplate code -(@pxref{Extension API Boilerplate}). +(@pxref{Extension API Boilerplate}): @example #ifdef HAVE_CONFIG_H @@ -34002,10 +34214,10 @@ int plugin_is_GPL_compatible; @cindex programming conventions, @command{gawk} extensions By convention, for an @command{awk} function @code{foo()}, the C function that implements it is called @code{do_foo()}. The function should have -two arguments: the first is an @code{int} usually called @code{nargs}, +two arguments. The first is an @code{int}, usually called @code{nargs}, that represents the number of actual arguments for the function. -The second is a pointer to an @code{awk_value_t}, usually named -@code{result}. +The second is a pointer to an @code{awk_value_t} structure, usually named +@code{result}: @example /* do_chdir --- provide dynamically loaded chdir() function for gawk */ @@ -34025,13 +34237,13 @@ do_chdir(int nargs, awk_value_t *result) @end example The @code{newdir} -variable represents the new directory to change to, retrieved +variable represents the new directory to change to, which is retrieved with @code{get_argument()}. Note that the first argument is numbered zero. If the argument is retrieved successfully, the function calls the @code{chdir()} system call. If the @code{chdir()} fails, @code{ERRNO} -is updated. +is updated: @example if (get_argument(0, AWK_STRING, & newdir)) @{ @@ -34050,7 +34262,7 @@ Finally, the function returns the return value to the @command{awk} level: The @code{stat()} extension is more involved. First comes a function that turns a numeric mode into a printable representation -(e.g., 644 becomes @samp{-rw-r--r--}). This is omitted here for brevity: +(e.g., octal @code{0644} becomes @samp{-rw-r--r--}). This is omitted here for brevity: @example /* format_mode --- turn a stat mode field into something readable */ @@ -34106,9 +34318,9 @@ array_set_numeric(awk_array_t array, const char *sub, double num) The following function does most of the work to fill in the @code{awk_array_t} result array with values obtained -from a valid @code{struct stat}. It is done in a separate function +from a valid @code{struct stat}. This work is done in a separate function to support the @code{stat()} function for @command{gawk} and also -to support the @code{fts()} extension which is included in +to support the @code{fts()} extension, which is included in the same file but whose code is not shown here (@pxref{Extension Sample File Functions}). @@ -34229,10 +34441,10 @@ the @code{stat()} system call instead of the @code{lstat()} system call. This is done by using a function pointer: @code{statfunc}. @code{statfunc} is initialized to point to @code{lstat()} (instead of @code{stat()}) to get the file information, in case the file is a -symbolic link. However, if there were three arguments, @code{statfunc} -is set point to @code{stat()}, instead. +symbolic link. However, if the third argument is included, @code{statfunc} +is set to point to @code{stat()}, instead. -Here is the @code{do_stat()} function. It starts with +Here is the @code{do_stat()} function, which starts with variable declarations and argument checking: @ignore @@ -34286,7 +34498,7 @@ Next, it gets the information for the file. If the called function /* always empty out the array */ clear_array(array); - /* stat the file, if error, set ERRNO and return */ + /* stat the file; if error, set ERRNO and return */ ret = statfunc(name, & sbuf); if (ret < 0) @{ update_ERRNO_int(errno); @@ -34308,7 +34520,9 @@ Finally, it's necessary to provide the ``glue'' that loads the new function(s) into @command{gawk}. The @code{filefuncs} extension also provides an @code{fts()} -function, which we omit here. For its sake there is an initialization +function, which we omit here +(@pxref{Extension Sample File Functions}). +For its sake, there is an initialization function: @example @@ -34347,7 +34561,7 @@ dl_load_func(func_table, filefuncs, "") And that's it! @node Using Internal File Ops -@subsection Integrating The Extensions +@subsection Integrating the Extensions @cindex @command{gawk}, interpreter@comma{} adding code to Now that the code is written, it must be possible to add it at @@ -34356,9 +34570,9 @@ code must be compiled. Assuming that the functions are in a file named @file{filefuncs.c}, and @var{idir} is the location of the @file{gawkapi.h} header file, the following steps@footnote{In practice, you would probably want to -use the GNU Autotools---Automake, Autoconf, Libtool, and @command{gettext}---to +use the GNU Autotools (Automake, Autoconf, Libtool, and @command{gettext}) to configure and build your libraries. Instructions for doing so are beyond -the scope of this @value{DOCUMENT}. @xref{gawkextlib}, for Internet links to +the scope of this @value{DOCUMENT}. @DBXREF{gawkextlib} for Internet links to the tools.} create a GNU/Linux shared library: @example @@ -34366,7 +34580,7 @@ $ @kbd{gcc -fPIC -shared -DHAVE_CONFIG_H -c -O -g -I@var{idir} filefuncs.c} $ @kbd{gcc -o filefuncs.so -shared filefuncs.o} @end example -Once the library exists, it is loaded by using the @code{@@load} keyword. +Once the library exists, it is loaded by using the @code{@@load} keyword: @example # file testff.awk @@ -34430,13 +34644,14 @@ $ @kbd{AWKLIBPATH=$PWD gawk -f testff.awk} @end example @node Extension Samples -@section The Sample Extensions In The @command{gawk} Distribution +@section The Sample Extensions in the @command{gawk} Distribution @cindex extensions distributed with @command{gawk} -This @value{SECTION} provides brief overviews of the sample extensions +This @value{SECTION} provides a brief overview of the sample extensions that come in the @command{gawk} distribution. Some of them are intended -for production use, such the @code{filefuncs}, @code{readdir} and @code{inplace} extensions. -Others mainly provide example code that shows how to use the extension API. +for production use (e.g., the @code{filefuncs}, @code{readdir}, and +@code{inplace} extensions). Others mainly provide example code that +shows how to use the extension API. @menu * Extension Sample File Functions:: The file functions sample. @@ -34457,9 +34672,9 @@ Others mainly provide example code that shows how to use the extension API. @end menu @node Extension Sample File Functions -@subsection File Related Functions +@subsection File-Related Functions -The @code{filefuncs} extension provides three different functions, as follows: +The @code{filefuncs} extension provides three different functions, as follows. The usage is: @table @asis @@ -34470,15 +34685,15 @@ This is how you load the extension. @item @code{result = chdir("/some/directory")} The @code{chdir()} function is a direct hook to the @code{chdir()} system call to change the current directory. It returns zero -upon success or less than zero upon error. In the latter case it updates -@code{ERRNO}. +upon success or a value less than zero upon error. +In the latter case, it updates @code{ERRNO}. @cindex @code{stat()} extension function @item @code{result = stat("/some/path", statdata} [@code{, follow}]@code{)} The @code{stat()} function provides a hook into the @code{stat()} system call. -It returns zero upon success or less than zero upon error. -In the latter case it updates @code{ERRNO}. +It returns zero upon success or a value less than zero upon error. +In the latter case, it updates @code{ERRNO}. By default, it uses the @code{lstat()} system call. However, if passed a third argument, it uses @code{stat()} instead. @@ -34504,10 +34719,10 @@ array with information retrieved from the filesystem, as follows: @item @code{"major"} @tab @code{st_major} @tab Device files @item @code{"minor"} @tab @code{st_minor} @tab Device files @item @code{"blksize"} @tab @code{st_blksize} @tab All -@item @code{"pmode"} @tab A human-readable version of the mode value, such as printed by -@command{ls}. For example, @code{"-rwxr-xr-x"} @tab All +@item @code{"pmode"} @tab A human-readable version of the mode value, like that printed by +@command{ls} (for example, @code{"-rwxr-xr-x"}) @tab All @item @code{"linkval"} @tab The value of the symbolic link @tab Symbolic links -@item @code{"type"} @tab The type of the file as a string. One of +@item @code{"type"} @tab The type of the file as a string---one of @code{"file"}, @code{"blockdev"}, @code{"chardev"}, @@ -34517,16 +34732,16 @@ array with information retrieved from the filesystem, as follows: @code{"symlink"}, @code{"door"}, or -@code{"unknown"}. -Not all systems support all file types. @tab All +@code{"unknown"} +(not all systems support all file types) @tab All @end multitable @cindex @code{fts()} extension function @item @code{flags = or(FTS_PHYSICAL, ...)} @itemx @code{result = fts(pathlist, flags, filedata)} Walk the file trees provided in @code{pathlist} and fill in the -@code{filedata} array as described below. @code{flags} is the bitwise -OR of several predefined values, also described below. +@code{filedata} array, as described next. @code{flags} is the bitwise +OR of several predefined values, also described in a moment. Return zero if there were no errors, otherwise return @minus{}1. @end table @@ -34574,17 +34789,18 @@ whether or not @code{FTS_LOGICAL} is set. By default, the C library @code{fts()} routines do not return entries for @file{.} (dot) and @file{..} (dot-dot). This option causes entries for dot-dot to also be included. (The extension always includes an entry -for dot, see below.) +for dot; more on this in a moment.) @item FTS_XDEV During a traversal, do not cross onto a different mounted filesystem. @end table @item filedata -The @code{filedata} array is first cleared. Then, @code{fts()} creates +The @code{filedata} array holds the results. +@code{fts()} first clears it. Then it creates an element in @code{filedata} for every element in @code{pathlist}. The index is the name of the directory or file given in @code{pathlist}. -The element for this index is itself an array. There are two cases. +The element for this index is itself an array. There are two cases: @c nested table @table @emph @@ -34610,8 +34826,8 @@ contain an element named @code{"error"}, which is a string describing the error. @item The path is a directory In this case, the array contains one element for each entry in the -directory. If an entry is a file, that element is as for files, just -described. If the entry is a directory, that element is (recursively), +directory. If an entry is a file, that element is the same as for files, just +described. If the entry is a directory, that element is (recursively) an array describing the subdirectory. If @code{FTS_SEEDOT} was provided in the flags, then there will also be an element named @code{".."}. This element will be an array containing the data as provided by @code{stat()}. @@ -34623,15 +34839,15 @@ for a file: @code{"path"}, @code{"stat"}, and @code{"error"}. @end table The @code{fts()} function returns zero if there were no errors. -Otherwise it returns @minus{}1. +Otherwise, it returns @minus{}1. @quotation NOTE The @code{fts()} extension does not exactly mimic the interface of the C library @code{fts()} routines, choosing instead to provide an interface that is based on associative arrays, which is more comfortable to use from an @command{awk} program. This includes the -lack of a comparison function, since @command{gawk} already provides -powerful array sorting facilities. While an @code{fts_read()}-like +lack of a comparison function, because @command{gawk} already provides +powerful array sorting facilities. Although an @code{fts_read()}-like interface could have been provided, this felt less natural than simply creating a multidimensional array to represent the file hierarchy and its information. @@ -34641,7 +34857,7 @@ See @file{test/fts.awk} in the @command{gawk} distribution for an example use of the @code{fts()} extension function. @node Extension Sample Fnmatch -@subsection Interface To @code{fnmatch()} +@subsection Interface to @code{fnmatch()} This extension provides an interface to the C library @code{fnmatch()} function. The usage is: @@ -34654,10 +34870,10 @@ This is how you load the extension. @item result = fnmatch(pattern, string, flags) The return value is zero on success, @code{FNM_NOMATCH} if the string did not match the pattern, or -a different non-zero value if an error occurred. +a different nonzero value if an error occurred. @end table -Besides the @code{fnmatch()} function, the @code{fnmatch} extension +In addition to the @code{fnmatch()} function, the @code{fnmatch} extension adds one constant (@code{FNM_NOMATCH}), and an array of flag values named @code{FNM}. @@ -34665,17 +34881,17 @@ The arguments to @code{fnmatch()} are: @table @code @item pattern -The @value{FN} wildcard to match. +The @value{FN} wildcard to match @item string -The @value{FN} string. +The @value{FN} string @item flag Either zero, or the bitwise OR of one or more of the -flags in the @code{FNM} array. +flags in the @code{FNM} array @end table -The flags are follows: +The flags are as follows: @multitable @columnfractions .25 .75 @headitem Array element @tab Corresponding flag defined by @code{fnmatch()} @@ -34698,9 +34914,9 @@ if (fnmatch("*.a", "foo.c", flags) == FNM_NOMATCH) @end example @node Extension Sample Fork -@subsection Interface To @code{fork()}, @code{wait()} and @code{waitpid()} +@subsection Interface to @code{fork()}, @code{wait()}, and @code{waitpid()} -The @code{fork} extension adds three functions, as follows. +The @code{fork} extension adds three functions, as follows: @table @code @item @@load "fork" @@ -34709,14 +34925,14 @@ This is how you load the extension. @cindex @code{fork()} extension function @item pid = fork() This function creates a new process. The return value is zero in the -child and the process-ID number of the child in the parent, or @minus{}1 +child and the process ID number of the child in the parent, or @minus{}1 upon error. In the latter case, @code{ERRNO} indicates the problem. In the child, @code{PROCINFO["pid"]} and @code{PROCINFO["ppid"]} are updated to reflect the correct values. @cindex @code{waitpid()} extension function @item ret = waitpid(pid) -This function takes a numeric argument, which is the process-ID to +This function takes a numeric argument, which is the process ID to wait for. The return value is that of the @code{waitpid()} system call. @@ -34744,8 +34960,8 @@ else @subsection Enabling In-Place File Editing @cindex @code{inplace} extension -The @code{inplace} extension emulates GNU @command{sed}'s @option{-i} option -which performs ``in place'' editing of each input file. +The @code{inplace} extension emulates GNU @command{sed}'s @option{-i} option, +which performs ``in-place'' editing of each input file. It uses the bundled @file{inplace.awk} include file to invoke the extension properly: @@ -34759,11 +34975,16 @@ properly: # Please set INPLACE_SUFFIX to make a backup copy. For example, you may # want to set INPLACE_SUFFIX to .bak on the command line or in a BEGIN rule. +# N.B. We call inplace_end() in the BEGINFILE and END rules so that any +# actions in an ENDFILE rule will be redirected as expected. + BEGINFILE @{ - inplace_begin(FILENAME, INPLACE_SUFFIX) + if (_inplace_filename != "") + inplace_end(_inplace_filename, INPLACE_SUFFIX) + inplace_begin(_inplace_filename = FILENAME, INPLACE_SUFFIX) @} -ENDFILE @{ +END @{ inplace_end(FILENAME, INPLACE_SUFFIX) @} @end group @@ -34778,6 +34999,10 @@ If @code{INPLACE_SUFFIX} is not an empty string, the original file is linked to a backup @value{FN} created by appending that suffix. Finally, the temporary file is renamed to the original @value{FN}. +The @code{_inplace_filename} variable serves to keep track of the +current filename so as to not invoke @code{inplace_end()} before +processing the first file. + If any error occurs, the extension issues a fatal error to terminate processing immediately without damaging the original file. @@ -34798,7 +35023,7 @@ $ @kbd{gawk -i inplace -v INPLACE_SUFFIX=.bak '@{ gsub(/foo/, "bar") @}} @subsection Character and Numeric values: @code{ord()} and @code{chr()} The @code{ordchr} extension adds two functions, named -@code{ord()} and @code{chr()}, as follows. +@code{ord()} and @code{chr()}, as follows: @table @code @item @@load "ordchr" @@ -34841,14 +35066,14 @@ they are read, with each entry returned as a record. The record consists of three fields. The first two are the inode number and the @value{FN}, separated by a forward slash character. On systems where the directory entry contains the file type, the record -has a third field (also separated by a slash) which is a single letter -indicating the type of the file. The letters are file types are shown -in @ref{table-readdir-file-types}. +has a third field (also separated by a slash), which is a single letter +indicating the type of the file. The letters and their corresponding file +types are shown in @ref{table-readdir-file-types}. @float Table,table-readdir-file-types -@caption{File Types Returned By The @code{readdir} Extension} +@caption{File types returned by the @code{readdir} extension} @multitable @columnfractions .1 .9 -@headitem Letter @tab File Type +@headitem Letter @tab File type @item @code{b} @tab Block device @item @code{c} @tab Character device @item @code{d} @tab Directory @@ -34876,14 +35101,14 @@ Here is an example: @@load "readdir" @dots{} BEGIN @{ FS = "/" @} -@{ print "file name is", $2 @} +@{ print "@value{FN} is", $2 @} @end example @node Extension Sample Revout @subsection Reversing Output The @code{revoutput} extension adds a simple output wrapper that reverses -the characters in each output line. It's main purpose is to show how to +the characters in each output line. Its main purpose is to show how to write an output wrapper, although it may be mildly amusing for the unwary. Here is an example: @@ -34897,15 +35122,14 @@ BEGIN @{ @} @end example -The output from this program is: -@samp{cinap t'nod}. +The output from this program is @samp{cinap t'nod}. @node Extension Sample Rev2way @subsection Two-Way I/O Example The @code{revtwoway} extension adds a simple two-way processor that reverses the characters in each line sent to it for reading back by -the @command{awk} program. It's main purpose is to show how to write +the @command{awk} program. Its main purpose is to show how to write a two-way processor, although it may also be mildly amusing. The following example shows how to use it: @@ -34932,7 +35156,7 @@ is: @samp{cinap t'nod}. @node Extension Sample Read write array -@subsection Dumping and Restoring An Array +@subsection Dumping and Restoring an Array The @code{rwarray} extension adds two functions, named @code{writea()} and @code{reada()}, as follows: @@ -34953,12 +35177,12 @@ success, or zero upon failure. @code{reada()} is the inverse of @code{writea()}; it reads the file named as its first argument, filling in the array named as the second argument. It clears the array first. -Here too, the return value is one on success and zero upon failure. +Here too, the return value is one on success, or zero upon failure. @end table The array created by @code{reada()} is identical to that written by @code{writea()} in the sense that the contents are the same. However, -due to implementation issues, the array traversal order of the recreated +due to implementation issues, the array traversal order of the re-created array is likely to be different from that of the original array. As array traversal order in @command{awk} is by default undefined, this is (technically) not a problem. If you need to guarantee a particular traversal @@ -34966,7 +35190,7 @@ order, use the array sorting features in @command{gawk} to do so (@pxref{Array Sorting}). The file contains binary data. All integral values are written in network -byte order. However, double precision floating-point values are written +byte order. However, double-precision floating-point values are written as native binary data. Thus, arrays containing only string data can theoretically be dumped on systems with one byte order and restored on systems with a different one, but this has not been tried. @@ -34982,7 +35206,7 @@ ret = reada("arraydump.bin", array) @end example @node Extension Sample Readfile -@subsection Reading An Entire File +@subsection Reading an Entire File The @code{readfile} extension adds a single function named @code{readfile()}, and an input parser: @@ -35029,7 +35253,7 @@ This is how you load the extension. @cindex @code{gettimeofday()} extension function @item the_time = gettimeofday() Return the time in seconds that has elapsed since 1970-01-01 UTC as a -floating point value. If the time is unavailable on this platform, return +floating-point value. If the time is unavailable on this platform, return @minus{}1 and set @code{ERRNO}. The returned time should have sub-second precision, but the actual precision may vary based on the platform. If the standard C @code{gettimeofday()} system call is available on this @@ -35041,7 +35265,7 @@ it tries to use @code{GetSystemTimeAsFileTime()}. Attempt to sleep for @var{seconds} seconds. If @var{seconds} is negative, or the attempt to sleep fails, return @minus{}1 and set @code{ERRNO}. Otherwise, return zero after sleeping for the indicated amount of time. -Note that @var{seconds} may be a floating-point (non-integral) value. +Note that @var{seconds} may be a floating-point (nonintegral) value. Implementation details: depending on platform availability, this function tries to use @code{nanosleep()} or @code{select()} to implement the delay. @end table @@ -35068,26 +35292,35 @@ project provides a number of @command{gawk} extensions, including one for processing XML files. This is the evolution of the original @command{xgawk} (XML @command{gawk}) project. -As of this writing, there are five extensions: +As of this writing, there are seven extensions: @itemize @value{BULLET} @item -GD graphics library extension. +@code{errno} extension + +@item +GD graphics library extension @item -PDF extension. +MPFR library extension +(this provides access to a number of MPFR functions that @command{gawk}'s +native MPFR support does not) @item -PostgreSQL extension. +PDF extension @item -MPFR library extension. -This provides access to a number of MPFR functions which @command{gawk}'s -native MPFR support does not. +PostgreSQL extension + +@item +Redis extension + +@item +Select extension @item XML parser extension, using the @uref{http://expat.sourceforge.net, Expat} -XML parsing library. +XML parsing library @end itemize @cindex @command{git} utility @@ -35133,14 +35366,14 @@ make install @ii{Install the extensions} If you have installed @command{gawk} in the standard way, then you will likely not need the @option{--with-gawk} option when configuring -@code{gawkextlib}. You may also need to use the @command{sudo} utility +@code{gawkextlib}. You may need to use the @command{sudo} utility to install both @command{gawk} and @code{gawkextlib}, depending upon how your system works. If you write an extension that you wish to share with other -@command{gawk} users, please consider doing so through the +@command{gawk} users, consider doing so through the @code{gawkextlib} project. -See the project's web site for more information. +See the project's website for more information. @node Extension summary @section Summary @@ -35148,7 +35381,7 @@ See the project's web site for more information. @itemize @value{BULLET} @item You can write extensions (sometimes called plug-ins) for @command{gawk} -in C or C++ using the Application Programming Interface (API) defined +in C or C++ using the application programming interface (API) defined by the @command{gawk} developers. @item @@ -35158,7 +35391,7 @@ named @code{plugin_is_GPL_compatible}. @item Communication between @command{gawk} and an extension is two-way. -@command{gawk} passes a @code{struct} to the extension which contains +@command{gawk} passes a @code{struct} to the extension that contains various data fields and function pointers. The extension can then call into @command{gawk} via the supplied function pointers to accomplish certain tasks. @@ -35171,7 +35404,7 @@ By convention, implementation functions are named @code{do_@var{XXXX}()} for some @command{awk}-level function @code{@var{XXXX}()}. @item -The API is defined in a header file named @file{gawkpi.h}. You must include +The API is defined in a header file named @file{gawkapi.h}. You must include a number of standard header files @emph{before} including it in your source file. @item @@ -35179,44 +35412,44 @@ API function pointers are provided for the following kinds of operations: @itemize @value{BULLET} @item -Allocating, reallocating, and releasing memory. +Allocating, reallocating, and releasing memory @item -Registration functions. You may register +Registration functions (you may register extension functions, exit callbacks, a version string, input parsers, output wrappers, -and two-way processors. +and two-way processors) @item -Printing fatal, warning, and ``lint'' warning messages. +Printing fatal, warning, and ``lint'' warning messages @item -Updating @code{ERRNO}, or unsetting it. +Updating @code{ERRNO}, or unsetting it @item Accessing parameters, including converting an undefined parameter into -an array. +an array @item -Symbol table access: retrieving a global variable, creating one, -or changing one. +Symbol table access (retrieving a global variable, creating one, +or changing one) @item Creating and releasing cached values; this provides an efficient way to use values for multiple variables and -can be a big performance win. +can be a big performance win @item -Manipulating arrays: -retrieving, adding, deleting, and modifying elements; +Manipulating arrays +(retrieving, adding, deleting, and modifying elements; getting the count of elements in an array; creating a new array; clearing an array; and -flattening an array for easy C style looping over all its indices and elements. +flattening an array for easy C-style looping over all its indices and elements) @end itemize @item @@ -35224,7 +35457,7 @@ The API defines a number of standard data types for representing @command{awk} values, array elements, and arrays. @item -The API provide convenience functions for constructing values. +The API provides convenience functions for constructing values. It also provides memory management functions to ensure compatibility between memory allocated by @command{gawk} and memory allocated by an extension. @@ -35245,13 +35478,13 @@ that loaded it. @item It is easiest to start a new extension by copying the boilerplate code -described in this @value{CHAPTER}. Macros in the @file{gawkapi.h} make -this easier to do. +described in this @value{CHAPTER}. Macros in the @file{gawkapi.h} header +file make this easier to do. @item The @command{gawk} distribution includes a number of small but useful -sample extensions. The @code{gawkextlib} project includes several more, -larger, extensions. If you wish to write an extension and contribute it +sample extensions. The @code{gawkextlib} project includes several more +(larger) extensions. If you wish to write an extension and contribute it to the community of @command{gawk} users, the @code{gawkextlib} project is the place to do so. @@ -35267,6 +35500,24 @@ Add functions to implement system calls such as @code{chown()}, @code{chmod()}, and @code{umask()} to the file operations extension presented in @ref{Internal File Ops}. +@c Idea from comp.lang.awk, February 2015 +@item +Write an input parser that prints a prompt if the input is +a from a ``terminal'' device. You can use the @code{isatty()} +function to tell if the input file is a terminal. (Hint: this function +is usually expensive to call; try to call it just once.) +The content of the prompt should come from a variable settable +by @command{awk}-level code. +You can write the prompt to stanard error. However, +for best results, open a new file descriptor (or file pointer) +on @file{/dev/tty} and print the prompt there, in case standard +error has been redirected. + +Why is standard error a better +choice than standard output for writing the prompt? +Which reading mechanism should you replace, the one to get +a record, or the one to read raw bytes? + @item (Hard.) How would you provide namespaces in @command{gawk}, so that the @@ -35295,34 +35546,34 @@ and the Glossary: @end ifclear @ifset FOR_PRINT -Part IV contains two appendices and the license that +Part IV contains three appendices, the last of which is the license that covers the @command{gawk} source code: @end ifset @itemize @value{BULLET} @item -@ref{Language History}. +@ref{Language History} @item -@ref{Installation}. +@ref{Installation} @ifclear FOR_PRINT @item -@ref{Notes}. +@ref{Notes} @item -@ref{Basic Concepts}. +@ref{Basic Concepts} @item -@ref{Glossary}. +@ref{Glossary} @end ifclear @item -@ref{Copying}. +@ref{Copying} @ifclear FOR_PRINT @item -@ref{GNU Free Documentation License}. +@ref{GNU Free Documentation License} @end ifclear @end itemize @end ifdocbook @@ -35331,7 +35582,7 @@ covers the @command{gawk} source code: @appendix The Evolution of the @command{awk} Language This @value{DOCUMENT} describes the GNU implementation of @command{awk}, -which follows the POSIX specification. Many long-time @command{awk} +which follows the POSIX specification. Many longtime @command{awk} users learned @command{awk} programming with the original @command{awk} implementation in Version 7 Unix. (This implementation was the basis for @command{awk} in Berkeley Unix, through 4.3-Reno. Subsequent versions @@ -35368,9 +35619,7 @@ online documentation}. @node V7/SVR3.1 @appendixsec Major Changes Between V7 and SVR3.1 -@c STARTOFRANGE gawkv @cindex @command{awk}, versions of -@c STARTOFRANGE gawkv1 @cindex @command{awk}, versions of, changes between V7 and SVR3.1 The @command{awk} language evolved considerably between the release of @@ -35381,83 +35630,82 @@ cross-references to further details: @itemize @value{BULLET} @item The requirement for @samp{;} to separate rules on a line -(@pxref{Statements/Lines}). +(@pxref{Statements/Lines}) @item User-defined functions and the @code{return} statement -(@pxref{User-defined}). +(@pxref{User-defined}) @item The @code{delete} statement (@pxref{Delete}). @item The @code{do}-@code{while} statement -(@pxref{Do Statement}). +(@pxref{Do Statement}) @item The built-in functions @code{atan2()}, @code{cos()}, @code{sin()}, @code{rand()}, and -@code{srand()} (@pxref{Numeric Functions}). +@code{srand()} (@pxref{Numeric Functions}) @item The built-in functions @code{gsub()}, @code{sub()}, and @code{match()} -(@pxref{String Functions}). +(@pxref{String Functions}) @item The built-in functions @code{close()} and @code{system()} -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @item The @code{ARGC}, @code{ARGV}, @code{FNR}, @code{RLENGTH}, @code{RSTART}, -and @code{SUBSEP} predefined variables (@pxref{Built-in Variables}). +and @code{SUBSEP} predefined variables (@pxref{Built-in Variables}) @item -Assignable @code{$0} (@pxref{Changing Fields}). +Assignable @code{$0} (@pxref{Changing Fields}) @item The conditional expression using the ternary operator @samp{?:} -(@pxref{Conditional Exp}). +(@pxref{Conditional Exp}) @item -The expression @samp{@var{index-variable} in @var{array}} outside of @code{for} -statements (@pxref{Reference to Elements}). +The expression @samp{@var{indx} in @var{array}} outside of @code{for} +statements (@pxref{Reference to Elements}) @item The exponentiation operator @samp{^} (@pxref{Arithmetic Ops}) and its assignment operator -form @samp{^=} (@pxref{Assignment Ops}). +form @samp{^=} (@pxref{Assignment Ops}) @item C-compatible operator precedence, which breaks some old @command{awk} -programs (@pxref{Precedence}). +programs (@pxref{Precedence}) @item Regexps as the value of @code{FS} (@pxref{Field Separators}) and as the third argument to the @code{split()} function (@pxref{String Functions}), rather than using only the first character -of @code{FS}. +of @code{FS} @item Dynamic regexps as operands of the @samp{~} and @samp{!~} operators -(@pxref{Computed Regexps}). +(@pxref{Computed Regexps}) @item The escape sequences @samp{\b}, @samp{\f}, and @samp{\r} -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @item Redirection of input for the @code{getline} function -(@pxref{Getline}). +(@pxref{Getline}) @item Multiple @code{BEGIN} and @code{END} rules -(@pxref{BEGIN/END}). +(@pxref{BEGIN/END}) @item Multidimensional arrays -(@pxref{Multidimensional}). +(@pxref{Multidimensional}) @end itemize -@c ENDOFRANGE gawkv1 @node SVR4 @appendixsec Changes Between SVR3.1 and SVR4 @@ -35468,54 +35716,54 @@ The System V Release 4 (1989) version of Unix @command{awk} added these features @itemize @value{BULLET} @item -The @code{ENVIRON} array (@pxref{Built-in Variables}). +The @code{ENVIRON} array (@pxref{Built-in Variables}) @c gawk and MKS awk @item Multiple @option{-f} options on the command line -(@pxref{Options}). +(@pxref{Options}) @c MKS awk @item The @option{-v} option for assigning variables before program execution begins -(@pxref{Options}). +(@pxref{Options}) @c GNU, Bell Laboratories & MKS together @item -The @option{--} signal for terminating command-line options. +The @option{--} signal for terminating command-line options @item The @samp{\a}, @samp{\v}, and @samp{\x} escape sequences -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @c GNU, for ANSI C compat @item A defined return value for the @code{srand()} built-in function -(@pxref{Numeric Functions}). +(@pxref{Numeric Functions}) @item The @code{toupper()} and @code{tolower()} built-in string functions for case translation -(@pxref{String Functions}). +(@pxref{String Functions}) @item -A cleaner specification for the @code{%c} format-control letter in the +A cleaner specification for the @samp{%c} format-control letter in the @code{printf} function -(@pxref{Control Letters}). +(@pxref{Control Letters}) @item The ability to dynamically pass the field width and precision (@code{"%*.*d"}) in the argument list of @code{printf} and @code{sprintf()} -(@pxref{Control Letters}). +(@pxref{Control Letters}) @item The use of regexp constants, such as @code{/foo/}, as expressions, where they are equivalent to using the matching operator, as in @samp{$0 ~ /foo/} -(@pxref{Using Constant Regexps}). +(@pxref{Using Constant Regexps}) @item Processing of escape sequences inside command-line variable assignments -(@pxref{Assignment Options}). +(@pxref{Assignment Options}) @end itemize @node POSIX @@ -35529,23 +35777,23 @@ introduced the following changes into the language: @itemize @value{BULLET} @item The use of @option{-W} for implementation-specific options -(@pxref{Options}). +(@pxref{Options}) @item The use of @code{CONVFMT} for controlling the conversion of numbers -to strings (@pxref{Conversion}). +to strings (@pxref{Conversion}) @item The concept of a numeric string and tighter comparison rules to go -with it (@pxref{Typing and Comparison}). +with it (@pxref{Typing and Comparison}) @item The use of predefined variables as function parameter names is forbidden -(@pxref{Definition Syntax}). +(@pxref{Definition Syntax}) @item More complete documentation of many of the previously undocumented -features of the language. +features of the language @end itemize In 2012, a number of extensions that had been commonly available for @@ -35554,25 +35802,24 @@ many years were finally added to POSIX. They are: @itemize @value{BULLET} @item The @code{fflush()} built-in function for flushing buffered output -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @item The @code{nextfile} statement -(@pxref{Nextfile Statement}). +(@pxref{Nextfile Statement}) @item The ability to delete all of an array at once with @samp{delete @var{array}} -(@pxref{Delete}). +(@pxref{Delete}) @end itemize -@xref{Common Extensions}, for a list of common extensions +@DBXREF{Common Extensions} for a list of common extensions not permitted by the POSIX standard. The 2008 POSIX standard can be found online at @url{http://www.opengroup.org/onlinepubs/9699919799/}. -@c ENDOFRANGE gawkv @node BTL @appendixsec Extensions in Brian Kernighan's @command{awk} @@ -35586,43 +35833,40 @@ has made his version available via his home page (@pxref{Other Versions}). This @value{SECTION} describes common extensions that -originally appeared in his version of @command{awk}. +originally appeared in his version of @command{awk}: @itemize @value{BULLET} @item The @samp{**} and @samp{**=} operators (@pxref{Arithmetic Ops} and -@ref{Assignment Ops}). +@ref{Assignment Ops}) @item The use of @code{func} as an abbreviation for @code{function} -(@pxref{Definition Syntax}). +(@pxref{Definition Syntax}) @item The @code{fflush()} built-in function for flushing buffered output -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @ignore @item The @code{SYMTAB} array, that allows access to @command{awk}'s internal symbol table. This feature was never documented for his @command{awk}, largely because it is somewhat shakily implemented. For instance, you cannot access arrays -or array elements through it. +or array elements through it @end ignore @end itemize -@xref{Common Extensions}, for a full list of the extensions +@DBXREF{Common Extensions} for a full list of the extensions available in his @command{awk}. @node POSIX/GNU @appendixsec Extensions in @command{gawk} Not in POSIX @command{awk} -@c STARTOFRANGE fripls @cindex compatibility mode (@command{gawk}), extensions -@c STARTOFRANGE exgnot @cindex extensions, in @command{gawk}, not in POSIX @command{awk} -@c STARTOFRANGE posnot @cindex POSIX, @command{gawk} extensions not included in The GNU implementation, @command{gawk}, adds a large number of features. They can all be disabled with either the @option{--traditional} or @@ -35641,7 +35885,7 @@ Additional predefined variables: @itemize @value{MINUS} @item The -@code{ARGIND} +@code{ARGIND}, @code{BINMODE}, @code{ERRNO}, @code{FIELDWIDTHS}, @@ -35653,7 +35897,7 @@ The and @code{TEXTDOMAIN} variables -(@pxref{Built-in Variables}). +(@pxref{Built-in Variables}) @end itemize @item @@ -35661,15 +35905,15 @@ Special files in I/O redirections: @itemize @value{MINUS} @item -The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr} and +The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr}, and @file{/dev/fd/@var{N}} special @value{FN}s -(@pxref{Special Files}). +(@pxref{Special Files}) @item The @file{/inet}, @file{/inet4}, and @samp{/inet6} special files for TCP/IP networking using @samp{|&} to specify which version of the -IP protocol to use. -(@pxref{TCP/IP Networking}). +IP protocol to use +(@pxref{TCP/IP Networking}) @end itemize @item @@ -35678,37 +35922,41 @@ Changes and/or additions to the language: @itemize @value{MINUS} @item The @samp{\x} escape sequence -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @item Full support for both POSIX and GNU regexps -(@pxref{Regexp}). +(@pxref{Regexp}) @item The ability for @code{FS} and for the third argument to @code{split()} to be null strings -(@pxref{Single Character Fields}). +(@pxref{Single Character Fields}) @item The ability for @code{RS} to be a regexp -(@pxref{Records}). +(@pxref{Records}) @item The ability to use octal and hexadecimal constants in @command{awk} program source code -(@pxref{Nondecimal-numbers}). +(@pxref{Nondecimal-numbers}) @item The @samp{|&} operator for two-way I/O to a coprocess -(@pxref{Two-way I/O}). +(@pxref{Two-way I/O}) @item Indirect function calls -(@pxref{Indirect Calls}). +(@pxref{Indirect Calls}) @item Directories on the command line produce a warning and are skipped -(@pxref{Command-line directories}). +(@pxref{Command-line directories}) + +@item +Output with @code{print} and @code{printf} need not be fatal +(@pxref{Nonfatal}) @end itemize @item @@ -35716,12 +35964,12 @@ New keywords: @itemize @value{MINUS} @item -The @code{BEGINFILE} and @code{ENDFILE} special patterns. -(@pxref{BEGINFILE/ENDFILE}). +The @code{BEGINFILE} and @code{ENDFILE} special patterns +(@pxref{BEGINFILE/ENDFILE}) @item The @code{switch} statement -(@pxref{Switch Statement}). +(@pxref{Switch Statement}) @end itemize @item @@ -35731,30 +35979,30 @@ Changes to standard @command{awk} functions: @item The optional second argument to @code{close()} that allows closing one end of a two-way pipe to a coprocess -(@pxref{Two-way I/O}). +(@pxref{Two-way I/O}) @item -POSIX compliance for @code{gsub()} and @code{sub()} with @option{--posix}. +POSIX compliance for @code{gsub()} and @code{sub()} with @option{--posix} @item The @code{length()} function accepts an array argument and returns the number of elements in the array -(@pxref{String Functions}). +(@pxref{String Functions}) @item The optional third argument to the @code{match()} function for capturing text-matching subexpressions within a regexp -(@pxref{String Functions}). +(@pxref{String Functions}) @item Positional specifiers in @code{printf} formats for making translations easier -(@pxref{Printf Ordering}). +(@pxref{Printf Ordering}) @item The @code{split()} function's additional optional fourth -argument which is an array to hold the text of the field separators. -(@pxref{String Functions}). +argument, which is an array to hold the text of the field separators +(@pxref{String Functions}) @end itemize @item @@ -35764,16 +36012,16 @@ Additional functions only in @command{gawk}: @item The @code{gensub()}, @code{patsplit()}, and @code{strtonum()} functions for more powerful text manipulation -(@pxref{String Functions}). +(@pxref{String Functions}) @item The @code{asort()} and @code{asorti()} functions for sorting arrays -(@pxref{Array Sorting}). +(@pxref{Array Sorting}) @item The @code{mktime()}, @code{systime()}, and @code{strftime()} functions for working with timestamps -(@pxref{Time Functions}). +(@pxref{Time Functions}) @item The @@ -35785,17 +36033,22 @@ The and @code{xor()} functions for bit manipulation -(@pxref{Bitwise Functions}). +(@pxref{Bitwise Functions}) @c In 4.1, and(), or() and xor() grew the ability to take > 2 arguments @item The @code{isarray()} function to check if a variable is an array or not -(@pxref{Type Functions}). +(@pxref{Type Functions}) @item -The @code{bindtextdomain()}, @code{dcgettext()} and @code{dcngettext()} +The @code{bindtextdomain()}, @code{dcgettext()}, and @code{dcngettext()} functions for internationalization -(@pxref{Programmer i18n}). +(@pxref{Programmer i18n}) + +@item +The @code{intdiv()} function for doing integer +division and remainder +(@pxref{Numeric Functions}) @end itemize @item @@ -35805,12 +36058,12 @@ Changes and/or additions in the command-line options: @item The @env{AWKPATH} environment variable for specifying a path search for the @option{-f} command-line option -(@pxref{Options}). +(@pxref{Options}) @item The @env{AWKLIBPATH} environment variable for specifying a path search for the @option{-l} command-line option -(@pxref{Options}). +(@pxref{Options}) @item The @@ -35839,7 +36092,7 @@ The and @option{-V} short options. Also, the -ability to use GNU-style long-named options that start with @option{--} +ability to use GNU-style long-named options that start with @option{--}, and the @option{--assign}, @option{--bignum}, @@ -35919,7 +36172,7 @@ GCC for VAX and Alpha has not been tested for a while. @end itemize @item -Support for the following obsolete systems was removed from the code +Support for the following obsolete system was removed from the code for @command{gawk} @value{PVERSION} 4.1: @c nested table @@ -35929,16 +36182,19 @@ Ultrix @end itemize @item -@c FIXME: Verify the version here. -Support for MirBSD was removed at @command{gawk} @value{PVERSION} 4.2. +Support for the following systems was removed from the code +for @command{gawk} @value{PVERSION} 4.2: + +@c nested table +@itemize @value{MINUS} +@item +MirBSD +@end itemize @end itemize @c XXX ADD MORE STUFF HERE -@c ENDOFRANGE fripls -@c ENDOFRANGE exgnot -@c ENDOFRANGE posnot @c This does not need to be in the formal book. @ifclear FOR_PRINT @@ -36520,11 +36776,11 @@ load @command{awk} library files. @item The @option{-l} and @option{--load} options load compiled dynamic extensions. -@item +@item The @option{-M} and @option{--bignum} options enable MPFR. @item -The @option{-o} only does pretty-printing. +The @option{-o} option only does pretty-printing. @item The @option{-p} option is used for profiling. @@ -36547,6 +36803,44 @@ with a minimum of two The dynamic extension interface was completely redone (@pxref{Dynamic Extensions}). +@item +Support for Ultrix was removed. + +@end itemize + +Version 4.2 introduced the following changes: + +@itemize @bullet +@item +Changes to @code{ENVIRON} are reflected into @command{gawk}'s +environment and that of programs that it runs. +@xref{Auto-set}. + +@item +The @option{--pretty-print} option no longer runs the @command{awk} +program too. +@xref{Options}. + +@item +The @command{igawk} program and its manual page are no longer +installed when @command{gawk} is built. +@xref{Igawk Program}. + +@item +The @code{intdiv()} function. +@xref{Numeric Functions}. + +@item +The maximum number of hexdecimal digits in @samp{\x} escapes +is now two. +@xref{Escape Sequences}. + +@item +Nonfatal output with @code{print} and @code{printf}. +@xref{Nonfatal}. + +@item +Support for MirBSD was removed. @end itemize @c XXX ADD MORE STUFF HERE @@ -36559,12 +36853,12 @@ The dynamic extension interface was completely redone @cindex extensions, @command{mawk} The following table summarizes the common extensions supported by @command{gawk}, Brian Kernighan's @command{awk}, and @command{mawk}, -the three most widely-used freely available versions of @command{awk} +the three most widely used freely available versions of @command{awk} (@pxref{Other Versions}). -@multitable {@file{/dev/stderr} special file} {BWK Awk} {Mawk} {GNU Awk} {Now standard} -@headitem Feature @tab BWK Awk @tab Mawk @tab GNU Awk @tab Now standard -@item @samp{\x} Escape sequence @tab X @tab X @tab X @tab +@multitable {@file{/dev/stderr} special file} {BWK @command{awk}} {@command{mawk}} {@command{gawk}} {Now standard} +@headitem Feature @tab BWK @command{awk} @tab @command{mawk} @tab @command{gawk} @tab Now standard +@item @samp{\x} escape sequence @tab X @tab X @tab X @tab @item @code{FS} as null string @tab X @tab X @tab X @tab @item @file{/dev/stdin} special file @tab X @tab X @tab X @tab @item @file{/dev/stdout} special file @tab X @tab X @tab X @tab @@ -36577,7 +36871,7 @@ the three most widely-used freely available versions of @command{awk} @item @code{func} keyword @tab X @tab @tab X @tab @item @code{BINMODE} variable @tab @tab X @tab X @tab @item @code{RS} as regexp @tab @tab X @tab X @tab -@item Time related functions @tab @tab X @tab X @tab +@item Time-related functions @tab @tab X @tab X @tab @end multitable @node Ranges and Locales @@ -36593,9 +36887,9 @@ the first character in the range and the last character in the range, inclusive. Ordering was based on the numeric value of each character in the machine's native character set. Thus, on ASCII-based systems, @samp{[a-z]} matched all the lowercase letters, and only the lowercase -letters, since the numeric values for the letters from @samp{a} through +letters, as the numeric values for the letters from @samp{a} through @samp{z} were contiguous. (On an EBCDIC system, the range @samp{[a-z]} -includes additional, non-alphabetic characters as well.) +includes additional nonalphabetic characters as well.) Almost all introductory Unix literature explained range expressions as working in this fashion, and in particular, would teach that the @@ -36604,8 +36898,8 @@ that @samp{[A-Z]} was the ``correct'' way to match uppercase letters. And indeed, this was true.@footnote{And Life was good.} The 1992 POSIX standard introduced the idea of locales (@pxref{Locales}). -Since many locales include other letters besides the plain twenty-six -letters of the American English alphabet, the POSIX standard added +Because many locales include other letters besides the plain 26 +letters of the English alphabet, the POSIX standard added character classes (@pxref{Bracket Expressions}) as a way to match different kinds of characters besides the traditional ones in the ASCII character set. @@ -36620,9 +36914,9 @@ What does that mean? In many locales, @samp{A} and @samp{a} are both less than @samp{B}. In other words, these locales sort characters in dictionary order, and @samp{[a-dx-z]} is typically not equivalent to @samp{[abcdxyz]}; -instead it might be equivalent to @samp{[ABCXYabcdxyz]}, for example. +instead, it might be equivalent to @samp{[ABCXYabcdxyz]}, for example. -This point needs to be emphasized: Much literature teaches that you should +This point needs to be emphasized: much literature teaches that you should use @samp{[a-z]} to match a lowercase character. But on systems with non-ASCII locales, this also matches all of the uppercase characters except @samp{A} or @samp{Z}! This was a continuous cause of confusion, even well @@ -36638,7 +36932,7 @@ $ @kbd{echo something1234abc | gawk-3.1.8 '@{ sub("[A-Z]*$", ""); print @}'} @end example @noindent -This output is unexpected, since the @samp{bc} at the end of +This output is unexpected, as the @samp{bc} at the end of @samp{something1234abc} should not normally match @samp{[A-Z]*}. This result is due to the locale setting (and thus you may not see it on your system). @@ -36649,13 +36943,13 @@ is perfectly valid in ASCII, but is not valid in many Unicode locales, such as @code{en_US.UTF-8}. Early versions of @command{gawk} used regexp matching code that was not -locale aware, so ranges had their traditional interpretation. +locale-aware, so ranges had their traditional interpretation. When @command{gawk} switched to using locale-aware regexp matchers, the problems began; especially as both GNU/Linux and commercial Unix vendors started implementing non-ASCII locales, @emph{and making them the default}. Perhaps the most frequently asked question became something -like ``why does @samp{[A-Z]} match lowercase letters?!?'' +like, ``Why does @samp{[A-Z]} match lowercase letters?!?'' @cindex Berry, Karl This situation existed for close to 10 years, if not more, and @@ -36665,7 +36959,7 @@ was in the user's locale. During the development of @value{PVERSION} 4.0, he modified @command{gawk} to always treat ranges in the original, pre-POSIX fashion, unless @option{--posix} was used (@pxref{Options}).@footnote{And thus was born the Campaign for Rational Range Interpretation (or -RRI). A number of GNU tools have either implemented this change, +RRI). A number of GNU tools have already implemented this change, or will soon. Thanks to Karl Berry for coining the phrase ``Rational Range Interpretation.''} @@ -36679,9 +36973,10 @@ and By using this lovely technical term, the standard gives license to implementors to implement ranges in whatever way they choose. -The @command{gawk} maintainer chose to apply the pre-POSIX meaning in all -cases: the default regexp matching; with @option{--traditional} and with -@option{--posix}; in all cases, @command{gawk} remains POSIX compliant. +The @command{gawk} maintainer chose to apply the pre-POSIX meaning +both with the default regexp matching and when @option{--traditional} or +@option{--posix} are used. +In all cases @command{gawk} remains POSIX-compliant. @node Contributors @appendixsec Major Contributors to @command{gawk} @@ -36727,7 +37022,7 @@ to around 90 pages. Richard Stallman helped finish the implementation and the initial draft of this @value{DOCUMENT}. -He is also the founder of the FSF and the GNU project. +He is also the founder of the FSF and the GNU Project. @item @cindex Woods, John @@ -36873,7 +37168,7 @@ Michael Benzinger contributed the initial code for @code{switch} statements. @cindex McPhee, Patrick Patrick T.J.@: McPhee contributed the code for dynamic loading in Windows32 environments. -(This is no longer supported) +(This is no longer supported.) @item @cindex Wallin, Anders @@ -36891,28 +37186,28 @@ John Haque made the following contributions: @itemize @value{MINUS} @item The modifications to convert @command{gawk} -into a byte-code interpreter, including the debugger. +into a byte-code interpreter, including the debugger @item -The addition of true arrays of arrays. +The addition of true arrays of arrays @item -The additional modifications for support of arbitrary precision arithmetic. +The additional modifications for support of arbitrary-precision arithmetic @item The initial text of -@ref{Arbitrary Precision Arithmetic}. +@ref{Arbitrary Precision Arithmetic} @item The work to merge the three versions of @command{gawk} -into one, for the 4.1 release. +into one, for the 4.1 release @item -Improved array internals for arrays indexed by integers. +Improved array internals for arrays indexed by integers @item -The improved array sorting features were driven by John together -with Pat Rankin. +The improved array sorting features were also driven by John, together +with Pat Rankin @end itemize @cindex Papadopoulos, Panos @@ -36953,10 +37248,10 @@ helping David Trueman, and as the primary maintainer since around 1994. @itemize @value{BULLET} @item The @command{awk} language has evolved over time. The first release -was with V7 Unix circa 1978. In 1987 for System V Release 3.1, +was with V7 Unix, circa 1978. In 1987, for System V Release 3.1, major additions, including user-defined functions, were made to the language. Additional changes were made for System V Release 4, in 1989. -Since then, further minor changes happen under the auspices of the +Since then, further minor changes have happened under the auspices of the POSIX standard. @item @@ -36972,7 +37267,7 @@ options. The interaction of POSIX locales and regexp matching in @command{gawk} has been confusing over the years. Today, @command{gawk} implements Rational Range Interpretation, where ranges of the form @samp{[a-z]} match @emph{only} the characters numerically between -@samp{a} through @samp{z} in the machine's native character set. Usually this is ASCII +@samp{a} through @samp{z} in the machine's native character set. Usually this is ASCII, but it can be EBCDIC on IBM S/390 systems. @item @@ -36987,16 +37282,14 @@ the appropriate credit where credit is due. @c last two commas are part of see also @cindex operating systems, See Also GNU/Linux@comma{} PC operating systems@comma{} Unix -@c STARTOFRANGE gligawk @cindex @command{gawk}, installing -@c STARTOFRANGE ingawk @cindex installing @command{gawk} This appendix provides instructions for installing @command{gawk} on the various platforms that are supported by the developers. The primary developer supports GNU/Linux (and Unix), whereas the other ports are contributed. -@xref{Bugs}, -for the electronic mail addresses of the people who maintain +@DBXREF{Bugs} +for the email addresses of the people who maintain the respective ports. @menu @@ -37050,7 +37343,7 @@ wget http://ftp.gnu.org/gnu/gawk/gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz The GNU software archive is mirrored around the world. The up-to-date list of mirror sites is available from -@uref{http://www.gnu.org/order/ftp.html, the main FSF web site}. +@uref{http://www.gnu.org/order/ftp.html, the main FSF website}. Try to use one of the mirrors; they will be less busy, and you can usually find one closer to your site. @@ -37059,9 +37352,9 @@ will be less busy, and you can usually find one closer to your site. @command{gawk} is distributed as several @code{tar} files compressed with different compression programs: @command{gzip}, @command{bzip2}, and @command{xz}. For simplicity, the rest of these instructions assume -you are using the one compressed with the GNU Zip program, @code{gzip}. +you are using the one compressed with the GNU Gzip program (@command{gzip}). -Once you have the distribution (for example, +Once you have the distribution (e.g., @file{gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz}), use @code{gzip} to expand the file and then use @code{tar} to extract it. You can use the following @@ -37099,7 +37392,6 @@ a local expert. @node Distribution contents @appendixsubsec Contents of the @command{gawk} Distribution -@c STARTOFRANGE gawdis @cindex @command{gawk}, distribution The @command{gawk} distribution has a number of C source files, @@ -37111,12 +37403,12 @@ operating systems: @table @asis @item Various @samp{.c}, @samp{.y}, and @samp{.h} files -The actual @command{gawk} source code. +These files contain the actual @command{gawk} source code. @end table @table @file @item ABOUT-NLS -Information about GNU @command{gettext} and translations. +A file containing information about GNU @command{gettext} and translations. @item AUTHORS A file with some information about the authorship of @command{gawk}. @@ -37146,7 +37438,7 @@ An older list of changes to @command{gawk}. The GNU General Public License. @item POSIX.STD -A description of behaviors in the POSIX standard for @command{awk} which +A description of behaviors in the POSIX standard for @command{awk} that are left undefined, or where @command{gawk} may not comply fully, as well as a list of things that the POSIX standard should describe but does not. @@ -37154,7 +37446,7 @@ as a list of things that the POSIX standard should describe but does not. @item doc/awkforai.txt Pointers to the original draft of a short article describing why @command{gawk} is a good language for -Artificial Intelligence (AI) programming. +artificial intelligence (AI) programming. @item doc/bc_notes A brief description of @command{gawk}'s ``byte code'' internals. @@ -37197,10 +37489,10 @@ The generated Info file for this @value{DOCUMENT}. @item doc/gawkinet.texi The Texinfo source file for @ifinfo -@inforef{Top, , General Introduction, gawkinet, TCP/IP Internetworking with @command{gawk}}. +@inforef{Top, , General Introduction, gawkinet, @value{GAWKINETTITLE}}. @end ifinfo @ifnotinfo -@cite{TCP/IP Internetworking with @command{gawk}}. +@cite{@value{GAWKINETTITLE}}. @end ifnotinfo It should be processed with @TeX{} (via @command{texi2dvi} or @command{texi2pdf}) @@ -37209,7 +37501,7 @@ with @command{makeinfo} to produce an Info or HTML file. @item doc/gawkinet.info The generated Info file for -@cite{TCP/IP Internetworking with @command{gawk}}. +@cite{@value{GAWKINETTITLE}}. @item doc/igawk.1 The @command{troff} source for a manual page describing the @command{igawk} @@ -37271,6 +37563,12 @@ The source code, manual pages, and infrastructure files for the sample extensions included with @command{gawk}. @xref{Dynamic Extensions}, for more information. +@item extras/* +Additional non-essential files. Currently, this directory contains some shell +startup files to be installed in @file{/etc/profile.d} to aid in manipulating +the @env{AWKPATH} and @env{AWKLIBPATH} environment variables. +@xref{Shell Startup Files}, for more information. + @item posix/* Files needed for building @command{gawk} on POSIX-compliant systems. @@ -37279,11 +37577,11 @@ Files needed for building @command{gawk} under MS-Windows @ifclear FOR_PRINT and OS/2 @end ifclear -(@pxref{PC Installation}, for details). +(@DBPXREF{PC Installation} for details). @item vms/* Files needed for building @command{gawk} under Vax/VMS and OpenVMS -(@pxref{VMS Installation}, for details). +(@DBPXREF{VMS Installation} for details). @item test/* A test suite for @@ -37292,10 +37590,9 @@ directory to run your version of @command{gawk} against the test suite. If @command{gawk} successfully passes @samp{make check}, then you can be confident of a successful port. @end table -@c ENDOFRANGE gawdis @node Unix Installation -@appendixsec Compiling and Installing @command{gawk} on Unix-like Systems +@appendixsec Compiling and Installing @command{gawk} on Unix-Like Systems Usually, you can compile and install @command{gawk} by typing only two commands. However, if you use an unusual system, you may need @@ -37303,12 +37600,13 @@ to configure @command{gawk} for your system yourself. @menu * Quick Installation:: Compiling @command{gawk} under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. @end menu @node Quick Installation -@appendixsubsec Compiling @command{gawk} for Unix-like Systems +@appendixsubsec Compiling @command{gawk} for Unix-Like Systems The normal installation steps should work on all modern commercial Unix-derived systems, GNU/Linux, BSD-based systems, and the Cygwin @@ -37325,7 +37623,7 @@ described fully in @cite{Autoconf---Generating Automatic Configuration Scripts}, which can be found online at @uref{http://www.gnu.org/software/autoconf/manual/index.html, -the Free Software Foundation's web site}.) +the Free Software Foundation's website}.) @end ifnotinfo @ifinfo (The Autoconf software is described fully starting with @@ -37372,7 +37670,7 @@ run @samp{make check}. All of the tests should succeed. If these steps do not work, or if any of the tests fail, check the files in the @file{README_d} directory to see if you've found a known problem. If the failure is not described there, -please send in a bug report (@pxref{Bugs}). +send in a bug report (@pxref{Bugs}). Of course, once you've built @command{gawk}, it is likely that you will wish to install it. To do so, you need to run the command @samp{make @@ -37383,6 +37681,44 @@ is likely that you will be asked for your password, and you will have to have been set up previously as a user who is allowed to run the @command{sudo} command. +@node Shell Startup Files +@appendixsubsec Shell Startup Files + +The distribution contains shell startup files @file{gawk.sh} and +@file{gawk.csh} containing functions to aid in manipulating +the @env{AWKPATH} and @env{AWKLIBPATH} environment variables. +On a Fedora system, these files should be installed in @file{/etc/profile.d}; +on other platforms, the appropriate location may be different. + +@table @command + +@cindex @command{gawkpath_default} shell function +@item gawkpath_default +Reset the @env{AWKPATH} environment variable to its default value. + +@cindex @command{gawkpath_prepend} shell function +@item gawkpath_prepend +Add the argument to the front of the @env{AWKPATH} environment variable. + +@cindex @command{gawkpath_append} shell function +@item gawkpath_append +Add the argument to the end of the @env{AWKPATH} environment variable. + +@cindex @command{gawklibpath_default} shell function +@item gawklibpath_default +Reset the @env{AWKLIBPATH} environment variable to its default value. + +@cindex @command{gawklibpath_prepend} shell function +@item gawklibpath_prepend +Add the argument to the front of the @env{AWKLIBPATH} environment variable. + +@cindex @command{gawklibpath_append} shell function +@item gawklibpath_append +Add the argument to the end of the @env{AWKLIBPATH} environment variable. + +@end table + + @node Additional Configuration Options @appendixsubsec Additional Configuration Options @cindex @command{gawk}, configuring, options @@ -37404,7 +37740,7 @@ can be configured and compiled. @cindex @option{--disable-lint} configuration option @cindex configuration option, @code{--disable-lint} @item --disable-lint -Disable all lint checking within @code{gawk}. The +Disable all lint checking within @command{gawk}. The @option{--lint} and @option{--lint-old} options (@pxref{Options}) are accepted, but silently do nothing. @@ -37412,14 +37748,17 @@ Similarly, setting the @code{LINT} variable (@pxref{User-modified}) has no effect on the running @command{awk} program. -When used with GCC's automatic dead-code-elimination, this option +When used with the GNU Compiler Collection's (GCC's) +automatic dead-code-elimination, this option cuts almost 23K bytes off the size of the @command{gawk} executable on GNU/Linux x86_64 systems. Results on other systems and with other compilers are likely to vary. Using this option may bring you some slight performance improvement. +@quotation CAUTION Using this option will cause some of the tests in the test suite to fail. This option may be removed at a later date. +@end quotation @cindex @option{--disable-nls} configuration option @cindex configuration option, @code{--disable-nls} @@ -37436,7 +37775,7 @@ function for deficient systems. @end table Use the command @samp{./configure --help} to see the full list of -options that @command{configure} supplies. +options supplied by @command{configure}. @node Configuration Philosophy @appendixsubsec The Configuration Process @@ -37470,19 +37809,19 @@ facts about your operating system. For example, there may not be an @cindex @code{custom.h} file It is possible for your C compiler to lie to @command{configure}. It may do so by not exiting with an error when a library function is not -available. To get around this, edit the file @file{custom.h}. +available. To get around this, edit the @file{custom.h} file. Use an @samp{#ifdef} that is appropriate for your system, and either @code{#define} any constants that @command{configure} should have defined but didn't, or @code{#undef} any constants that @command{configure} defined and -should not have. @file{custom.h} is automatically included by -@file{config.h}. +should not have. The @file{custom.h} file is automatically included by +the @file{config.h} file. It is also possible that the @command{configure} program generated by Autoconf will not work on your system in some other fashion. -If you do have a problem, the file @file{configure.ac} is the input for +If you do have a problem, the @file{configure.ac} file is the input for Autoconf. You may be able to change this file and generate a new version of @command{configure} that works on your system -(@pxref{Bugs}, +(@DBPXREF{Bugs} for information on how to report problems in configuring @command{gawk}). The same mechanism may be used to send in updates to @file{configure.ac} and/or @file{custom.h}. @@ -37516,13 +37855,13 @@ running MS-DOS, any version of MS-Windows, or OS/2. running MS-DOS and any version of MS-Windows. @end ifset In this @value{SECTION}, the term ``Windows32'' -refers to any of Microsoft Windows-95/98/ME/NT/2000/XP/Vista/7/8. +refers to any of Microsoft Windows 95/98/ME/NT/2000/XP/Vista/7/8. The limitations of MS-DOS (and MS-DOS shells under the other operating -systems) has meant that various ``DOS extenders'' are often used with +systems) have meant that various ``DOS extenders'' are often used with programs such as @command{gawk}. The varying capabilities of Microsoft Windows 3.1 and Windows32 can add to the confusion. For an overview -of the considerations, please refer to @file{README_d/README.pc} in +of the considerations, refer to @file{README_d/README.pc} in the distribution. @menu @@ -37685,7 +38024,7 @@ Ancient OS/2 ports of GNU @command{make} are not able to handle the Makefiles of this package. If you encounter any problems with @command{make}, try GNU Make 3.79.1 or later versions. You should find the latest version on -@uref{ftp://hobbes.nmsu.edu/pub/os2/}.@footnote{As of May, 2014, +@uref{ftp://hobbes.nmsu.edu/pub/os2/}.@footnote{As of November 2014, this site is still there, but the author could not find a package for GNU Make.} @end quotation @@ -37718,9 +38057,7 @@ multibyte functionality is not available. @node PC Using @appendixsubsubsec Using @command{gawk} on PC Operating Systems -@c STARTOFRANGE opgawx @cindex operating systems, PC, @command{gawk} on -@c STARTOFRANGE pcgawon @cindex PC operating systems, @command{gawk} on Under MS-DOS and MS-Windows, the Cygwin and MinGW environments support @@ -37734,8 +38071,8 @@ EMX (OS/2 only) supports at least the @samp{|&} operator. @cindex search paths, for source files @cindex @command{gawk}, MS-DOS version of @cindex @command{gawk}, MS-Windows version of -@cindex @code{;} (semicolon), @code{AWKPATH} variable and -@cindex semicolon (@code{;}), @code{AWKPATH} variable and +@cindex @code{;} (semicolon), @env{AWKPATH} variable and +@cindex semicolon (@code{;}), @env{AWKPATH} variable and @cindex @env{AWKPATH} environment variable The MS-DOS and MS-Windows versions of @command{gawk} search for program files as described in @ref{AWKPATH Variable}. However, @@ -37780,7 +38117,7 @@ Under MS-Windows, OS/2 and MS-DOS, Under MS-Windows and MS-DOS, @end ifset @command{gawk} (and many other text programs) silently -translate end-of-line @samp{\r\n} to @samp{\n} on input and @samp{\n} +translates end-of-line @samp{\r\n} to @samp{\n} on input and @samp{\n} to @samp{\r\n} on output. A special @code{BINMODE} variable @value{COMMONEXT} allows control over these translations and is interpreted as follows: @@ -37814,7 +38151,7 @@ Setting @code{BINMODE} for standard input or standard output is accomplished by using an appropriate @samp{-v BINMODE=@var{N}} option on the command line. @code{BINMODE} is set at the time a file or pipe is opened and cannot be -changed mid-stream. +changed midstream. The name @code{BINMODE} was chosen to match @command{mawk} (@pxref{Other Versions}). @@ -37870,8 +38207,8 @@ moved into the @code{BEGIN} rule. @command{gawk} can be built and used ``out of the box'' under MS-Windows if you are using the @uref{http://www.cygwin.com, Cygwin environment}. -This environment provides an excellent simulation of GNU/Linux, using the -GNU tools, such as Bash, the GNU Compiler Collection (GCC), GNU Make, +This environment provides an excellent simulation of GNU/Linux, using +Bash, GCC, GNU Make, and other GNU programs. Compilation and installation for Cygwin is the same as for a Unix system: @@ -37890,12 +38227,12 @@ and then the @samp{make} proceeds as usual. @appendixsubsubsec Using @command{gawk} In The MSYS Environment In the MSYS environment under MS-Windows, @command{gawk} automatically -uses binary mode for reading and writing files. Thus there is no +uses binary mode for reading and writing files. Thus, there is no need to use the @code{BINMODE} variable. This can cause problems with other Unix-like components that have been ported to MS-Windows that expect @command{gawk} to do automatic -translation of @code{"\r\n"}, since it won't. +translation of @code{"\r\n"}, because it won't. @node VMS Installation @appendixsubsec Compiling and Installing @command{gawk} on Vax/VMS and OpenVMS @@ -37954,19 +38291,19 @@ With ODS-5 volumes and extended parsing enabled, the case of the target parameter may need to be exact. @command{gawk} has been tested under VAX/VMS 7.3 and Alpha/VMS 7.3-1 -using Compaq C V6.4, and Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3. +using Compaq C V6.4, and under Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3. The most recent builds used HP C V7.3 on Alpha VMS 8.3 and both Alpha and IA64 VMS 8.4 used HP C 7.3.@footnote{The IA64 architecture is also known as ``Itanium.''} -@xref{VMS GNV}, for information on building +@DBXREF{VMS GNV} for information on building @command{gawk} as a PCSI kit that is compatible with the GNV product. @node VMS Dynamic Extensions @appendixsubsubsec Compiling @command{gawk} Dynamic Extensions on VMS The extensions that have been ported to VMS can be built using one of -the following commands. +the following commands: @example $ @kbd{MMS/DESCRIPTION=[.vms]descrip.mms extensions} @@ -37983,7 +38320,7 @@ $ @kbd{MMK/DESCRIPTION=[.vms]descrip.mms extensions} or a logical name to find the dynamic extensions. Dynamic extensions need to be compiled with the same compiler options for -floating point, pointer size, and symbol name handling as were used +floating-point, pointer size, and symbol name handling as were used to compile @command{gawk} itself. Alpha and Itanium should use IEEE floating point. The pointer size is 32 bits, and the symbol name handling should be exact case with CRC shortening for @@ -38002,7 +38339,7 @@ For VAX: /name=(as_is,short) @end example -Compile time macros need to be defined before the first VMS-supplied +Compile-time macros need to be defined before the first VMS-supplied header file is included, as follows: @example @@ -38049,7 +38386,7 @@ If your @command{gawk} was installed by a PCSI kit into the @file{GNV$GNU:[vms_help]gawk.hlp}. The PCSI kit also installs a @file{GNV$GNU:[vms_bin]gawk_verb.cld} file -which can be used to add @command{gawk} and @command{awk} as DCL commands. +that can be used to add @command{gawk} and @command{awk} as DCL commands. For just the current process you can use: @@ -38058,7 +38395,7 @@ $ @kbd{set command gnv$gnu:[vms_bin]gawk_verb.cld} @end example Or the system manager can use @file{GNV$GNU:[vms_bin]gawk_verb.cld} to -add the @command{gawk} and @command{awk} to the system wide @samp{DCLTABLES}. +add the @command{gawk} and @command{awk} to the system-wide @samp{DCLTABLES}. The DCL syntax is documented in the @file{gawk.hlp} file. @@ -38113,7 +38450,7 @@ Note that uppercase and mixed-case text must be quoted. The VMS port of @command{gawk} includes a @code{DCL}-style interface in addition to the original shell-style interface (see the help entry for details). One side effect of dual command-line parsing is that if there is only a -single parameter (as in the quoted string program above), the command +single parameter (as in the quoted string program), the command becomes ambiguous. To work around this, the normally optional @option{--} flag is required to force Unix-style parsing rather than @code{DCL} parsing. If any other dash-type options (or multiple parameters such as @value{DF}s to @@ -38124,14 +38461,14 @@ The @code{exit} value is a Unix-style value and is encoded into a VMS exit status value when the program exits. The VMS severity bits will be set based on the @code{exit} value. -A failure is indicated by 1 and VMS sets the @code{ERROR} status. -A fatal error is indicated by 2 and VMS sets the @code{FATAL} status. +A failure is indicated by 1, and VMS sets the @code{ERROR} status. +A fatal error is indicated by 2, and VMS sets the @code{FATAL} status. All other values will have the @code{SUCCESS} status. The exit value is encoded to comply with VMS coding standards and will have the @code{C_FACILITY_NO} of @code{0x350000} with the constant @code{0xA000} added to the number shifted over by 3 bits to make room for the severity codes. -To extract the actual @command{gawk} exit code from the VMS status use: +To extract the actual @command{gawk} exit code from the VMS status, use: @example unix_status = (vms_status .and. &x7f8) / 8 @@ -38150,7 +38487,7 @@ VAX/VMS floating point uses unbiased rounding. @xref{Round Function}. VMS reports time values in GMT unless one of the @code{SYS$TIMEZONE_RULE} or @code{TZ} logical names is set. Older versions of VMS, such as VAX/VMS -7.3 do not set these logical names. +7.3, do not set these logical names. @c @cindex directory search @c @cindex path, search @@ -38168,7 +38505,7 @@ translation and not a multitranslation @code{RMS} searchlist. The VMS GNV package provides a build environment similar to POSIX with ports of a collection of open source tools. The @command{gawk} found in the GNV -base kit is an older port. Currently the GNV project is being reorganized +base kit is an older port. Currently, the GNV project is being reorganized to supply individual PCSI packages for each component. See @w{@uref{https://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/}.} @@ -38228,40 +38565,36 @@ $ @kbd{gawk :== $sys$common:[syshlp.examples.tcpip.snmp]gawk.exe} This is apparently @value{PVERSION} 2.15.6, which is extremely old. We recommend compiling and using the current version. -@c ENDOFRANGE opgawx -@c ENDOFRANGE pcgawon @node Bugs @appendixsec Reporting Problems and Bugs -@cindex archeologists +@cindex archaeologists @quotation -@i{There is nothing more dangerous than a bored archeologist.} -@author The Hitchhiker's Guide to the Galaxy +@i{There is nothing more dangerous than a bored archaeologist.} +@author Douglas Adams, @cite{The Hitchhiker's Guide to the Galaxy} @end quotation @c the radio show, not the book. :-) -@c STARTOFRANGE dbugg @cindex debugging @command{gawk}, bug reports -@c STARTOFRANGE tblgawb @cindex troubleshooting, @command{gawk}, bug reports If you have problems with @command{gawk} or think that you have found a bug, -please report it to the developers; we cannot promise to do anything +report it to the developers; we cannot promise to do anything, but we might well want to fix it. -Before reporting a bug, please make sure you have really found a genuine bug. +Before reporting a bug, make sure you have really found a genuine bug. Carefully reread the documentation and see if it says you can do what you're trying to do. If it's not clear whether you should be able to do something or not, report that too; it's a bug in the documentation! Before reporting a bug or trying to fix it yourself, try to isolate it to the smallest possible @command{awk} program and input @value{DF} that -reproduces the problem. Then send us the program and @value{DF}, +reproduce the problem. Then send us the program and @value{DF}, some idea of what kind of Unix system you're using, the compiler you used to compile @command{gawk}, and the exact results @command{gawk} gave you. Also say what you expected to occur; this helps us decide whether the problem is really in the documentation. -Please include the version number of @command{gawk} you are using. +Make sure to include the version number of @command{gawk} you are using. You can get this information with the command @samp{gawk --version}. @cindex @code{bug-gawk@@gnu.org} bug reporting address @@ -38270,10 +38603,10 @@ You can get this information with the command @samp{gawk --version}. Once you have a precise problem description, send email to @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org}. -The @command{gawk} maintainers subscribe to this address and +The @command{gawk} maintainers subscribe to this address, and thus they will receive your bug report. Although you can send mail to the maintainers directly, -the bug reporting address is preferred since the +the bug reporting address is preferred because the email list is archived at the GNU Project. @emph{All email must be in English. This is the only language understood in common by all the maintainers.} @@ -38282,32 +38615,32 @@ understood in common by all the maintainers.} @quotation CAUTION Do @emph{not} try to report bugs in @command{gawk} by posting to the Usenet/Internet newsgroup @code{comp.lang.awk}. -While the @command{gawk} developers do occasionally read this newsgroup, -there is no guarantee that we will see your posting. The steps described -above are the only official recognized way for reporting bugs. +The @command{gawk} developers do occasionally read this newsgroup, +but there is no guarantee that we will see your posting. The steps described +here are the only officially recognized way for reporting bugs. Really. @end quotation @quotation NOTE Many distributions of GNU/Linux and the various BSD-based operating systems have their own bug reporting systems. If you report a bug using your distribution's -bug reporting system, @emph{please} also send a copy to +bug reporting system, you should also send a copy to @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org}. -This is for two reasons. First, while some distributions forward +This is for two reasons. First, although some distributions forward bug reports ``upstream'' to the GNU mailing list, many don't, so there is a good chance that the @command{gawk} maintainers won't even see the bug report! Second, -mail to the GNU list is archived, and having everything at the GNU project -keeps things self-contained and not dependant on other organizations. +mail to the GNU list is archived, and having everything at the GNU Project +keeps things self-contained and not dependent on other organizations. @end quotation Non-bug suggestions are always welcome as well. If you have questions about things that are unclear in the documentation or are just obscure features, ask on the bug list; we will try to help you out if we can. -If you find bugs in one of the non-Unix ports of @command{gawk}, please -send an electronic mail message to the bug list, with a copy to the -person who maintains that port. They are named in the following list, +If you find bugs in one of the non-Unix ports of @command{gawk}, +send an email to the bug list, with a copy to the +person who maintains that port. The maintainers are named in the following list, as well as in the @file{README} file in the @command{gawk} distribution. Information in the @file{README} file should be considered authoritative if it conflicts with this @value{DOCUMENT}. @@ -38322,30 +38655,26 @@ The people maintaining the various @command{gawk} ports are: @cindex Robbins, Arnold @cindex Zaretskii, Eli @multitable {MS-Windows with MinGW} {123456789012345678901234567890123456789001234567890} -@item Unix and POSIX systems @tab Arnold Robbins, @EMAIL{arnold@@skeeve.com,arnold at skeeve dot com}. +@item Unix and POSIX systems @tab Arnold Robbins, @EMAIL{arnold@@skeeve.com,arnold at skeeve dot com} -@item MS-DOS with DJGPP @tab Scott Deifik, @EMAIL{scottd.mail@@sbcglobal.net,scottd dot mail at sbcglobal dot net}. +@item MS-DOS with DJGPP @tab Scott Deifik, @EMAIL{scottd.mail@@sbcglobal.net,scottd dot mail at sbcglobal dot net} -@item MS-Windows with MinGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org}. +@item MS-Windows with MinGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org} @c Leave this in the print version on purpose. @c OS/2 is not mentioned anywhere else in the print version though. -@item OS/2 @tab Andreas Buening, @EMAIL{andreas.buening@@nexgo.de,andreas dot buening at nexgo dot de}. +@item OS/2 @tab Andreas Buening, @EMAIL{andreas.buening@@nexgo.de,andreas dot buening at nexgo dot de} -@item VMS @tab John Malmberg, @EMAIL{wb8tyw@@qsl.net,wb8tyw at qsl.net}. +@item VMS @tab John Malmberg, @EMAIL{wb8tyw@@qsl.net,wb8tyw at qsl.net} -@item z/OS (OS/390) @tab Dave Pitts, @EMAIL{dpitts@@cozx.com,dpitts at cozx dot com}. +@item z/OS (OS/390) @tab Dave Pitts, @EMAIL{dpitts@@cozx.com,dpitts at cozx dot com} @end multitable -If your bug is also reproducible under Unix, please send a copy of your -report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email -list as well. -@c ENDOFRANGE dbugg -@c ENDOFRANGE tblgawb +If your bug is also reproducible under Unix, send a copy of your +report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email list as well. @node Other Versions @appendixsec Other Freely Available @command{awk} Implementations -@c STARTOFRANGE awkim @cindex @command{awk}, implementations @ignore From: emory!amc.com!brennan (Michael Brennan) @@ -38357,7 +38686,7 @@ Date: Wed, 4 Sep 1996 08:11:48 -0700 (PDT) @cindex Brennan, Michael @ifnotdocbook @quotation -@i{It's kind of fun to put comments like this in your awk code.}@* +@i{It's kind of fun to put comments like this in your awk code:}@* @ @ @ @ @ @ @code{// Do C++ comments work? answer: yes! of course} @author Michael Brennan @end quotation @@ -38370,8 +38699,6 @@ Date: Wed, 4 Sep 1996 08:11:48 -0700 (PDT) </blockquote> @end docbook - - There are a number of other freely available @command{awk} implementations. This @value{SECTION} briefly describes where to get them: @@ -38384,7 +38711,7 @@ This @value{SECTION} briefly describes where to get them: Brian Kernighan, one of the original designers of Unix @command{awk}, has made his implementation of @command{awk} freely available. -You can retrieve this version via the World Wide Web from +You can retrieve this version via @uref{http://www.cs.princeton.edu/~bwk, his home page}. It is available in several archive formats: @@ -38400,14 +38727,14 @@ It is available in several archive formats: @end table @cindex @command{git} utility -You can also retrieve it from Git Hub: +You can also retrieve it from GitHub: @example git clone git://github.com/onetrueawk/awk bwkawk @end example @noindent -The above command creates a copy of the @uref{http://www.git-scm.com, Git} +This command creates a copy of the @uref{http://git-scm.com, Git} repository in a directory named @file{bwkawk}. If you leave that argument off the @command{git} command line, the repository copy is created in a directory named @file{awk}. @@ -38415,9 +38742,13 @@ directory named @file{awk}. This version requires an ISO C (1990 standard) compiler; the C compiler from GCC (the GNU Compiler Collection) works quite nicely. -@xref{Common Extensions}, +@DBXREF{Common Extensions} for a list of extensions in this @command{awk} that are not in POSIX @command{awk}. +As a side note, Dan Bornstein has created a Git repository tracking +all the versions of BWK @command{awk} that he could find. It's +available at @uref{git://github.com/danfuzz/one-true-awk}. + @cindex Brennan, Michael @cindex @command{mawk} utility @cindex source code, @command{mawk} @@ -38447,7 +38778,7 @@ Once you have it, is similar to @command{gawk}'s (@pxref{Unix Installation}). -@xref{Common Extensions}, +@DBXREF{Common Extensions} for a list of extensions in @command{mawk} that are not in POSIX @command{awk}. @cindex Sumner, Andrew @@ -38456,7 +38787,7 @@ for a list of extensions in @command{mawk} that are not in POSIX @command{awk}. @item @command{awka} Written by Andrew Sumner, @command{awka} translates @command{awk} programs into C, compiles them, -and links them with a library of functions that provides the core +and links them with a library of functions that provide the core @command{awk} functionality. It also has a number of extensions. @@ -38468,7 +38799,7 @@ To get @command{awka}, go to @url{http://sourceforge.net/projects/awka}. @c andrewsumner@@yahoo.net The project seems to be frozen; no new code changes have been made -since approximately 2003. +since approximately 2001. @cindex Beebe, Nelson H.F.@: @cindex @command{pawk} (profiling version of Brian Kernighan's @command{awk}) @@ -38477,17 +38808,17 @@ since approximately 2003. Nelson H.F.@: Beebe at the University of Utah has modified BWK @command{awk} to provide timing and profiling information. It is different from @command{gawk} with the @option{--profile} option -(@pxref{Profiling}), +(@pxref{Profiling}) in that it uses CPU-based profiling, not line-count profiling. You may find it at either @uref{ftp://ftp.math.utah.edu/pub/pawk/pawk-20030606.tar.gz} or @uref{http://www.math.utah.edu/pub/pawk/pawk-20030606.tar.gz}. -@item Busybox Awk -@cindex Busybox Awk -@cindex source code, Busybox Awk -Busybox is a GPL-licensed program providing small versions of many +@item BusyBox @command{awk} +@cindex BusyBox Awk +@cindex source code, BusyBox Awk +BusyBox is a GPL-licensed program providing small versions of many applications within a single executable. It is aimed at embedded systems. It includes a full implementation of POSIX @command{awk}. When building it, be careful not to do @samp{make install} as it will overwrite @@ -38499,7 +38830,7 @@ information, see the @uref{http://busybox.net, project's home page}. @cindex source code, Solaris @command{awk} @item The OpenSolaris POSIX @command{awk} The versions of @command{awk} in @file{/usr/xpg4/bin} and -@file{/usr/xpg6/bin} on Solaris are more-or-less POSIX-compliant. +@file{/usr/xpg6/bin} on Solaris are more or less POSIX-compliant. They are based on the @command{awk} from Mortice Kern Systems for PCs. We were able to make this code compile and work under GNU/Linux with 1--2 hours of work. Making it more generally portable (using @@ -38509,8 +38840,8 @@ has not been done, at least to our knowledge. @cindex Illumos @cindex Illumos, POSIX-compliant @command{awk} @cindex source code, Illumos @command{awk} -The source code used to be available from the OpenSolaris web site. -However, that project was ended and the web site shut down. Fortunately, the +The source code used to be available from the OpenSolaris website. +However, that project was ended and the website shut down. Fortunately, the @uref{http://wiki.illumos.org/display/illumos/illumos+Home, Illumos project} makes this implementation available. You can view the files one at a time from @uref{https://github.com/joyent/illumos-joyent/blob/master/usr/src/cmd/awk_xpg4}. @@ -38529,7 +38860,7 @@ from POSIX @command{awk}. More information is available on the @cindex libmawk @cindex source code, libmawk This is an embeddable @command{awk} interpreter derived from -@command{mawk}. For more information see +@command{mawk}. For more information, see @uref{http://repo.hu/projects/libmawk/}. @item @code{pawk} @@ -38540,10 +38871,10 @@ features to Python. See @uref{https://github.com/alecthomas/pawk} for more information. (This is not related to Nelson Beebe's modified version of BWK @command{awk}, described earlier.) -@item @w{QSE Awk} -@cindex QSE Awk -@cindex source code, QSE Awk -This is an embeddable @command{awk} interpreter. For more information +@item @w{QSE @command{awk}} +@cindex QSE @command{awk} +@cindex source code, QSE @command{awk} +This is an embeddable @command{awk} interpreter. For more information, see @uref{http://code.google.com/p/qse/} and @uref{http://awk.info/?tools/qse}. @item @command{QTawk} @@ -38558,19 +38889,19 @@ including the manual and a download link. The project may also be frozen; no new code changes have been made since approximately 2008. -@item Other Versions -See also the @uref{http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations, -Wikipedia article}, for information on additional versions. +@item Other versions +See also the ``Versions and implementations'' section of the +@uref{http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations, +Wikipedia article} on @command{awk} for information on additional versions. @end table -@c ENDOFRANGE awkim @node Installation summary @appendixsec Summary @itemize @value{BULLET} @item -The @command{gawk} distribution is available from GNU project's main +The @command{gawk} distribution is available from the GNU Project's main distribution site, @code{ftp.gnu.org}. The canonical build recipe is: @example @@ -38582,34 +38913,30 @@ cd gawk-@value{VERSION}.@value{PATCHLEVEL} @item @command{gawk} may be built on non-POSIX systems as well. The currently -supported systems are MS-Windows using DJGPP, MSYS, MinGW and Cygwin, +supported systems are MS-Windows using DJGPP, MSYS, MinGW, and Cygwin, @ifclear FOR_PRINT OS/2 using EMX, @end ifclear and both Vax/VMS and OpenVMS. -Instructions for each system are included in this @value{CHAPTER}. +Instructions for each system are included in this @value{APPENDIX}. @item Bug reports should be sent via email to @email{bug-gawk@@gnu.org}. -Bug reports should be in English, and should include the version of @command{gawk}, -how it was compiled, and a short program and @value{DF} which demonstrate +Bug reports should be in English and should include the version of @command{gawk}, +how it was compiled, and a short program and @value{DF} that demonstrate the problem. @item There are a number of other freely available @command{awk} -implementations. Many are POSIX compliant; others are less so. +implementations. Many are POSIX-compliant; others are less so. @end itemize -@c ENDOFRANGE gligawk -@c ENDOFRANGE ingawk @ifclear FOR_PRINT @node Notes @appendix Implementation Notes -@c STARTOFRANGE gawii @cindex @command{gawk}, implementation issues -@c STARTOFRANGE impis @cindex implementation issues, @command{gawk} This appendix contains information mainly of interest to implementers and @@ -38685,7 +39012,7 @@ However, if you want to modify @command{gawk} and contribute back your changes, you will probably wish to work with the development version. To do so, you will need to access the @command{gawk} source code repository. The code is maintained using the -@uref{http://git-scm.com/, Git distributed version control system}. +@uref{http://git-scm.com, Git distributed version control system}. You will need to install it if your system doesn't have it. Once you have done so, use the command: @@ -38714,11 +39041,8 @@ that has a Git plug-in for working with Git repositories. @node Adding Code @appendixsubsec Adding New Features -@c STARTOFRANGE adfgaw @cindex adding, features to @command{gawk} -@c STARTOFRANGE fadgaw @cindex features, adding to @command{gawk} -@c STARTOFRANGE gawadf @cindex @command{gawk}, features, adding You are free to add any new features you like to @command{gawk}. However, if you want your changes to be incorporated into the @command{gawk} @@ -38753,7 +39077,7 @@ for information on getting the latest version of @command{gawk}.) @item @ifnotinfo -Follow the @uref{http://www.gnu.org/prep/standards/, @cite{GNU Coding Standards}}. +Follow the @cite{GNU Coding Standards}. @end ifnotinfo @ifinfo See @inforef{Top, , Version, standards, GNU Coding Standards}. @@ -38762,7 +39086,7 @@ This document describes how GNU software should be written. If you haven't read it, please do so, preferably @emph{before} starting to modify @command{gawk}. (The @cite{GNU Coding Standards} are available from the GNU Project's -@uref{http://www.gnu.org/prep/standards_toc.html, web site}. +@uref{http://www.gnu.org/prep/standards/, website}. Texinfo, Info, and DVI versions are also available.) @cindex @command{gawk}, coding style in @@ -38885,9 +39209,6 @@ Although this sounds like a lot of work, please remember that while you may write the new code, I have to maintain it and support it. If it isn't possible for me to do that with a minimum of extra work, then I probably will not. -@c ENDOFRANGE adfgaw -@c ENDOFRANGE gawadf -@c ENDOFRANGE fadgaw @node New Ports @appendixsubsec Porting @command{gawk} to a New Operating System @@ -39021,7 +39342,6 @@ coding style and brace layout that suits your taste. @node Derived Files @appendixsubsec Why Generated Files Are Kept In Git -@c STARTOFRANGE gawkgit @cindex Git, use of for @command{gawk} source code @c From emails written March 22, 2012, to the gawk developers list. @@ -39210,7 +39530,6 @@ wget http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-@var{branchname}.ta @noindent to retrieve a snapshot of the given branch. -@c ENDOFRANGE gawkgit @node Future Extensions @appendixsec Probable Future Extensions @@ -39591,13 +39910,10 @@ of @command{gawk}, but it @emph{will} be removed in the next major release. @end itemize -@c ENDOFRANGE impis -@c ENDOFRANGE gawii @node Basic Concepts @appendix Basic Programming Concepts @cindex programming, concepts -@c STARTOFRANGE procon @cindex programming, concepts This @value{APPENDIX} attempts to define some of the basic concepts @@ -39835,7 +40151,6 @@ standard for C. This standard became an ISO standard in 1990. In 1999, a revised ISO C standard was approved and released. Where it makes sense, POSIX @command{awk} is compatible with 1999 ISO C. -@c ENDOFRANGE procon @node Glossary @unnumbered Glossary @@ -39847,6 +40162,13 @@ pattern matches an input record, @command{awk} executes the rule's action. Actions are always enclosed in braces. (@xref{Action Overview}.) +@cindex Ada programming language +@cindex programming languages, Ada +@item Ada +A programming language originally defined by the U.S.@: Department of +Defense for embedded programming. It was designed to enforce good +Software Engineering practices. + @cindex Spencer, Henry @cindex @command{sed} utility @cindex amazing @command{awk} assembler (@command{aaa}) @@ -39858,13 +40180,6 @@ microcomputers. It is a good example of a program that would have been better written in another language. You can get it from @uref{http://awk.info/?awk100/aaa}. -@cindex Ada programming language -@cindex programming languages, Ada -@item Ada -A programming language originally defined by the U.S.@: Department of -Defense for embedded programming. It was designed to enforce good -Software Engineering practices. - @cindex amazingly workable formatter (@command{awf}) @cindex @command{awf} (amazingly workable formatter) program @item Amazingly Workable Formatter (@command{awf}) @@ -39886,6 +40201,21 @@ languages. These standards often become international standards as well. See also ``ISO.'' +@item Argument +An argument can be two different things. It can be an option or a +@value{FN} passed to a command while invoking it from the command line, or +it can be something passed to a @dfn{function} inside a program, e.g. +inside @command{awk}. + +In the latter case, an argument can be passed to a function in two ways. +Either it is given to the called function by value, i.e., a copy of the +value of the variable is made available to the called function, but the +original variable cannot be modified by the function itself; or it is +given by reference, i.e., a pointer to the interested variable is passed to +the function, which can then directly modify it. In @command{awk} +scalars are passed by value, and arrays are passed by reference. +See ``Pass By Value/Reference.'' + @item Array A grouping of multiple values under the same name. Most languages just provide sequential arrays. @@ -39927,6 +40257,25 @@ The GNU version of the standard shell @end ifinfo See also ``Bourne Shell.'' +@item Binary +Base-two notation, where the digits are @code{0}--@code{1}. Since +electronic circuitry works ``naturally'' in base 2 (just think of Off/On), +everything inside a computer is calculated using base 2. Each digit +represents the presence (or absence) of a power of 2 and is called a +@dfn{bit}. So, for example, the base-two number @code{10101} is +the same as decimal 21, ((1 x 16) + (1 x 4) + (1 x 1)). + +Since base-two numbers quickly become +very long to read and write, they are usually grouped by 3 (i.e., they are +read as octal numbers), or by 4 (i.e., they are read as hexadecimal +numbers). There is no direct way to insert base 2 numbers in a C program. +If need arises, such numbers are usually inserted as octal or hexadecimal +numbers. The number of base-two digits that fit into registers used for +representing integer numbers in computers is a rough indication of the +computing power of the computer itself. Most computers nowadays use 64 +bits for representing integer numbers in their registers, but 32-bit, +16-bit and 8-bit registers have been widely used in the past. +@xref{Nondecimal-numbers}. @item Bit Short for ``Binary Digit.'' All values in computer memory ultimately reduce to binary digits: values @@ -39953,6 +40302,24 @@ originally written by Steven R.@: Bourne at Bell Laboratories. Many shells (Bash, @command{ksh}, @command{pdksh}, @command{zsh}) are generally upwardly compatible with the Bourne shell. +@item Braces +The characters @samp{@{} and @samp{@}}. Braces are used in +@command{awk} for delimiting actions, compound statements, and function +bodies. + +@item Bracket Expression +Inside a @dfn{regular expression}, an expression included in square +brackets, meant to designate a single character as belonging to a +specified character class. A bracket expression can contain a list of one +or more characters, like @samp{[abc]}, a range of characters, like +@samp{[A-Z]}, or a name, delimited by @samp{:}, that designates a known set +of characters, like @samp{[:digit:]}. The form of bracket expression +enclosed between @samp{:} is independent of the underlying representation +of the character themselves, which could utilize the ASCII, ECBDIC, or +Unicode codesets, depending on the architecture of the computer system, and on +localization. +See also ``Regular Expression.'' + @item Built-in Function The @command{awk} language provides built-in functions that perform various numerical, I/O-related, and string computations. Examples are @@ -39998,11 +40365,6 @@ are the variables that have special meaning to @command{gawk}. Changing some of them affects @command{awk}'s running environment. (@xref{Built-in Variables}.) -@item Braces -The characters @samp{@{} and @samp{@}}. Braces are used in -@command{awk} for delimiting actions, compound statements, and function -bodies. - @item C The system programming language that most GNU software is written in. The @command{awk} programming language has C-like syntax, and this @value{DOCUMENT} @@ -40011,9 +40373,25 @@ points out similarities between @command{awk} and C when appropriate. In general, @command{gawk} attempts to be as similar to the 1990 version of ISO C as makes sense. +@item C Shell +The C Shell (@command{csh} or its improved version, @command{tcsh}) is a Unix shell that was +created by Bill Joy in the late 1970s. The C shell was differentiated from +other shells by its interactive features and overall style, which +looks more like C. The C Shell is not backward compatible with the Bourne +Shell, so special attention is required when converting scripts +written for other Unix shells to the C shell, especially with regard to the management of +shell variables. +See also ``Bourne Shell.'' + @item C++ A popular object-oriented programming language derived from C. +@item Character Class +See ``Bracket Expression.'' + +@item Character List +See ``Bracket Expression.'' + @cindex ASCII @cindex ISO 8859-1 @cindex ISO Latin-1 @@ -40037,7 +40415,59 @@ A preprocessor for @command{pic} that reads descriptions of molecules and produces @command{pic} input for drawing them. It was written in @command{awk} by Brian Kernighan and Jon Bentley, and is available from -@uref{http://netlib.sandia.gov/netlib/typesetting/chem.gz}. +@uref{http://netlib.org/typesetting/chem}. + +@item Comparison Expression +A relation that is either true or false, such as @samp{a < b}. +Comparison expressions are used in @code{if}, @code{while}, @code{do}, +and @code{for} +statements, and in patterns to select which input records to process. +(@xref{Typing and Comparison}.) + +@cindex compiled programs +@item Compiler +A program that translates human-readable source code into +machine-executable object code. The object code is then executed +directly by the computer. +See also ``Interpreter.'' + +@item Complemented Bracket Expression +The negation of a @dfn{bracket expression}. All that is @emph{not} +described by a given bracket expression. The symbol @samp{^} precedes +the negated bracket expression. E.g.: @samp{[[^:digit:]} +designates whatever character is not a digit. @samp{[^bad]} +designates whatever character is not one of the letters @samp{b}, @samp{a}, +or @samp{d}. +See ``Bracket Expression.'' + +@item Compound Statement +A series of @command{awk} statements, enclosed in curly braces. Compound +statements may be nested. +(@xref{Statements}.) + +@item Computed Regexps +See ``Dynamic Regular Expressions.'' + +@item Concatenation +Concatenating two strings means sticking them together, one after another, +producing a new string. For example, the string @samp{foo} concatenated with +the string @samp{bar} gives the string @samp{foobar}. +(@xref{Concatenation}.) + +@item Conditional Expression +An expression using the @samp{?:} ternary operator, such as +@samp{@var{expr1} ? @var{expr2} : @var{expr3}}. The expression +@var{expr1} is evaluated; if the result is true, the value of the whole +expression is the value of @var{expr2}; otherwise the value is +@var{expr3}. In either case, only one of @var{expr2} and @var{expr3} +is evaluated. (@xref{Conditional Exp}.) + +@item Control Statement +A control statement is an instruction to perform a given operation or a set +of operations inside an @command{awk} program, if a given condition is +true. Control statements are: @code{if}, @code{for}, @code{while}, and +@code{do} +(@pxref{Statements}). @cindex McIlroy, Doug @cindex cookie @@ -40087,39 +40517,6 @@ Doug @item Coprocess A subordinate program with which two-way communications is possible. -@cindex compiled programs -@item Compiler -A program that translates human-readable source code into -machine-executable object code. The object code is then executed -directly by the computer. -See also ``Interpreter.'' - -@item Compound Statement -A series of @command{awk} statements, enclosed in curly braces. Compound -statements may be nested. -(@xref{Statements}.) - -@item Concatenation -Concatenating two strings means sticking them together, one after another, -producing a new string. For example, the string @samp{foo} concatenated with -the string @samp{bar} gives the string @samp{foobar}. -(@xref{Concatenation}.) - -@item Conditional Expression -An expression using the @samp{?:} ternary operator, such as -@samp{@var{expr1} ? @var{expr2} : @var{expr3}}. The expression -@var{expr1} is evaluated; if the result is true, the value of the whole -expression is the value of @var{expr2}; otherwise the value is -@var{expr3}. In either case, only one of @var{expr2} and @var{expr3} -is evaluated. (@xref{Conditional Exp}.) - -@item Comparison Expression -A relation that is either true or false, such as @samp{a < b}. -Comparison expressions are used in @code{if}, @code{while}, @code{do}, -and @code{for} -statements, and in patterns to select which input records to process. -(@xref{Typing and Comparison}.) - @item Curly Braces See ``Braces.'' @@ -40165,15 +40562,15 @@ ordinary expression. It could be a string constant, such as @code{"foo"}, but it may also be an expression whose value can vary. (@xref{Computed Regexps}.) +@item Empty String +See ``Null String.'' + @item Environment A collection of strings, of the form @samp{@var{name}=@var{val}}, that each program has available to it. Users generally place values into the environment in order to provide information to various programs. Typical examples are the environment variables @env{HOME} and @env{PATH}. -@item Empty String -See ``Null String.'' - @cindex epoch, definition of @item Epoch The date used as the ``beginning of time'' for timestamps. @@ -40226,19 +40623,15 @@ Format strings control the appearance of output in the are controlled by the format strings contained in the predefined variables @code{CONVFMT} and @code{OFMT}. (@xref{Control Letters}.) +@item Fortran +Shorthand for FORmula TRANslator, one of the first programming languages +available for scientific calculations. It was created by John Backus, +and has been available since 1957. It is still in use today. + @item Free Documentation License This document describes the terms under which this @value{DOCUMENT} is published and may be copied. (@xref{GNU Free Documentation License}.) -@item Function -A specialized group of statements used to encapsulate general -or program-specific tasks. @command{awk} has a number of built-in -functions, and also allows you to define your own. -(@xref{Functions}.) - -@item FSF -See ``Free Software Foundation.'' - @cindex FSF (Free Software Foundation) @cindex Free Software Foundation (FSF) @cindex Stallman, Richard @@ -40248,6 +40641,26 @@ to the production and distribution of freely distributable software. It was founded by Richard M.@: Stallman, the author of the original Emacs editor. GNU Emacs is the most widely used version of Emacs today. +@item FSF +See ``Free Software Foundation.'' + +@item Function +A part of an @command{awk} program that can be invoked from every point of +the program, to perform a task. @command{awk} has several built-in +functions. +Users can define their own functions in every part of the program. +Function can be recursive, i.e., they may invoke themselves. +@xref{Functions}. +In @command{gawk} it is also possible to have functions shared +among different programs, and included where required using the +@code{@@include} directive +(@pxref{Include Files}). +In @command{gawk} the name of the function that should be invoked +can be generated at run time, i.e., dynamically. +The @command{gawk} extension API provides constructor functions +(@pxref{Constructor Functions}). + + @item @command{gawk} The GNU implementation of @command{awk}. @@ -40370,6 +40783,12 @@ meaning. Keywords are reserved and may not be used as variable names. and @code{while}. +@item Korn Shell +The Korn Shell (@command{ksh}) is a Unix shell which was developed by David Korn at Bell +Laboratories in the early 1980s. The Korn Shell is backward-compatible with the Bourne +shell and includes many features of the C shell. +See also ``Bourne Shell.'' + @cindex LGPL (Lesser General Public License) @cindex Lesser General Public License (LGPL) @cindex GNU Lesser General Public License @@ -40378,12 +40797,12 @@ This document describes the terms under which binary library archives or shared objects, and their source code may be distributed. -@item Linux -See ``GNU/Linux.'' - @item LGPL See ``Lesser General Public License.'' +@item Linux +See ``GNU/Linux.'' + @item Localization The process of providing the data necessary for an internationalized program to work in a particular language. @@ -40409,6 +40828,14 @@ Characters used within a regexp that do not stand for themselves. Instead, they denote regular expression operations, such as repetition, grouping, or alternation. +@item Nesting +Nesting is where information is organized in layers, or where objects +contain other similar objects. +In @command{gawk} the @code{@@include} +directive can be nested. The ``natural'' nesting of arithmetic and +logical operations can be changed using parentheses +(@pxref{Precedence}). + @item No-op An operation that does nothing. @@ -40429,6 +40856,11 @@ Octal numbers are written in C using a leading @samp{0}, to indicate their base. Thus, @code{013} is 11 ((1 x 8) + 3). @xref{Nondecimal-numbers}. +@item Output Record +A single chunk of data that is written out by @command{awk}. Usually, an +@command{awk} output record consists of one or more lines of text. +@xref{Records}. + @item Pattern Patterns tell @command{awk} which input records are interesting to which rules. @@ -40443,6 +40875,9 @@ An acronym describing what is possibly the most frequent source of computer usage problems. (Problem Exists Between Keyboard And Chair.) +@item Plug-in +See ``Extensions.'' + @item POSIX The name for a series of standards that specify a Portable Operating System interface. The ``IX'' denotes @@ -40467,6 +40902,9 @@ A sequence of consecutive lines from the input file(s). A pattern can specify ranges of input lines for @command{awk} to process or it can specify single lines. (@xref{Pattern Overview}.) +@item Record +See ``Input record'' and ``Output record.'' + @item Recursion When a function calls itself, either directly or indirectly. If this is clear, stop, and proceed to the next entry. @@ -40484,6 +40922,15 @@ operators. (@xref{Getline}, and @ref{Redirection}.) +@item Reference Counts +An internal mechanism in @command{gawk} to minimize the amount of memory +needed to store the value of string variables. If the value assumed by +a variable is used in more than one place, only one copy of the value +itself is kept, and the associated reference count is increased when the +same value is used by an additional variable, and decresed when the related +variable is no longer in use. When the reference count goes to zero, +the memory space used to store the value of the variable is freed. + @item Regexp See ``Regular Expression.'' @@ -40501,6 +40948,15 @@ slashes, such as @code{/foo/}. This regular expression is chosen when you write the @command{awk} program and cannot be changed during its execution. (@xref{Regexp Usage}.) +@item Regular Expression Operators +See ``Metacharacters.'' + +@item Rounding +Rounding the result of an arithmetic operation can be tricky. +More than one way of rounding exists, and in @command{gawk} +it is possible to choose which method should be used in a program. +@xref{Setting the rounding mode}. + @item Rule A segment of an @command{awk} program that specifies how to process single input records. A rule consists of a @dfn{pattern} and an @dfn{action}. @@ -40521,12 +40977,12 @@ Regular variables are scalars; arrays and functions are not. In @command{gawk}, a list of directories to search for @command{awk} program source files. In the shell, a list of directories to search for executable programs. -@item Seed -The initial value, or starting point, for a sequence of random numbers. - @item @command{sed} See ``Stream Editor.'' +@item Seed +The initial value, or starting point, for a sequence of random numbers. + @item Shell The command interpreter for Unix and POSIX-compliant systems. The shell works both interactively, and as a programming language @@ -40560,6 +41016,12 @@ A @value{FN} interpreted internally by @command{gawk}, instead of being handed directly to the underlying operating system---for example, @file{/dev/stderr}. (@xref{Special Files}.) +@item Statement +An expression inside an @command{awk} program in the action part +of a pattern--action rule, or inside an +@command{awk} function. A statement can be a variable assignment, +an array operation, a loop, etc. + @item Stream Editor A program that reads records from an input stream and processes them one or more at a time. This is in contrast with batch programs, which may @@ -40610,9 +41072,14 @@ This is standard time in Greenwich, England, which is used as a reference time for day and date calculations. See also ``Epoch'' and ``GMT.'' +@item Variable +A name for a value. In @command{awk}, variables may be either scalars +or arrays. + @item Whitespace A sequence of space, TAB, or newline characters occurring inside an input record or a string. + @end table @end ifclear @@ -40628,7 +41095,7 @@ record or a string. @end docbook @c This file is intended to be included within another document, -@c hence no sectioning command or @node. +@c hence no sectioning command or @node. @display Copyright @copyright{} 2007 Free Software Foundation, Inc. @url{http://fsf.org/} @@ -40850,7 +41317,7 @@ terms of section 4, provided that you also meet all of these conditions: @enumerate a -@item +@item The work must carry prominent notices stating that you modified it, and giving a relevant date. @@ -41300,7 +41767,7 @@ state the exclusion of warranty; and each file should have at least the ``copyright'' line and a pointer to where the full notice is found. @smallexample -@var{one line to give the program's name and a brief idea of what it does.} +@var{one line to give the program's name and a brief idea of what it does.} Copyright (C) @var{year} @var{name of author} This program is free software: you can redistribute it and/or modify @@ -41323,7 +41790,7 @@ If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: @smallexample -@var{program} Copyright (C) @var{year} @var{name of author} +@var{program} Copyright (C) @var{year} @var{name of author} This program comes with ABSOLUTELY NO WARRANTY; for details type @samp{show w}. This is free software, and you are welcome to redistribute it under certain conditions; type @samp{show c} for details. @@ -42007,3 +42474,4 @@ But to use it you have to say which sorta sucks. TODO: +Check that all dark corners are indexed properly. diff --git a/doc/gawkinet.info b/doc/gawkinet.info index 0a0d69d8..d726be0b 100644 --- a/doc/gawkinet.info +++ b/doc/gawkinet.info @@ -6,7 +6,7 @@ START-INFO-DIR-ENTRY * Gawkinet: (gawkinet). TCP/IP Internetworking With `gawk'. END-INFO-DIR-ENTRY - This is Edition 1.3 of `TCP/IP Internetworking With `gawk'', for the + This is Edition 1.3 of `TCP/IP Internetworking with `gawk'', for the 4.0.0 (or later) version of the GNU implementation of AWK. @@ -30,7 +30,7 @@ texts being (a) (see below), and with the Back-Cover Texts being (b) This file documents the networking features in GNU `awk'. - This is Edition 1.3 of `TCP/IP Internetworking With `gawk'', for the + This is Edition 1.3 of `TCP/IP Internetworking with `gawk'', for the 4.0.0 (or later) version of the GNU implementation of AWK. @@ -61,7 +61,7 @@ General Introduction This file documents the networking features in GNU Awk (`gawk') version 4.0 and later. - This is Edition 1.3 of `TCP/IP Internetworking With `gawk'', for the + This is Edition 1.3 of `TCP/IP Internetworking with `gawk'', for the 4.0.0 (or later) version of the GNU implementation of AWK. diff --git a/doc/gawkinet.texi b/doc/gawkinet.texi index 40198e1d..10223239 100644 --- a/doc/gawkinet.texi +++ b/doc/gawkinet.texi @@ -60,7 +60,7 @@ @c fit into that chapter, thus this separate document. At over 50 @c pages, I think this is the right decision. ADR. -@set TITLE TCP/IP Internetworking With @command{gawk} +@set TITLE TCP/IP Internetworking with @command{gawk} @set EDITION 1.3 @set UPDATE-MONTH December, 2010 @c gawk versions: diff --git a/doc/gawktexi.in b/doc/gawktexi.in index 875893f1..4cd04763 100644 --- a/doc/gawktexi.in +++ b/doc/gawktexi.in @@ -32,13 +32,11 @@ @ifnotdocbook @set BULLET @bullet{} @set MINUS @minus{} -@set NUL @sc{nul} @end ifnotdocbook @ifdocbook @set BULLET @set MINUS -@set NUL NUL @end ifdocbook @set xref-automatic-section-title @@ -48,12 +46,13 @@ @c applies to and all the info about who's publishing this edition @c These apply across the board. -@set UPDATE-MONTH September, 2014 +@set UPDATE-MONTH February, 2015 @set VERSION 4.1 @set PATCHLEVEL 2 +@set GAWKINETTITLE TCP/IP Internetworking with @command{gawk} @ifset FOR_PRINT -@set TITLE Effective AWK Programming +@set TITLE Effective awk Programming @end ifset @ifclear FOR_PRINT @set TITLE GAWK: Effective AWK Programming @@ -172,19 +171,31 @@ @macro DBREF{text} @ref{\text\} @end macro +@macro DBXREF{text} +@xref{\text\} +@end macro +@macro DBPXREF{text} +@pxref{\text\} +@end macro @end ifdocbook @ifnotdocbook @macro DBREF{text} @ref{\text\}, @end macro +@macro DBXREF{text} +@xref{\text\}, +@end macro +@macro DBPXREF{text} +@pxref{\text\}, +@end macro @end ifnotdocbook @ifclear FOR_PRINT @set FN file name -@set FFN File Name +@set FFN File name @set DF data file -@set DDF Data File +@set DDF Data file @set PVERSION version @end ifclear @ifset FOR_PRINT @@ -192,7 +203,7 @@ @set FFN Filename @set DF datafile @set DDF Datafile -@set PVERSION Version +@set PVERSION version @end ifset @c For HTML, spell out email addresses, to avoid problems with @@ -283,13 +294,13 @@ Fax: +1-617-542-2652 Email: <email>gnu@@gnu.org</email> URL: <ulink url="http://www.gnu.org">http://www.gnu.org/</ulink></literallayout> -<literallayout class="normal">Copyright © 1989, 1991, 1992, 1993, 1996–2005, 2007, 2009–2014 +<literallayout class="normal">Copyright © 1989, 1991, 1992, 1993, 1996–2005, 2007, 2009–2015 Free Software Foundation, Inc. All Rights Reserved.</literallayout> @end docbook @ifnotdocbook -Copyright @copyright{} 1989, 1991, 1992, 1993, 1996--2005, 2007, 2009--2014 @* +Copyright @copyright{} 1989, 1991, 1992, 1993, 1996--2005, 2007, 2009--2015 @* Free Software Foundation, Inc. @end ifnotdocbook @sp 2 @@ -312,7 +323,7 @@ A copy of the license is included in the section entitled A copy of the license may be found on the Internet at @uref{http://www.gnu.org/software/gawk/manual/html_node/GNU-Free-Documentation-License.html, -the GNU Project's web site}. +the GNU Project's website}. @end ifset @enumerate a @@ -380,10 +391,10 @@ ISBN 1-882114-28-0 @* @sp 9 @center @i{To my parents, for their love, and for the wonderful example they set for me.} @sp 1 -@center @i{To my wife Miriam, for making me complete. +@center @i{To my wife, Miriam, for making me complete. Thank you for building your life together with me.} @sp 1 -@center @i{To our children Chana, Rivka, Nachum and Malka, for enrichening our lives in innumerable ways.} +@center @i{To our children, Chana, Rivka, Nachum, and Malka, for enrichening our lives in innumerable ways.} @sp 1 @w{ } @page @@ -428,8 +439,9 @@ particular records in a file and perform operations upon them. @end ifnottex @menu -* Foreword:: Some nice words about this +* Foreword3:: Some nice words about this @value{DOCUMENT}. +* Foreword4:: More nice words. * Preface:: What this @value{DOCUMENT} is about; brief history and acknowledgments. * Getting Started:: A basic introduction to using @@ -456,7 +468,7 @@ particular records in a file and perform operations upon them. @command{gawk}. * Internationalization:: Getting @command{gawk} to speak your language. -* Debugger:: The @code{gawk} debugger. +* Debugger:: The @command{gawk} debugger. * Arbitrary Precision Arithmetic:: Arbitrary precision arithmetic with @command{gawk}. * Dynamic Extensions:: Adding new built-in functions to @@ -587,6 +599,7 @@ particular records in a file and perform operations upon them. @code{getline}. * Getline Summary:: Summary of @code{getline} Variants. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -616,6 +629,7 @@ particular records in a file and perform operations upon them. * Special Caveats:: Things to watch out for. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. * Values:: Constants, Variables, and Regular @@ -927,6 +941,7 @@ particular records in a file and perform operations upon them. * Array Functions:: Functions for working with arrays. * Flattening Arrays:: How to flatten arrays. * Creating Arrays:: How to create and populate arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension Versioning:: API Version information. * Extension API Informational Variables:: Variables providing information about @@ -939,7 +954,7 @@ particular records in a file and perform operations upon them. * Internal File Ops:: The code for internal file operations. * Using Internal File Ops:: How to use an external extension. * Extension Samples:: The sample extensions that ship with - @code{gawk}. + @command{gawk}. * Extension Sample File Functions:: The file functions sample. * Extension Sample Fnmatch:: An interface to @code{fnmatch()}. * Extension Sample Fork:: An interface to @code{fork()} and @@ -985,6 +1000,7 @@ particular records in a file and perform operations upon them. * Unix Installation:: Installing @command{gawk} under various versions of Unix. * Quick Installation:: Compiling @command{gawk} under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. * Non-Unix Installation:: Installation on Other Operating @@ -1057,8 +1073,8 @@ for enrichening our lives in innumerable ways. @summarycontents @contents -@node Foreword -@unnumbered Foreword +@node Foreword3 +@unnumbered Foreword to the Third Edition @c This bit is post-processed by a script which turns the chapter @c tag into a preface tag, and moves this stuff to before the title. @@ -1071,7 +1087,7 @@ for enrichening our lives in innumerable ways. <!-- can't put mawk into command tags. sigh. --> <affiliation><jobtitle>Author of mawk</jobtitle></affiliation> </author> - <date>March, 2001</date> + <date>March 2001</date> </prefaceinfo> @end docbook @@ -1083,21 +1099,23 @@ The circumstances started a couple of years earlier. I was working at a new job and noticed an unplugged Unix computer sitting in the corner. No one knew how to use it, and neither did I. However, -a couple of days later it was running, and +a couple of days later, it was running, and I was @code{root} and the one-and-only user. That day, I began the transition from statistician to Unix programmer. On one of many trips to the library or bookstore in search of -books on Unix, I found the gray AWK book, a.k.a.@: Aho, Kernighan and -Weinberger, @cite{The AWK Programming Language}, Addison-Wesley, -1988. AWK's simple programming paradigm---find a pattern in the +books on Unix, I found the gray AWK book, a.k.a.@: +Alfred V.@: Aho, Brian W.@: Kernighan, and +Peter J.@: Weinberger's @cite{The AWK Programming Language} (Addison-Wesley, +1988). @command{awk}'s simple programming paradigm---find a pattern in the input and then perform an action---often reduced complex or tedious data manipulations to a few lines of code. I was excited to try my hand at programming in AWK. Alas, the @command{awk} on my computer was a limited version of the -language described in the AWK book. I discovered that my computer -had ``old @command{awk}'' and the AWK book described ``new @command{awk}.'' +language described in the gray book. I discovered that my computer +had ``old @command{awk}'' and the book described +``new @command{awk}.'' I learned that this was typical; the old version refused to step aside or relinquish its name. If a system had a new @command{awk}, it was invariably called @command{nawk}, and few systems had it. @@ -1115,7 +1133,7 @@ My Unix system started out unplugged from the wall; it certainly was not plugged into a network. So, oblivious to the existence of @command{gawk} and the Unix community in general, and desiring a new @command{awk}, I wrote my own, called @command{mawk}. -Before I was finished I knew about @command{gawk}, +Before I was finished, I knew about @command{gawk}, but it was too late to stop, so I eventually posted to a @code{comp.sources} newsgroup. @@ -1124,7 +1142,7 @@ from Arnold introducing himself. He suggested we share design and algorithms and attached a draft of the POSIX standard so that I could update @command{mawk} to support language extensions added -after publication of the AWK book. +after publication of @cite{The AWK Programming Language}. Frankly, if our roles had been reversed, I would not have been so open and we probably would @@ -1143,7 +1161,7 @@ standard. On the other hand, the novice AWK programmer can study a wealth of practical programs that emphasize the power of AWK's basic idioms: -data driven control-flow, pattern matching with regular expressions, +data-driven control flow, pattern matching with regular expressions, and associative arrays. Those looking for something new can try out @command{gawk}'s interface to network protocols via special @file{/inet} files. @@ -1151,7 +1169,7 @@ interface to network protocols via special @file{/inet} files. The programs in this book make clear that an AWK program is typically much smaller and faster to develop than a counterpart written in C. -Consequently, there is often a payoff to prototype an +Consequently, there is often a payoff to prototyping an algorithm or design in AWK to get it running quickly and expose problems early. Often, the interpreted performance is adequate and the AWK prototype becomes the product. @@ -1204,7 +1222,62 @@ AWK or want to learn how, then read this book. @display Michael Brennan Author of @command{mawk} -March, 2001 +March 2001 +@end display +@end ifnotdocbook + +@node Foreword4 +@unnumbered Foreword to the Fourth Edition + +@c This bit is post-processed by a script which turns the chapter +@c tag into a preface tag, and moves this stuff to before the title. +@c Bleah. +@docbook + <prefaceinfo> + <author> + <firstname>Michael</firstname> + <surname>Brennan</surname> + <!-- can't put mawk into command tags. sigh. --> + <affiliation><jobtitle>Author of mawk</jobtitle></affiliation> + </author> + <date>October 2014</date> + </prefaceinfo> +@end docbook + +Some things don't change. Thirteen years ago I wrote: +``If you use AWK or want to learn how, then read this book.'' +True then, and still true today. + +Learning to use a programming language is about more than mastering the +syntax. One needs to acquire an understanding of how to use the +features of the language to solve practical programming problems. +A focus of this book is many examples that show how to use AWK. + +Some things do change. Our computers are much faster and have more memory. +Consequently, speed and storage inefficiencies of a high-level language +matter less. Prototyping in AWK and then rewriting in C for performance +reasons happens less, because more often the prototype is fast enough. + +Of course, there are computing operations that are best done in C or C++. +With @command{gawk} 4.1 and later, you do not have to choose between writing +your program in AWK or in C/C++. You can write most of your +program in AWK and the aspects that require C/C++ capabilities can be written +in C/C++, and then the pieces glued together when the @command{gawk} module loads +the C/C++ module as a dynamic plug-in. +@c Chapter 16 +@ref{Dynamic Extensions}, +has all the +details, and, as expected, many examples to help you learn the ins and outs. + +I enjoy programming in AWK and had fun (re)reading this book. +I think you will too. + +@ifnotdocbook +@cindex Brennan, Michael +@display +Michael Brennan +Author of @command{mawk} +October 2014 @end display @end ifnotdocbook @@ -1224,9 +1297,9 @@ March, 2001 <firstname>Arnold</firstname> <surname>Robbins</surname> <affiliation><jobtitle>Nof Ayalon</jobtitle></affiliation> - <affiliation><jobtitle>ISRAEL</jobtitle></affiliation> + <affiliation><jobtitle>Israel</jobtitle></affiliation> </author> - <date>December, 2014</date> + <date>February 2015</date> </prefaceinfo> @end docbook @@ -1238,7 +1311,7 @@ The @command{awk} utility interprets a special-purpose programming language that makes it easy to handle simple data-reformatting jobs. The GNU implementation of @command{awk} is called @command{gawk}; if you -invoke it with the proper options or environment variables +invoke it with the proper options or environment variables, it is fully compatible with the POSIX@footnote{The 2008 POSIX standard is accessible online at @w{@url{http://www.opengroup.org/onlinepubs/9699919799/}.}} @@ -1269,7 +1342,7 @@ Generate reports Validate data @item -Produce indexes and perform other document preparation tasks +Produce indexes and perform other document-preparation tasks @item Experiment with algorithms that you can adapt later to other computer @@ -1345,7 +1418,7 @@ has been removed.} @unnumberedsec History of @command{awk} and @command{gawk} @cindex recipe for a programming language @cindex programming language, recipe for -@sidebar Recipe For A Programming Language +@sidebar Recipe for a Programming Language @multitable {2 parts} {1 part @code{egrep}} {1 part @code{snobol}} @item @tab 1 part @code{egrep} @tab 1 part @code{snobol} @@ -1364,7 +1437,7 @@ more parts C. Document very well and release. @cindex Kernighan, Brian @cindex @command{awk}, history of The name @command{awk} comes from the initials of its designers: Alfred V.@: -Aho, Peter J.@: Weinberger and Brian W.@: Kernighan. The original version of +Aho, Peter J.@: Weinberger, and Brian W.@: Kernighan. The original version of @command{awk} was written in 1977 at AT&T Bell Laboratories. In 1985, a new version made the programming language more powerful, introducing user-defined functions, multiple input @@ -1388,23 +1461,23 @@ help from me, thoroughly reworked @command{gawk} for compatibility with the newer @command{awk}. Circa 1994, I became the primary maintainer. Current development focuses on bug fixes, -performance improvements, standards compliance and, occasionally, new features. +performance improvements, standards compliance, and, occasionally, new features. -In May of 1997, J@"urgen Kahrs felt the need for network access +In May 1997, J@"urgen Kahrs felt the need for network access from @command{awk}, and with a little help from me, set about adding features to do this for @command{gawk}. At that time, he also wrote the bulk of -@cite{TCP/IP Internetworking with @command{gawk}} +@cite{@value{GAWKINETTITLE}} (a separate document, available as part of the @command{gawk} distribution). His code finally became part of the main @command{gawk} distribution with @command{gawk} @value{PVERSION} 3.1. John Haque rewrote the @command{gawk} internals, in the process providing an @command{awk}-level debugger. This version became available as -@command{gawk} @value{PVERSION} 4.0, in 2011. +@command{gawk} @value{PVERSION} 4.0 in 2011. -@xref{Contributors}, -for a full list of those who made important contributions to @command{gawk}. +@DBXREF{Contributors} +for a full list of those who have made important contributions to @command{gawk}. @node Names @unnumberedsec A Rose by Any Other Name @@ -1413,11 +1486,11 @@ for a full list of those who made important contributions to @command{gawk}. The @command{awk} language has evolved over the years. Full details are provided in @ref{Language History}. The language described in this @value{DOCUMENT} -is often referred to as ``new @command{awk}''. +is often referred to as ``new @command{awk}.'' By analogy, the original version of @command{awk} is referred to as ``old @command{awk}.'' -Today, on most systems, when you run the @command{awk} utility, +On most current systems, when you run the @command{awk} utility you get some version of new @command{awk}.@footnote{Only Solaris systems still use an old @command{awk} for the default @command{awk} utility. A more modern @command{awk} lives in @@ -1477,7 +1550,9 @@ the POSIX standard for @command{awk}. This @value{DOCUMENT} has the difficult task of being both a tutorial and a reference. If you are a novice, feel free to skip over details that seem too complex. You should also ignore the many cross-references; they are for the -expert user and for the online Info and HTML versions of the @value{DOCUMENT}. +expert user and for the Info and +@uref{http://www.gnu.org/software/gawk/manual/, HTML} +versions of the @value{DOCUMENT}. @end ifnotinfo There are sidebars @@ -1492,12 +1567,15 @@ Most of the time, the examples use complete @command{awk} programs. Some of the more advanced sections show only the part of the @command{awk} program that illustrates the concept being described. -While this @value{DOCUMENT} is aimed principally at people who have not been +Although this @value{DOCUMENT} is aimed principally at people who have not been exposed to @command{awk}, there is a lot of information here that even the @command{awk} expert should find useful. In particular, the description of POSIX @command{awk} and the example programs in -@ref{Library Functions}, and in +@ref{Library Functions}, and +@ifnotdocbook +in +@end ifnotdocbook @ref{Sample Programs}, should be of interest. @@ -1505,22 +1583,30 @@ This @value{DOCUMENT} is split into several parts, as follows: @c FULLXREF ON -Part I describes the @command{awk} language and @command{gawk} program in detail. +@itemize @value{BULLET} +@item +Part I describes the @command{awk} language and the @command{gawk} program in detail. It starts with the basics, and continues through all of the features of @command{awk}. It contains the following chapters: +@c nested +@itemize @value{MINUS} +@item @ref{Getting Started}, provides the essentials you need to know to begin using @command{awk}. +@item @ref{Invoking Gawk}, describes how to run @command{gawk}, the meaning of its command-line options, and how it finds @command{awk} program source files. +@item @ref{Regexp}, introduces regular expressions in general, and in particular the flavors supported by POSIX @command{awk} and @command{gawk}. +@item @ref{Reading Files}, describes how @command{awk} reads your data. It introduces the concepts of records and fields, as well @@ -1528,46 +1614,62 @@ as the @code{getline} command. I/O redirection is first described here. Network I/O is also briefly introduced here. +@item @ref{Printing}, describes how @command{awk} programs can produce output with @code{print} and @code{printf}. +@item @ref{Expressions}, describes expressions, which are the basic building blocks for getting most things done in a program. +@item @ref{Patterns and Actions}, describes how to write patterns for matching records, actions for doing something when a record is matched, and the predefined variables @command{awk} and @command{gawk} use. +@item @ref{Arrays}, -covers @command{awk}'s one-and-only data structure: associative arrays. -Deleting array elements and whole arrays is also described, as well as -sorting arrays in @command{gawk}. It also describes how @command{gawk} -provides arrays of arrays. +covers @command{awk}'s one-and-only data structure: the associative array. +Deleting array elements and whole arrays is described, as well as +sorting arrays in @command{gawk}. The @value{CHAPTER} also describes how +@command{gawk} provides arrays of arrays. +@item @ref{Functions}, describes the built-in functions @command{awk} and @command{gawk} provide, as well as how to define your own functions. It also discusses how @command{gawk} lets you call functions indirectly. +@end itemize +@item Part II shows how to use @command{awk} and @command{gawk} for problem solving. There is lots of code here for you to read and learn from. -It contains the following chapters: +This part contains the following chapters: -@ref{Library Functions}, which provides a number of functions meant to +@c nested +@itemize @value{MINUS} +@item +@ref{Library Functions}, provides a number of functions meant to be used from main @command{awk} programs. +@item @ref{Sample Programs}, -which provides many sample @command{awk} programs. +provides many sample @command{awk} programs. +@end itemize Reading these two chapters allows you to see @command{awk} solving real problems. +@item Part III focuses on features specific to @command{gawk}. It contains the following chapters: +@c nested +@itemize @value{MINUS} +@item @ref{Advanced Features}, describes a number of advanced features. Of particular note @@ -1576,18 +1678,24 @@ have two-way communications with another process, perform TCP/IP networking, and profile your @command{awk} programs. +@item @ref{Internationalization}, describes special features for translating program messages into different languages at runtime. +@item @ref{Debugger}, describes the @command{gawk} debugger. +@item @ref{Arbitrary Precision Arithmetic}, describes advanced arithmetic facilities. +@item @ref{Dynamic Extensions}, describes how to add new variables and functions to @command{gawk} by writing extensions in C or C++. +@end itemize +@item @ifclear FOR_PRINT Part IV provides the appendices, the Glossary, and two licenses that cover the @command{gawk} source code and this @value{DOCUMENT}, respectively. @@ -1598,11 +1706,14 @@ Part IV provides the following appendices, including the GNU General Public License: @end ifset +@itemize @value{MINUS} +@item @ref{Language History}, describes how the @command{awk} language has evolved since -its first release to present. It also describes how @command{gawk} +its first release to the present. It also describes how @command{gawk} has acquired features over time. +@item @ref{Installation}, describes how to get @command{gawk}, how to compile it on POSIX-compatible systems, @@ -1612,15 +1723,44 @@ in @command{gawk} and where to get other freely available @command{awk} implementations. @ifset FOR_PRINT - +@item @ref{Copying}, presents the license that covers the @command{gawk} source code. +@end ifset + +@ifclear FOR_PRINT +@item +@ref{Notes}, +describes how to disable @command{gawk}'s extensions, as +well as how to contribute new code to @command{gawk}, +and some possible future directions for @command{gawk} development. + +@item +@ref{Basic Concepts}, +provides some very cursory background material for those who +are completely unfamiliar with computer programming. + +The @ref{Glossary}, defines most, if not all, of the significant terms used +throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, +try looking them up here. +@item +@ref{Copying}, and +@ref{GNU Free Documentation License}, +present the licenses that cover the @command{gawk} source code +and this @value{DOCUMENT}, respectively. +@end ifclear +@end itemize +@end itemize + +@ifset FOR_PRINT The version of this @value{DOCUMENT} distributed with @command{gawk} contains additional appendices and other end material. To save space, we have omitted them from the printed edition. You may find them online, as follows: +@itemize @value{BULLET} +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Notes.html, The appendix on implementation notes} describes how to disable @command{gawk}'s extensions, how to contribute @@ -1628,45 +1768,29 @@ new code to @command{gawk}, where to find information on some possible future directions for @command{gawk} development, and the design decisions behind the extension API. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Basic-Concepts.html, The appendix on basic concepts} provides some very cursory background material for those who are completely unfamiliar with computer programming. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/Glossary.html, The Glossary} -defines most, if not all, the significant terms used +defines most, if not all, of the significant terms used throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, try looking them up here. +@item @uref{http://www.gnu.org/software/gawk/manual/html_node/GNU-Free-Documentation-License.html, The GNU FDL} is the license that covers this @value{DOCUMENT}. +@end itemize Some of the chapters have exercise sections; these have also been omitted from the print edition but are available online. @end ifset -@ifclear FOR_PRINT -@ref{Notes}, -describes how to disable @command{gawk}'s extensions, as -well as how to contribute new code to @command{gawk}, -and some possible future directions for @command{gawk} development. - -@ref{Basic Concepts}, -provides some very cursory background material for those who -are completely unfamiliar with computer programming. - -The @ref{Glossary}, defines most, if not all, the significant terms used -throughout the @value{DOCUMENT}. If you find terms that you aren't familiar with, -try looking them up here. - -@ref{Copying}, and -@ref{GNU Free Documentation License}, -present the licenses that cover the @command{gawk} source code -and this @value{DOCUMENT}, respectively. -@end ifclear - @c FULLXREF OFF @node Conventions @@ -1697,7 +1821,7 @@ This typically represents the command's standard output. Output from the command, usually its standard output, appears @code{like this}. @end ifset -Error messages, and other output on the command's standard error, are preceded +Error messages and other output on the command's standard error are preceded by the glyph ``@error{}''. For example: @example @@ -1708,15 +1832,23 @@ $ @kbd{echo hello on stderr 1>&2} @end example @ifnotinfo -In the text, command names appear in @code{this font}, while code segments +In the text, almost anything related to programming, such as +command names, +variable and function names, and string, numeric and regexp constants +appear in @code{this font}. Code fragments appear in the same font and quoted, @samp{like this}. +Things that are replaced by the user or programmer +appear in @var{this font}. Options look like this: @option{-f}. +@value{FFN}s are indicated like this: @file{/path/to/ourfile}. +@ifclear FOR_PRINT Some things are emphasized @emph{like this}, and if a point needs to be made -strongly, it is done @strong{like this}. The first occurrence of +strongly, it is done @strong{like this}. +@end ifclear +The first occurrence of a new term is usually its @dfn{definition} and appears in the same font as the previous occurrence of ``definition'' in this sentence. -Finally, @value{FN}s are indicated like this: @file{/path/to/ourfile}. @end ifnotinfo Characters that you type at the keyboard look @kbd{like this}. In particular, @@ -1724,11 +1856,11 @@ there are special characters called ``control characters.'' These are characters that you type by holding down both the @kbd{CONTROL} key and another key, at the same time. For example, a @kbd{Ctrl-d} is typed by first pressing and holding the @kbd{CONTROL} key, next -pressing the @kbd{d} key and finally releasing both keys. +pressing the @kbd{d} key, and finally releasing both keys. For the sake of brevity, throughout this @value{DOCUMENT}, we refer to Brian Kernighan's version of @command{awk} as ``BWK @command{awk}.'' -(@xref{Other Versions}, for information on his and other versions.) +(@DBXREF{Other Versions} for information on his and other versions.) @ifset FOR_PRINT @quotation NOTE @@ -1744,7 +1876,7 @@ Cautionary or warning notes look like this. @unnumberedsubsec Dark Corners @cindex Kernighan, Brian @quotation -@i{Dark corners are basically fractal --- no matter how much +@i{Dark corners are basically fractal---no matter how much you illuminate, there's always a smaller but darker one.} @author Brian Kernighan @end quotation @@ -1760,7 +1892,7 @@ the picture of a flashlight in the margin, as shown here. @value{DARKCORNER} @end iftex @ifnottex -``(d.c.)''. +``(d.c.).'' @end ifnottex @ifclear FOR_PRINT They also appear in the index under the heading ``dark corner.'' @@ -1795,12 +1927,12 @@ Emacs editor. GNU Emacs is the most widely used version of Emacs today. @cindex GPL (General Public License) @cindex General Public License, See GPL @cindex documentation, online -The GNU@footnote{GNU stands for ``GNU's not Unix.''} +The GNU@footnote{GNU stands for ``GNU's Not Unix.''} Project is an ongoing effort on the part of the Free Software Foundation to create a complete, freely distributable, POSIX-compliant computing environment. -The FSF uses the ``GNU General Public License'' (GPL) to ensure that -their software's +The FSF uses the GNU General Public License (GPL) to ensure that +its software's source code is always available to the end user. @ifclear FOR_PRINT A copy of the GPL is included @@ -1814,7 +1946,7 @@ The GPL applies to the C language source code for @command{gawk}. To find out more about the FSF and the GNU Project online, see @uref{http://www.gnu.org, the GNU Project's home page}. This @value{DOCUMENT} may also be read from -@uref{http://www.gnu.org/software/gawk/manual/, their web site}. +@uref{http://www.gnu.org/software/gawk/manual/, GNU's website}. @ifclear FOR_PRINT A shell, an editor (Emacs), highly portable optimizing C, C++, and @@ -1851,16 +1983,16 @@ License in @ref{GNU Free Documentation License}.) @cindex Close, Diane The @value{DOCUMENT} itself has gone through multiple previous editions. Paul Rubin wrote the very first draft of @cite{The GAWK Manual}; -it was around 40 pages in size. +it was around 40 pages long. Diane Close and Richard Stallman improved it, yielding a version that was -around 90 pages long and barely described the original, ``old'' +around 90 pages and barely described the original, ``old'' version of @command{awk}. I started working with that version in the fall of 1988. As work on it progressed, the FSF published several preliminary versions (numbered 0.@var{x}). -In 1996, Edition 1.0 was released with @command{gawk} 3.0.0. +In 1996, edition 1.0 was released with @command{gawk} 3.0.0. The FSF published the first two editions under the title @cite{The GNU Awk User's Guide}. @ifset FOR_PRINT @@ -1872,7 +2004,7 @@ the third edition in 2001. This edition maintains the basic structure of the previous editions. For FSF edition 4.0, the content was thoroughly reviewed and updated. All references to @command{gawk} versions prior to 4.0 were removed. -Of significant note for that edition was @ref{Debugger}. +Of significant note for that edition was the addition of @ref{Debugger}. For FSF edition @ifclear FOR_PRINT @@ -1887,17 +2019,17 @@ and the major new additions are @ref{Arbitrary Precision Arithmetic}, and @ref{Dynamic Extensions}. This @value{DOCUMENT} will undoubtedly continue to evolve. If you -find an error in this @value{DOCUMENT}, please report it! @xref{Bugs}, +find an error in the @value{DOCUMENT}, please report it! @DBXREF{Bugs} for information on submitting problem reports electronically. @ifset FOR_PRINT @c fakenode --- for prepinfo @unnumberedsec How to Stay Current -It may be you have a version of @command{gawk} which is newer than the +You may have a newer version of @command{gawk} than the one described here. To find out what has changed, you should first look at the @file{NEWS} file in the @command{gawk} -distribution, which provides a high level summary of what changed in +distribution, which provides a high-level summary of the changes in each release. You can then look at the @uref{http://www.gnu.org/software/gawk/manual/, @@ -1922,16 +2054,18 @@ contributed code: the archive did not grow and the domain went unused for several years. Late in 2008, a volunteer took on the task of setting up -an @command{awk}-related web site---@uref{http://awk.info}---and did a very +an @command{awk}-related website---@uref{http://awk.info}---and did a very nice job. If you have written an interesting @command{awk} program, or have written a @command{gawk} extension that you would like to share with the rest of the world, please see @uref{http://awk.info/?contribute} for how to -contribute it to the web site. +contribute it to the website. +@ignore As of this writing, this website is in search of a maintainer; please contact me if you are interested. +@end ignore @ignore Other links: @@ -1949,7 +2083,7 @@ The initial draft of @cite{The GAWK Manual} had the following acknowledgments: Many people need to be thanked for their assistance in producing this manual. Jay Fenlason contributed many ideas and sample programs. Richard Mlynarik and Robert Chassell gave helpful comments on drafts of this -manual. The paper @cite{A Supplemental Document for @command{awk}} by John W.@: +manual. The paper @cite{A Supplemental Document for AWK} by John W.@: Pierce of the Chemistry Department at UC San Diego, pinpointed several issues relevant both to @command{awk} implementation and to this manual, that would otherwise have escaped us. @@ -1960,12 +2094,18 @@ I would like to acknowledge Richard M.@: Stallman, for his vision of a better world and for his courage in founding the FSF and starting the GNU Project. +@ifclear FOR_PRINT Earlier editions of this @value{DOCUMENT} had the following acknowledgements: +@end ifclear +@ifset FOR_PRINT +The previous edition of this @value{DOCUMENT} had +the following acknowledgements: +@end ifset @quotation The following people (in alphabetical order) provided helpful comments on various -versions of this book, +versions of this book: Rick Adams, Dr.@: Nelson H.F. Beebe, Karl Berry, @@ -1993,7 +2133,7 @@ Robert J.@: Chassell provided much valuable advice on the use of Texinfo. He also deserves special thanks for convincing me @emph{not} to title this @value{DOCUMENT} -@cite{How To Gawk Politely}. +@cite{How to Gawk Politely}. Karl Berry helped significantly with the @TeX{} part of Texinfo. @cindex Hartholz, Marshall @@ -2073,38 +2213,39 @@ portable program it is today. It has been and continues to be a pleasure working with this team of fine people. Notable code and documentation contributions were made by -a number of people. @xref{Contributors}, for the full list. +a number of people. @DBXREF{Contributors} for the full list. @ifset FOR_PRINT @cindex Oram, Andy -Thanks to Andy Oram, of O'Reilly Media, for initiating +Thanks to Andy Oram of O'Reilly Media for initiating the fourth edition and for his support during the work. +Thanks to Jasmine Kwityn for her copyediting work. @end ifset -Thanks to Michael Brennan for the Foreword. +Thanks to Michael Brennan for the Forewords. @cindex Duman, Patrice @cindex Berry, Karl Thanks to Patrice Dumas for the new @command{makeinfo} program. -Thanks to Karl Berry who continues to work to keep +Thanks to Karl Berry, who continues to work to keep the Texinfo markup language sane. @cindex Kernighan, Brian @cindex Brennan, Michael @cindex Day, Robert P.J.@: -Robert P.J.@: Day, Michael Brennan and Brian Kernighan kindly acted as +Robert P.J.@: Day, Michael Brennan, and Brian Kernighan kindly acted as reviewers for the 2015 edition of this @value{DOCUMENT}. Their feedback helped improve the final work. -I would like to thank Brian Kernighan for invaluable assistance during the -testing and debugging of @command{gawk}, and for ongoing +I would also like to thank Brian Kernighan for his invaluable assistance during the +testing and debugging of @command{gawk}, and for his ongoing help and advice in clarifying numerous points about the language. We could not have done nearly as good a job on either @command{gawk} or its documentation without his help. Brian is in a class by himself as a programmer and technical author. I have to thank him (yet again) for his ongoing friendship -and the role model he has been for me for close to 30 years! +and for being a role model to me for close to 30 years! Having him as a reviewer is an exciting privilege. It has also been extremely humbling@enddots{} @@ -2120,14 +2261,14 @@ which they raised and educated me. Finally, I also must acknowledge my gratitude to G-d, for the many opportunities He has sent my way, as well as for the gifts He has given me with which to take advantage of those opportunities. -@iftex +@ifnotdocbook @sp 2 @noindent Arnold Robbins @* Nof Ayalon @* -ISRAEL @* -December, 2014 -@end iftex +Israel @* +February 2015 +@end ifnotdocbook @ifnotinfo @part @value{PART1}The @command{awk} Language @@ -2143,31 +2284,31 @@ following chapters: @itemize @value{BULLET} @item -@ref{Getting Started}. +@ref{Getting Started} @item -@ref{Invoking Gawk}. +@ref{Invoking Gawk} @item -@ref{Regexp}. +@ref{Regexp} @item -@ref{Reading Files}. +@ref{Reading Files} @item -@ref{Printing}. +@ref{Printing} @item -@ref{Expressions}. +@ref{Expressions} @item -@ref{Patterns and Actions}. +@ref{Patterns and Actions} @item -@ref{Arrays}. +@ref{Arrays} @item -@ref{Functions}. +@ref{Functions} @end itemize @end ifdocbook @@ -2182,17 +2323,17 @@ following chapters: The basic function of @command{awk} is to search files for lines (or other units of text) that contain certain patterns. When a line matches one of the patterns, @command{awk} performs specified actions on that line. -@command{awk} keeps processing input lines in this way until it reaches +@command{awk} continues to process input lines in this way until it reaches the end of the input files. @cindex @command{awk}, uses for @cindex programming languages@comma{} data-driven vs.@: procedural @cindex @command{awk} programs Programs in @command{awk} are different from programs in most other languages, -because @command{awk} programs are @dfn{data-driven}; that is, you describe -the data you want to work with and then what to do when you find it. +because @command{awk} programs are @dfn{data driven} (i.e., you describe +the data you want to work with and then what to do when you find it). Most other languages are @dfn{procedural}; you have to describe, in great -detail, every step the program is to take. When working with procedural +detail, every step the program should take. When working with procedural languages, it is usually much harder to clearly describe the data your program will process. For this reason, @command{awk} programs are often refreshingly easy to @@ -2202,15 +2343,15 @@ read and write. @cindex rule, definition of When you run @command{awk}, you specify an @command{awk} @dfn{program} that tells @command{awk} what to do. The program consists of a series of -@dfn{rules}. (It may also contain @dfn{function definitions}, -an advanced feature that we will ignore for now. -@xref{User-defined}.) Each rule specifies one +@dfn{rules} (it may also contain @dfn{function definitions}, +an advanced feature that we will ignore for now; +@pxref{User-defined}). Each rule specifies one pattern to search for and one action to perform upon finding the pattern. -Syntactically, a rule consists of a pattern followed by an action. The -action is enclosed in braces to separate it from the pattern. -Newlines usually separate rules. Therefore, an @command{awk} +Syntactically, a rule consists of a @dfn{pattern} followed by an +@dfn{action}. The action is enclosed in braces to separate it from the +pattern. Newlines usually separate rules. Therefore, an @command{awk} program looks like this: @example @@ -2284,8 +2425,8 @@ awk '@var{program}' @var{input-file1} @var{input-file2} @dots{} @end example @noindent -where @var{program} consists of a series of @var{patterns} and -@var{actions}, as described earlier. +where @var{program} consists of a series of patterns and +actions, as described earlier. @cindex single quote (@code{'}) @cindex @code{'} (single quote) @@ -2304,11 +2445,12 @@ programs from shell scripts, because it avoids the need for a separate file for the @command{awk} program. A self-contained shell script is more reliable because there are no other files to misplace. +Later in this chapter, in +@ifdocbook +the section +@end ifdocbook @ref{Very Simple}, -@ifnotinfo -later in this @value{CHAPTER}, -@end ifnotinfo -presents several short, +we'll see examples of several short, self-contained programs. @node Read Terminal @@ -2329,10 +2471,10 @@ awk '@var{program}' which usually means whatever you type on the keyboard. This continues until you indicate end-of-file by typing @kbd{Ctrl-d}. @ifset FOR_PRINT -(On other operating systems, the end-of-file character may be different.) +(On non-POSIX operating systems, the end-of-file character may be different.) @end ifset @ifclear FOR_PRINT -(On other operating systems, the end-of-file character may be different. +(On non-POSIX operating systems, the end-of-file character may be different. For example, on OS/2, it is @kbd{Ctrl-z}.) @end ifclear @@ -2366,7 +2508,7 @@ startup file. This next simple @command{awk} program emulates the @command{cat} utility; it copies whatever you type on the -keyboard to its standard output (why this works is explained shortly). +keyboard to its standard output (why this works is explained shortly): @example $ @kbd{awk '@{ print @}'} @@ -2432,11 +2574,9 @@ for programs that are provided on the @command{awk} command line. (Also, placing the program in a file allows us to use a literal single quote in the program text, instead of the magic @samp{\47}.) -@c STARTOFRANGE sq1x @cindex single quote (@code{'}) in @command{gawk} command lines -@c STARTOFRANGE qs2x @cindex @code{'} (single quote) in @command{gawk} command lines -If you want to clearly identify your @command{awk} program files as such, +If you want to clearly identify an @command{awk} program file as such, you can add the extension @file{.awk} to the @value{FN}. This doesn't affect the execution of the @command{awk} program but it does make ``housekeeping'' easier. @@ -2451,7 +2591,7 @@ affect the execution of the @command{awk} program but it does make Once you have learned @command{awk}, you may want to write self-contained @command{awk} scripts, using the @samp{#!} script mechanism. You can do this on many systems.@footnote{The @samp{#!} mechanism works on -GNU/Linux systems, BSD-based systems and commercial Unix systems.} +GNU/Linux systems, BSD-based systems, and commercial Unix systems.} For example, you could update the file @file{advice} to look like this: @example @@ -2490,7 +2630,7 @@ according to the instructions in your program. (This is different from a @dfn{compiled} language such as C, where your program is first compiled into machine code that is executed directly by your system's processor.) The @command{awk} utility is thus termed an @dfn{interpreter}. -Many modern languages are interperted. +Many modern languages are interpreted. The line beginning with @samp{#!} lists the full @value{FN} of an interpreter to run and a single optional initial command-line argument @@ -2536,14 +2676,14 @@ can explain what the program does and how it works. Nearly all programming languages have provisions for comments, as programs are typically hard to understand without them. -In the @command{awk} language, a comment starts with the sharp sign +In the @command{awk} language, a comment starts with the number sign character (@samp{#}) and continues to the end of the line. The @samp{#} does not have to be the first character on the line. The -@command{awk} language ignores the rest of a line following a sharp sign. +@command{awk} language ignores the rest of a line following a number sign. For example, we could have put the following into @file{advice}: @example -# This program prints a nice friendly message. It helps +# This program prints a nice, friendly message. It helps # keep novice users from being afraid of the computer. BEGIN @{ print "Don't Panic!" @} @end example @@ -2559,7 +2699,8 @@ when reading it at a later time. @quotation CAUTION As mentioned in @ref{One-shot}, -you can enclose small to medium programs in single quotes, in order to keep +you can enclose short to medium-sized programs in single quotes, +in order to keep your shell scripts self-contained. When doing so, @emph{don't} put an apostrophe (i.e., a single quote) into a comment (or anywhere else in your program). The shell interprets the quote as the closing @@ -2588,19 +2729,19 @@ $ @kbd{awk '@{ print "hello" @} # let's be cute'} @cindex @code{\} (backslash) @cindex backslash (@code{\}) Putting a backslash before the single quote in @samp{let's} wouldn't help, -since backslashes are not special inside single quotes. +because backslashes are not special inside single quotes. The next @value{SUBSECTION} describes the shell's quoting rules. @end quotation @node Quoting -@subsection Shell-Quoting Issues +@subsection Shell Quoting Issues @cindex shell quoting, rules for @menu * DOS Quoting:: Quoting in Windows Batch Files. @end menu -For short to medium length @command{awk} programs, it is most convenient +For short to medium-length @command{awk} programs, it is most convenient to enter the program on the @command{awk} command line. This is best done by enclosing the entire program in single quotes. This is true whether you are entering the program interactively at @@ -2624,8 +2765,8 @@ or empty, string. The null string is character data that has no value. In other words, it is empty. It is written in @command{awk} programs like this: @code{""}. In the shell, it can be written using single -or double quotes: @code{""} or @code{''}. While the null string has -no characters in it, it does exist. Consider this command: +or double quotes: @code{""} or @code{''}. Although the null string has +no characters in it, it does exist. For example, consider this command: @example $ @kbd{echo ""} @@ -2635,8 +2776,7 @@ $ @kbd{echo ""} Here, the @command{echo} utility receives a single argument, even though that argument has no characters in it. In the rest of this @value{DOCUMENT}, we use the terms @dfn{null string} and @dfn{empty string} -interchangeably. Now, on to the quoting rules. - +interchangeably. Now, on to the quoting rules: @itemize @value{BULLET} @item @@ -2659,7 +2799,7 @@ The shell does no interpretation of the quoted text, passing it on verbatim to the command. It is @emph{impossible} to embed a single quote inside single-quoted text. Refer back to -@ref{Comments}, +@DBREF{Comments} for an example of what happens if you try. @item @@ -2669,7 +2809,7 @@ Double quotes protect most things between the opening and closing quotes. The shell does at least variable and command substitution on the quoted text. Different shells may do additional kinds of processing on double-quoted text. -Since certain characters within double-quoted text are processed by the shell, +Because certain characters within double-quoted text are processed by the shell, they must be @dfn{escaped} within the text. Of note are the characters @samp{$}, @samp{`}, @samp{\}, and @samp{"}, all of which must be preceded by a backslash within double-quoted text if they are to be passed on literally @@ -2731,7 +2871,7 @@ $ @kbd{awk 'BEGIN @{ print "Here is a single quote <'"'"'>" @}'} @noindent This program consists of three concatenated quoted strings. The first and the -third are single-quoted, the second is double-quoted. +third are single-quoted, and the second is double-quoted. This can be ``simplified'' to: @@ -2752,8 +2892,6 @@ $ @kbd{awk "BEGIN @{ print \"Here is a single quote <'>\" @}"} @end example @noindent -@c ENDOFRANGE sq1x -@c ENDOFRANGE qs2x This option is also painful, because double quotes, backslashes, and dollar signs are very common in more advanced @command{awk} programs. @@ -2770,22 +2908,22 @@ $ @kbd{awk 'BEGIN @{ print "Here is a double quote <\42>" @}'} @end example @noindent -This works nicely, except that you should comment clearly what the +This works nicely, but you should comment clearly what the escapes mean. A fourth option is to use command-line variable assignment, like this: @example -$ awk -v sq="'" 'BEGIN @{ print "Here is a single quote <" sq ">" @}' +$ @kbd{awk -v sq="'" 'BEGIN @{ print "Here is a single quote <" sq ">" @}'} @print{} Here is a single quote <'> @end example (Here, the two string constants and the value of @code{sq} are concatenated -into a single string which is printed by @code{print}.) +into a single string that is printed by @code{print}.) If you really need both single and double quotes in your @command{awk} program, it is probably best to move it into a separate file, where -the shell won't be part of the picture, and you can say what you mean. +the shell won't be part of the picture and you can say what you mean. @node DOS Quoting @subsubsection Quoting in MS-Windows Batch Files @@ -2847,7 +2985,7 @@ information about monthly shipments. In both files, each line is considered to be one @dfn{record}. In @file{mail-list}, each record contains the name of a person, -his/her phone number, his/her email-address, and a code for their relationship +his/her phone number, his/her email address, and a code for his/her relationship with the author of the list. The columns are aligned using spaces. An @samp{A} in the last column @@ -2884,7 +3022,7 @@ of green crates shipped, the number of red boxes shipped, the number of orange bags shipped, and the number of blue packages shipped, respectively. There are 16 entries, covering the 12 months of last year and the first four months of the current year. -An empty line separates the data for the two years. +An empty line separates the data for the two years: @example @c file eg/data/inventory-shipped @@ -2918,7 +3056,7 @@ The following command runs a simple @command{awk} program that searches the input file @file{mail-list} for the character string @samp{li} (a grouping of characters is usually called a @dfn{string}; the term @dfn{string} is based on similar usage in English, such -as ``a string of pearls,'' or ``a string of cars in a train''): +as ``a string of pearls'' or ``a string of cars in a train''): @example awk '/li/ @{ print $0 @}' mail-list @@ -2965,11 +3103,11 @@ omitting the @code{print} statement but retaining the braces makes an empty action that does nothing (i.e., no lines are printed). @cindex @command{awk} programs, one-line examples -Many practical @command{awk} programs are just a line or two. Following is a +Many practical @command{awk} programs are just a line or two long. Following is a collection of useful, short programs to get you started. Some of these programs contain constructs that haven't been covered yet. (The description -of the program will give you a good idea of what is going on, but please -read the rest of the @value{DOCUMENT} to become an @command{awk} expert!) +of the program will give you a good idea of what is going on, but you'll +need to read the rest of the @value{DOCUMENT} to become an @command{awk} expert!) Most of the examples use a @value{DF} named @file{data}. This is just a placeholder; if you use these programs yourself, substitute your own @value{FN}s for @file{data}. @@ -2986,7 +3124,7 @@ Print every line that is longer than 80 characters: awk 'length($0) > 80' data @end example -The sole rule has a relational expression as its pattern and it has no +The sole rule has a relational expression as its pattern and has no action---so it uses the default action, printing the record. @item @@ -3010,7 +3148,7 @@ expand data | awk '@{ if (x < length($0)) x = length($0) @} @end example This example differs slightly from the previous one: -The input is processed by the @command{expand} utility to change TABs +the input is processed by the @command{expand} utility to change TABs into spaces, so the widths compared are actually the right-margin columns, as opposed to the number of input characters on each line. @@ -3073,7 +3211,7 @@ Print the even-numbered lines in the @value{DF}: awk 'NR % 2 == 0' data @end example -If you use the expression @samp{NR % 2 == 1} instead, +If you used the expression @samp{NR % 2 == 1} instead, the program would print the odd-numbered lines. @end itemize @@ -3089,8 +3227,13 @@ no actions run. After processing all the rules that match the line (and perhaps there are none), @command{awk} reads the next line. (However, -@pxref{Next Statement}, +@DBPXREF{Next Statement} +@ifdocbook +and @DBREF{Nextfile Statement}.) +@end ifdocbook +@ifnotdocbook and also @pxref{Nextfile Statement}.) +@end ifnotdocbook This continues until the program reaches the end of the file. For example, the following @command{awk} program contains two rules: @@ -3264,7 +3407,7 @@ lines in the middle of a regular expression or a string. with the C shell.} It works for @command{awk} programs in files and for one-shot programs, @emph{provided} you are using a POSIX-compliant shell, such as the Unix Bourne shell or Bash. But the C shell behaves -differently! There, you must use two backslashes in a row, followed by +differently! There you must use two backslashes in a row, followed by a newline. Note also that when using the C shell, @emph{every} newline in your @command{awk} program must be escaped with a backslash. To illustrate: @@ -3305,9 +3448,9 @@ starts a comment, it ignores @emph{everything} on the rest of the line. For example: @example -$ gawk 'BEGIN @{ print "dont panic" # a friendly \ -> BEGIN rule -> @}' +$ @kbd{gawk 'BEGIN @{ print "dont panic" # a friendly \} +> @kbd{ BEGIN rule} +> @kbd{@}'} @error{} gawk: cmd. line:2: BEGIN rule @error{} gawk: cmd. line:2: ^ syntax error @end example @@ -3355,9 +3498,9 @@ performing bit manipulation, for runtime string translation (internationalizatio determining the type of a variable, and array sorting. -As we develop our presentation of the @command{awk} language, we introduce +As we develop our presentation of the @command{awk} language, we will introduce most of the variables and many of the functions. They are described -systematically in @ref{Built-in Variables}, and in +systematically in @DBREF{Built-in Variables} and in @ref{Built-in}. @node When @@ -3391,8 +3534,8 @@ eight-bit microprocessors, @end ifset and a microcode assembler for a special-purpose Prolog computer. -While the original @command{awk}'s capabilities were strained by tasks -of such complexity, modern versions are more capable. +The original @command{awk}'s capabilities were strained by tasks +of such complexity, but modern versions are more capable. @cindex @command{awk} programs, complex If you find yourself writing @command{awk} scripts of more than, say, @@ -3409,7 +3552,7 @@ and Perl.} @c FIXME: Review this chapter for summary of builtin functions called. @itemize @value{BULLET} @item -Programs in @command{awk} consist of @var{pattern}-@var{action} pairs. +Programs in @command{awk} consist of @var{pattern}--@var{action} pairs. @item An @var{action} without a @var{pattern} always runs. The default @@ -3438,7 +3581,7 @@ part of a larger shell script (or MS-Windows batch file). You may use backslash continuation to continue a source line. Lines are automatically continued after a comma, open brace, question mark, colon, -@samp{||}, @samp{&&}, @code{do} and @code{else}. +@samp{||}, @samp{&&}, @code{do}, and @code{else}. @end itemize @node Invoking Gawk @@ -3447,7 +3590,7 @@ a comma, open brace, question mark, colon, This @value{CHAPTER} covers how to run @command{awk}, both POSIX-standard and @command{gawk}-specific command-line options, and what @command{awk} and -@command{gawk} do with non-option arguments. +@command{gawk} do with nonoption arguments. It then proceeds to cover how @command{gawk} searches for source files, reading standard input along with other files, @command{gawk}'s environment variables, @command{gawk}'s exit status, using include files, @@ -3491,7 +3634,7 @@ enclosed in [@dots{}] in these templates are optional: @cindex GNU long options @cindex long options @cindex options, long -Besides traditional one-letter POSIX-style options, @command{gawk} also +In addition to traditional one-letter POSIX-style options, @command{gawk} also supports GNU long options. @cindex dark corner, invoking @command{awk} @@ -3513,20 +3656,16 @@ warning that the program is empty. @node Options @section Command-Line Options -@c STARTOFRANGE ocl @cindex options, command-line -@c STARTOFRANGE clo @cindex command line, options -@c STARTOFRANGE gnulo @cindex GNU long options -@c STARTOFRANGE longo @cindex options, long Options begin with a dash and consist of a single character. GNU-style long options consist of two dashes and a keyword. The keyword can be abbreviated, as long as the abbreviation allows the option -to be uniquely identified. If the option takes an argument, then the -keyword is either immediately followed by an equals sign (@samp{=}) and the +to be uniquely identified. If the option takes an argument, either the +keyword is immediately followed by an equals sign (@samp{=}) and the argument's value, or the keyword and the argument's value are separated by whitespace. If a particular option with a value is given more than once, it is the @@ -3553,8 +3692,8 @@ Set the @code{FS} variable to @var{fs} @cindex @option{-f} option @cindex @option{--file} option @cindex @command{awk} programs, location of -Read @command{awk} program source from @var{source-file} -instead of in the first non-option argument. +Read the @command{awk} program source from @var{source-file} +instead of in the first nonoption argument. This option may be given multiple times; the @command{awk} program consists of the concatenation of the contents of each specified @var{source-file}. @@ -3608,8 +3747,6 @@ by the user that could start with @samp{-}. It is also useful for passing options on to the @command{awk} program; see @ref{Getopt Function}. @end table -@c ENDOFRANGE gnulo -@c ENDOFRANGE longo The following list describes @command{gawk}-specific options: @@ -3621,14 +3758,14 @@ The following list describes @command{gawk}-specific options: @cindex @option{--characters-as-bytes} option Cause @command{gawk} to treat all input data as single-byte characters. In addition, all output written with @code{print} or @code{printf} -are treated as single-byte characters. +is treated as single-byte characters. Normally, @command{gawk} follows the POSIX standard and attempts to process its input data according to the current locale (@pxref{Locales}). This can often involve converting multibyte characters into wide characters (internally), and can lead to problems or confusion if the input data does not contain valid -multibyte characters. This option is an easy way to tell @command{gawk}: -``hands off my data!''. +multibyte characters. This option is an easy way to tell @command{gawk}, +``Hands off my data!'' @item @option{-c} @itemx @option{--traditional} @@ -3682,9 +3819,10 @@ names like @code{i}, @code{j}, etc.) @cindex @command{awk} debugging, enabling Enable debugging of @command{awk} programs (@pxref{Debugging}). -By default, the debugger reads commands interactively from the keyboard. +By default, the debugger reads commands interactively from the keyboard +(standard input). The optional @var{file} argument allows you to specify a file with a list -of commands for the debugger to execute non-interactively. +of commands for the debugger to execute noninteractively. No space is allowed between the @option{-D} and @var{file}, if @var{file} is supplied. @@ -3722,7 +3860,13 @@ Command-line variable assignments of the form This option is particularly necessary for World Wide Web CGI applications that pass arguments through the URL; using this option prevents a malicious (or other) user from passing in options, assignments, or @command{awk} source -code (via @option{-e}) to the CGI application. This option should be used +code (via @option{-e}) to the CGI application.@footnote{For more detail, +please see Section 4.4 of @uref{http://www.ietf.org/rfc/rfc3875, +RFC 3875}. Also see the +@uref{http://lists.gnu.org/archive/html/bug-gawk/2014-11/msg00022.html, +explanatory note sent to the @command{gawk} bug +mailing list}.} +This option should be used with @samp{#!} scripts (@pxref{Executable Scripts}), like so: @example @@ -3738,7 +3882,7 @@ with @samp{#!} scripts (@pxref{Executable Scripts}), like so: @cindex portable object files, generating @cindex files, portable object, generating Analyze the source program and -generate a GNU @command{gettext} Portable Object Template file on standard +generate a GNU @command{gettext} portable object template file on standard output for all string constants that have been marked for translation. @xref{Internationalization}, for information about this option. @@ -3750,7 +3894,7 @@ for information about this option. @cindex GNU long options, printing list of @cindex options, printing list of @cindex printing, list of options -Print a ``usage'' message summarizing the short and long style options +Print a ``usage'' message summarizing the short- and long-style options that @command{gawk} accepts and then exit. @item @option{-i} @var{source-file} @@ -3760,7 +3904,7 @@ that @command{gawk} accepts and then exit. @cindex @command{awk} programs, location of Read an @command{awk} source library from @var{source-file}. This option is completely equivalent to using the @code{@@include} directive inside -your program. This option is very similar to the @option{-f} option, +your program. It is very similar to the @option{-f} option, but there are two important differences. First, when @option{-i} is used, the program source is not loaded if it has been previously loaded, whereas with @option{-f}, @command{gawk} always loads the file. @@ -3813,7 +3957,7 @@ care to search for all occurrences of each inappropriate construct. As @itemx @option{--bignum} @cindex @option{-M} option @cindex @option{--bignum} option -Force arbitrary precision arithmetic on numbers. This option has no effect +Force arbitrary-precision arithmetic on numbers. This option has no effect if @command{gawk} is not compiled to use the GNU MPFR and MP libraries (@pxref{Arbitrary Precision Arithmetic}). @@ -3829,10 +3973,8 @@ values in input data (@pxref{Nondecimal Data}). @quotation CAUTION -This option can severely break old programs. -Use with care. - -This option may disappear in a future version of @command{gawk}. +This option can severely break old programs. Use with care. Also note +that this option may disappear in a future version of @command{gawk}. @end quotation @item @option{-N} @@ -3847,7 +3989,7 @@ when parsing numeric input data (@pxref{Locales}). @cindex @option{-o} option @cindex @option{--pretty-print} option Enable pretty-printing of @command{awk} programs. -By default, output program is created in a file named @file{awkprof.out} +By default, the output program is created in a file named @file{awkprof.out} (@pxref{Profiling}). The optional @var{file} argument allows you to specify a different @value{FN} for the output. @@ -3864,7 +4006,7 @@ This is no longer the case. @cindex @option{--optimize} option @cindex @option{-O} option Enable some optimizations on the internal representation of the program. -At the moment this includes just simple constant folding. +At the moment, this includes just simple constant folding. @item @option{-p}[@var{file}] @itemx @option{--profile}[@code{=}@var{file}] @@ -3891,7 +4033,7 @@ in the left margin, and function call counts for each function. Operate in strict POSIX mode. This disables all @command{gawk} extensions (just like @option{--traditional}) and disables all extensions not allowed by POSIX. -@xref{Common Extensions}, for a summary of the extensions +@DBXREF{Common Extensions} for a summary of the extensions in @command{gawk} that are disabled by this option. Also, the following additional @@ -3941,8 +4083,8 @@ Allow interval expressions (@pxref{Regexp Operators}) in regexps. This is now @command{gawk}'s default behavior. -Nevertheless, this option remains both for backward compatibility, -and for use in combination with @option{--traditional}. +Nevertheless, this option remains (both for backward compatibility +and for use in combination with @option{--traditional}). @item @option{-S} @itemx @option{--sandbox} @@ -3995,7 +4137,7 @@ If it is, @command{awk} reads its program source from all of the named files, as if they had been concatenated together into one big file. This is useful for creating libraries of @command{awk} functions. These functions can be written once and then retrieved from a standard place, instead -of having to be included into each individual program. +of having to be included in each individual program. The @option{-i} option is similar in this regard. (As mentioned in @ref{Definition Syntax}, @@ -4006,20 +4148,20 @@ if the program is entered at the keyboard, by specifying @samp{-f /dev/tty}. After typing your program, type @kbd{Ctrl-d} (the end-of-file character) to terminate it. (You may also use @samp{-f -} to read program source from the standard -input but then you will not be able to also use the standard input as a +input, but then you will not be able to also use the standard input as a source of data.) Because it is clumsy using the standard @command{awk} mechanisms to mix source file and command-line @command{awk} programs, @command{gawk} provides the @option{-e} option. This does not require you to -pre-empt the standard input for your source code; it allows you to easily +preempt the standard input for your source code; it allows you to easily mix command-line and library source code (@pxref{AWKPATH Variable}). As with @option{-f}, the @option{-e} and @option{-i} options may also be used multiple times on the command line. @cindex @option{-e} option If no @option{-f} or @option{-e} option is specified, then @command{gawk} -uses the first non-option command-line argument as the text of the +uses the first nonoption command-line argument as the text of the program source code. @cindex @env{POSIXLY_CORRECT} environment variable @@ -4058,8 +4200,6 @@ setenv POSIXLY_CORRECT true Having @env{POSIXLY_CORRECT} set is not recommended for daily use, but it is good for testing the portability of your programs to other environments. -@c ENDOFRANGE ocl -@c ENDOFRANGE clo @node Other Arguments @section Other Command-Line Arguments @@ -4086,7 +4226,7 @@ All the command-line arguments are made available to your @command{awk} program and the program text (if present) are omitted from @code{ARGV}. All other arguments, including variable assignments, are included. As each element of @code{ARGV} is processed, @command{gawk} -sets the variable @code{ARGIND} to the index in @code{ARGV} of the +sets @code{ARGIND} to the index in @code{ARGV} of the current element. @c FIXME: One day, move the ARGC and ARGV node closer to here. @@ -4191,30 +4331,26 @@ behaves. @cindex @env{AWKPATH} environment variable @cindex directories, searching for source files @cindex search paths, for source files -@cindex differences in @command{awk} and @command{gawk}, @code{AWKPATH} environment variable +@cindex differences in @command{awk} and @command{gawk}, @env{AWKPATH} environment variable @ifinfo The previous @value{SECTION} described how @command{awk} program files can be named on the command line with the @option{-f} option. @end ifinfo In most @command{awk} -implementations, you must supply a precise path name for each program +implementations, you must supply a precise pathname for each program file, unless the file is in the current directory. -But in @command{gawk}, if the @value{FN} supplied to the @option{-f} +But with @command{gawk}, if the @value{FN} supplied to the @option{-f} or @option{-i} options does not contain a directory separator @samp{/}, then @command{gawk} searches a list of -directories (called the @dfn{search path}), one by one, looking for a +directories (called the @dfn{search path}) one by one, looking for a file with the specified name. The search path is a string consisting of directory names -separated by colons@footnote{Semicolons on MS-Windows and MS-DOS.}. @command{gawk} gets its search path from the +separated by colons.@footnote{Semicolons on MS-Windows and MS-DOS.} +@command{gawk} gets its search path from the @env{AWKPATH} environment variable. If that variable does not exist, -@command{gawk} uses a default path, -@samp{.:/usr/local/share/awk}.@footnote{Your version of @command{gawk} -may use a different directory; it -will depend upon how @command{gawk} was built and installed. The actual -directory is the value of @code{$(datadir)} generated when -@command{gawk} was configured. You probably don't need to worry about this, -though.} +or if it has an empty value, +@command{gawk} uses a default path (described shortly). The search path feature is particularly helpful for building libraries of useful @command{awk} functions. The library files can be placed in a @@ -4222,7 +4358,7 @@ standard directory in the default path and then specified on the command line with a short @value{FN}. Otherwise, you would have to type the full @value{FN} for each file. -By using the @option{-i} option, or the @option{-e} and @option{-f} options, your command-line +By using the @option{-i} or @option{-f} options, your command-line @command{awk} programs can use facilities in @command{awk} library files (@pxref{Library Functions}). Path searching is not done if @command{gawk} is in compatibility mode. @@ -4230,7 +4366,7 @@ This is true for both @option{--traditional} and @option{--posix}. @xref{Options}. If the source code file is not found after the initial search, the path is searched -again after adding the default @samp{.awk} suffix to the @value{FN}. +again after adding the suffix @samp{.awk} to the @value{FN}. @command{gawk}'s path search mechanism is similar to the shell's. @@ -4242,21 +4378,35 @@ directory. colon or by placing two colons next to each other [@samp{::}].) @quotation NOTE -@command{gawk} always looks in the current directory @emph{before} -searching @env{AWKPATH}. Thus, while you can include the current directory -in the search path, either explicitly or with a null entry, there is no -real reason to do so. -@c Prior to 4.0, gawk searched the current directory after the -@c path search, but it's not worth documenting it. +To include the current directory in the path, either place @file{.} +as an entry in the path or write a null entry in the path. + +Different past versions of @command{gawk} would also look explicitly in +the current directory, either before or after the path search. As of +@value{PVERSION} 4.1.2, this no longer happens; if you wish to look +in the current directory, you must include @file{.} either as a separate +entry or as a null entry in the search path. @end quotation -If @env{AWKPATH} is not defined in the -environment, @command{gawk} places its default search path into -@code{ENVIRON["AWKPATH"]}. This makes it easy to determine -the actual search path that @command{gawk} used -from within an @command{awk} program. +The default value for @env{AWKPATH} is +@samp{.:/usr/local/share/awk}.@footnote{Your version of @command{gawk} +may use a different directory; it +will depend upon how @command{gawk} was built and installed. The actual +directory is the value of @code{$(datadir)} generated when +@command{gawk} was configured. You probably don't need to worry about this, +though.} Since @file{.} is included at the beginning, @command{gawk} +searches first in the current directory and then in @file{/usr/local/share/awk}. +In practice, this means that you will rarely need to change the +value of @env{AWKPATH}. -While you can change @code{ENVIRON["AWKPATH"]} within your @command{awk} +@xref{Shell Startup Files}, for information on functions that help to +manipulate the @env{AWKPATH} variable. + +@command{gawk} places the value of the search path that it used into +@code{ENVIRON["AWKPATH"]}. This provides access to the actual search +path value from within an @command{awk} program. + +Although you can change @code{ENVIRON["AWKPATH"]} within your @command{awk} program, this has no effect on the running program's behavior. This makes sense: the @env{AWKPATH} environment variable is used to find the program source files. Once your program is running, all the files have been @@ -4278,12 +4428,24 @@ the platform. For example, on GNU/Linux systems, the suffix @samp{.so} is used. The search path specified is also used for extensions loaded via the @code{@@load} keyword (@pxref{Loading Shared Libraries}). +If @env{AWKLIBPATH} does not exist in the environment, or if it has +an empty value, @command{gawk} uses a default path; this +is typically @samp{/usr/local/lib/gawk}, although it can vary depending +upon how @command{gawk} was built. + +@xref{Shell Startup Files}, for information on functions that help to +manipulate the @env{AWKLIBPATH} variable. + +@command{gawk} places the value of the search path that it used into +@code{ENVIRON["AWKLIBPATH"]}. This provides access to the actual search +path value from within an @command{awk} program. + @node Other Environment Variables @subsection Other Environment Variables A number of other environment variables affect @command{gawk}'s behavior, but they are more specialized. Those in the following -list are meant to be used by regular users. +list are meant to be used by regular users: @table @env @item GAWK_MSEC_SLEEP @@ -4301,9 +4463,11 @@ wait for input before returning with an error. Controls the number of times @command{gawk} attempts to retry a two-way TCP/IP (socket) connection before giving up. @xref{TCP/IP Networking}. +Note that when nonfatal I/O is enabled (@pxref{Nonfatal}), +@command{gawk} only tries to open a TCP/IP socket once. @item POSIXLY_CORRECT -Causes @command{gawk} to switch to POSIX compatibility +Causes @command{gawk} to switch to POSIX-compatibility mode, disabling all traditional and GNU extensions. @xref{Options}. @end table @@ -4335,11 +4499,11 @@ for debugging problems on filesystems on non-POSIX operating systems where I/O is performed in records, not in blocks. @item GAWK_MSG_SRC -If this variable exists, @command{gawk} includes the file -name and line number within the @command{gawk} source code +If this variable exists, @command{gawk} includes the @value{FN} +and line number within the @command{gawk} source code from which warning and/or fatal messages are generated. Its purpose is to help isolate the source of a -message, since there are multiple places which produce the +message, as there are multiple places that produce the same warning or error message. @item GAWK_NO_DFA @@ -4355,16 +4519,16 @@ This specifies the amount by which @command{gawk} should grow its internal evaluation stack, when needed. @item INT_CHAIN_MAX -The intended maximum number of items @command{gawk} will maintain on a +This specifies intended maximum number of items @command{gawk} will maintain on a hash chain for managing arrays indexed by integers. @item STR_CHAIN_MAX -The intended maximum number of items @command{gawk} will maintain on a +This specifies intended maximum number of items @command{gawk} will maintain on a hash chain for managing arrays indexed by strings. @item TIDYMEM If this variable exists, @command{gawk} uses the @code{mtrace()} library -calls from GNU LIBC to help track down possible memory leaks. +calls from the GNU C library to help track down possible memory leaks. @end table @node Exit Status @@ -4383,11 +4547,11 @@ If an error occurs, @command{gawk} exits with the value of the C constant @code{EXIT_FAILURE}. This is usually one. If @command{gawk} exits because of a fatal error, the exit -status is 2. On non-POSIX systems, this value may be mapped +status is two. On non-POSIX systems, this value may be mapped to @code{EXIT_FAILURE}. @node Include Files -@section Including Other Files Into Your Program +@section Including Other Files into Your Program @c Panos Papadopoulos <panos1962@gmail.com> contributed the original @c text for this section. @@ -4401,7 +4565,7 @@ The @code{@@include} keyword can be used to read external @command{awk} source files. This gives you the ability to split large @command{awk} source files into smaller, more manageable pieces, and also lets you reuse common @command{awk} code from various @command{awk} scripts. In other words, you can group -together @command{awk} functions, used to carry out specific tasks, +together @command{awk} functions used to carry out specific tasks into external files. These files can be used just like function libraries, using the @code{@@include} keyword in conjunction with the @env{AWKPATH} environment variable. Note that source files may also be included @@ -4436,9 +4600,9 @@ $ @kbd{gawk -f test2} @print{} This is script test2. @end example -@code{gawk} runs the @file{test2} script which includes @file{test1} +@command{gawk} runs the @file{test2} script, which includes @file{test1} using the @code{@@include} -keyword. So, to include external @command{awk} source files you just +keyword. So, to include external @command{awk} source files, you just use @code{@@include} followed by the name of the file to be included, enclosed in double quotes. @@ -4475,27 +4639,28 @@ The @value{FN} can, of course, be a pathname. For example: @end example @noindent -or: +and: @example @@include "/usr/awklib/network" @end example @noindent -are valid. The @code{AWKPATH} environment variable can be of great +are both valid. The @env{AWKPATH} environment variable can be of great value when using @code{@@include}. The same rules for the use -of the @code{AWKPATH} variable in command-line file searches +of the @env{AWKPATH} variable in command-line file searches (@pxref{AWKPATH Variable}) apply to @code{@@include} also. This is very helpful in constructing @command{gawk} function libraries. -If you have a large script with useful, general purpose @command{awk} +If you have a large script with useful, general-purpose @command{awk} functions, you can break it down into library files and put those files -in a special directory. You can then include those ``libraries,'' using -either the full pathnames of the files, or by setting the @code{AWKPATH} +in a special directory. You can then include those ``libraries,'' +either by using the full pathnames of the files, or by setting the @env{AWKPATH} environment variable accordingly and then using @code{@@include} with -just the file part of the full pathname. Of course you can have more -than one directory to keep library files; the more complex the working +just the file part of the full pathname. Of course, +you can keep library files in more than one directory; +the more complex the working environment is, the more directories you may need to organize the files to be included. @@ -4508,11 +4673,11 @@ In particular, @code{@@include} is very useful for writing CGI scripts to be run from web pages. As mentioned in @ref{AWKPATH Variable}, the current directory is always -searched first for source files, before searching in @env{AWKPATH}, -and this also applies to files named with @code{@@include}. +searched first for source files, before searching in @env{AWKPATH}; +this also applies to files named with @code{@@include}. @node Loading Shared Libraries -@section Loading Dynamic Extensions Into Your Program +@section Loading Dynamic Extensions into Your Program This @value{SECTION} describes a feature that is specific to @command{gawk}. @@ -4522,7 +4687,7 @@ This @value{SECTION} describes a feature that is specific to @command{gawk}. The @code{@@load} keyword can be used to read external @command{awk} extensions (stored as system shared libraries). This allows you to link in compiled code that may offer superior -performance and/or give you access to extended capabilities not supported +performance and/or give you access to extended capabilities not supported by the @command{awk} language. The @env{AWKLIBPATH} variable is used to search for the extension. Using @code{@@load} is completely equivalent to using the @option{-l} command-line option. @@ -4530,7 +4695,7 @@ to using the @option{-l} command-line option. If the extension is not initially found in @env{AWKLIBPATH}, another search is conducted after appending the platform's default shared library suffix to the @value{FN}. For example, on GNU/Linux systems, the suffix -@samp{.so} is used. +@samp{.so} is used: @example $ @kbd{gawk '@@load "ordchr"; BEGIN @{print chr(65)@}'} @@ -4563,8 +4728,8 @@ It also describes the @code{ordchr} extension. @cindex features, deprecated @cindex obsolete features This @value{SECTION} describes features and/or command-line options from -previous releases of @command{gawk} that are either not available in the -current version or that are still supported but deprecated (meaning that +previous releases of @command{gawk} that either are not available in the +current version or are still supported but deprecated (meaning that they will @emph{not} be in the next release). The process-related special files @file{/dev/pid}, @file{/dev/ppid}, @@ -4644,7 +4809,7 @@ This seems to have been a long-undocumented feature in Unix @command{awk}. Similarly, you may use @code{print} or @code{printf} statements in the @var{init} and @var{increment} parts of a @code{for} loop. This is another -long-undocumented ``feature'' of Unix @code{awk}. +long-undocumented ``feature'' of Unix @command{awk}. @end ignore @@ -4661,17 +4826,17 @@ to run @command{awk}. @item The three standard options for all versions of @command{awk} are -@option{-f}, @option{-F} and @option{-v}. @command{gawk} supplies these +@option{-f}, @option{-F}, and @option{-v}. @command{gawk} supplies these and many others, as well as corresponding GNU-style long options. @item -Non-option command-line arguments are usually treated as @value{FN}s, +Nonoption command-line arguments are usually treated as @value{FN}s, unless they have the form @samp{@var{var}=@var{value}}, in which case they are taken as variable assignments to be performed at that point in processing the input. @item -All non-option command-line arguments, excluding the program text, +All nonoption command-line arguments, excluding the program text, are placed in the @code{ARGV} array. Adjusting @code{ARGC} and @code{ARGV} affects how @command{awk} processes input. @@ -4698,13 +4863,12 @@ and @option{-f} command-line options. @item @command{gawk} allows you to load additional functions written in C or C++ using the @code{@@load} statement and/or the @option{-l} option. -(This advanced feature is described later on in @ref{Dynamic Extensions}.) +(This advanced feature is described later, in @ref{Dynamic Extensions}.) @end itemize @node Regexp @chapter Regular Expressions @cindex regexp -@c STARTOFRANGE regexp @cindex regular expressions A @dfn{regular expression}, or @dfn{regexp}, is a way of describing a @@ -4720,7 +4884,7 @@ belongs to that set. The simplest regular expression is a sequence of letters, numbers, or both. Such a regexp matches any string that contains that sequence. Thus, the regexp @samp{foo} matches any string containing @samp{foo}. -Therefore, the pattern @code{/foo/} matches any input record containing +Thus, the pattern @code{/foo/} matches any input record containing the three adjacent characters @samp{foo} @emph{anywhere} in the record. Other kinds of regexps let you specify more complicated classes of strings. @@ -4783,17 +4947,16 @@ and @samp{!~} perform regular expression comparisons. Expressions using these operators can be used as patterns, or in @code{if}, @code{while}, @code{for}, and @code{do} statements. (@xref{Statements}.) -For example: +For example, the following is true if the expression @var{exp} (taken +as a string) matches @var{regexp}: @example @var{exp} ~ /@var{regexp}/ @end example @noindent -is true if the expression @var{exp} (taken as a string) -matches @var{regexp}. The following example matches, or selects, -all input records with the uppercase letter @samp{J} somewhere in the -first field: +This example matches, or selects, all input records with the uppercase +letter @samp{J} somewhere in the first field: @example $ @kbd{awk '$1 ~ /J/' inventory-shipped} @@ -4863,9 +5026,9 @@ string or regexp. Thus, the string whose contents are the two characters @samp{"} and @samp{\} must be written @code{"\"\\"}. Other escape sequences represent unprintable characters -such as TAB or newline. While there is nothing to stop you from entering most +such as TAB or newline. There is nothing to stop you from entering most unprintable characters directly in a string constant or regexp constant, -they may look ugly. +but they may look ugly. The following list presents all the escape sequences used in @command{awk} and @@ -4912,7 +5075,7 @@ Horizontal TAB, @kbd{Ctrl-i}, ASCII code 9 (HT). @cindex @code{\} (backslash), @code{\v} escape sequence @cindex backslash (@code{\}), @code{\v} escape sequence @item \v -Vertical tab, @kbd{Ctrl-k}, ASCII code 11 (VT). +Vertical TAB, @kbd{Ctrl-k}, ASCII code 11 (VT). @cindex @code{\} (backslash), @code{\}@var{nnn} escape sequence @cindex backslash (@code{\}), @code{\}@var{nnn} escape sequence @@ -4933,15 +5096,18 @@ of hexadecimal digits (@samp{0}--@samp{9}, and either @samp{A}--@samp{F} or @samp{a}--@samp{f}). A maximum of two digts are allowed after the @samp{\x}. Any further hexadecimal digits are treated as simple letters or numbers. @value{COMMONEXT} +(The @samp{\x} escape sequence is not allowed in POSIX awk.) @quotation CAUTION In ISO C, the escape sequence continues until the first nonhexadecimal digit is seen. -@c FIXME: Add exact version here. For many years, @command{gawk} would continue incorporating hexadecimal digits into the value until a non-hexadecimal digit or the end of the string was encountered. -However, using more than two hexadecimal digits produces +However, using more than two hexadecimal digits produced +undefined results. +As of @value{PVERSION} 4.2, only two digits +are processed. @end quotation @cindex @code{\} (backslash), @code{\/} escape sequence @@ -4951,7 +5117,7 @@ A literal slash (necessary for regexp constants only). This sequence is used when you want to write a regexp constant that contains a slash (such as @code{/.*:\/home\/[[:alnum:]]+:.*/}; the @samp{[[:alnum:]]} -notation is discussed shortly, in @ref{Bracket Expressions}). +notation is discussed in @ref{Bracket Expressions}). Because the regexp is delimited by slashes, you need to escape any slash that is part of the pattern, in order to tell @command{awk} to keep processing the rest of the regexp. @@ -4974,7 +5140,7 @@ with a backslash have special meaning in regexps. In a regexp, a backslash before any character that is not in the previous list and not listed in -@ref{GNU Regexp Operators}, +@DBREF{GNU Regexp Operators} means that the next character should be taken literally, even if it would normally be a regexp operator. For example, @code{/a\+b/} matches the three characters @samp{a+b}. @@ -4983,27 +5149,9 @@ characters @samp{a+b}. @cindex @code{\} (backslash), in escape sequences @cindex portability For complete portability, do not use a backslash before any character not -shown in the previous list and that is not an operator. - -To summarize: - -@itemize @value{BULLET} -@item -The escape sequences in the list above are always processed first, -for both string constants and regexp constants. This happens very early, -as soon as @command{awk} reads your program. - -@item -@command{gawk} processes both regexp constants and dynamic regexps -(@pxref{Computed Regexps}), -for the special operators listed in -@ref{GNU Regexp Operators}. - -@item -A backslash before any other character means to treat that character -literally. -@end itemize +shown in the previous list or that is not an operator. +@c 11/2014: Moved so as to not stack sidebars @sidebar Backslash Before Regular Characters @cindex portability, backslash in escape sequences @cindex POSIX @command{awk}, backslashes in string constants @@ -5039,6 +5187,25 @@ In such implementations, typing @code{"a\qc"} is the same as typing @end table @end sidebar +To summarize: + +@itemize @value{BULLET} +@item +The escape sequences in the preceding list are always processed first, +for both string constants and regexp constants. This happens very early, +as soon as @command{awk} reads your program. + +@item +@command{gawk} processes both regexp constants and dynamic regexps +(@pxref{Computed Regexps}), +for the special operators listed in +@ref{GNU Regexp Operators}. + +@item +A backslash before any other character means to treat that character +literally. +@end itemize + @sidebar Escape Sequences for Metacharacters @cindex metacharacters, escape sequences for @@ -5061,7 +5228,6 @@ escape sequences literally when used in regexp constants. Thus, @node Regexp Operators @section Regular Expression Operators -@c STARTOFRANGE regexpo @cindex regular expressions, operators @cindex metacharacters in regular expressions @@ -5079,14 +5245,14 @@ are recognized and converted into corresponding real characters as the very first step in processing regexps. Here is a list of metacharacters. All characters that are not escape -sequences and that are not listed in the following stand for themselves: +sequences and that are not listed here stand for themselves: @c Use @asis so the docbook comes out ok. Sigh. @table @asis @cindex backslash (@code{\}), regexp operator @cindex @code{\} (backslash), regexp operator @item @code{\} -This is used to suppress the special meaning of a character when +This suppresses the special meaning of a character when matching. For example, @samp{\$} matches the character @samp{$}. @@ -5095,8 +5261,9 @@ matches the character @samp{$}. @cindex @code{^} (caret), regexp operator @cindex caret (@code{^}), regexp operator @item @code{^} -This matches the beginning of a string. For example, @samp{^@@chapter} -matches @samp{@@chapter} at the beginning of a string and can be used +This matches the beginning of a string. @samp{^@@chapter} +matches @samp{@@chapter} at the beginning of a string, +for example, and can be used to identify chapter beginnings in Texinfo source files. The @samp{^} is known as an @dfn{anchor}, because it anchors the pattern to match only at the beginning of the string. @@ -5136,10 +5303,10 @@ with @samp{A}. @cindex POSIX @command{awk}, period (@code{.})@comma{} using In strict POSIX mode (@pxref{Options}), -@samp{.} does not match the @value{NUL} +@samp{.} does not match the @sc{nul} character, which is a character with all bits equal to zero. -Otherwise, @value{NUL} is just another character. Other versions of @command{awk} -may not be able to match the @value{NUL} character. +Otherwise, @sc{nul} is just another character. Other versions of @command{awk} +may not be able to match the @sc{nul} character. @cindex @code{[]} (square brackets), regexp operator @cindex square brackets (@code{[]}), regexp operator @@ -5201,8 +5368,8 @@ just @samp{p} if no @samp{h}s are present. There are two subtle points to understand about how @samp{*} works. First, the @samp{*} applies only to the single preceding regular expression component (e.g., in @samp{ph*}, it applies just to the @samp{h}). -To cause @samp{*} to apply to a larger sub-expression, use parentheses: -@samp{(ph)*} matches @samp{ph}, @samp{phph}, @samp{phphph} and so on. +To cause @samp{*} to apply to a larger subexpression, use parentheses: +@samp{(ph)*} matches @samp{ph}, @samp{phph}, @samp{phphph}, and so on. Second, @samp{*} finds as many repetitions as possible. If the text to be matched is @samp{phhhhhhhhhhhhhhooey}, @samp{ph*} matches all of @@ -5240,10 +5407,10 @@ is repeated at least @var{n} times: Matches @samp{whhhy}, but not @samp{why} or @samp{whhhhy}. @item wh@{3,5@}y -Matches @samp{whhhy}, @samp{whhhhy}, or @samp{whhhhhy}, only. +Matches @samp{whhhy}, @samp{whhhhy}, or @samp{whhhhhy} only. @item wh@{2,@}y -Matches @samp{whhy} or @samp{whhhy}, and so on. +Matches @samp{whhy}, @samp{whhhy}, and so on. @end table @cindex POSIX @command{awk}, interval expressions in @@ -5292,17 +5459,15 @@ usage as a syntax error. If @command{gawk} is in compatibility mode (@pxref{Options}), interval expressions are not available in regular expressions. -@c ENDOFRANGE regexpo @node Bracket Expressions @section Using Bracket Expressions -@c STARTOFRANGE charlist @cindex bracket expressions @cindex bracket expressions, range expressions @cindex range expressions (regexps) @cindex character lists in regular expression -As mentioned earlier, a bracket expression matches any character amongst +As mentioned earlier, a bracket expression matches any character among those listed between the opening and closing square brackets. Within a bracket expression, a @dfn{range expression} consists of two @@ -5360,23 +5525,23 @@ a keyword denoting the class, and @samp{:]}. POSIX standard. @float Table,table-char-classes -@caption{POSIX Character Classes} +@caption{POSIX character classes} @multitable @columnfractions .15 .85 @headitem Class @tab Meaning -@item @code{[:alnum:]} @tab Alphanumeric characters. -@item @code{[:alpha:]} @tab Alphabetic characters. -@item @code{[:blank:]} @tab Space and TAB characters. -@item @code{[:cntrl:]} @tab Control characters. -@item @code{[:digit:]} @tab Numeric characters. -@item @code{[:graph:]} @tab Characters that are both printable and visible. -(A space is printable but not visible, whereas an @samp{a} is both.) -@item @code{[:lower:]} @tab Lowercase alphabetic characters. -@item @code{[:print:]} @tab Printable characters (characters that are not control characters). +@item @code{[:alnum:]} @tab Alphanumeric characters +@item @code{[:alpha:]} @tab Alphabetic characters +@item @code{[:blank:]} @tab Space and TAB characters +@item @code{[:cntrl:]} @tab Control characters +@item @code{[:digit:]} @tab Numeric characters +@item @code{[:graph:]} @tab Characters that are both printable and visible +(a space is printable but not visible, whereas an @samp{a} is both) +@item @code{[:lower:]} @tab Lowercase alphabetic characters +@item @code{[:print:]} @tab Printable characters (characters that are not control characters) @item @code{[:punct:]} @tab Punctuation characters (characters that are not letters, digits, -control characters, or space characters). -@item @code{[:space:]} @tab Space characters (such as space, TAB, and formfeed, to name a few). -@item @code{[:upper:]} @tab Uppercase alphabetic characters. -@item @code{[:xdigit:]} @tab Characters that are hexadecimal digits. +control characters, or space characters) +@item @code{[:space:]} @tab Space characters (such as space, TAB, and formfeed, to name a few) +@item @code{[:upper:]} @tab Uppercase alphabetic characters +@item @code{[:xdigit:]} @tab Characters that are hexadecimal digits @end multitable @end float @@ -5391,12 +5556,12 @@ and numeric characters in your character set. @c Thanks to @c Date: Tue, 01 Jul 2014 07:39:51 +0200 @c From: Hermann Peifer <peifer@gmx.eu> -Some utilities that match regular expressions provide a non-standard -@code{[:ascii:]} character class; @command{awk} does not. However, you -can simulate such a construct using @code{[\x00-\x7F]}. This matches +Some utilities that match regular expressions provide a nonstandard +@samp{[:ascii:]} character class; @command{awk} does not. However, you +can simulate such a construct using @samp{[\x00-\x7F]}. This matches all values numerically between zero and 127, which is the defined range of the ASCII character set. Use a complemented character list -(@code{[^\x00-\x7F]}) to match any single-byte characters that are not +(@samp{[^\x00-\x7F]}) to match any single-byte characters that are not in the ASCII range. @cindex bracket expressions, collating elements @@ -5425,8 +5590,8 @@ Locale-specific names for a list of characters that are equal. The name is enclosed between @samp{[=} and @samp{=]}. For example, the name @samp{e} might be used to represent all of -``e,'' ``@`e,'' and ``@'e.'' In this case, @samp{[[=e=]]} is a regexp -that matches any of @samp{e}, @samp{@'e}, or @samp{@`e}. +``e,'' ``@^e,'' ``@`e,'' and ``@'e.'' In this case, @samp{[[=e=]]} is a regexp +that matches any of @samp{e}, @samp{@^e}, @samp{@'e}, or @samp{@`e}. @end table These features are very valuable in non-English-speaking locales. @@ -5440,7 +5605,6 @@ expression matching currently recognize only POSIX character classes; they do not recognize collating symbols or equivalence classes. @end quotation @c maybe one day ... -@c ENDOFRANGE charlist @node Leftmost Longest @section How Much Text Matches? @@ -5456,7 +5620,7 @@ echo aaaabcd | awk '@{ sub(/a+/, "<A>"); print @}' This example uses the @code{sub()} function to make a change to the input record. (@code{sub()} replaces the first instance of any text matched by the first argument with the string provided as the second argument; -@pxref{String Functions}). Here, the regexp @code{/a+/} indicates ``one +@pxref{String Functions}.) Here, the regexp @code{/a+/} indicates ``one or more @samp{a} characters,'' and the replacement text is @samp{<A>}. The input contains four @samp{a} characters. @@ -5484,9 +5648,7 @@ and also @pxref{Field Separators}). @node Computed Regexps @section Using Dynamic Regexps -@c STARTOFRANGE dregexp @cindex regular expressions, computed -@c STARTOFRANGE regexpd @cindex regular expressions, dynamic @cindex @code{~} (tilde), @code{~} operator @cindex tilde (@code{~}), @code{~} operator @@ -5512,14 +5674,14 @@ and tests whether the input record matches this regexp. @quotation NOTE When using the @samp{~} and @samp{!~} -operators, there is a difference between a regexp constant +operators, be aware that there is a difference between a regexp constant enclosed in slashes and a string constant enclosed in double quotes. If you are going to use a string constant, you have to understand that the string is, in essence, scanned @emph{twice}: the first time when @command{awk} reads your program, and the second time when it goes to match the string on the lefthand side of the operator with the pattern on the right. This is true of any string-valued expression (such as -@code{digits_regexp}, shown previously), not just string constants. +@code{digits_regexp}, shown in the previous example), not just string constants. @end quotation @cindex regexp constants, slashes vs.@: quotes @@ -5577,7 +5739,7 @@ $ @kbd{awk '$0 ~ "[ \t\n]"'} @error{} ]... @error{} source line number 1 @error{} context is -@error{} $0 ~ "[ >>> \t\n]" <<< +@error{} $0 ~ "[ >>> \t\n]" <<< @end example @cindex newlines, in regexp constants @@ -5593,17 +5755,13 @@ $ @kbd{awk '$0 ~ /[ \t\n]/'} @command{gawk} does not have this problem, and it isn't likely to occur often in practice, but it's worth noting for future reference. @end sidebar -@c ENDOFRANGE dregexp -@c ENDOFRANGE regexpd @node GNU Regexp Operators @section @command{gawk}-Specific Regexp Operators @c This section adapted (long ago) from the regex-0.12 manual -@c STARTOFRANGE regexpg @cindex regular expressions, operators, @command{gawk} -@c STARTOFRANGE gregexp @cindex @command{gawk}, regular expressions, operators @cindex operators, GNU-specific @cindex regular expressions, operators, for words @@ -5679,7 +5837,7 @@ matches either @samp{ball} or @samp{balls}, as a separate word. @item \B Matches the empty string that occurs between two word-constituent characters. For example, -@code{/\Brat\B/} matches @samp{crate} but it does not match @samp{dirty rat}. +@code{/\Brat\B/} matches @samp{crate}, but it does not match @samp{dirty rat}. @samp{\B} is essentially the opposite of @samp{\y}. @end table @@ -5687,9 +5845,9 @@ word-constituent characters. For example, @cindex regular expressions, operators, for buffers @cindex operators, string-matching, for buffers There are two other operators that work on buffers. In Emacs, a -@dfn{buffer} is, naturally, an Emacs buffer. For other programs, -@command{gawk}'s regexp library routines consider the entire -string to match as the buffer. +@dfn{buffer} is, naturally, an Emacs buffer. +Other GNU programs, including @command{gawk}, +consider the entire string to match as the buffer. The operators are: @table @code @@ -5698,14 +5856,14 @@ The operators are: @cindex backslash (@code{\}), @code{\`} operator (@command{gawk}) @cindex @code{\} (backslash), @code{\`} operator (@command{gawk}) Matches the empty string at the -beginning of a buffer (string). +beginning of a buffer (string) @c @cindex operators, @code{\'} (@command{gawk}) @cindex backslash (@code{\}), @code{\'} operator (@command{gawk}) @cindex @code{\} (backslash), @code{\'} operator (@command{gawk}) @item \' Matches the empty string at the -end of a buffer (string). +end of a buffer (string) @end table @cindex @code{^} (caret), regexp operator @@ -5750,16 +5908,16 @@ in @ref{Regexp Operators}. @end ifnottex @item @code{--posix} -Only POSIX regexps are supported; the GNU operators are not special +Match only POSIX regexps; the GNU operators are not special (e.g., @samp{\w} matches a literal @samp{w}). Interval expressions are allowed. @cindex Brian Kernighan's @command{awk} @item @code{--traditional} -Traditional Unix @command{awk} regexps are matched. The GNU operators +Match traditional Unix @command{awk} regexps. The GNU operators are not special, and interval expressions are not available. -The POSIX character classes (@samp{[[:alnum:]]}, etc.) are supported, -as BWK @command{awk} supports them. +Because BWK @command{awk} supports them, +the POSIX character classes (@samp{[[:alnum:]]}, etc.) are available. Characters described by octal and hexadecimal escape sequences are treated literally, even if they represent regexp metacharacters. @@ -5768,15 +5926,11 @@ Allow interval expressions in regexps, if @option{--traditional} has been provided. Otherwise, interval expressions are available by default. @end table -@c ENDOFRANGE gregexp -@c ENDOFRANGE regexpg @node Case-sensitivity @section Case Sensitivity in Matching -@c STARTOFRANGE regexpcs @cindex regular expressions, case sensitivity -@c STARTOFRANGE csregexp @cindex case sensitivity, regexps and Case is normally significant in regular expressions, both when matching ordinary characters (i.e., not metacharacters) and inside bracket @@ -5819,7 +5973,7 @@ When @code{IGNORECASE} is not zero, @emph{all} regexp and string operations ignore case. Changing the value of @code{IGNORECASE} dynamically controls the -case-sensitivity of the program as it runs. Case is significant by +case sensitivity of the program as it runs. Case is significant by default because @code{IGNORECASE} (like most variables) is initialized to zero: @@ -5832,7 +5986,7 @@ if (x ~ /ab/) @dots{} # now it will succeed @end example In general, you cannot use @code{IGNORECASE} to make certain rules -case-insensitive and other rules case-sensitive, because there is no +case insensitive and other rules case sensitive, as there is no straightforward way to set @code{IGNORECASE} just for the pattern of a particular rule.@footnote{Experienced C and C++ programmers will note @@ -5843,13 +5997,13 @@ and However, this is somewhat obscure and we don't recommend it.} To do this, use either bracket expressions or @code{tolower()}. However, one thing you can do with @code{IGNORECASE} only is dynamically turn -case-sensitivity on or off for all the rules at once. +case sensitivity on or off for all the rules at once. @code{IGNORECASE} can be set on the command line or in a @code{BEGIN} rule (@pxref{Other Arguments}; also @pxref{Using BEGIN/END}). Setting @code{IGNORECASE} from the command line is a way to make -a program case-insensitive without having to edit it. +a program case insensitive without having to edit it. @c @cindex ISO 8859-1 @c @cindex ISO Latin-1 @@ -5868,8 +6022,6 @@ the right thing.} The value of @code{IGNORECASE} has no effect if @command{gawk} is in compatibility mode (@pxref{Options}). Case is always significant in compatibility mode. -@c ENDOFRANGE csregexp -@c ENDOFRANGE regexpcs @node Regexp Summary @section Summary @@ -5886,12 +6038,12 @@ in conditional expressions, or as part of matching expressions using the @samp{~} and @samp{!~} operators. @item -Escape sequences let you represent non-printable characters and +Escape sequences let you represent nonprintable characters and also let you represent regexp metacharacters as literal characters to be matched. @item -Regexp operators provide grouping, alternation and repetition. +Regexp operators provide grouping, alternation, and repetition. @item Bracket expressions give you a shorthand for specifying sets @@ -5906,8 +6058,8 @@ the match, such as for text substitution and when the record separator is a regexp. @item -Matching expressions may use dynamic regexps, that is, string values -treated as regular expressions. +Matching expressions may use dynamic regexps (i.e., string values +treated as regular expressions). @item @command{gawk}'s @code{IGNORECASE} variable lets you control the @@ -5916,12 +6068,10 @@ versions, use @code{tolower()} or @code{toupper()}. @end itemize -@c ENDOFRANGE regexp @node Reading Files @chapter Reading Input Files -@c STARTOFRANGE infir @cindex reading input files @cindex input files, reading @cindex input files @@ -5946,7 +6096,7 @@ This makes it more convenient for programs to work on the parts of a record. @cindex @code{getline} command On rare occasions, you may need to use the @code{getline} command. -The @code{getline} command is valuable, both because it +The @code{getline} command is valuable both because it can do explicit input from any number of files, and because the files used with it do not have to be named on the @command{awk} command line (@pxref{Getline}). @@ -5963,6 +6113,7 @@ used with it do not have to be named on the @command{awk} command line * Getline:: Reading files under explicit program control using the @code{getline} function. * Read Timeout:: Reading input with a timeout. +* Retrying Input:: Retrying input after certain errors. * Command-line directories:: What happens if you put a directory on the command line. * Input Summary:: Input summary. @@ -5972,16 +6123,14 @@ used with it do not have to be named on the @command{awk} command line @node Records @section How Input Is Split into Records -@c STARTOFRANGE inspl @cindex input, splitting into records -@c STARTOFRANGE recspl @cindex records, splitting input into @cindex @code{NR} variable @cindex @code{FNR} variable @command{awk} divides the input for your program into records and fields. It keeps track of the number of records that have been read so far from the current input file. This value is stored in a predefined variable -called @code{FNR} which is reset to zero every time a new file is started. +called @code{FNR}, which is reset to zero every time a new file is started. Another predefined variable, @code{NR}, records the total number of input records read so far from all @value{DF}s. It starts at zero, but is never automatically reset to zero. @@ -5992,15 +6141,15 @@ never automatically reset to zero. @end menu @node awk split records -@subsection Record Splitting With Standard @command{awk} +@subsection Record Splitting with Standard @command{awk} @cindex separators, for records @cindex record separators Records are separated by a character called the @dfn{record separator}. By default, the record separator is the newline character. This is why records are, by default, single lines. -A different character can be used for the record separator by -assigning the character to the predefined variable @code{RS}. +To use a different character for the record separator, +simply assign that character to the predefined variable @code{RS}. @cindex newlines, as record separators @cindex @code{RS} variable @@ -6009,7 +6158,7 @@ the value of @code{RS} can be changed in the @command{awk} program with the assignment operator, @samp{=} (@pxref{Assignment Ops}). The new record-separator character should be enclosed in quotation marks, -which indicate a string constant. Often the right time to do this is +which indicate a string constant. Often, the right time to do this is at the beginning of execution, before any input is processed, so that the very first record is read with the proper separator. To do this, use the special @code{BEGIN} pattern @@ -6023,8 +6172,8 @@ awk 'BEGIN @{ RS = "u" @} @noindent changes the value of @code{RS} to @samp{u}, before reading any input. -This is a string whose first character is the letter ``u;'' as a result, records -are separated by the letter ``u.'' Then the input file is read, and the second +The new value is a string whose first character is the letter ``u''; as a result, records +are separated by the letter ``u''. Then the input file is read, and the second rule in the @command{awk} program (the action with no pattern) prints each record. Because each @code{print} statement adds a newline at the end of its output, this @command{awk} program copies the input @@ -6071,7 +6220,7 @@ $ @kbd{awk 'BEGIN @{ RS = "u" @}} @print{} m@@ny @print{} .ed @print{} R -@print{} +@print{} @end example @noindent @@ -6085,8 +6234,8 @@ Bill 555-1675 bill.drowning@@hotmail.com A @end example @noindent -It contains no @samp{u} so there is no reason to split the record, -unlike the others which have one or more occurrences of the @samp{u}. +It contains no @samp{u}, so there is no reason to split the record, +unlike the others, which each have one or more occurrences of the @samp{u}. In fact, this record is treated as part of the previous record; the newline separating them in the output is the original newline in the @value{DF}, not the one added by @@ -6117,7 +6266,7 @@ being fully POSIX-compliant (@pxref{Options}). Then, the following (extreme) pipeline prints a surprising @samp{1}: @example -$ echo | gawk --posix 'BEGIN @{ RS = "a" @} ; @{ print NF @}' +$ @kbd{echo | gawk --posix 'BEGIN @{ RS = "a" @} ; @{ print NF @}'} @print{} 1 @end example @@ -6139,7 +6288,7 @@ The empty string @code{""} (a string without any characters) has a special meaning as the value of @code{RS}. It means that records are separated by one or more blank lines and nothing else. -@xref{Multiple Line}, for more details. +@DBXREF{Multiple Line} for more details. If you change the value of @code{RS} in the middle of an @command{awk} run, the new value is used to delimit subsequent records, but the record @@ -6159,7 +6308,7 @@ sets the variable @code{RT} to the text in the input that matched @code{RS}. @node gawk split records -@subsection Record Splitting With @command{gawk} +@subsection Record Splitting with @command{gawk} @cindex common extensions, @code{RS} as a regexp @cindex extensions, common@comma{} @code{RS} as a regexp @@ -6181,7 +6330,7 @@ contains the same single character. However, when @code{RS} is a regular expression, @code{RT} contains the actual input text that matched the regular expression. -If the input file ended without any text that matches @code{RS}, +If the input file ends without any text matching @code{RS}, @command{gawk} sets @code{RT} to the null string. The following example illustrates both of these features. @@ -6203,11 +6352,11 @@ $ @kbd{echo record 1 AAAA record 2 BBBB record 3 |} The square brackets delineate the contents of @code{RT}, letting you see the leading and trailing whitespace. The final value of @code{RT} is a newline. -@xref{Simple Sed}, for a more useful example +@DBXREF{Simple Sed} for a more useful example of @code{RS} as a regexp and @code{RT}. If you set @code{RS} to a regular expression that allows optional -trailing text, such as @samp{RS = "abc(XYZ)?"} it is possible, due +trailing text, such as @samp{RS = "abc(XYZ)?"}, it is possible, due to implementation constraints, that @command{gawk} may match the leading part of the regular expression, but not the trailing part, particularly if the input text that could match the trailing part is fairly long. @@ -6221,7 +6370,7 @@ the beginning and end of a @emph{line}. As a result, something like @samp{RS = "^[[:upper:]]"} can only match at the beginning of a file. This is because @command{gawk} views the input file as one long string that happens to contain newline characters. -It is thus best to avoid anchor characters in the value of @code{RS}. +It is thus best to avoid anchor metacharacters in the value of @code{RS}. @end quotation @cindex differences in @command{awk} and @command{gawk}, @code{RS}/@code{RT} variables @@ -6240,7 +6389,7 @@ a value that you know doesn't occur in the input file. This is hard to do in a general way, such that a program always works for arbitrary input files. -You might think that for text files, the @value{NUL} character, which +You might think that for text files, the @sc{nul} character, which consists of a character with all bits equal to zero, is a good value to use for @code{RS} in this case: @@ -6249,33 +6398,31 @@ BEGIN @{ RS = "\0" @} # whole file becomes one record? @end example @cindex differences in @command{awk} and @command{gawk}, strings, storing -@command{gawk} in fact accepts this, and uses the @value{NUL} +@command{gawk} in fact accepts this, and uses the @sc{nul} character for the record separator. This works for certain special files, such as @file{/proc/environ} on -GNU/Linux systems, where the @value{NUL} character is in fact the record separator. +GNU/Linux systems, where the @sc{nul} character is in fact the record separator. However, this usage is @emph{not} portable to most other @command{awk} implementations. @cindex dark corner, strings, storing Almost all other @command{awk} implementations@footnote{At least that we know about.} store strings internally as C-style strings. C strings use the -@value{NUL} character as the string terminator. In effect, this means that +@sc{nul} character as the string terminator. In effect, this means that @samp{RS = "\0"} is the same as @samp{RS = ""}. @value{DARKCORNER} -It happens that recent versions of @command{mawk} can use the @value{NUL} +It happens that recent versions of @command{mawk} can use the @sc{nul} character as a record separator. However, this is a special case: -@command{mawk} does not allow embedded @value{NUL} characters in strings. +@command{mawk} does not allow embedded @sc{nul} characters in strings. (This may change in a future version of @command{mawk}.) @cindex records, treating files as @cindex treating files, as single records -@xref{Readfile Function}, for an interesting way to read -whole files. If you are using @command{gawk}, see @ref{Extension Sample -Readfile}, for another option. +@DBXREF{Readfile Function} for an interesting way to read +whole files. If you are using @command{gawk}, see @DBREF{Extension Sample +Readfile} for another option. @end sidebar -@c ENDOFRANGE inspl -@c ENDOFRANGE recspl @node Fields @section Examining Fields @@ -6283,7 +6430,6 @@ Readfile}, for another option. @cindex examining fields @cindex fields @cindex accessing fields -@c STARTOFRANGE fiex @cindex fields, examining @cindex POSIX @command{awk}, field separators and @cindex field separators, POSIX and @@ -6294,9 +6440,9 @@ called @dfn{fields}. By default, fields are separated by @dfn{whitespace}, like words in a line. Whitespace in @command{awk} means any string of one or more spaces, TABs, or newlines;@footnote{In POSIX @command{awk}, newlines are not -considered whitespace for separating fields.} other characters, such as -formfeed, vertical tab, etc., that are -considered whitespace by other languages, are @emph{not} considered +considered whitespace for separating fields.} other characters +that are considered whitespace by other languages +(such as formfeed, vertical tab, etc.) are @emph{not} considered whitespace by @command{awk}. The purpose of fields is to make it more convenient for you to refer to @@ -6308,12 +6454,12 @@ simple @command{awk} programs so powerful. @cindex @code{$} (dollar sign), @code{$} field operator @cindex dollar sign (@code{$}), @code{$} field operator @cindex field operators@comma{} dollar sign as -You use a dollar-sign (@samp{$}) +You use a dollar sign (@samp{$}) to refer to a field in an @command{awk} program, followed by the number of the field you want. Thus, @code{$1} refers to the first field, @code{$2} to the second, and so on. -(Unlike the Unix shells, the field numbers are not limited to single digits. -@code{$127} is the one hundred twenty-seventh field in the record.) +(Unlike in the Unix shells, the field numbers are not limited to single digits. +@code{$127} is the 127th field in the record.) For example, suppose the following is a line of input: @example @@ -6338,7 +6484,7 @@ If you try to reference a field beyond the last one (such as @code{$8} when the record has only seven fields), you get the empty string. (If used in a numeric operation, you get zero.) -The use of @code{$0}, which looks like a reference to the ``zero-th'' field, is +The use of @code{$0}, which looks like a reference to the ``zeroth'' field, is a special case: it represents the whole input record. Use it when you are not interested in specific fields. Here are some more examples: @@ -6364,7 +6510,6 @@ $ @kbd{awk '/li/ @{ print $1, $NF @}' mail-list} @print{} Julie F @print{} Samuel A @end example -@c ENDOFRANGE fiex @node Nonconstant Fields @section Nonconstant Field Numbers @@ -6383,7 +6528,7 @@ awk '@{ print $NR @}' @noindent Recall that @code{NR} is the number of records read so far: one in the -first record, two in the second, etc. So this example prints the first +first record, two in the second, and so on. So this example prints the first field of the first record, the second field of the second record, and so on. For the twentieth record, field number 20 is printed; most likely, the record has fewer than 20 fields, so this prints a blank line. @@ -6394,13 +6539,13 @@ awk '@{ print $(2*2) @}' mail-list @end example @command{awk} evaluates the expression @samp{(2*2)} and uses -its value as the number of the field to print. The @samp{*} sign +its value as the number of the field to print. The @samp{*} represents multiplication, so the expression @samp{2*2} evaluates to four. The parentheses are used so that the multiplication is done before the @samp{$} operation; they are necessary whenever there is a binary operator@footnote{A @dfn{binary operator}, such as @samp{*} for multiplication, is one that takes two operands. The distinction -is required, since @command{awk} also has unary (one-operand) +is required because @command{awk} also has unary (one-operand) and ternary (three-operand) operators.} in the field-number expression. This example, then, prints the type of relationship (the fourth field) for every line of the file @@ -6425,7 +6570,6 @@ evaluating @code{NF} and using its value as a field number. @node Changing Fields @section Changing the Contents of a Field -@c STARTOFRANGE ficon @cindex fields, changing contents of The contents of a field, as seen by @command{awk}, can be changed within an @command{awk} program; this changes what @command{awk} perceives as the @@ -6474,7 +6618,7 @@ $ @kbd{awk '@{ $2 = $2 - 10; print $0 @}' inventory-shipped} @dots{} @end example -It is also possible to also assign contents to fields that are out +It is also possible to assign contents to fields that are out of range. For example: @example @@ -6525,9 +6669,9 @@ else @noindent should print @samp{everything is normal}, because @code{NF+1} is certain -to be out of range. (@xref{If Statement}, +to be out of range. (@DBXREF{If Statement} for more information about @command{awk}'s @code{if-else} statements. -@xref{Typing and Comparison}, +@DBXREF{Typing and Comparison} for more information about the @samp{!=} operator.) It is important to note that making an assignment to an existing field @@ -6567,8 +6711,8 @@ after the new value of @code{NF} and recomputes @code{$0}. Here is an example: @example -$ echo a b c d e f | awk '@{ print "NF =", NF; -> NF = 3; print $0 @}' +$ @kbd{echo a b c d e f | awk '@{ print "NF =", NF;} +> @kbd{ NF = 3; print $0 @}'} @print{} NF = 6 @print{} a b c @end example @@ -6581,7 +6725,7 @@ rebuild @code{$0} when @code{NF} is decremented. Finally, there are times when it is convenient to force @command{awk} to rebuild the entire record, using the current -value of the fields and @code{OFS}. To do this, use the +values of the fields and @code{OFS}. To do this, use the seemingly innocuous assignment: @example @@ -6605,19 +6749,18 @@ such as @code{sub()} and @code{gsub()} It is important to remember that @code{$0} is the @emph{full} record, exactly as it was read from the input. This includes any leading or trailing whitespace, and the exact whitespace (or other -characters) that separate the fields. +characters) that separates the fields. It is a common error to try to change the field separators in a record simply by setting @code{FS} and @code{OFS}, and then expecting a plain @samp{print} or @samp{print $0} to print the modified record. -But this does not work, since nothing was done to change the record +But this does not work, because nothing was done to change the record itself. Instead, you must force the record to be rebuilt, typically with a statement such as @samp{$1 = $1}, as described earlier. @end sidebar -@c ENDOFRANGE ficon @node Field Separators @section Specifying How Fields Are Separated @@ -6633,9 +6776,7 @@ with a statement such as @samp{$1 = $1}, as described earlier. @cindex @code{FS} variable @cindex fields, separating -@c STARTOFRANGE fisepr @cindex field separators -@c STARTOFRANGE fisepg @cindex fields, separating The @dfn{field separator}, which is either a single character or a regular expression, controls the way @command{awk} splits an input record into fields. @@ -6664,7 +6805,7 @@ the Unix Bourne shell, @command{sh}, or Bash). @cindex @code{FS} variable, changing value of The value of @code{FS} can be changed in the @command{awk} program with the assignment operator, @samp{=} (@pxref{Assignment Ops}). -Often the right time to do this is at the beginning of execution +Often, the right time to do this is at the beginning of execution before any input has been processed, so that the very first record is read with the proper separator. To do this, use the special @code{BEGIN} pattern @@ -6701,7 +6842,7 @@ John Q. Smith, LXIX, 29 Oak St., Walamazoo, MI 42139 @end example @noindent -The same program would extract @samp{@bullet{}LXIX}, instead of +The same program would extract @samp{@bullet{}LXIX} instead of @samp{@bullet{}29@bullet{}Oak@bullet{}St.}. If you were expecting the program to print the address, you would be surprised. The moral is to choose your data layout and @@ -6735,9 +6876,7 @@ rules. @node Regexp Field Splitting @subsection Using Regular Expressions to Separate Fields -@c STARTOFRANGE regexpfs @cindex regular expressions, as field separators -@c STARTOFRANGE fsregexp @cindex field separators, regular expressions as The previous @value{SUBSECTION} discussed the use of single characters or simple strings as the @@ -6820,7 +6959,7 @@ statement prints the new @code{$0}. @cindex dark corner, @code{^}, in @code{FS} There is an additional subtlety to be aware of when using regular expressions for field splitting. -It is not well-specified in the POSIX standard, or anywhere else, what @samp{^} +It is not well specified in the POSIX standard, or anywhere else, what @samp{^} means when splitting fields. Does the @samp{^} match only at the beginning of the entire record? Or is each field separator a new string? It turns out that different @command{awk} versions answer this question differently, and you @@ -6841,8 +6980,6 @@ $ @kbd{echo 'xxAA xxBxx C' |} @print{} -->xxBxx<-- @print{} -->C<-- @end example -@c ENDOFRANGE regexpfs -@c ENDOFRANGE fsregexp @node Single Character Fields @subsection Making Each Character a Separate Field @@ -6966,7 +7103,7 @@ choosing your field and record separators. @cindex Unix @command{awk}, password files@comma{} field separators and Perhaps the most common use of a single character as the field separator occurs when processing the Unix system password file. On many Unix -systems, each user has a separate entry in the system password file, one +systems, each user has a separate entry in the system password file, with one line per user. The information in these lines is separated by colons. The first field is the user's login name and the second is the user's encrypted or shadow password. (A shadow password is indicated by the @@ -6986,11 +7123,11 @@ awk -F: '$5 == ""' /etc/passwd @end example @node Full Line Fields -@subsection Making The Full Line Be A Single Field +@subsection Making the Full Line Be a Single Field Occasionally, it's useful to treat the whole input line as a single field. This can be done easily and portably simply by -setting @code{FS} to @code{"\n"} (a newline).@footnote{Thanks to +setting @code{FS} to @code{"\n"} (a newline):@footnote{Thanks to Andrew Schorr for this tip.} @example @@ -7000,42 +7137,6 @@ awk -F'\n' '@var{program}' @var{files @dots{}} @noindent When you do this, @code{$1} is the same as @code{$0}. -@node Field Splitting Summary -@subsection Field-Splitting Summary - -It is important to remember that when you assign a string constant -as the value of @code{FS}, it undergoes normal @command{awk} string -processing. For example, with Unix @command{awk} and @command{gawk}, -the assignment @samp{FS = "\.."} assigns the character string @code{".."} -to @code{FS} (the backslash is stripped). This creates a regexp meaning -``fields are separated by occurrences of any two characters.'' -If instead you want fields to be separated by a literal period followed -by any single character, use @samp{FS = "\\.."}. - -The following list summarizes how fields are split, based on the value -of @code{FS} (@samp{==} means ``is equal to''): - -@table @code -@item FS == " " -Fields are separated by runs of whitespace. Leading and trailing -whitespace are ignored. This is the default. - -@item FS == @var{any other single character} -Fields are separated by each occurrence of the character. Multiple -successive occurrences delimit empty fields, as do leading and -trailing occurrences. -The character can even be a regexp metacharacter; it does not need -to be escaped. - -@item FS == @var{regexp} -Fields are separated by occurrences of characters that match @var{regexp}. -Leading and trailing matches of @var{regexp} delimit empty fields. - -@item FS == "" -Each individual character in the record becomes a separate field. -(This is a common extension; it is not specified by the POSIX standard.) -@end table - @sidebar Changing @code{FS} Does Not Affect the Fields @cindex POSIX @command{awk}, field separators and @@ -7043,7 +7144,7 @@ Each individual character in the record becomes a separate field. According to the POSIX standard, @command{awk} is supposed to behave as if each record is split into fields at the time it is read. In particular, this means that if you change the value of @code{FS} -after a record is read, the value of the fields (i.e., how they were split) +after a record is read, the values of the fields (i.e., how they were split) should reflect the old value of @code{FS}, not the new one. @cindex dark corner, field separators @@ -7056,10 +7157,7 @@ using the @emph{current} value of @code{FS}! @value{DARKCORNER} This behavior can be difficult to diagnose. The following example illustrates the difference -between the two methods. -(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' -Its behavior is also defined by the POSIX standard.} -command prints just the first line of @file{/etc/passwd}.) +between the two methods: @example sed 1q /etc/passwd | awk '@{ FS = ":" ; print $1 @}' @@ -7077,10 +7175,50 @@ on an incorrect implementation of @command{awk}, while @command{gawk} prints the full first line of the file, something like: @example -root:nSijPlPhZZwgE:0:0:Root:/: +root:x:0:0:Root:/: @end example + +(The @command{sed}@footnote{The @command{sed} utility is a ``stream editor.'' +Its behavior is also defined by the POSIX standard.} +command prints just the first line of @file{/etc/passwd}.) @end sidebar +@node Field Splitting Summary +@subsection Field-Splitting Summary + +It is important to remember that when you assign a string constant +as the value of @code{FS}, it undergoes normal @command{awk} string +processing. For example, with Unix @command{awk} and @command{gawk}, +the assignment @samp{FS = "\.."} assigns the character string @code{".."} +to @code{FS} (the backslash is stripped). This creates a regexp meaning +``fields are separated by occurrences of any two characters.'' +If instead you want fields to be separated by a literal period followed +by any single character, use @samp{FS = "\\.."}. + +The following list summarizes how fields are split, based on the value +of @code{FS} (@samp{==} means ``is equal to''): + +@table @code +@item FS == " " +Fields are separated by runs of whitespace. Leading and trailing +whitespace are ignored. This is the default. + +@item FS == @var{any other single character} +Fields are separated by each occurrence of the character. Multiple +successive occurrences delimit empty fields, as do leading and +trailing occurrences. +The character can even be a regexp metacharacter; it does not need +to be escaped. + +@item FS == @var{regexp} +Fields are separated by occurrences of characters that match @var{regexp}. +Leading and trailing matches of @var{regexp} delimit empty fields. + +@item FS == "" +Each individual character in the record becomes a separate field. +(This is a common extension; it is not specified by the POSIX standard.) +@end table + @sidebar @code{FS} and @code{IGNORECASE} The @code{IGNORECASE} variable @@ -7099,30 +7237,28 @@ print $1 @noindent The output is @samp{aCa}. If you really want to split fields on an alphabetic character while ignoring case, use a regexp that will -do it for you. E.g., @samp{FS = "[c]"}. In this case, @code{IGNORECASE} +do it for you (e.g., @samp{FS = "[c]"}). In this case, @code{IGNORECASE} will take effect. @end sidebar -@c ENDOFRANGE fisepr -@c ENDOFRANGE fisepg @node Constant Size @section Reading Fixed-Width Data -@quotation NOTE +@cindex data, fixed-width +@cindex fixed-width data +@cindex advanced features, fixed-width data + +@c O'Reilly doesn't like it as a note the first thing in the section. This @value{SECTION} discusses an advanced feature of @command{gawk}. If you are a novice @command{awk} user, you might want to skip it on the first reading. -@end quotation -@cindex data, fixed-width -@cindex fixed-width data -@cindex advanced features, fixed-width data -@command{gawk} provides a facility for dealing with -fixed-width fields with no distinctive field separator. For example, -data of this nature arises in the input for old Fortran programs where -numbers are run together, or in the output of programs that did not -anticipate the use of their output as input for other programs. +@command{gawk} provides a facility for dealing with fixed-width fields +with no distinctive field separator. For example, data of this nature +arises in the input for old Fortran programs where numbers are run +together, or in the output of programs that did not anticipate the use +of their output as input for other programs. An example of the latter is a table where all the columns are lined up by the use of a variable number of spaces and @emph{empty fields are just @@ -7142,7 +7278,7 @@ variable @code{FIELDWIDTHS}. Each number specifies the width of the field, @emph{including} columns between fields. If you want to ignore the columns between fields, you can specify the width as a separate field that is subsequently ignored. -It is a fatal error to supply a field width that is not a positive number. +It is a fatal error to supply a field width that has a negative value. The following data is the output of the Unix @command{w} utility. It is useful to illustrate the use of @code{FIELDWIDTHS}: @@ -7161,15 +7297,10 @@ dave ttyq4 26Jun9115days 46 46 wnewmail @end group @end example -The following program takes the above input, converts the idle time to +The following program takes this input, converts the idle time to number of seconds, and prints out the first two fields and the calculated idle time: -@quotation NOTE -This program uses a number of @command{awk} features that -haven't been introduced yet. -@end quotation - @example BEGIN @{ FIELDWIDTHS = "9 6 10 6 7 7 35" @} NR > 2 @{ @@ -7188,6 +7319,11 @@ NR > 2 @{ @} @end example +@quotation NOTE +The preceding program uses a number of @command{awk} features that +haven't been introduced yet. +@end quotation + Running the program on the data produces the following results: @example @@ -7219,7 +7355,7 @@ In order to tell which kind of field splitting is in effect, use @code{PROCINFO["FS"]} (@pxref{Auto-set}). The value is @code{"FS"} if regular field splitting is being used, -or it is @code{"FIELDWIDTHS"} if fixed-width field splitting is being used: +or @code{"FIELDWIDTHS"} if fixed-width field splitting is being used: @example if (PROCINFO["FS"] == "FS") @@ -7233,17 +7369,16 @@ else This information is useful when writing a function that needs to temporarily change @code{FS} or @code{FIELDWIDTHS}, read some records, and then restore the original settings -(@pxref{Passwd Functions}, +(@DBPXREF{Passwd Functions} for an example of such a function). @node Splitting By Content -@section Defining Fields By Content +@section Defining Fields by Content -@quotation NOTE +@c O'Reilly doesn't like it as a note the first thing in the section. This @value{SECTION} discusses an advanced feature of @command{gawk}. If you are a novice @command{awk} user, you might want to skip it on the first reading. -@end quotation @cindex advanced features, specifying field content Normally, when using @code{FS}, @command{gawk} defines the fields as the @@ -7254,14 +7389,16 @@ However, there are times when you really want to define the fields by what they are, and not by what they are not. The most notorious such case -is so-called @dfn{comma separated value} (CSV) data. Many spreadsheet programs, +is so-called @dfn{comma-separated values} (CSV) data. Many spreadsheet programs, for example, can export their data into text files, where each record is -terminated with a newline, and fields are separated by commas. If only -commas separated the data, there wouldn't be an issue. The problem comes when -one of the fields contains an @emph{embedded} comma. While there is no -formal standard specification for CSV data@footnote{At least, we don't know of one.}, -in such cases, most programs embed the field in double quotes. So we might -have data like this: +terminated with a newline, and fields are separated by commas. If +commas only separated the data, there wouldn't be an issue. The problem comes when +one of the fields contains an @emph{embedded} comma. +In such cases, most programs embed the field in double quotes.@footnote{The +CSV format lacked a formal standard definition for many years. +@uref{http://www.ietf.org/rfc/rfc4180.txt, RFC 4180} +standardizes the most common practices.} +So, we might have data like this: @example @c file eg/misc/addresses.csv @@ -7275,7 +7412,7 @@ The @code{FPAT} variable offers a solution for cases like this. The value of @code{FPAT} should be a string that provides a regular expression. This regular expression describes the contents of each field. -In the case of CSV data as presented above, each field is either ``anything that +In the case of CSV data as presented here, each field is either ``anything that is not a comma,'' or ``a double quote, anything that is not a double quote, and a closing double quote.'' If written as a regular expression constant (@pxref{Regexp}), @@ -7340,15 +7477,15 @@ will be @code{"FPAT"} if content-based field splitting is being used. @quotation NOTE Some programs export CSV data that contains embedded newlines between the double quotes. @command{gawk} provides no way to deal with this. -Since there is no formal specification for CSV data, there isn't much +Even though a formal specification for CSV data exists, there isn't much more to be done; the @code{FPAT} mechanism provides an elegant solution for the majority -of cases, and the @command{gawk} developers are satisfied with that. +of cases, and the @command{gawk} developers are satisfied with that. @end quotation As written, the regexp used for @code{FPAT} requires that each field -have a least one character. A straightforward modification -(changing changed the first @samp{+} to @samp{*}) allows fields to be empty: +contain at least one character. A straightforward modification +(changing the first @samp{+} to @samp{*}) allows fields to be empty: @example FPAT = "([^,]*)|(\"[^\"]+\")" @@ -7358,20 +7495,17 @@ Finally, the @code{patsplit()} function makes the same functionality available for splitting regular strings (@pxref{String Functions}). To recap, @command{gawk} provides three independent methods -to split input records into fields. @command{gawk} uses whichever -mechanism was last chosen based on which of the three -variables---@code{FS}, @code{FIELDWIDTHS}, and @code{FPAT}---was +to split input records into fields. +The mechanism used is based on which of the three +variables---@code{FS}, @code{FIELDWIDTHS}, or @code{FPAT}---was last assigned to. @node Multiple Line @section Multiple-Line Records @cindex multiple-line records -@c STARTOFRANGE recm @cindex records, multiline -@c STARTOFRANGE imr @cindex input, multiline records -@c STARTOFRANGE frm @cindex files, reading, multiline records @cindex input, files, See input files In some databases, a single line cannot conveniently hold all the @@ -7406,7 +7540,7 @@ at the end of the record and one or more blank lines after the record. In addition, a regular expression always matches the longest possible sequence when there is a choice (@pxref{Leftmost Longest}). -So the next record doesn't start until +So, the next record doesn't start until the first nonblank line that follows---no matter how many blank lines appear in a row, they are considered one record separator. @@ -7421,10 +7555,10 @@ In the second case, this special processing is not done. @cindex field separator, in multiline records @cindex @code{FS}, in multiline records Now that the input is separated into records, the second step is to -separate the fields in the record. One way to do this is to divide each +separate the fields in the records. One way to do this is to divide each of the lines into fields in the normal manner. This happens by default as the result of a special feature. When @code{RS} is set to the empty -string, @emph{and} @code{FS} is set to a single character, +string @emph{and} @code{FS} is set to a single character, the newline character @emph{always} acts as a field separator. This is in addition to whatever field separations result from @code{FS}.@footnote{When @code{FS} is the null string (@code{""}) @@ -7439,7 +7573,7 @@ want the newline character to separate fields, because there is no way to prevent it. However, you can work around this by using the @code{split()} function to break up the record manually (@pxref{String Functions}). -If you have a single character field separator, you can work around +If you have a single-character field separator, you can work around the special feature in a different way, by making @code{FS} into a regexp for that single character. For example, if the field separator is a percent character, instead of @@ -7447,10 +7581,10 @@ separator is a percent character, instead of Another way to separate fields is to put each field on a separate line: to do this, just set the -variable @code{FS} to the string @code{"\n"}. (This single -character separator matches a single newline.) +variable @code{FS} to the string @code{"\n"}. +(This single-character separator matches a single newline.) A practical example of a @value{DF} organized this way might be a mailing -list, where each entry is separated by blank lines. Consider a mailing +list, where blank lines separate the entries. Consider a mailing list in a file named @file{addresses}, which looks like this: @example @@ -7497,7 +7631,7 @@ $ @kbd{awk -f addrs.awk addresses} @dots{} @end example -@xref{Labels Program}, for a more realistic program that deals with +@DBXREF{Labels Program} for a more realistic program dealing with address lists. The following list summarizes how records are split, based on the value of @ifinfo @@ -7538,20 +7672,15 @@ If not in compatibility mode (@pxref{Options}), @command{gawk} sets @code{RT} to the input text that matched the value specified by @code{RS}. But if the input file ended without any text that matches @code{RS}, then @command{gawk} sets @code{RT} to the null string. -@c ENDOFRANGE recm -@c ENDOFRANGE imr -@c ENDOFRANGE frm @node Getline @section Explicit Input with @code{getline} -@c STARTOFRANGE getl @cindex @code{getline} command, explicit input with -@c STARTOFRANGE inex @cindex input, explicit So far we have been getting our input data from @command{awk}'s main input stream---either the standard input (usually your keyboard, sometimes -the output from another program) or from the +the output from another program) or the files specified on the command line. The @command{awk} language has a special built-in command called @code{getline} that can be used to read input under your explicit control. @@ -7587,12 +7716,19 @@ a record, such as a file that cannot be opened, then @code{getline} returns @minus{}1. In this case, @command{gawk} sets the variable @code{ERRNO} to a string describing the error that occurred. +If @code{ERRNO} indicates that the I/O operation may be +retried, and @code{PROCINFO["@var{input}", "RETRY"]} is set, +then @code{getline} returns @minus{}2 +instead of @minus{}1, and further calls to @code{getline} +may be attemped. @DBXREF{Retrying Input} for further information about +this feature. + In the following examples, @var{command} stands for a string value that represents a shell command. @quotation NOTE When @option{--sandbox} is specified (@pxref{Options}), -reading lines from files, pipes and coprocesses is disabled. +reading lines from files, pipes, and coprocesses is disabled. @end quotation @menu @@ -7735,7 +7871,7 @@ free @end example The @code{getline} command used in this way sets only the variables -@code{NR}, @code{FNR} and @code{RT} (and of course, @var{var}). +@code{NR}, @code{FNR}, and @code{RT} (and, of course, @var{var}). The record is not split into fields, so the values of the fields (including @code{$0}) and the value of @code{NF} do not change. @@ -7750,7 +7886,7 @@ the value of @code{NF} do not change. @cindex left angle bracket (@code{<}), @code{<} operator (I/O) @cindex operators, input/output Use @samp{getline < @var{file}} to read the next record from @var{file}. -Here @var{file} is a string-valued expression that +Here, @var{file} is a string-valued expression that specifies the @value{FN}. @samp{< @var{file}} is called a @dfn{redirection} because it directs input to come from a different place. For example, the following @@ -7789,7 +7925,7 @@ you want your program to be portable to all @command{awk} implementations. Use @samp{getline @var{var} < @var{file}} to read input from the file -@var{file}, and put it in the variable @var{var}. As above, @var{file} +@var{file}, and put it in the variable @var{var}. As earlier, @var{file} is a string-valued expression that specifies the file from which to read. In this version of @code{getline}, none of the predefined variables are @@ -7825,7 +7961,7 @@ One deficiency of this program is that it does not process nested @code{@@include} statements (i.e., @code{@@include} statements in included files) the way a true macro preprocessor would. -@xref{Igawk Program}, for a program +@DBXREF{Igawk Program} for a program that does handle nested @code{@@include} statements. @node Getline/Pipe @@ -7928,7 +8064,7 @@ of a construct like @samp{@w{"echo "} "date" | getline}. Most versions, including the current version, treat it at as @samp{@w{("echo "} "date") | getline}. (This is also how BWK @command{awk} behaves.) -Some versions changed and treated it as +Some versions instead treat it as @samp{@w{"echo "} ("date" | getline)}. (This is how @command{mawk} behaves.) In short, @emph{always} use explicit parentheses, and then you won't @@ -7976,7 +8112,7 @@ program to be portable to other @command{awk} implementations. @cindex operators, input/output @cindex differences in @command{awk} and @command{gawk}, input/output operators -Input into @code{getline} from a pipe is a one-way operation. +Reading input into @code{getline} from a pipe is a one-way operation. The command that is started with @samp{@var{command} | getline} only sends data @emph{to} your @command{awk} program. @@ -7986,7 +8122,7 @@ for processing and then read the results back. communications are possible. This is done with the @samp{|&} operator. Typically, you write data to the coprocess first and then -read results back, as shown in the following: +read the results back, as shown in the following: @example print "@var{some query}" |& "db_server" @@ -8069,17 +8205,23 @@ also @pxref{Auto-set}.) @item Using @code{FILENAME} with @code{getline} (@samp{getline < FILENAME}) -is likely to be a source for +is likely to be a source of confusion. @command{awk} opens a separate input stream from the current input file. However, by not using a variable, @code{$0} -and @code{NR} are still updated. If you're doing this, it's +and @code{NF} are still updated. If you're doing this, it's probably by accident, and you should reconsider what it is you're trying to accomplish. @item -@DBREF{Getline Summary} presents a table summarizing the +@ifdocbook +The next section +@end ifdocbook +@ifnotdocbook +@ref{Getline Summary}, +@end ifnotdocbook +presents a table summarizing the @code{getline} variants and which variables they can affect. -It is worth noting that those variants which do not use redirection +It is worth noting that those variants that do not use redirection can cause @code{FILENAME} to be updated if they cause @command{awk} to start reading a new input file. @@ -8088,7 +8230,7 @@ can cause @code{FILENAME} to be updated if they cause If the variable being assigned is an expression with side effects, different versions of @command{awk} behave differently upon encountering end-of-file. Some versions don't evaluate the expression; many versions -(including @command{gawk}) do. Here is an example, due to Duncan Moore: +(including @command{gawk}) do. Here is an example, courtesy of Duncan Moore: @ignore Date: Sun, 01 Apr 2012 11:49:33 +0100 @@ -8105,7 +8247,7 @@ BEGIN @{ @noindent Here, the side effect is the @samp{++c}. Is @code{c} incremented if -end of file is encountered, before the element in @code{a} is assigned? +end-of-file is encountered before the element in @code{a} is assigned? @command{gawk} treats @code{getline} like a function call, and evaluates the expression @samp{a[++c]} before attempting to read from @file{f}. @@ -8124,7 +8266,7 @@ and whether the variant is standard or a @command{gawk} extension. Note: for each variant, @command{gawk} sets the @code{RT} predefined variable. @float Table,table-getline-variants -@caption{@code{getline} Variants and What They Set} +@caption{@code{getline} variants and what they set} @multitable @columnfractions .33 .38 .27 @headitem Variant @tab Effect @tab @command{awk} / @command{gawk} @item @code{getline} @tab Sets @code{$0}, @code{NF}, @code{FNR}, @code{NR}, and @code{RT} @tab @command{awk} @@ -8137,12 +8279,9 @@ Note: for each variant, @command{gawk} sets the @code{RT} predefined variable. @item @var{command} @code{|& getline} @var{var} @tab Sets @var{var} and @code{RT} @tab @command{gawk} @end multitable @end float -@c ENDOFRANGE getl -@c ENDOFRANGE inex -@c ENDOFRANGE infir @node Read Timeout -@section Reading Input With A Timeout +@section Reading Input with a Timeout @cindex timeout, reading input @cindex differences in @command{awk} and @command{gawk}, read timeouts @@ -8150,8 +8289,8 @@ This @value{SECTION} describes a feature that is specific to @command{gawk}. You may specify a timeout in milliseconds for reading input from the keyboard, a pipe, or two-way communication, including TCP/IP sockets. This can be done -on a per input, command or connection basis, by setting a special element -in the @code{PROCINFO} array (@pxref{Auto-set}): +on a per-input, per-command, or per-connection basis, by setting a special +element in the @code{PROCINFO} array (@pxref{Auto-set}): @example PROCINFO["input_name", "READ_TIMEOUT"] = @var{timeout in milliseconds} @@ -8182,7 +8321,7 @@ while ((getline < "/dev/stdin") > 0) @end example @command{gawk} terminates the read operation if input does not -arrive after waiting for the timeout period, returns failure +arrive after waiting for the timeout period, returns failure, and sets @code{ERRNO} to an appropriate string value. A negative or zero value for the timeout is the same as specifying no timeout at all. @@ -8192,7 +8331,7 @@ loop that reads input records and matches them against patterns, like so: @example -$ @kbd{ gawk 'BEGIN @{ PROCINFO["-", "READ_TIMEOUT"] = 5000 @}} +$ @kbd{gawk 'BEGIN @{ PROCINFO["-", "READ_TIMEOUT"] = 5000 @}} > @kbd{@{ print "You entered: " $0 @}'} @kbd{gawk} @print{} You entered: gawk @@ -8215,7 +8354,7 @@ for the input to arrive: PROCINFO[Service, "READ_TIMEOUT"] = 1000 while ((Service |& getline) > 0) @{ print $0 - PROCINFO[S, "READ_TIMEOUT"] -= 100 + PROCINFO[Service, "READ_TIMEOUT"] -= 100 @} @end example @@ -8224,21 +8363,22 @@ You should not assume that the read operation will block exactly after the tenth record has been printed. It is possible that @command{gawk} will read and buffer more than one record's worth of data the first time. Because of this, changing the value -of timeout like in the above example is not very useful. +of timeout like in the preceding example is not very useful. @end quotation -If the @code{PROCINFO} element is not present and the environment -variable @env{GAWK_READ_TIMEOUT} exists, +If the @code{PROCINFO} element is not present and the +@env{GAWK_READ_TIMEOUT} environment variable exists, @command{gawk} uses its value to initialize the timeout value. The exclusive use of the environment variable to specify timeout has the disadvantage of not being able to control it -on a per command or connection basis. +on a per-command or per-connection basis. @command{gawk} considers a timeout event to be an error even though the attempt to read from the underlying device may succeed in a later attempt. This is a limitation, and it also means that you cannot use this to multiplex input from -two or more sources. +two or more sources. @DBXREF{Retrying Input} for a way to enable +later I/O attempts to succeed. Assigning a timeout value prevents read operations from blocking indefinitely. But bear in mind that there are other ways @@ -8248,8 +8388,38 @@ a connection before it can start reading any data, or the attempt to open a FIFO special file for reading can block indefinitely until some other process opens it for writing. +@node Retrying Input +@section Retrying Reads After Certain Input Errors +@cindex retrying input + +@cindex differences in @command{awk} and @command{gawk}, retrying input +This @value{SECTION} describes a feature that is specific to @command{gawk}. + +When @command{gawk} encounters an error while reading input, by +default @code{getline} returns @minus{}1, and subsequent attempts to +read from that file result in an end-of-file indication. However, you +may optionally instruct @command{gawk} to allow I/O to be retried when +certain errors are encountered by setting a special element in +the @code{PROCINFO} array (@pxref{Auto-set}): + +@example +PROCINFO["@var{input_name}", "RETRY"] = 1 +@end example + +When this element exists, @command{gawk} checks the value of the system +(C language) +@code{errno} variable when an I/O error occurs. If @code{errno} indicates +a subsequent I/O attempt may succeed, @code{getline} instead returns +@minus{}2 and +further calls to @code{getline} may succeed. This applies to the @code{errno} +values @code{EAGAIN}, @code{EWOULDBLOCK}, @code{EINTR}, or @code{ETIMEDOUT}. + +This feature is useful in conjunction with +@code{PROCINFO["@var{input_name}", "READ_TIMEOUT"]} or situations where a file +descriptor has been configured to behave in a non-blocking fashion. + @node Command-line directories -@section Directories On The Command Line +@section Directories on the Command Line @cindex differences in @command{awk} and @command{gawk}, command-line directories @cindex directories, command-line @cindex command line, directories on @@ -8264,14 +8434,14 @@ command line, but otherwise ignores it. This makes it easier to use shell wildcards with your @command{awk} program: @example -$ @kbd{gawk -f whizprog.awk *} @ii{Directories could kill this progam} +$ @kbd{gawk -f whizprog.awk *} @ii{Directories could kill this program} @end example If either of the @option{--posix} or @option{--traditional} options is given, then @command{gawk} reverts to treating a directory on the command line as a fatal error. -@xref{Extension Sample Readdir}, for a way to treat directories +@DBXREF{Extension Sample Readdir} for a way to treat directories as usable data from an @command{awk} program. @node Input Summary @@ -8283,7 +8453,7 @@ Input is split into records based on the value of @code{RS}. The possibilities are as follows: @multitable @columnfractions .25 .35 .40 -@headitem Value of @code{RS} @tab Records are split on @tab @command{awk} / @command{gawk} +@headitem Value of @code{RS} @tab Records are split on @dots{} @tab @command{awk} / @command{gawk} @item Any single character @tab That character @tab @command{awk} @item The empty string (@code{""}) @tab Runs of two or more newlines @tab @command{awk} @item A regexp @tab Text that matches the regexp @tab @command{gawk} @@ -8298,7 +8468,7 @@ The possibilities are as follows: @item After splitting the input into records, @command{awk} further splits -the record into individual fields, named @code{$1}, @code{$2} and so +the records into individual fields, named @code{$1}, @code{$2}, and so on. @code{$0} is the whole record, and @code{NF} indicates how many fields there are. The default way to split fields is between whitespace characters. @@ -8312,14 +8482,14 @@ greater than @code{NF} creates the field and rebuilds the record, using thing. Decrementing @code{NF} throws away fields and rebuilds the record. @item -Field splitting is more complicated than record splitting. +Field splitting is more complicated than record splitting: -@multitable @columnfractions .40 .45 .15 +@multitable @columnfractions .40 .40 .20 @headitem Field separator value @tab Fields are split @dots{} @tab @command{awk} / @command{gawk} @item @code{FS == " "} @tab On runs of whitespace @tab @command{awk} @item @code{FS == @var{any single character}} @tab On that character @tab @command{awk} @item @code{FS == @var{regexp}} @tab On text matching the regexp @tab @command{awk} -@item @code{FS == ""} @tab Each individual character is a separate field @tab @command{gawk} +@item @code{FS == ""} @tab Such that each individual character is a separate field @tab @command{gawk} @item @code{FIELDWIDTHS == @var{list of columns}} @tab Based on character position @tab @command{gawk} @item @code{FPAT == @var{regexp}} @tab On the text surrounding text matching the regexp @tab @command{gawk} @end multitable @@ -8336,11 +8506,11 @@ This can also be done using command-line variable assignment. Use @code{PROCINFO["FS"]} to see how fields are being split. @item -Use @code{getline} in its various forms to read additional records, +Use @code{getline} in its various forms to read additional records from the default input stream, from a file, or from a pipe or coprocess. @item -Use @code{PROCINFO[@var{file}, "READ_TIMEOUT"]} to cause reads to timeout +Use @code{PROCINFO[@var{file}, "READ_TIMEOUT"]} to cause reads to time out for @var{file}. @item @@ -8374,7 +8544,6 @@ That can be fixed by making one simple change. What is it? @node Printing @chapter Printing Output -@c STARTOFRANGE prnt @cindex printing @cindex output, printing, See printing One of the most common programming actions is to @dfn{print}, or output, @@ -8385,12 +8554,11 @@ The @code{print} statement is not limited when computing @emph{which} values to print. However, with two exceptions, you cannot specify @emph{how} to print them---how many columns, whether to use exponential notation or not, and so on. -(For the exceptions, @pxref{Output Separators}, and +(For the exceptions, @DBPXREF{Output Separators} and @ref{OFMT}.) For printing with specifications, you need the @code{printf} statement (@pxref{Printf}). -@c STARTOFRANGE prnts @cindex @code{print} statement @cindex @code{printf} statement Besides basic and formatted printing, this @value{CHAPTER} @@ -8411,6 +8579,7 @@ and discusses the @code{close()} built-in function. @command{gawk} allows access to inherited file descriptors. * Close Files And Pipes:: Closing Input and Output Files and Pipes. +* Nonfatal:: Enabling Nonfatal Output. * Output Summary:: Output summary. * Output Exercises:: Exercises. @end menu @@ -8451,7 +8620,7 @@ space is printed between any two items. Note that the @code{print} statement is a statement and not an expression---you can't use it in the pattern part of a -@var{pattern}-@var{action} statement, for example. +pattern--action statement, for example. @node Print Examples @section @code{print} Statement Examples @@ -8570,7 +8739,6 @@ You can continue either a @code{print} or @code{printf} statement simply by putting a newline after any comma (@pxref{Statements/Lines}). @end quotation -@c ENDOFRANGE prnts @node Output Separators @section Output Separators @@ -8582,14 +8750,14 @@ separated by single spaces. However, this doesn't need to be the case; a single space is simply the default. Any string of characters may be used as the @dfn{output field separator} by setting the predefined variable @code{OFS}. The initial value of this variable -is the string @w{@code{" "}}---that is, a single space. +is the string @w{@code{" "}} (i.e., a single space). -The output from an entire @code{print} statement is called an -@dfn{output record}. Each @code{print} statement outputs one output -record, and then outputs a string called the @dfn{output record separator} -(or @code{ORS}). The initial -value of @code{ORS} is the string @code{"\n"}; i.e., a newline -character. Thus, each @code{print} statement normally makes a separate line. +The output from an entire @code{print} statement is called an @dfn{output +record}. Each @code{print} statement outputs one output record, and +then outputs a string called the @dfn{output record separator} (or +@code{ORS}). The initial value of @code{ORS} is the string @code{"\n"} +(i.e., a newline character). Thus, each @code{print} statement normally +makes a separate line. @cindex output, records @cindex output record separator, See @code{ORS} variable @@ -8612,27 +8780,27 @@ newline: $ @kbd{awk 'BEGIN @{ OFS = ";"; ORS = "\n\n" @}} > @kbd{@{ print $1, $2 @}' mail-list} @print{} Amelia;555-5553 -@print{} +@print{} @print{} Anthony;555-3412 -@print{} +@print{} @print{} Becky;555-7685 -@print{} +@print{} @print{} Bill;555-1675 -@print{} +@print{} @print{} Broderick;555-0542 -@print{} +@print{} @print{} Camilla;555-2912 -@print{} +@print{} @print{} Fabius;555-1234 -@print{} +@print{} @print{} Julie;555-6699 -@print{} +@print{} @print{} Martin;555-6480 -@print{} +@print{} @print{} Samuel;555-3430 -@print{} +@print{} @print{} Jean-Paul;555-2127 -@print{} +@print{} @end example If the value of @code{ORS} does not contain a newline, the program's output @@ -8643,7 +8811,7 @@ runs together on a single line. @cindex numeric, output format @cindex formats@comma{} numeric output When printing numeric values with the @code{print} statement, -@command{awk} internally converts the number to a string of characters +@command{awk} internally converts each number to a string of characters and prints that string. @command{awk} uses the @code{sprintf()} function to do this conversion (@pxref{String Functions}). @@ -8683,7 +8851,6 @@ if @code{OFMT} contains anything but a floating-point conversion specification. @node Printf @section Using @code{printf} Statements for Fancier Printing -@c STARTOFRANGE printfs @cindex @code{printf} statement @cindex output, formatted @cindex formatting output @@ -8713,9 +8880,9 @@ printf @var{format}, @var{item1}, @var{item2}, @dots{} @end example @noindent -As print @code{print}, the entire list of arguments may optionally be +As for @code{print}, the entire list of arguments may optionally be enclosed in parentheses. Here too, the parentheses are necessary if any -of the item expressions use the @samp{>} relational operator; otherwise, +of the item expressions uses the @samp{>} relational operator; otherwise, it can be confused with an output redirection (@pxref{Redirection}). @cindex format specifiers @@ -8746,7 +8913,7 @@ $ @kbd{awk 'BEGIN @{} @end example @noindent -Here, neither the @samp{+} nor the @samp{OUCH} appear in +Here, neither the @samp{+} nor the @samp{OUCH!} appears in the output message. @node Control Letters @@ -8790,11 +8957,11 @@ a single byte (0--255). @item @code{%d}, @code{%i} Print a decimal integer. The two control letters are equivalent. -(The @code{%i} specification is for compatibility with ISO C.) +(The @samp{%i} specification is for compatibility with ISO C.) @item @code{%e}, @code{%E} -Print a number in scientific (exponential) notation; -for example: +Print a number in scientific (exponential) notation. +For example: @example printf "%4.3e\n", 1950 @@ -8805,7 +8972,7 @@ prints @samp{1.950e+03}, with a total of four significant figures, three of which follow the decimal point. (The @samp{4.3} represents two modifiers, discussed in the next @value{SUBSECTION}.) -@code{%E} uses @samp{E} instead of @samp{e} in the output. +@samp{%E} uses @samp{E} instead of @samp{e} in the output. @item @code{%f} Print a number in floating-point notation. @@ -8821,26 +8988,26 @@ which follow the decimal point. (The @samp{4.3} represents two modifiers, discussed in the next @value{SUBSECTION}.) -On systems supporting IEEE 754 floating point format, values +On systems supporting IEEE 754 floating-point format, values representing negative infinity are formatted as @samp{-inf} or @samp{-infinity}, and positive infinity as -@samp{inf} and @samp{infinity}. +@samp{inf} or @samp{infinity}. The special ``not a number'' value formats as @samp{-nan} or @samp{nan} (@pxref{Math Definitions}). @item @code{%F} -Like @code{%f} but the infinity and ``not a number'' values are spelled +Like @samp{%f}, but the infinity and ``not a number'' values are spelled using uppercase letters. -The @code{%F} format is a POSIX extension to ISO C; not all systems -support it. On those that don't, @command{gawk} uses @code{%f} instead. +The @samp{%F} format is a POSIX extension to ISO C; not all systems +support it. On those that don't, @command{gawk} uses @samp{%f} instead. @item @code{%g}, @code{%G} Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters; if the result is printed in -scientific notation, @code{%G} uses @samp{E} instead of @samp{e}. +scientific notation, @samp{%G} uses @samp{E} instead of @samp{e}. @item @code{%o} Print an unsigned octal integer @@ -8852,11 +9019,11 @@ Print a string. @item @code{%u} Print an unsigned decimal integer. (This format is of marginal use, because all numbers in @command{awk} -are floating-point; it is provided primarily for compatibility with C.) +are floating point; it is provided primarily for compatibility with C.) @item @code{%x}, @code{%X} Print an unsigned hexadecimal integer; -@code{%X} uses the letters @samp{A} through @samp{F} +@samp{%X} uses the letters @samp{A} through @samp{F} instead of @samp{a} through @samp{f} (@pxref{Nondecimal-numbers}). @@ -8871,7 +9038,7 @@ argument and it ignores any modifiers. @quotation NOTE When using the integer format-control letters for values that are outside the range of the widest C integer type, @command{gawk} switches to -the @code{%g} format specifier. If @option{--lint} is provided on the +the @samp{%g} format specifier. If @option{--lint} is provided on the command line (@pxref{Options}), @command{gawk} warns about this. Other versions of @command{awk} may print invalid values or do something else entirely. @@ -8881,7 +9048,6 @@ values or do something else entirely. @node Format Modifiers @subsection Modifiers for @code{printf} Formats -@c STARTOFRANGE pfm @cindex @code{printf} statement, modifiers @cindex modifiers@comma{} in format specifiers A format specification can also include @dfn{modifiers} that can control @@ -8892,12 +9058,12 @@ represent spaces in the output. Here are the possible modifiers, in the order in which they may appear: -@table @code +@table @asis @cindex differences in @command{awk} and @command{gawk}, @code{print}/@code{printf} statements @cindex @code{printf} statement, positional specifiers @c the code{} does NOT start a secondary @cindex positional specifiers, @code{printf} statement -@item @var{N}$ +@item @code{@var{N}$} An integer constant followed by a @samp{$} is a @dfn{positional specifier}. Normally, format specifications are applied to arguments in the order given in the format string. With a positional specifier, the format @@ -8920,7 +9086,7 @@ messages at runtime. which describes how and why to use positional specifiers. For now, we ignore them. -@item - +@item @code{-} (Minus) The minus sign, used before the width modifier (see later on in this list), says to left-justify @@ -8938,31 +9104,31 @@ prints @samp{foo@bullet{}}. For numeric conversions, prefix positive values with a space and negative values with a minus sign. -@item + +@item @code{+} The plus sign, used before the width modifier (see later on in this list), says to always supply a sign for numeric conversions, even if the data to format is positive. The @samp{+} overrides the space modifier. -@item # -Use an ``alternate form'' for certain control letters. -For @code{%o}, supply a leading zero. -For @code{%x} and @code{%X}, supply a leading @code{0x} or @samp{0X} for +@item @code{#} +Use an ``alternative form'' for certain control letters. +For @samp{%o}, supply a leading zero. +For @samp{%x} and @samp{%X}, supply a leading @samp{0x} or @samp{0X} for a nonzero result. -For @code{%e}, @code{%E}, @code{%f}, and @code{%F}, the result always +For @samp{%e}, @samp{%E}, @samp{%f}, and @samp{%F}, the result always contains a decimal point. -For @code{%g} and @code{%G}, trailing zeros are not removed from the result. +For @samp{%g} and @samp{%G}, trailing zeros are not removed from the result. -@item 0 +@item @code{0} A leading @samp{0} (zero) acts as a flag indicating that output should be padded with zeros instead of spaces. This applies only to the numeric output formats. This flag only has an effect when the field width is wider than the value to print. -@item ' +@item @code{'} A single quote or apostrophe character is a POSIX extension to ISO C. -It indicates that the integer part of a floating point value, or the +It indicates that the integer part of a floating-point value, or the entire part of an integer decimal value, should have a thousands-separator character in it. This only works in locales that support such characters. For example: @@ -9013,7 +9179,7 @@ prints @samp{foobar}. Preceding the @var{width} with a minus sign causes the output to be padded with spaces on the right, instead of on the left. -@item .@var{prec} +@item @code{.@var{prec}} A period followed by an integer constant specifies the precision to use when printing. The meaning of the precision varies by control letter: @@ -9043,7 +9209,7 @@ prints @samp{foob}. @end table The C library @code{printf}'s dynamic @var{width} and @var{prec} -capability (for example, @code{"%*.*s"}) is supported. Instead of +capability (e.g., @code{"%*.*s"}) is supported. Instead of supplying explicit @var{width} and/or @var{prec} values in the format string, they are passed in the argument list. For example: @@ -9076,7 +9242,7 @@ printf "%" w "." p "s\n", s @end example @noindent -This is not particularly easy to read but it does work. +This is not particularly easy to read, but it does work. @c @cindex lint checks @cindex troubleshooting, fatal errors, @code{printf} format strings @@ -9087,7 +9253,6 @@ format strings. These are not valid in @command{awk}. Most @command{awk} implementations silently ignore them. If @option{--lint} is provided on the command line (@pxref{Options}), @command{gawk} warns about their use. If @option{--posix} is supplied, their use is a fatal error. -@c ENDOFRANGE pfm @node Printf Examples @subsection Examples Using @code{printf} @@ -9123,7 +9288,7 @@ $ @kbd{awk '@{ printf "%-10s %s\n", $1, $2 @}' mail-list} @end example In this case, the phone numbers had to be printed as strings because -the numbers are separated by a dash. Printing the phone numbers as +the numbers are separated by dashes. Printing the phone numbers as numbers would have produced just the first three digits: @samp{555}. This would have been pretty confusing. @@ -9143,7 +9308,7 @@ awk 'BEGIN @{ print "Name Number" @{ printf "%-10s %s\n", $1, $2 @}' mail-list @end example -The above example mixes @code{print} and @code{printf} statements in +The preceding example mixes @code{print} and @code{printf} statements in the same program. Using just @code{printf} statements can produce the same results: @@ -9168,14 +9333,11 @@ awk 'BEGIN @{ format = "%-10s %s\n" @{ printf format, $1, $2 @}' mail-list @end example -@c ENDOFRANGE printfs @node Redirection @section Redirecting Output of @code{print} and @code{printf} -@c STARTOFRANGE outre @cindex output redirection -@c STARTOFRANGE reout @cindex redirection of output @cindex @option{--sandbox} option, output redirection with @code{print}, @code{printf} So far, the output from @code{print} and @code{printf} has gone @@ -9186,7 +9348,7 @@ This is called @dfn{redirection}. @quotation NOTE When @option{--sandbox} is specified (@pxref{Options}), -redirecting output to files, pipes and coprocesses is disabled. +redirecting output to files, pipes, and coprocesses is disabled. @end quotation A redirection appears after the @code{print} or @code{printf} statement. @@ -9239,7 +9401,7 @@ Each output file contains one name or number per line. @cindex @code{>} (right angle bracket), @code{>>} operator (I/O) @cindex right angle bracket (@code{>}), @code{>>} operator (I/O) @item print @var{items} >> @var{output-file} -This redirection prints the items into the pre-existing output file +This redirection prints the items into the preexisting output file named @var{output-file}. The difference between this and the single-@samp{>} redirection is that the old contents (if any) of @var{output-file} are not erased. Instead, the @command{awk} output is @@ -9278,7 +9440,7 @@ The unsorted list is written with an ordinary redirection, while the sorted list is written by piping through the @command{sort} utility. The next example uses redirection to mail a message to the mailing -list @samp{bug-system}. This might be useful when trouble is encountered +list @code{bug-system}. This might be useful when trouble is encountered in an @command{awk} script run periodically for system maintenance: @example @@ -9290,7 +9452,7 @@ close(report) The @code{close()} function is called here because it's a good idea to close the pipe as soon as all the intended output has been sent to it. -@xref{Close Files And Pipes}, +@DBXREF{Close Files And Pipes} for more information. This example also illustrates the use of a variable to represent @@ -9309,15 +9471,23 @@ This redirection prints the items to the input of @var{command}. The difference between this and the single-@samp{|} redirection is that the output from @var{command} can be read with @code{getline}. -Thus @var{command} is a @dfn{coprocess}, which works together with, -but subsidiary to, the @command{awk} program. +Thus, @var{command} is a @dfn{coprocess}, which works together with +but is subsidiary to the @command{awk} program. This feature is a @command{gawk} extension, and is not available in POSIX @command{awk}. +@ifnotdocbook @xref{Getline/Coprocess}, for a brief discussion. @xref{Two-way I/O}, for a more complete discussion. +@end ifnotdocbook +@ifdocbook +@DBXREF{Getline/Coprocess} +for a brief discussion and +@DBREF{Two-way I/O} +for a more complete discussion. +@end ifdocbook @end table Redirecting output using @samp{>}, @samp{>>}, @samp{|}, or @samp{|&} @@ -9340,9 +9510,9 @@ print "Avoid improbability generators" >> "guide.txt" @noindent This is indeed how redirections must be used from the shell. But in @command{awk}, it isn't necessary. In this kind of case, a program should -use @samp{>} for all the @code{print} statements, since the output file +use @samp{>} for all the @code{print} statements, because the output file is only opened once. (It happens that if you mix @samp{>} and @samp{>>} -that output is produced in the expected order. However, mixing the operators +output is produced in the expected order. However, mixing the operators for the same file is definitely poor style, and is confusing to readers of your program.) @@ -9389,14 +9559,12 @@ The program builds up a list of command lines, using the @command{mv} utility to rename the files. It then sends the list to the shell for execution. -@xref{Shell Quoting}, for a function that can help in generating +@DBXREF{Shell Quoting} for a function that can help in generating command lines to be fed to the shell. @end sidebar -@c ENDOFRANGE outre -@c ENDOFRANGE reout @node Special FD -@section Special Files for Standard Pre-Opened Data Streams +@section Special Files for Standard Preopened Data Streams @cindex standard input @cindex input, standard @cindex standard output @@ -9409,7 +9577,7 @@ command lines to be fed to the shell. Running programs conventionally have three input and output streams already available to them for reading and writing. These are known as the @dfn{standard input}, @dfn{standard output}, and @dfn{standard -error output}. These open streams (and any other open file or pipe) +error output}. These open streams (and any other open files or pipes) are often referred to by the technical term @dfn{file descriptors}. These streams are, by default, connected to your keyboard and screen, but @@ -9447,14 +9615,14 @@ that is connected to your keyboard and screen. It represents the ``terminal,''@footnote{The ``tty'' in @file{/dev/tty} stands for ``Teletype,'' a serial terminal.} which on modern systems is a keyboard and screen, not a serial console.) -This generally has the same effect but not always: although the +This generally has the same effect, but not always: although the standard error stream is usually the screen, it can be redirected; when that happens, writing to the screen is not correct. In fact, if @command{awk} is run from a background job, it may not have a terminal at all. Then opening @file{/dev/tty} fails. -@command{gawk}, BWK @command{awk} and @command{mawk} provide +@command{gawk}, BWK @command{awk}, and @command{mawk} provide special @value{FN}s for accessing the three standard streams. If the @value{FN} matches one of these special names when @command{gawk} (or one of the others) redirects input or output, then it directly uses @@ -9492,21 +9660,20 @@ print "Serious error detected!" > "/dev/stderr" @cindex troubleshooting, quotes with file names Note the use of quotes around the @value{FN}. -Like any other redirection, the value must be a string. +Like with any other redirection, the value must be a string. It is a common error to omit the quotes, which leads to confusing results. @command{gawk} does not treat these @value{FN}s as special when -in POSIX compatibility mode. However, since BWK @command{awk} +in POSIX-compatibility mode. However, because BWK @command{awk} supports them, @command{gawk} does support them even when invoked with the @option{--traditional} option (@pxref{Options}). @node Special Files @section Special @value{FFN}s in @command{gawk} -@c STARTOFRANGE gfn @cindex @command{gawk}, file names in -Besides access to standard input, stanard output, and standard error, +Besides access to standard input, standard output, and standard error, @command{gawk} provides access to any open file descriptor. Additionally, there are special @value{FN}s reserved for TCP/IP networking. @@ -9519,7 +9686,7 @@ TCP/IP networking. @end menu @node Other Inherited Files -@subsection Accessing Other Open Files With @command{gawk} +@subsection Accessing Other Open Files with @command{gawk} Besides the @code{/dev/stdin}, @code{/dev/stdout}, and @code{/dev/stderr} special @value{FN}s mentioned earlier, @command{gawk} provides syntax @@ -9555,7 +9722,7 @@ This is done using a special @value{FN} of the form: @file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}} @end example -The @var{net-type} is one of @samp{inet}, @samp{inet4} or @samp{inet6}. +The @var{net-type} is one of @samp{inet}, @samp{inet4}, or @samp{inet6}. The @var{protocol} is one of @samp{tcp} or @samp{udp}, and the other fields represent the other essential pieces of information for making a networking connection. @@ -9576,7 +9743,7 @@ special @value{FN}s that @command{gawk} provides: @cindex compatibility mode (@command{gawk}), file names @cindex file names, in compatibility mode @item -Recognition of the @value{FN}s for the three standard pre-opened +Recognition of the @value{FN}s for the three standard preopened files is disabled only in POSIX mode. @item @@ -9589,23 +9756,18 @@ compatibility mode (either @option{--traditional} or @option{--posix}; interprets these special @value{FN}s. For example, using @samp{/dev/fd/4} for output actually writes on file descriptor 4, and not on a new -file descriptor that is @code{dup()}'ed from file descriptor 4. Most of +file descriptor that is @code{dup()}ed from file descriptor 4. Most of the time this does not matter; however, it is important to @emph{not} close any of the files related to file descriptors 0, 1, and 2. Doing so results in unpredictable behavior. @end itemize -@c ENDOFRANGE gfn @node Close Files And Pipes @section Closing Input and Output Redirections @cindex files, output, See output files -@c STARTOFRANGE ifc @cindex input files, closing -@c STARTOFRANGE ofc @cindex output, files@comma{} closing -@c STARTOFRANGE pc @cindex pipe, closing -@c STARTOFRANGE cc @cindex coprocesses, closing @cindex @code{getline} command, coprocesses@comma{} using from @@ -9744,7 +9906,7 @@ is not closed and released until @code{close()} is called or @command{awk} exits. @code{close()} silently does nothing if given an argument that -does not represent a file, pipe or coprocess that was opened with +does not represent a file, pipe, or coprocess that was opened with a redirection. In such a case, it returns a negative value, indicating an error. In addition, @command{gawk} sets @code{ERRNO} to a string indicating the error. @@ -9778,9 +9940,10 @@ which describes it in more detail and gives an example. @cindex Unix @command{awk}, @code{close()} function and In many older versions of Unix @command{awk}, the @code{close()} function -is actually a statement. It is a syntax error to try and use the return -value from @code{close()}: +is actually a statement. @value{DARKCORNER} +It is a syntax error to try and use the return +value from @code{close()}: @example command = "@dots{}" @@ -9810,18 +9973,79 @@ This value is zero if the close succeeds, or @minus{}1 if it fails. The POSIX standard is very vague; it says that @code{close()} -returns zero on success and nonzero otherwise. In general, +returns zero on success and a nonzero value otherwise. In general, different implementations vary in what they report when closing -pipes; thus the return value cannot be used portably. +pipes; thus, the return value cannot be used portably. @value{DARKCORNER} In POSIX mode (@pxref{Options}), @command{gawk} just returns zero when closing a pipe. @end sidebar -@c ENDOFRANGE ifc -@c ENDOFRANGE ofc -@c ENDOFRANGE pc -@c ENDOFRANGE cc + +@node Nonfatal +@section Enabling Nonfatal Output + +This @value{SECTION} describes a @command{gawk}-specific feature. + +In standard @command{awk}, output with @code{print} or @code{printf} +to a nonexistent file, or some other I/O error (such as filling up the +disk) is a fatal error. + +@example +$ @kbd{gawk 'BEGIN @{ print "hi" > "/no/such/file" @}'} +@error{} gawk: cmd. line:1: fatal: can't redirect to `/no/such/file' (No such file or directory) +@end example + +@command{gawk} makes it possible to detect that an error has +occurred, allowing you to possibly recover from the error, or +at least print an error message of your choosing before exiting. +You can do this in one of two ways: + +@itemize @bullet +@item +For all output files, by assigning any value to @code{PROCINFO["NONFATAL"]}. + +@item +On a per-file basis, by assigning any value to +@code{PROCINFO[@var{filename}, "NONFATAL"]}. +Here, @var{filename} is the name of the file to which +you wish output to be nonfatal. +@end itemize + +Once you have enabled nonfatal output, you must check @code{ERRNO} +after every relevant @code{print} or @code{printf} statement to +see if something went wrong. It is also a good idea to initialize +@code{ERRNO} to zero before attempting the output. For example: + +@example +$ @kbd{gawk '} +> @kbd{BEGIN @{} +> @kbd{ PROCINFO["NONFATAL"] = 1} +> @kbd{ ERRNO = 0} +> @kbd{ print "hi" > "/no/such/file"} +> @kbd{ if (ERRNO) @{} +> @kbd{ print("Output failed:", ERRNO) > "/dev/stderr"} +> @kbd{ exit 1} +> @kbd{ @}} +> @kbd{@}'} +@error{} Output failed: No such file or directory +@end example + +Here, @command{gawk} did not produce a fatal error; instead +it let the @command{awk} program code detect the problem and handle it. + +This mechanism works also for standard output and standard error. +For standard output, you may use @code{PROCINFO["-", "NONFATAL"]} +or @code{PROCINFO["/dev/stdout", "NONFATAL"]}. For standard error, use +@code{PROCINFO["/dev/stderr", "NONFATAL"]}. + +When attempting to open a TCP/IP socket (@pxref{TCP/IP Networking}), +@command{gawk} tries multiple times. The @env{GAWK_SOCK_RETRIES} +environment variable (@pxref{Other Environment Variables}) allows you to +override @command{gawk}'s builtin default number of attempts. However, +once nonfatal I/O is enabled for a given socket, @command{gawk} only +retries once, relying on @command{awk}-level code to notice that there +was a problem. @node Output Summary @section Summary @@ -9835,22 +10059,28 @@ for numeric values for the @code{print} statement. @item The @code{printf} statement provides finer-grained control over output, -with format control letters for different data types and various flags -that modify the behavior of the format control letters. +with format-control letters for different data types and various flags +that modify the behavior of the format-control letters. @item Output from both @code{print} and @code{printf} may be redirected to files, pipes, and coprocesses. @item -@command{gawk} provides special file names for access to standard input, -output and error, and for network communications. +@command{gawk} provides special @value{FN}s for access to standard input, +output, and error, and for network communications. @item -Use @code{close()} to close open file, pipe and coprocess redirections. +Use @code{close()} to close open file, pipe, and coprocess redirections. For coprocesses, it is possible to close only one direction of the communications. +@item +Normally errors with @code{print} or @code{printf} are fatal. +@command{gawk} lets you make output errors be nonfatal either for +all files or on a per-file basis. You must then check for errors +after every relevant output statement. + @end itemize @c EXCLUDE START @@ -9885,11 +10115,9 @@ BEGIN @{ print "Serious error detected!" > /dev/stderr @} @end enumerate @c EXCLUDE END -@c ENDOFRANGE prnt @node Expressions @chapter Expressions -@c STARTOFRANGE exps @cindex expressions Expressions are the basic building blocks of @command{awk} patterns @@ -9900,7 +10128,7 @@ can assign a new value to a variable or a field by using an assignment operator. An expression can serve as a pattern or action statement on its own. Most other kinds of statements contain one or more expressions that specify the data on which to -operate. As in other languages, expressions in @command{awk} include +operate. As in other languages, expressions in @command{awk} can include variables, array references, constants, and function calls, as well as combinations of these with various operators. @@ -9915,11 +10143,11 @@ combinations of these with various operators. @end menu @node Values -@section Constants, Variables and Conversions +@section Constants, Variables, and Conversions Expressions are built up from values and the operations performed upon them. This @value{SECTION} describes the elementary objects -which provide the values used in expressions. +that provide the values used in expressions. @menu * Constants:: String, numeric and regexp constants. @@ -9932,7 +10160,6 @@ which provide the values used in expressions. @node Constants @subsection Constant Expressions -@c STARTOFRANGE cnst @cindex constants, types of The simplest type of expression is the @dfn{constant}, which always has @@ -9941,7 +10168,7 @@ string, and regular expression. Each is used in the appropriate context when you need a data value that isn't going to change. Numeric constants can -have different forms, but are stored identically internally. +have different forms, but are internally stored in an identical manner. @menu * Scalar Constants:: Numeric and string constants. @@ -9957,7 +10184,7 @@ have different forms, but are stored identically internally. A @dfn{numeric constant} stands for a number. This number can be an integer, a decimal fraction, or a number in scientific (exponential) notation.@footnote{The internal representation of all numbers, -including integers, uses double precision floating-point numbers. +including integers, uses double-precision floating-point numbers. On most modern systems, these are in IEEE 754 standard format. @xref{Arbitrary Precision Arithmetic}, for much more information.} Here are some examples of numeric constants that all @@ -9970,8 +10197,8 @@ have the same value: @end example @cindex string constants -A string constant consists of a sequence of characters enclosed in -double-quotation marks. For example: +A @dfn{string constant} consists of a sequence of characters enclosed in +double quotation marks. For example: @example "parrot" @@ -9982,7 +10209,7 @@ double-quotation marks. For example: @cindex strings, length limitations represents the string whose contents are @samp{parrot}. Strings in @command{gawk} can be of any length, and they can contain any of the possible -eight-bit ASCII characters including ASCII @value{NUL} (character code zero). +eight-bit ASCII characters, including ASCII @sc{nul} (character code zero). Other @command{awk} implementations may have difficulty with some character codes. @@ -9993,19 +10220,19 @@ implementations may have difficulty with some character codes. @cindex numbers, octal @cindex numbers, hexadecimal -In @command{awk}, all numbers are in decimal; i.e., base 10. Many other +In @command{awk}, all numbers are in decimal (i.e., base 10). Many other programming languages allow you to specify numbers in other bases, often octal (base 8) and hexadecimal (base 16). -In octal, the numbers go 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, etc. -Just as @samp{11}, in decimal, is 1 times 10 plus 1, so -@samp{11}, in octal, is 1 times 8, plus 1. This equals 9 in decimal. -In hexadecimal, there are 16 digits. Since the everyday decimal +In octal, the numbers go 0, 1, 2, 3, 4, 5, 6, 7, 10, 11, 12, and so on. +Just as @samp{11} in decimal is 1 times 10 plus 1, so +@samp{11} in octal is 1 times 8 plus 1. This equals 9 in decimal. +In hexadecimal, there are 16 digits. Because the everyday decimal number system only has ten digits (@samp{0}--@samp{9}), the letters @samp{a} through @samp{f} are used to represent the rest. (Case in the letters is usually irrelevant; hexadecimal @samp{a} and @samp{A} have the same value.) -Thus, @samp{11}, in -hexadecimal, is 1 times 16 plus 1, which equals 17 in decimal. +Thus, @samp{11} in +hexadecimal is 1 times 16 plus 1, which equals 17 in decimal. Just by looking at plain @samp{11}, you can't tell what base it's in. So, in C, C++, and other languages derived from C, @@ -10016,13 +10243,13 @@ and hexadecimal numbers start with a leading @samp{0x} or @samp{0X}: @table @code @item 11 -Decimal value 11. +Decimal value 11 @item 011 -Octal 11, decimal value 9. +Octal 11, decimal value 9 @item 0x11 -Hexadecimal 11, decimal value 17. +Hexadecimal 11, decimal value 17 @end table This example shows the difference: @@ -10050,12 +10277,13 @@ you can use the @code{strtonum()} function (@pxref{String Functions}) to convert the data into a number. Most of the time, you will want to use octal or hexadecimal constants -when working with the built-in bit manipulation functions; -see @ref{Bitwise Functions}, +when working with the built-in bit-manipulation functions; +see @DBREF{Bitwise Functions} for more information. -Unlike some early C implementations, @samp{8} and @samp{9} are not valid -in octal constants; e.g., @command{gawk} treats @samp{018} as decimal 18: +Unlike in some early C implementations, @samp{8} and @samp{9} are not +valid in octal constants. For example, @command{gawk} treats @samp{018} +as decimal 18: @example $ @kbd{gawk 'BEGIN @{ print "021 is", 021 ; print 018 @}'} @@ -10088,19 +10316,17 @@ $ @kbd{gawk 'BEGIN @{ printf "0x11 is <%s>\n", 0x11 @}'} @node Regexp Constants @subsubsection Regular Expression Constants -@c STARTOFRANGE rec @cindex regexp constants @cindex @code{~} (tilde), @code{~} operator @cindex tilde (@code{~}), @code{~} operator @cindex @code{!} (exclamation point), @code{!~} operator @cindex exclamation point (@code{!}), @code{!~} operator -A regexp constant is a regular expression description enclosed in +A @dfn{regexp constant} is a regular expression description enclosed in slashes, such as @code{@w{/^beginning and end$/}}. Most regexps used in @command{awk} programs are constant, but the @samp{~} and @samp{!~} matching operators can also match computed or dynamic regexps (which are typically just ordinary strings or variables that contain a regexp, -but could be a more complex expression). -@c ENDOFRANGE cnst +but could be more complex expressions). @node Using Constant Regexps @subsection Using Regular Expression Constants @@ -10112,7 +10338,7 @@ matched. However, regexp constants (such as @code{/foo/}) may be used like simple expressions. When a regexp constant appears by itself, it has the same meaning as if it appeared -in a pattern, i.e., @samp{($0 ~ /foo/)} +in a pattern (i.e., @samp{($0 ~ /foo/)}). @value{DARKCORNER} @xref{Expression Patterns}. This means that the following two code segments: @@ -10180,7 +10406,7 @@ the third argument of @code{split()} to be a regexp constant, but some older implementations do not. @value{DARKCORNER} Because some built-in functions accept regexp constants as arguments, -it can be confusing when attempting to use regexp constants as arguments +confusion can arise when attempting to use regexp constants as arguments to user-defined functions (@pxref{User-defined}). For example: @example @@ -10206,19 +10432,18 @@ function mysub(pat, repl, str, global) In this example, the programmer wants to pass a regexp constant to the user-defined function @code{mysub()}, which in turn passes it on to either @code{sub()} or @code{gsub()}. However, what really happens is that -the @code{pat} parameter is either one or zero, depending upon whether +the @code{pat} parameter is assigned a value of either one or zero, depending upon whether or not @code{$0} matches @code{/hi/}. @command{gawk} issues a warning when it sees a regexp constant used as -a parameter to a user-defined function, since passing a truth value in +a parameter to a user-defined function, because passing a truth value in this way is probably not what was intended. -@c ENDOFRANGE rec @node Variables @subsection Variables @cindex variables, user-defined @cindex user-defined, variables -Variables are ways of storing values at one point in your program for +@dfn{Variables} are ways of storing values at one point in your program for use later in another part of your program. They can be manipulated entirely within the program text, and they can also be assigned values on the @command{awk} command line. @@ -10246,18 +10471,18 @@ are distinct variables. A variable name is a valid expression by itself; it represents the variable's current value. Variables are given new values with @dfn{assignment operators}, @dfn{increment operators}, and -@dfn{decrement operators}. -@xref{Assignment Ops}. +@dfn{decrement operators} +(@pxref{Assignment Ops}). In addition, the @code{sub()} and @code{gsub()} functions can -change a variable's value, and the @code{match()}, @code{split()} +change a variable's value, and the @code{match()}, @code{split()}, and @code{patsplit()} functions can change the contents of their -array parameters. @xref{String Functions}. +array parameters (@pxref{String Functions}). @cindex variables, built-in @cindex variables, initializing A few variables have special built-in meanings, such as @code{FS} (the -field separator), and @code{NF} (the number of fields in the current input -record). @xref{Built-in Variables}, for a list of the predefined variables. +field separator) and @code{NF} (the number of fields in the current input +record). @DBXREF{Built-in Variables} for a list of the predefined variables. These predefined variables can be used and assigned just like all other variables, but their values are also used or changed automatically by @command{awk}. All predefined variables' names are entirely uppercase. @@ -10298,7 +10523,7 @@ as in the following: the variable is set at the very beginning, even before the @code{BEGIN} rules execute. The @option{-v} option and its assignment must precede all the @value{FN} arguments, as well as the program text. -(@xref{Options}, for more information about +(@DBXREF{Options} for more information about the @option{-v} option.) Otherwise, the variable assignment is performed at a time determined by its position among the input file arguments---after the processing of the @@ -10338,7 +10563,7 @@ sequences @node Conversion @subsection Conversion of Strings and Numbers -Number to string and string to number conversion are generally +Number-to-string and string-to-number conversion are generally straightforward. There can be subtleties to be aware of; this @value{SECTION} discusses this important facet of @command{awk}. @@ -10349,7 +10574,7 @@ this @value{SECTION} discusses this important facet of @command{awk}. @end menu @node Strings And Numbers -@subsubsection How @command{awk} Converts Between Strings And Numbers +@subsubsection How @command{awk} Converts Between Strings and Numbers @cindex converting, strings to numbers @cindex strings, converting @@ -10380,7 +10605,7 @@ string, concatenate that number with the empty string, @code{""}. To force a string to be converted to a number, add zero to that string. A string is converted to a number by interpreting any numeric prefix of the string as numerals: -@code{"2.5"} converts to 2.5, @code{"1e3"} converts to 1000, and @code{"25fix"} +@code{"2.5"} converts to 2.5, @code{"1e3"} converts to 1,000, and @code{"25fix"} has a numeric value of 25. Strings that can't be interpreted as valid numbers convert to zero. @@ -10420,7 +10645,7 @@ b = a "" @code{b} has the value @code{"12"}, not @code{"12.00"}. @value{DARKCORNER} -@sidebar Pre-POSIX @command{awk} Used @code{OFMT} For String Conversion +@sidebar Pre-POSIX @command{awk} Used @code{OFMT} for String Conversion @cindex POSIX @command{awk}, @code{OFMT} variable and @cindex @code{OFMT} variable @cindex portability, new @command{awk} vs.@: old @command{awk} @@ -10432,7 +10657,7 @@ specifies the output format to use when printing numbers with @code{print}. conversion from the semantics of printing. Both @code{CONVFMT} and @code{OFMT} have the same default value: @code{"%.6g"}. In the vast majority of cases, old @command{awk} programs do not change their behavior. -@xref{Print}, for more information on the @code{print} statement. +@DBXREF{Print} for more information on the @code{print} statement. @end sidebar @node Locale influences conversions @@ -10454,7 +10679,7 @@ The POSIX standard says that @command{awk} always uses the period as the decimal point when reading the @command{awk} program source code, and for command-line variable assignments (@pxref{Other Arguments}). However, when interpreting input data, for @code{print} and @code{printf} output, -and for number to string conversion, the local decimal point character +and for number-to-string conversion, the local decimal point character is used. @value{DARKCORNER} In all cases, numbers in source code and in input data cannot have a thousands separator. Here are some examples indicating the difference in behavior, on a GNU/Linux system: @@ -10479,12 +10704,12 @@ as the full number including the fractional part, 4.321. Some earlier versions of @command{gawk} fully complied with this aspect of the standard. However, many users in non-English locales complained -about this behavior, since their data used a period as the decimal +about this behavior, because their data used a period as the decimal point, so the default behavior was restored to use a period as the decimal point character. You can use the @option{--use-lc-numeric} option (@pxref{Options}) to force @command{gawk} to use the locale's decimal point character. (@command{gawk} also uses the locale's decimal -point character when in POSIX mode, either via @option{--posix}, or the +point character when in POSIX mode, either via @option{--posix} or the @env{POSIXLY_CORRECT} environment variable, as shown previously.) @ref{table-locale-affects} describes the cases in which the locale's decimal @@ -10492,7 +10717,7 @@ point character is used and when a period is used. Some of these features have not been described yet. @float Table,table-locale-affects -@caption{Locale Decimal Point versus A Period} +@caption{Locale decimal point versus a period} @multitable @columnfractions .15 .20 .45 @headitem Feature @tab Default @tab @option{--posix} or @option{--use-lc-numeric} @item @code{%'g} @tab Use locale @tab Use locale @@ -10502,15 +10727,15 @@ features have not been described yet. @end multitable @end float -Finally, modern day formal standards and IEEE standard floating point +Finally, modern-day formal standards and the IEEE standard floating-point representation can have an unusual but important effect on the way @command{gawk} converts some special string values to numbers. The details are presented in @ref{POSIX Floating Point Problems}. @node All Operators -@section Operators: Doing Something With Values +@section Operators: Doing Something with Values -This @value{SECTION} introduces the @dfn{operators} which make use +This @value{SECTION} introduces the @dfn{operators} that make use of the values provided by constants and variables. @menu @@ -10587,7 +10812,7 @@ Multiplication. Division; because all numbers in @command{awk} are floating-point numbers, the result is @emph{not} rounded to an integer---@samp{3 / 4} has the value 0.75. (It is a common mistake, especially for C programmers, -to forget that @emph{all} numbers in @command{awk} are floating-point, +to forget that @emph{all} numbers in @command{awk} are floating point, and that division of integer-looking constants produces a real number, not an integer.) @@ -10672,7 +10897,7 @@ $ @kbd{awk '@{ print "Field number one:" $1 @}' mail-list} @cindex troubleshooting, string concatenation Because string concatenation does not have an explicit operator, it is -often necessary to insure that it happens at the right time by using +often necessary to ensure that it happens at the right time by using parentheses to enclose the items to concatenate. For example, you might expect that the following code fragment concatenates @code{file} and @code{name}: @@ -10688,7 +10913,7 @@ print "something meaningful" > file name @noindent This produces a syntax error with some versions of Unix @command{awk}.@footnote{It happens that BWK -@command{awk}, @command{gawk} and @command{mawk} all ``get it right,'' +@command{awk}, @command{gawk}, and @command{mawk} all ``get it right,'' but you should not rely on this.} It is necessary to use the following: @@ -10777,11 +11002,8 @@ you're never quite sure what you'll get. @node Assignment Ops @subsection Assignment Expressions -@c STARTOFRANGE asop @cindex assignment operators -@c STARTOFRANGE opas @cindex operators, assignment -@c STARTOFRANGE exas @cindex expressions, assignment @cindex @code{=} (equals sign), @code{=} operator @cindex equals sign (@code{=}), @code{=} operator @@ -10934,7 +11156,14 @@ The indices of @code{bar} are practically guaranteed to be different, because @code{rand()} returns different values each time it is called. (Arrays and the @code{rand()} function haven't been covered yet. @xref{Arrays}, -and see @ref{Numeric Functions}, for more information). +and +@ifnotdocbook +@DBPXREF{Numeric Functions} +@end ifnotdocbook +@ifdocbook +@DBREF{Numeric Functions} +@end ifdocbook +for more information.) This example illustrates an important fact about assignment operators: the lefthand expression is only evaluated @emph{once}. @@ -10967,7 +11196,7 @@ to a number. @cindex @code{*} (asterisk), @code{**=} operator @cindex asterisk (@code{*}), @code{**=} operator @float Table,table-assign-ops -@caption{Arithmetic Assignment Operators} +@caption{Arithmetic assignment operators} @multitable @columnfractions .30 .70 @headitem Operator @tab Effect @item @var{lvalue} @code{+=} @var{increment} @tab Add @var{increment} to the value of @var{lvalue}. @@ -10979,7 +11208,7 @@ to a number. @cindex extensions, common@comma{} @code{**=} operator @cindex @command{awk} language, POSIX version @cindex POSIX @command{awk} -@item @var{lvalue} @code{^=} @var{power} @tab +@item @var{lvalue} @code{^=} @var{power} @tab Raise @var{lvalue} to the power @var{power}. @item @var{lvalue} @code{**=} @var{power} @tab Raise @var{lvalue} to the power @var{power}. @value{COMMONEXT} @end multitable @end float @@ -11011,7 +11240,7 @@ This is most notable in some commercial @command{awk} versions. For example: @example -$ awk /==/ /dev/null +$ @kbd{awk /==/ /dev/null} @error{} awk: syntax error at source line 1 @error{} context is @error{} >>> /= <<< @@ -11028,16 +11257,11 @@ awk '/[=]=/' /dev/null @command{gawk} does not have this problem; BWK @command{awk} and @command{mawk} also do not. @end sidebar -@c ENDOFRANGE exas -@c ENDOFRANGE opas -@c ENDOFRANGE asop @node Increment Ops @subsection Increment and Decrement Operators -@c STARTOFRANGE inop @cindex increment operators -@c STARTOFRANGE opde @cindex operators, decrement/increment @dfn{Increment} and @dfn{decrement operators} increase or decrease the value of a variable by one. An assignment operator can do the same thing, so @@ -11061,13 +11285,14 @@ has the value four, but it changes the value of @code{foo} to five. In other words, the operator returns the old value of the variable, but with the side effect of incrementing it. +@c FIXME: Use @sup here for superscript The post-increment @samp{foo++} is nearly the same as writing @samp{(foo += 1) - 1}. It is not perfectly equivalent because all numbers in -@command{awk} are floating-point---in floating-point, @samp{foo + 1 - 1} does +@command{awk} are floating point---in floating point, @samp{foo + 1 - 1} does not necessarily equal @code{foo}. But the difference is minute as long as you stick to numbers that are fairly small (less than @iftex -@math{10^12}). +@math{10^{12}}). @end iftex @ifnottex @ifnotdocbook @@ -11085,7 +11310,6 @@ just like variables. (Use @samp{$(i++)} when you want to do a field reference and a variable increment at the same time. The parentheses are necessary because of the precedence of the field reference operator @samp{$}.) -@c STARTOFRANGE deop @cindex decrement operators The decrement operator @samp{--} works just like @samp{++}, except that it subtracts one instead of adding it. As with @samp{++}, it can be used before @@ -11125,8 +11349,8 @@ like @samp{@var{lvalue}++}, but instead of adding, it subtracts.) @cindex evaluation order @cindex Marx, Groucho @quotation -@i{Doctor, doctor! It hurts when I do this!@* -So don't do that!} +@i{Doctor, it hurts when I do this!@* +Then don't do that!} @author Groucho Marx @end quotation @@ -11150,7 +11374,7 @@ print b @cindex side effects In other words, when do the various side effects prescribed by the postfix operators (@samp{b++}) take effect? -When side effects happen is @dfn{implementation defined}. +When side effects happen is @dfn{implementation-defined}. In other words, it is up to the particular version of @command{awk}. The result for the first example may be 12 or 13, and for the second, it may be 22 or 23. @@ -11161,14 +11385,11 @@ You should avoid such things in your own programs. @c You'll sleep better at night and be able to look at yourself @c in the mirror in the morning. @end sidebar -@c ENDOFRANGE inop -@c ENDOFRANGE opde -@c ENDOFRANGE deop @node Truth Values and Conditions @section Truth Values and Conditions -In certain contexts, expression values also serve as ``truth values;'' i.e., +In certain contexts, expression values also serve as ``truth values''; i.e., they determine what should happen next as the program runs. This @value{SECTION} describes how @command{awk} defines ``true'' and ``false'' and how values are compared. @@ -11225,22 +11446,21 @@ the string constant @code{"0"} is actually true, because it is non-null. @subsection Variable Typing and Comparison Expressions @quotation @i{The Guide is definitive. Reality is frequently inaccurate.} -@author The Hitchhiker's Guide to the Galaxy +@author Douglas Adams, @cite{The Hitchhiker's Guide to the Galaxy} @end quotation +@c 2/2015: Antonio Colombo points out that this is really from +@c The Restaurant at the End of the Universe. But I'm going to +@c leave it alone. -@c STARTOFRANGE comex @cindex comparison expressions -@c STARTOFRANGE excom @cindex expressions, comparison @cindex expressions, matching, See comparison expressions @cindex matching, expressions, See comparison expressions @cindex relational operators, See comparison operators @cindex operators, relational, See operators@comma{} comparison -@c STARTOFRANGE varting @cindex variable typing -@c STARTOFRANGE vartypc @cindex variables, types of, comparison expressions and -Unlike other programming languages, @command{awk} variables do not have a +Unlike in other programming languages, in @command{awk} variables do not have a fixed type. Instead, they can be either a number or a string, depending upon the value that is assigned to them. We look now at how variables are typed, and how @command{awk} @@ -11253,7 +11473,7 @@ compares variables. @end menu @node Variable Typing -@subsubsection String Type Versus Numeric Type +@subsubsection String Type versus Numeric Type @cindex numeric, strings @cindex strings, numeric @@ -11269,20 +11489,20 @@ Variable typing follows these rules: @itemize @value{BULLET} @item -A numeric constant or the result of a numeric operation has the @var{numeric} +A numeric constant or the result of a numeric operation has the @dfn{numeric} attribute. @item -A string constant or the result of a string operation has the @var{string} +A string constant or the result of a string operation has the @dfn{string} attribute. @item Fields, @code{getline} input, @code{FILENAME}, @code{ARGV} elements, @code{ENVIRON} elements, and the elements of an array created by -@code{match()}, @code{split()} and @code{patsplit()} that are numeric -strings have the @var{strnum} attribute. Otherwise, they have -the @var{string} attribute. Uninitialized variables also have the -@var{strnum} attribute. +@code{match()}, @code{split()}, and @code{patsplit()} that are numeric +strings have the @dfn{strnum} attribute. Otherwise, they have +the @dfn{string} attribute. Uninitialized variables also have the +@dfn{strnum} attribute. @item Attributes propagate across assignments but are not changed by @@ -11426,13 +11646,13 @@ constant, then a string comparison is performed. Otherwise, a numeric comparison is performed. This point bears additional emphasis: All user input is made of characters, -and so is first and foremost of @var{string} type; input strings -that look numeric are additionally given the @var{strnum} attribute. +and so is first and foremost of string type; input strings +that look numeric are additionally given the strnum attribute. Thus, the six-character input string @w{@samp{ +3.14}} receives the -@var{strnum} attribute. In contrast, the eight characters +strnum attribute. In contrast, the eight characters @w{@code{" +3.14"}} appearing in program text comprise a string constant. The following examples print @samp{1} when the comparison between -the two different constants is true, @samp{0} otherwise: +the two different constants is true, and @samp{0} otherwise: @c 22.9.2014: Tested with mawk and BWK awk, got same results. @example @@ -11480,18 +11700,18 @@ operators}, which are a superset of those in C. @cindex exclamation point (@code{!}), @code{!~} operator @cindex @code{in} operator @float Table,table-relational-ops -@caption{Relational Operators} +@caption{Relational operators} @multitable @columnfractions .25 .75 @headitem Expression @tab Result -@item @var{x} @code{<} @var{y} @tab True if @var{x} is less than @var{y}. -@item @var{x} @code{<=} @var{y} @tab True if @var{x} is less than or equal to @var{y}. -@item @var{x} @code{>} @var{y} @tab True if @var{x} is greater than @var{y}. -@item @var{x} @code{>=} @var{y} @tab True if @var{x} is greater than or equal to @var{y}. -@item @var{x} @code{==} @var{y} @tab True if @var{x} is equal to @var{y}. -@item @var{x} @code{!=} @var{y} @tab True if @var{x} is not equal to @var{y}. -@item @var{x} @code{~} @var{y} @tab True if the string @var{x} matches the regexp denoted by @var{y}. -@item @var{x} @code{!~} @var{y} @tab True if the string @var{x} does not match the regexp denoted by @var{y}. -@item @var{subscript} @code{in} @var{array} @tab True if the array @var{array} has an element with the subscript @var{subscript}. +@item @var{x} @code{<} @var{y} @tab True if @var{x} is less than @var{y} +@item @var{x} @code{<=} @var{y} @tab True if @var{x} is less than or equal to @var{y} +@item @var{x} @code{>} @var{y} @tab True if @var{x} is greater than @var{y} +@item @var{x} @code{>=} @var{y} @tab True if @var{x} is greater than or equal to @var{y} +@item @var{x} @code{==} @var{y} @tab True if @var{x} is equal to @var{y} +@item @var{x} @code{!=} @var{y} @tab True if @var{x} is not equal to @var{y} +@item @var{x} @code{~} @var{y} @tab True if the string @var{x} matches the regexp denoted by @var{y} +@item @var{x} @code{!~} @var{y} @tab True if the string @var{x} does not match the regexp denoted by @var{y} +@item @var{subscript} @code{in} @var{array} @tab True if the array @var{array} has an element with the subscript @var{subscript} @end multitable @end float @@ -11529,24 +11749,24 @@ The following list of expressions illustrates the kinds of comparisons @table @code @item 1.5 <= 2.0 -numeric comparison (true) +Numeric comparison (true) @item "abc" >= "xyz" -string comparison (false) +String comparison (false) @item 1.5 != " +2" -string comparison (true) +String comparison (true) @item "1e2" < "3" -string comparison (true) +String comparison (true) @item a = 2; b = "2" @itemx a == b -string comparison (true) +String comparison (true) @item a = 2; b = " +2" @itemx a == b -string comparison (false) +String comparison (false) @end table In this example: @@ -11562,7 +11782,7 @@ $ @kbd{echo 1e2 3 | awk '@{ print ($1 < $2) ? "true" : "false" @}'} @noindent the result is @samp{false} because both @code{$1} and @code{$2} are user input. They are numeric strings---therefore both have -the @var{strnum} attribute, dictating a numeric comparison. +the strnum attribute, dictating a numeric comparison. The purpose of the comparison rules and the use of numeric strings is to attempt to produce the behavior that is ``least surprising,'' while still ``doing the right thing.'' @@ -11599,7 +11819,7 @@ dynamic regexp (@pxref{Regexp Usage}; also @cindex @command{awk}, regexp constants and @cindex regexp constants A constant regular -expression in slashes by itself is also an expression. The regexp +expression in slashes by itself is also an expression. @code{/@var{regexp}/} is an abbreviation for the following comparison expression: @example @@ -11613,7 +11833,7 @@ One special place where @code{/foo/} is @emph{not} an abbreviation for where this is discussed in more detail. @node POSIX String Comparison -@subsubsection String Comparison With POSIX Rules +@subsubsection String Comparison with POSIX Rules The POSIX standard says that string comparison is performed based on the locale's @dfn{collating order}. This is the order in which @@ -11621,7 +11841,7 @@ characters sort, as defined by the locale (for more discussion, @pxref{Locales}). This order is usually very different from the results obtained when doing straight character-by-character comparison.@footnote{Technically, string comparison is supposed -to behave the same way as if the strings are compared with the C +to behave the same way as if the strings were compared with the C @code{strcoll()} function.} Because this behavior differs considerably from existing practice, @@ -11638,19 +11858,13 @@ $ @kbd{gawk --posix 'BEGIN @{ printf("ABC < abc = %s\n",} @print{} ABC < abc = FALSE @end example -@c ENDOFRANGE comex -@c ENDOFRANGE excom -@c ENDOFRANGE vartypc -@c ENDOFRANGE varting @node Boolean Ops @subsection Boolean Expressions @cindex and Boolean-logic operator @cindex or Boolean-logic operator @cindex not Boolean-logic operator -@c STARTOFRANGE exbo @cindex expressions, Boolean -@c STARTOFRANGE boex @cindex Boolean expressions @cindex operators, Boolean, See Boolean expressions @cindex Boolean operators, See Boolean expressions @@ -11734,7 +11948,7 @@ BEGIN @{ if (! ("HOME" in ENVIRON)) @cindex vertical bar (@code{|}), @code{||} operator The @samp{&&} and @samp{||} operators are called @dfn{short-circuit} operators because of the way they work. Evaluation of the full expression -is ``short-circuited'' if the result can be determined part way through +is ``short-circuited'' if the result can be determined partway through its evaluation. @cindex line continuations @@ -11796,8 +12010,6 @@ next record, and start processing the rules over again at the top. The reason it's there is to avoid printing the bracketing @samp{START} and @samp{END} lines. @end quotation -@c ENDOFRANGE exbo -@c ENDOFRANGE boex @node Conditional Exp @subsection Conditional Expressions @@ -11808,8 +12020,8 @@ The reason it's there is to avoid printing the bracketing A @dfn{conditional expression} is a special kind of expression that has three operands. It allows you to use one expression's value to select one of two other expressions. -The conditional expression is the same as in the C language, -as shown here: +The conditional expression in @command{awk} is the same as in the C +language, as shown here: @example @var{selector} ? @var{if-true-exp} : @var{if-false-exp} @@ -11818,8 +12030,8 @@ as shown here: @noindent There are three subexpressions. The first, @var{selector}, is always computed first. If it is ``true'' (not zero or not null), then -@var{if-true-exp} is computed next and its value becomes the value of -the whole expression. Otherwise, @var{if-false-exp} is computed next +@var{if-true-exp} is computed next, and its value becomes the value of +the whole expression. Otherwise, @var{if-false-exp} is computed next, and its value becomes the value of the whole expression. For example, the following expression produces the absolute value of @code{x}: @@ -11867,15 +12079,15 @@ ask for it by name at any point in the program. For example, the function @code{sqrt()} computes the square root of a number. @cindex functions, built-in -A fixed set of functions are @dfn{built-in}, which means they are +A fixed set of functions are @dfn{built in}, which means they are available in every @command{awk} program. The @code{sqrt()} function is one -of these. @xref{Built-in}, for a list of built-in +of these. @DBXREF{Built-in} for a list of built-in functions and their descriptions. In addition, you can define functions for use in your program. -@xref{User-defined}, +@DBXREF{User-defined} for instructions on how to do this. Finally, @command{gawk} lets you write functions in C or C++ -that may be called from your program: see @ref{Dynamic Extensions}. +that may be called from your program (@pxref{Dynamic Extensions}). @cindex arguments, in function calls The way to use a function is with a @dfn{function call} expression, @@ -11894,7 +12106,7 @@ rand() @ii{no arguments} @cindex troubleshooting, function call syntax @quotation CAUTION -Do not put any space between the function name and the open-parenthesis! +Do not put any space between the function name and the opening parenthesis! A user-defined function name looks just like the name of a variable---a space would make the expression look like concatenation of a variable with an expression inside parentheses. @@ -11915,7 +12127,7 @@ Some of the built-in functions have one or more optional arguments. If those arguments are not supplied, the functions use a reasonable default value. -@xref{Built-in}, for full details. If arguments +@DBXREF{Built-in} for full details. If arguments are omitted in calls to user-defined functions, then those arguments are treated as local variables. Such local variables act like the empty string if referenced where a string value is required, @@ -11976,9 +12188,7 @@ $ @kbd{awk -f matchit.awk} @node Precedence @section Operator Precedence (How Operators Nest) -@c STARTOFRANGE prec @cindex precedence -@c STARTOFRANGE oppr @cindex operators, precedence @dfn{Operator precedence} determines how operators are grouped when @@ -12043,7 +12253,7 @@ Increment, decrement. @cindex @code{*} (asterisk), @code{**} operator @cindex asterisk (@code{*}), @code{**} operator @item @code{^ **} -Exponentiation. These operators group right-to-left. +Exponentiation. These operators group right to left. @cindex @code{+} (plus sign), @code{+} operator @cindex plus sign (@code{+}), @code{+} operator @@ -12070,7 +12280,7 @@ Multiplication, division, remainder. @item @code{+ -} Addition, subtraction. -@item String Concatenation +@item String concatenation There is no special symbol for concatenation. The operands are simply written side by side (@pxref{Concatenation}). @@ -12109,7 +12319,7 @@ statements belong to the statement level, not to expressions. The redirection does not produce an expression that could be the operand of another operator. As a result, it does not make sense to use a redirection operator near another operator of lower precedence without -parentheses. Such combinations (for example, @samp{print foo > a ? b : c}), +parentheses. Such combinations (e.g., @samp{print foo > a ? b : c}) result in syntax errors. The correct way to write this statement is @samp{print foo > (a ? b : c)}. @@ -12127,17 +12337,17 @@ Array membership. @cindex @code{&} (ampersand), @code{&&} operator @cindex ampersand (@code{&}), @code{&&} operator @item @code{&&} -Logical ``and''. +Logical ``and.'' @cindex @code{|} (vertical bar), @code{||} operator @cindex vertical bar (@code{|}), @code{||} operator @item @code{||} -Logical ``or''. +Logical ``or.'' @cindex @code{?} (question mark), @code{?:} operator @cindex question mark (@code{?}), @code{?:} operator @item @code{?:} -Conditional. This operator groups right-to-left. +Conditional. This operator groups right to left. @cindex @code{+} (plus sign), @code{+=} operator @cindex plus sign (@code{+}), @code{+=} operator @@ -12154,7 +12364,7 @@ Conditional. This operator groups right-to-left. @cindex @code{^} (caret), @code{^=} operator @cindex caret (@code{^}), @code{^=} operator @item @code{= += -= *= /= %= ^= **=} -Assignment. These operators group right-to-left. +Assignment. These operators group right to left. @end table @cindex POSIX @command{awk}, @code{**} operator and @@ -12163,11 +12373,9 @@ Assignment. These operators group right-to-left. The @samp{|&}, @samp{**}, and @samp{**=} operators are not specified by POSIX. For maximum portability, do not use them. @end quotation -@c ENDOFRANGE prec -@c ENDOFRANGE oppr @node Locales -@section Where You Are Makes A Difference +@section Where You Are Makes a Difference @cindex locale, definition of Modern systems support the notion of @dfn{locales}: a way to tell the @@ -12187,8 +12395,8 @@ character}, to find the record terminator. Locales can affect how dates and times are formatted (@pxref{Time Functions}). For example, a common way to abbreviate the date September -4, 2015 in the United States is ``9/4/15.'' In many countries in -Europe, however, it is abbreviated ``4.9.15.'' Thus, the @code{%x} +4, 2015, in the United States is ``9/4/15.'' In many countries in +Europe, however, it is abbreviated ``4.9.15.'' Thus, the @samp{%x} specification in a @code{"US"} locale might produce @samp{9/4/15}, while in a @code{"EUROPE"} locale, it might produce @samp{4.9.15}. @@ -12206,13 +12414,13 @@ in @ref{Conversion}. @itemize @value{BULLET} @item Expressions are the basic elements of computation in programs. They are -built from constants, variables, function calls and combinations of the +built from constants, variables, function calls, and combinations of the various kinds of values with operators. @item @command{awk} supplies three kinds of constants: numeric, string, and regexp. @command{gawk} lets you specify numeric constants in octal -and hexadecimal (bases 8 and 16) in addition to decimal (base 10). +and hexadecimal (bases 8 and 16) as well as decimal (base 10). In certain contexts, a standalone regexp constant such as @code{/foo/} has the same meaning as @samp{$0 ~ /foo/}. @@ -12230,8 +12438,8 @@ Locales can influence the conversions. @item @command{awk} provides the usual arithmetic operators (addition, subtraction, multiplication, division, modulus), and unary plus and minus. -It also provides comparison operators, boolean operators, array membership -testing, and regexp +It also provides comparison operators, Boolean operators, an array membership +testing operator, and regexp matching operators. String concatenation is accomplished by placing two expressions next to each other; there is no explicit operator. The three-operand @samp{?:} operator provides an ``if-else'' test within @@ -12242,7 +12450,7 @@ Assignment operators provide convenient shorthands for common arithmetic operations. @item -In @command{awk}, a value is considered to be true if it is non-zero +In @command{awk}, a value is considered to be true if it is nonzero @emph{or} non-null. Otherwise, the value is false. @item @@ -12251,11 +12459,11 @@ lifetime. The type determines how it behaves in comparisons (string or numeric). @item -Function calls return a value which may be used as part of a larger +Function calls return a value that may be used as part of a larger expression. Expressions used to pass parameter values are fully evaluated before the function is called. @command{awk} provides -built-in and user-defined functions; this is described later on in this -@value{DOCUMENT}. +built-in and user-defined functions; this is described in +@ref{Functions}. @item Operator precedence specifies the order in which operations are performed, @@ -12268,11 +12476,9 @@ program, and occasionally the format for data read as input. @end itemize -@c ENDOFRANGE exps @node Patterns and Actions @chapter Patterns, Actions, and Variables -@c STARTOFRANGE pat @cindex patterns As you have already seen, each @command{awk} statement consists of @@ -12280,7 +12486,7 @@ a pattern with an associated action. This @value{CHAPTER} describes how you build patterns and actions, what kinds of things you can do within actions, and @command{awk}'s predefined variables. -The pattern-action rules and the statements available for use +The pattern--action rules and the statements available for use within actions form the core of @command{awk} programming. In a sense, everything covered up to here has been the foundation @@ -12416,8 +12622,8 @@ $ @kbd{awk '$1 ~ /li/ @{ print $2 @}' mail-list} @cindex regexp constants, as patterns @cindex patterns, regexp constants as A regexp constant as a pattern is also a special case of an expression -pattern. The expression @samp{/li/} has the value one if @samp{li} -appears in the current input record. Thus, as a pattern, @samp{/li/} +pattern. The expression @code{/li/} has the value one if @samp{li} +appears in the current input record. Thus, as a pattern, @code{/li/} matches any record containing @samp{li}. @cindex Boolean expressions, as patterns @@ -12468,11 +12674,11 @@ The subexpressions of a Boolean operator in a pattern can be constant regular expressions, comparisons, or any other @command{awk} expressions. Range patterns are not expressions, so they cannot appear inside Boolean patterns. Likewise, the special patterns @code{BEGIN}, @code{END}, -@code{BEGINFILE} and @code{ENDFILE}, +@code{BEGINFILE}, and @code{ENDFILE}, which never match any input record, are not expressions and cannot appear inside Boolean patterns. -The precedence of the different operators which can appear in +The precedence of the different operators that can appear in patterns is described in @ref{Precedence}. @node Ranges @@ -12498,7 +12704,7 @@ prints every record in @file{myfile} between @samp{on}/@samp{off} pairs, inclusi A range pattern starts out by matching @var{begpat} against every input record. When a record matches @var{begpat}, the range pattern is -@dfn{turned on} and the range pattern matches this record as well. As long as +@dfn{turned on}, and the range pattern matches this record as well. As long as the range pattern stays turned on, it automatically matches every input record read. The range pattern also matches @var{endpat} against every input record; when this succeeds, the range pattern is @dfn{turned off} again @@ -12569,9 +12775,7 @@ a range pattern. @value{DARKCORNER} @node BEGIN/END @subsection The @code{BEGIN} and @code{END} Special Patterns -@c STARTOFRANGE beg @cindex @code{BEGIN} pattern -@c STARTOFRANGE end @cindex @code{END} pattern All the patterns described so far are for matching input records. The @code{BEGIN} and @code{END} special patterns are different. @@ -12579,7 +12783,7 @@ They supply startup and cleanup actions for @command{awk} programs. @code{BEGIN} and @code{END} rules must have actions; there is no default action for these rules because there is no current record when they run. @code{BEGIN} and @code{END} rules are often referred to as -``@code{BEGIN} and @code{END} blocks'' by long-time @command{awk} +``@code{BEGIN} and @code{END} blocks'' by longtime @command{awk} programmers. @menu @@ -12610,7 +12814,7 @@ $ @kbd{awk '} This program finds the number of records in the input file @file{mail-list} that contain the string @samp{li}. The @code{BEGIN} rule prints a title for the report. There is no need to use the @code{BEGIN} rule to -initialize the counter @code{n} to zero, since @command{awk} does this +initialize the counter @code{n} to zero, as @command{awk} does this automatically (@pxref{Variables}). The second rule increments the variable @code{n} every time a record containing the pattern @samp{li} is read. The @code{END} rule @@ -12638,13 +12842,13 @@ The order in which library functions are named on the command line controls the order in which their @code{BEGIN} and @code{END} rules are executed. Therefore, you have to be careful when writing such rules in library files so that the order in which they are executed doesn't matter. -@xref{Options}, for more information on +@DBXREF{Options} for more information on using library functions. @xref{Library Functions}, for a number of useful library functions. If an @command{awk} program has only @code{BEGIN} rules and no -other rules, then the program exits after the @code{BEGIN} rule is +other rules, then the program exits after the @code{BEGIN} rules are run.@footnote{The original version of @command{awk} kept reading and ignoring input until the end of the file was seen.} However, if an @code{END} rule exists, then the input is read, even if there are @@ -12672,7 +12876,7 @@ Another way is simply to assign a value to @code{$0}. @cindex @code{print} statement, @code{BEGIN}/@code{END} patterns and @cindex @code{BEGIN} pattern, @code{print} statement and @cindex @code{END} pattern, @code{print} statement and -The second point is similar to the first but from the other direction. +The second point is similar to the first, but from the other direction. Traditionally, due largely to implementation issues, @code{$0} and @code{NF} were @emph{undefined} inside an @code{END} rule. The POSIX standard specifies that @code{NF} is available in an @code{END} @@ -12687,11 +12891,11 @@ of Unix @command{awk} do not. The third point follows from the first two. The meaning of @samp{print} inside a @code{BEGIN} or @code{END} rule is the same as always: @samp{print $0}. If @code{$0} is the null string, then this prints an -empty record. Many long time @command{awk} programmers use an unadorned +empty record. Many longtime @command{awk} programmers use an unadorned @samp{print} in @code{BEGIN} and @code{END} rules, to mean @samp{@w{print ""}}, relying on @code{$0} being null. Although one might generally get away with this in @code{BEGIN} rules, it is a very bad idea in @code{END} rules, -at least in @command{gawk}. It is also poor style, since if an empty +at least in @command{gawk}. It is also poor style, because if an empty line is needed in the output, the program should print one explicitly. @cindex @code{next} statement, @code{BEGIN}/@code{END} patterns and @@ -12701,11 +12905,14 @@ line is needed in the output, the program should print one explicitly. Finally, the @code{next} and @code{nextfile} statements are not allowed in a @code{BEGIN} rule, because the implicit read-a-record-and-match-against-the-rules loop has not started yet. Similarly, those statements -are not valid in an @code{END} rule, since all the input has been read. -(@xref{Next Statement}, and see -@ref{Nextfile Statement}.) -@c ENDOFRANGE beg -@c ENDOFRANGE end +are not valid in an @code{END} rule, because all the input has been read. +(@DBXREF{Next Statement} and +@ifnotdocbook +@DBPXREF{Nextfile Statement}.) +@end ifnotdocbook +@ifdocbook +@DBREF{Nextfile Statement}.) +@end ifdocbook @node BEGINFILE/ENDFILE @subsection The @code{BEGINFILE} and @code{ENDFILE} Special Patterns @@ -12758,7 +12965,7 @@ fatal error. @item If you have written extensions that modify the record handling (by -inserting an ``input parser,'' @pxref{Input Parsers}), you can invoke +inserting an ``input parser''; @pxref{Input Parsers}), you can invoke them at this point, before @command{gawk} has started processing the file. (This is a @emph{very} advanced feature, currently used only by the @uref{http://gawkextlib.sourceforge.net, @code{gawkextlib} project}.) @@ -12769,8 +12976,8 @@ the last record in an input file. For the last input file, it will be called before any @code{END} rules. The @code{ENDFILE} rule is executed even for empty input files. -Normally, when an error occurs when reading input in the normal input -processing loop, the error is fatal. However, if an @code{ENDFILE} +Normally, when an error occurs when reading input in the normal +input-processing loop, the error is fatal. However, if an @code{ENDFILE} rule is present, the error becomes non-fatal, and instead @code{ERRNO} is set. This makes it possible to catch and process I/O errors at the level of the @command{awk} program. @@ -12779,7 +12986,7 @@ level of the @command{awk} program. The @code{next} statement (@pxref{Next Statement}) is not allowed inside either a @code{BEGINFILE} or an @code{ENDFILE} rule. The @code{nextfile} statement is allowed only inside a -@code{BEGINFILE} rule, but not inside an @code{ENDFILE} rule. +@code{BEGINFILE} rule, not inside an @code{ENDFILE} rule. @cindex @code{getline} statement, @code{BEGINFILE}/@code{ENDFILE} patterns and The @code{getline} statement (@pxref{Getline}) is restricted inside @@ -12826,7 +13033,6 @@ awk '@{ print $1 @}' mail-list @noindent prints the first field of every record. -@c ENDOFRANGE pat @node Using Shell Variables @section Using Shell Variables in Programs @@ -12860,7 +13066,7 @@ The first part is double-quoted, which allows substitution of the @code{pattern} shell variable inside the quotes. The second part is single-quoted. -Variable substitution via quoting works, but can be potentially +Variable substitution via quoting works, but can potentially be messy. It requires a good understanding of the shell's quoting rules (@pxref{Quoting}), and it's often difficult to correctly @@ -12887,7 +13093,7 @@ The assignment @samp{-v pat="$pattern"} still requires double quotes, in case there is whitespace in the value of @code{$pattern}. The @command{awk} variable @code{pat} could be named @code{pattern} too, but that would be more confusing. Using a variable also -provides more flexibility, since the variable can be used anywhere inside +provides more flexibility, as the variable can be used anywhere inside the program---for printing, as an array subscript, or for any other use---without requiring the quoting tricks at every point in the program. @@ -12960,7 +13166,7 @@ is used in order to put several statements together in the body of an Use the @code{getline} command (@pxref{Getline}). Also supplied in @command{awk} are the @code{next} -statement (@pxref{Next Statement}), +statement (@pxref{Next Statement}) and the @code{nextfile} statement (@pxref{Nextfile Statement}). @@ -12975,11 +13181,8 @@ For deleting array elements. @node Statements @section Control Statements in Actions -@c STARTOFRANGE csta @cindex control statements -@c STARTOFRANGE acs @cindex statements, control, in actions -@c STARTOFRANGE accs @cindex actions, control statements in @dfn{Control statements}, such as @code{if}, @code{while}, and so on, @@ -13048,7 +13251,7 @@ else print "x is odd" @end example -In this example, if the expression @samp{x % 2 == 0} is true (that is, +In this example, if the expression @samp{x % 2 == 0} is true (i.e., if the value of @code{x} is evenly divisible by two), then the first @code{print} statement is executed; otherwise, the second @code{print} statement is executed. @@ -13122,13 +13325,13 @@ The body of this loop is a compound statement enclosed in braces, containing two statements. The loop works in the following manner: first, the value of @code{i} is set to one. Then, the @code{while} statement tests whether @code{i} is less than or equal to -three. This is true when @code{i} equals one, so the @code{i}-th +three. This is true when @code{i} equals one, so the @code{i}th field is printed. Then the @samp{i++} increments the value of @code{i} and the loop repeats. The loop terminates when @code{i} reaches four. A newline is not required between the condition and the -body; however using one makes the program clearer unless the body is a -compound statement or else is very simple. The newline after the open-brace +body; however, using one makes the program clearer unless the body is a +compound statement or else is very simple. The newline after the open brace that begins the compound statement is not required either, but the program is harder to read without it. @@ -13158,9 +13361,9 @@ while (@var{condition}) @end example @noindent -This statement does not execute @var{body} even once if the @var{condition} -is false to begin with. -The following is an example of a @code{do} statement: +This statement does not execute the @var{body} even once if the +@var{condition} is false to begin with. The following is an example of +a @code{do} statement: @example @{ @@ -13174,7 +13377,7 @@ The following is an example of a @code{do} statement: @noindent This program prints each input record 10 times. However, it isn't a very -realistic example, since in this case an ordinary @code{while} would do +realistic example, because in this case an ordinary @code{while} would do just as well. This situation reflects actual experience; only occasionally is there a real use for a @code{do} statement. @@ -13227,7 +13430,7 @@ their assignments as separate statements preceding the @code{for} loop.) The same is true of the @var{increment} part. Incrementing additional variables requires separate statements at the end of the loop. The C compound expression, using C's comma operator, is useful in -this context but it is not supported in @command{awk}. +this context, but it is not supported in @command{awk}. Most often, @var{increment} is an increment expression, as in the previous example. But this is not required; it can be any expression @@ -13271,7 +13474,7 @@ very common in loops. It can be easier to think of this counting as part of looping rather than as something to do inside the loop. @cindex @code{in} operator -There is an alternate version of the @code{for} loop, for iterating over +There is an alternative version of the @code{for} loop, for iterating over all the indices of an array: @example @@ -13280,7 +13483,7 @@ for (i in array) @end example @noindent -@xref{Scanning an Array}, +@DBXREF{Scanning an Array} for more information on this version of the @code{for} loop. @node Switch Statement @@ -13300,7 +13503,7 @@ are checked for a match in the order they are defined. If no suitable Each @code{case} contains a single constant, be it numeric, string, or regexp. The @code{switch} expression is evaluated, and then each -@code{case}'s constant is compared against the result in turn. The type of constant +@code{case}'s constant is compared against the result in turn. The type of constant determines the comparison: numeric or string do the usual comparisons. A regexp constant does a regular expression match against the string value of the original expression. The general form of the @code{switch} @@ -13318,7 +13521,7 @@ default: Control flow in the @code{switch} statement works as it does in C. Once a match to a given case is made, the case statement bodies execute until a @code{break}, -@code{continue}, @code{next}, @code{nextfile} or @code{exit} is encountered, +@code{continue}, @code{next}, @code{nextfile}, or @code{exit} is encountered, or the end of the @code{switch} statement itself. For example: @example @@ -13347,9 +13550,9 @@ while ((c = getopt(ARGC, ARGV, "aksx")) != -1) @{ @} @end example -Note that if none of the statements specified above halt execution +Note that if none of the statements specified here halt execution of a matched @code{case} statement, execution falls through to the -next @code{case} until execution halts. In the above example, the +next @code{case} until execution halts. In this example, the @code{case} for @code{"?"} falls through to the @code{default} case, which is to call a function named @code{usage()}. (The @code{getopt()} function being called here is @@ -13370,12 +13573,12 @@ numbers: # find smallest divisor of num @{ num = $1 - for (div = 2; div * div <= num; div++) @{ - if (num % div == 0) + for (divisor = 2; divisor * divisor <= num; divisor++) @{ + if (num % divisor == 0) break @} - if (num % div == 0) - printf "Smallest divisor of %d is %d\n", num, div + if (num % divisor == 0) + printf "Smallest divisor of %d is %d\n", num, divisor else printf "%d is prime\n", num @} @@ -13396,12 +13599,12 @@ an @code{if}: # find smallest divisor of num @{ num = $1 - for (div = 2; ; div++) @{ - if (num % div == 0) @{ - printf "Smallest divisor of %d is %d\n", num, div + for (divisor = 2; ; divisor++) @{ + if (num % divisor == 0) @{ + printf "Smallest divisor of %d is %d\n", num, divisor break @} - if (div * div > num) @{ + if (divisor * divisor > num) @{ printf "%d is prime\n", num break @} @@ -13476,7 +13679,7 @@ BEGIN @{ @end example @noindent -This program loops forever once @code{x} reaches 5, since +This program loops forever once @code{x} reaches 5, because the increment (@samp{x++}) is never reached. @c @cindex @code{continue}, outside of loops @@ -13492,7 +13695,12 @@ body of a loop. Historical versions of @command{awk} treated a @code{continue} statement outside a loop the same way they treated a @code{break} statement outside a loop: as if it were a @code{next} statement +@ifset FOR_PRINT +(discussed in the following section). +@end ifset +@ifclear FOR_PRINT (@pxref{Next Statement}). +@end ifclear @value{DARKCORNER} Recent versions of BWK @command{awk} no longer work this way, nor does @command{gawk}. @@ -13537,7 +13745,7 @@ Because of the @code{next} statement, the program's subsequent rules won't see the bad record. The error message is redirected to the standard error output stream, as error messages should be. -For more detail see +For more detail, see @ref{Special Files}. If the @code{next} statement causes the end of the input to be reached, @@ -13603,7 +13811,7 @@ rule to skip over a file that would otherwise cause @command{gawk} to exit with a fatal error. In this case, @code{ENDFILE} rules are not executed. @xref{BEGINFILE/ENDFILE}. -While one might think that @samp{close(FILENAME)} would accomplish +Although it might seem that @samp{close(FILENAME)} would accomplish the same as @code{nextfile}, this isn't true. @code{close()} is reserved for closing files, pipes, and coprocesses that are opened with redirections. It is not related to the main processing that @@ -13611,7 +13819,7 @@ opened with redirections. It is not related to the main processing that @quotation NOTE For many years, @code{nextfile} was a -common extension. In September, 2012, it was accepted for +common extension. In September 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=607, the Austin Group website}. @end quotation @@ -13620,7 +13828,7 @@ See @uref{http://austingroupbugs.net/view.php?id=607, the Austin Group website}. @cindex @code{nextfile} statement, user-defined functions and @cindex Brian Kernighan's @command{awk} @cindex @command{mawk} utility -The current version of BWK @command{awk}, and @command{mawk} +The current version of BWK @command{awk} and @command{mawk} also support @code{nextfile}. However, they don't allow the @code{nextfile} statement inside function bodies (@pxref{User-defined}). @command{gawk} does; a @code{nextfile} inside a function body reads the @@ -13658,9 +13866,9 @@ any @code{ENDFILE} rules; they do not execute. In such a case, if you don't want the @code{END} rule to do its job, set a variable -to nonzero before the @code{exit} statement and check that variable in +to a nonzero value before the @code{exit} statement and check that variable in the @code{END} rule. -@xref{Assert Function}, +@DBXREF{Assert Function} for an example that does this. @cindex dark corner, @code{exit} statement @@ -13671,7 +13879,7 @@ In the case where an argument is supplied to a first @code{exit} statement, and then @code{exit} is called a second time from an @code{END} rule with no argument, @command{awk} uses the previously supplied exit value. @value{DARKCORNER} -@xref{Exit Status}, for more information. +@DBXREF{Exit Status} for more information. @cindex programming conventions, @code{exit} statement For example, suppose an error condition occurs that is difficult or @@ -13697,15 +13905,10 @@ Negative values, and values of 127 or greater, may not produce consistent results across different operating systems. @end quotation -@c ENDOFRANGE csta -@c ENDOFRANGE acs -@c ENDOFRANGE accs @node Built-in Variables @section Predefined Variables -@c STARTOFRANGE bvar @cindex predefined variables -@c STARTOFRANGE varb @cindex variables, predefined Most @command{awk} variables are available to use for your own @@ -13732,9 +13935,7 @@ their areas of activity. @node User-modified @subsection Built-in Variables That Control @command{awk} -@c STARTOFRANGE bvaru @cindex predefined variables, user-modifiable -@c STARTOFRANGE nmbv @cindex user-modifiable variables The following is an alphabetical list of variables that you can change to @@ -13762,7 +13963,7 @@ respectively, should use binary I/O. A string value of @code{"rw"} or @code{"wr"} indicates that all files should use binary I/O. Any other string value is treated the same as @code{"rw"}, but causes @command{gawk} to generate a warning message. @code{BINMODE} is described in more -detail in @ref{PC Using}. @command{mawk} (@pxref{Other Versions}), +detail in @ref{PC Using}. @command{mawk} (@pxref{Other Versions}) also supports this variable, but only using numeric values. @cindex @code{CONVFMT} variable @@ -13770,7 +13971,7 @@ also supports this variable, but only using numeric values. @cindex numbers, converting, to strings @cindex strings, converting, numbers to @item @code{CONVFMT} -This string controls conversion of numbers to +A string that controls the conversion of numbers to strings (@pxref{Conversion}). It works by being passed, in effect, as the first argument to the @code{sprintf()} function @@ -13788,7 +13989,7 @@ A space-separated list of columns that tells @command{gawk} how to split input with fixed columnar boundaries. Assigning a value to @code{FIELDWIDTHS} overrides the use of @code{FS} and @code{FPAT} for field splitting. -@xref{Constant Size}, for more information. +@DBXREF{Constant Size} for more information. @cindex @command{gawk}, @code{FPAT} variable in @cindex @code{FPAT} variable @@ -13800,7 +14001,7 @@ A regular expression (as a string) that tells @command{gawk} to create the fields based on text that matches the regular expression. Assigning a value to @code{FPAT} overrides the use of @code{FS} and @code{FIELDWIDTHS} for field splitting. -@xref{Splitting By Content}, for more information. +@DBXREF{Splitting By Content} for more information. @cindex @code{FS} variable @cindex separators, field @@ -13845,12 +14046,13 @@ is to simply say @samp{FS = FS}, perhaps with an explanatory comment. @cindex regular expressions, case sensitivity @item IGNORECASE # If @code{IGNORECASE} is nonzero or non-null, then all string comparisons -and all regular expression matching are case independent. Thus, regexp -matching with @samp{~} and @samp{!~}, as well as the @code{gensub()}, -@code{gsub()}, @code{index()}, @code{match()}, @code{patsplit()}, -@code{split()}, and @code{sub()} -functions, record termination with @code{RS}, and field splitting with -@code{FS} and @code{FPAT}, all ignore case when doing their particular regexp operations. +and all regular expression matching are case-independent. +This applies to +regexp matching with @samp{~} and @samp{!~}, +the @code{gensub()}, @code{gsub()}, @code{index()}, @code{match()}, +@code{patsplit()}, @code{split()}, and @code{sub()} functions, +record termination with @code{RS}, and field splitting with +@code{FS} and @code{FPAT}. However, the value of @code{IGNORECASE} does @emph{not} affect array subscripting and it does not affect field splitting when using a single-character field separator. @@ -13871,7 +14073,7 @@ Any other true value prints nonfatal warnings. Assigning a false value to @code{LINT} turns off the lint warnings. This variable is a @command{gawk} extension. It is not special -in other @command{awk} implementations. Unlike the other special variables, +in other @command{awk} implementations. Unlike with the other special variables, changing @code{LINT} does affect the production of lint warnings, even if @command{gawk} is in compatibility mode. Much as the @option{--lint} and @option{--traditional} options independently @@ -13883,7 +14085,7 @@ of @command{awk} being executed. @cindex numbers, converting, to strings @cindex strings, converting, numbers to @item OFMT -Controls conversion of numbers to +A string that controls conversion of numbers to strings (@pxref{Conversion}) for printing with the @code{print} statement. It works by being passed as the first argument to the @code{sprintf()} function @@ -13898,7 +14100,7 @@ strings in general expressions; this is now done by @code{CONVFMT}. @cindex separators, field @cindex field separators @item OFS -This is the output field separator (@pxref{Output Separators}). It is +The output field separator (@pxref{Output Separators}). It is output between the fields printed by a @code{print} statement. Its default value is @w{@code{" "}}, a string consisting of a single space. @@ -13910,13 +14112,13 @@ character. (@xref{Output Separators}.) @cindex @code{PREC} variable @item PREC # -The working precision of arbitrary precision floating-point numbers, +The working precision of arbitrary-precision floating-point numbers, 53 bits by default (@pxref{Setting precision}). @cindex @code{ROUNDMODE} variable @item ROUNDMODE # -The rounding mode to use for arbitrary precision arithmetic on -numbers, by default @code{"N"} (@samp{roundTiesToEven} in +The rounding mode to use for arbitrary-precision arithmetic on +numbers, by default @code{"N"} (@code{roundTiesToEven} in the IEEE 754 standard; @pxref{Setting the rounding mode}). @cindex @code{RS} variable @@ -13945,7 +14147,7 @@ just the first character of @code{RS}'s value is used. @item @code{SUBSEP} The subscript separator. It has the default value of @code{"\034"} and is used to separate the parts of the indices of a -multidimensional array. Thus, the expression @code{@w{foo["A", "B"]}} +multidimensional array. Thus, the expression @samp{@w{foo["A", "B"]}} really accesses @code{foo["A\034B"]} (@pxref{Multidimensional}). @@ -13957,21 +14159,15 @@ really accesses @code{foo["A\034B"]} Used for internationalization of programs at the @command{awk} level. It sets the default text domain for specially marked string constants in the source text, as well as for the -@code{dcgettext()}, @code{dcngettext()} and @code{bindtextdomain()} functions +@code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()} functions (@pxref{Internationalization}). The default value of @code{TEXTDOMAIN} is @code{"messages"}. @end table -@c ENDOFRANGE bvar -@c ENDOFRANGE varb -@c ENDOFRANGE bvaru -@c ENDOFRANGE nmbv @node Auto-set @subsection Built-in Variables That Convey Information -@c STARTOFRANGE bvconi @cindex predefined variables, conveying information -@c STARTOFRANGE vbconi @cindex variables, predefined conveying information The following is an alphabetical list of variables that @command{awk} sets automatically on certain occasions in order to provide @@ -13980,7 +14176,7 @@ information to your program. The variables that are specific to @command{gawk} are marked with a pound sign (@samp{#}). These variables are @command{gawk} extensions. In other @command{awk} implementations or if @command{gawk} is in compatibility -mode (@pxref{Options}), they are not special. +mode (@pxref{Options}), they are not special: @c @asis for docbook @table @asis @@ -14021,7 +14217,7 @@ method of accessing command-line arguments. The value of @code{ARGV[0]} can vary from system to system. Also, you should note that the program text is @emph{not} included in @code{ARGV}, nor are any of @command{awk}'s command-line options. -@xref{ARGC and ARGV}, for information +@DBXREF{ARGC and ARGV} for information about how @command{awk} uses these variables. @value{DARKCORNER} @@ -14063,12 +14259,23 @@ that it creates. You should therefore be especially careful if you modify @code{ENVIRON["PATH"]"}, which is the search path for finding executable programs. +This can also affect the running @command{gawk} program, since some of the +built-in functions may pay attention to certain environment variables. +The most notable instance of this is @code{mktime()} (@pxref{Time +Functions}), which pays attention the value of the @env{TZ} environment +variable on many systems. + Some operating systems may not have environment variables. On such systems, the @code{ENVIRON} array is empty (except for @w{@code{ENVIRON["AWKPATH"]}} and @w{@code{ENVIRON["AWKLIBPATH"]}}; -@pxref{AWKPATH Variable}, and +@DBPXREF{AWKPATH Variable} and +@ifdocbook +@DBREF{AWKLIBPATH Variable}). +@end ifdocbook +@ifnotdocbook @pxref{AWKLIBPATH Variable}). +@end ifnotdocbook @cindex @command{gawk}, @code{ERRNO} variable in @cindex @code{ERRNO} variable @@ -14090,6 +14297,11 @@ value to be meaningful when an I/O operation returns a failure value, such as @code{getline} returning @minus{}1. You are, of course, free to clear it yourself before doing an I/O operation. +If the value of @code{ERRNO} corresponds to a system error in the C +@code{errno} variable, then @code{PROCINFO["errno"]} will be set to the value +of @code{errno}. For non-system errors, @code{PROCINFO["errno"]} will +be zero. + @cindex @code{FILENAME} variable @cindex dark corner, @code{FILENAME} variable @item @code{FILENAME} @@ -14097,7 +14309,7 @@ The name of the current input file. When no @value{DF}s are listed on the command line, @command{awk} reads from the standard input and @code{FILENAME} is set to @code{"-"}. @code{FILENAME} changes each time a new file is read (@pxref{Reading Files}). Inside a @code{BEGIN} -rule, the value of @code{FILENAME} is @code{""}, since there are no input +rule, the value of @code{FILENAME} is @code{""}, because there are no input files being processed yet.@footnote{Some early implementations of Unix @command{awk} initialized @code{FILENAME} to @code{"-"}, even if there were @value{DF}s to be processed. This behavior was incorrect and should @@ -14116,12 +14328,12 @@ input file. @item @code{NF} The number of fields in the current input record. @code{NF} is set each time a new record is read, when a new field is -created or when @code{$0} changes (@pxref{Fields}). +created, or when @code{$0} changes (@pxref{Fields}). Unlike most of the variables described in this @value{SUBSECTION}, assigning a value to @code{NF} has the potential to affect @command{awk}'s internal workings. In particular, assignments -to @code{NF} can be used to create or remove fields from the +to @code{NF} can be used to create fields in or remove fields from the current record. @xref{Changing Fields}. @cindex @code{FUNCTAB} array @@ -14129,7 +14341,7 @@ current record. @xref{Changing Fields}. @cindex differences in @command{awk} and @command{gawk}, @code{FUNCTAB} variable @item @code{FUNCTAB #} An array whose indices and corresponding values are the names of all -the built-in, user-defined and extension functions in the program. +the built-in, user-defined, and extension functions in the program. @quotation NOTE Attempting to use the @code{delete} statement with the @code{FUNCTAB} @@ -14158,6 +14370,10 @@ are guaranteed to be available: @item PROCINFO["egid"] The value of the @code{getegid()} system call. +@item PROCINFO["errno"] +The value of the C @code{errno} variable when @code{ERRNO} is set to +the associated error message. + @item PROCINFO["euid"] @cindex effective user ID of @command{gawk} user The value of the @code{geteuid()} system call. @@ -14171,7 +14387,7 @@ or @code{"FPAT"} if field matching with @code{FPAT} is in effect. @item PROCINFO["identifiers"] @cindex program identifiers A subarray, indexed by the names of all identifiers used in the text of -the AWK program. An @dfn{identifier} is simply the name of a variable +the @command{awk} program. An @dfn{identifier} is simply the name of a variable (be it scalar or array), built-in function, user-defined function, or extension function. For each identifier, the value of the element is one of the following: @@ -14191,7 +14407,7 @@ The identifier is an extension function loaded via The identifier is a scalar. @item "untyped" -The identifier is untyped (could be used as a scalar or array, +The identifier is untyped (could be used as a scalar or an array; @command{gawk} doesn't know yet). @item "user" @@ -14208,7 +14424,7 @@ while the program runs. The value of the @code{getgid()} system call. @item PROCINFO["pgrpid"] -@cindex process group idIDof @command{gawk} process +@cindex process group ID of @command{gawk} process The process group ID of the current process. @item PROCINFO["pid"] @@ -14223,7 +14439,7 @@ The parent process ID of the current process. If this element exists in @code{PROCINFO}, its value controls the order in which array indices will be processed by @samp{for (@var{indx} in @var{array})} loops. -Since this is an advanced feature, we defer the +This is an advanced feature, so we defer the full description until later; see @ref{Scanning an Array}. @@ -14243,10 +14459,10 @@ The version of @command{gawk}. The following additional elements in the array are available to provide information about the MPFR and GMP libraries -if your version of @command{gawk} supports arbitrary precision arithmetic -(@pxref{Arbitrary Precision Arithmetic}): +if your version of @command{gawk} supports arbitrary-precision arithmetic +(@pxref{Arbitrary Precision Arithmetic}): -@table @code +@table @code @cindex version of GNU MPFR library @item PROCINFO["mpfr_version"] The version of the GNU MPFR library. @@ -14260,7 +14476,7 @@ The version of the GNU MP library. The maximum precision supported by MPFR. @item PROCINFO["prec_min"] -@cindex minimum precision supported by MPFR library +@cindex minimum precision required by MPFR library The minimum precision required by MPFR. @end table @@ -14294,7 +14510,12 @@ The @code{PROCINFO} array has the following additional uses: @item It may be used to provide a timeout when reading from any open input file, pipe, or coprocess. -@xref{Read Timeout}, for more information. +@DBXREF{Read Timeout} for more information. + +@item +It may be used to indicate that input may be retried when it fails due to +certain errors. +@DBXREF{Retrying Input} for more information. @item It may be used to cause coprocesses to communicate over pseudo-ttys @@ -14312,7 +14533,7 @@ is the length of the matched string, or @minus{}1 if no match is found. @cindex @code{RSTART} variable @item @code{RSTART} -The start-index in characters of the substring that is matched by the +The start index in characters of the substring that is matched by the @code{match()} function (@pxref{String Functions}). @code{RSTART} is set by invoking the @code{match()} function. Its value @@ -14379,11 +14600,9 @@ function multiply(variable, amount) @quotation NOTE In order to avoid severe time-travel paradoxes,@footnote{Not to mention difficult implementation issues.} neither @code{FUNCTAB} nor @code{SYMTAB} -are available as elements within the @code{SYMTAB} array. +is available as an element within the @code{SYMTAB} array. @end quotation @end table -@c ENDOFRANGE bvconi -@c ENDOFRANGE vbconi @sidebar Changing @code{NR} and @code{FNR} @cindex @code{NR} variable, changing @@ -14493,8 +14712,14 @@ use the @code{delete} statement to remove elements from All of these actions are typically done in the @code{BEGIN} rule, before actual processing of the input begins. -@xref{Split Program}, and see -@ref{Tee Program}, for examples +@DBXREF{Split Program} and +@ifnotdocbook +@DBPXREF{Tee Program} +@end ifnotdocbook +@ifdocbook +@DBREF{Tee Program} +@end ifdocbook +for examples of each way of removing elements from @code{ARGV}. To actually get options into an @command{awk} program, @@ -14506,7 +14731,7 @@ awk -f myprog.awk -- -v -q file1 file2 @dots{} @end example The following fragment processes @code{ARGV} in order to examine, and -then remove, the above command-line options: +then remove, the previously mentioned command-line options: @example BEGIN @{ @@ -14542,29 +14767,36 @@ gawk -f myprog.awk -q -v file1 file2 @dots{} @noindent Because @option{-q} is not a valid @command{gawk} option, it and the following @option{-v} are passed on to the @command{awk} program. -(@xref{Getopt Function}, for an @command{awk} library function that +(@DBXREF{Getopt Function} for an @command{awk} library function that parses command-line options.) When designing your program, you should choose options that don't -conflict with @command{gawk}'s, since it will process any options +conflict with @command{gawk}'s, because it will process any options that it accepts before passing the rest of the command line on to your program. Using @samp{#!} with the @option{-E} option may help -(@pxref{Executable Scripts}, and @pxref{Options}). +(@DBPXREF{Executable Scripts} +and +@ifnotdocbook +@DBPXREF{Options}). +@end ifnotdocbook +@ifdocbook +@DBREF{Options}). +@end ifdocbook @node Pattern Action Summary @section Summary @itemize @value{BULLET} @item -Pattern-action pairs make up the basic elements of an @command{awk} +Pattern--action pairs make up the basic elements of an @command{awk} program. Patterns are either normal expressions, range expressions, -regexp constants, one of the special keywords @code{BEGIN}, @code{END}, -@code{BEGINFILE}, @code{ENDFILE}, or empty. The action executes if +or regexp constants; one of the special keywords @code{BEGIN}, @code{END}, +@code{BEGINFILE}, or @code{ENDFILE}; or empty. The action executes if the current record matches the pattern. Empty (missing) patterns match all records. @item -I/O from @code{BEGIN} and @code{END} rules have certain constraints. +I/O from @code{BEGIN} and @code{END} rules has certain constraints. This is also true, only more so, for @code{BEGINFILE} and @code{ENDFILE} rules. The latter two give you ``hooks'' into @command{gawk}'s file processing, allowing you to recover from a file that otherwise would @@ -14594,12 +14826,12 @@ iteration of a loop (or get out of a @code{switch}). @item @code{next} and @code{nextfile} let you read the next record and start -over at the top of your program, or skip to the next input file and +over at the top of your program or skip to the next input file and start over, respectively. @item The @code{exit} statement terminates your program. When executed -from an action (or function body) it transfers control to the +from an action (or function body), it transfers control to the @code{END} statements. From an @code{END} statement body, it exits immediately. You may pass an optional numeric value to be used as @command{awk}'s exit status. @@ -14617,7 +14849,6 @@ control how @command{awk} will process the provided @value{DF}s. @node Arrays @chapter Arrays in @command{awk} -@c STARTOFRANGE arrs @cindex arrays An @dfn{array} is a table of values called @dfn{elements}. The @@ -14692,7 +14923,7 @@ In most other languages, arrays must be @dfn{declared} before use, including a specification of how many elements or components they contain. In such languages, the declaration causes a contiguous block of memory to be allocated for that -many elements. Usually, an index in the array must be a positive integer. +many elements. Usually, an index in the array must be a nonnegative integer. For example, the index zero specifies the first element in the array, which is actually stored at the beginning of the block of memory. Index one specifies the second element, which is stored in memory right after the @@ -14703,19 +14934,21 @@ the declaration. indices---e.g., @samp{15 .. 27}---but the size of the array is still fixed when the array is declared.) -A contiguous array of four elements might look like the following example, -conceptually, if the element values are 8, @code{"foo"}, -@code{""}, and 30 +@c 1/2015: Do not put the numeric values into @code. Array element +@c values are no different than scalar variable values. +A contiguous array of four elements might look like @ifnotdocbook -as shown in @ref{figure-array-elements}: +@ref{figure-array-elements}, @end ifnotdocbook @ifdocbook -as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: +@inlineraw{docbook, <xref linkend="figure-array-elements"/>}, @end ifdocbook +conceptually, if the element values are eight, @code{"foo"}, +@code{""}, and 30. @ifnotdocbook @float Figure,figure-array-elements -@caption{A Contiguous Array} +@caption{A contiguous array} @ifinfo @center @image{array-elements, , , Basic Program Stages, txt} @end ifinfo @@ -14727,7 +14960,7 @@ as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: @docbook <figure id="figure-array-elements" float="0"> -<title>A Contiguous Array</title> +<title>A contiguous array</title> <mediaobject> <imageobject role="web"><imagedata fileref="array-elements.png" format="PNG"/></imageobject> </mediaobject> @@ -14736,33 +14969,33 @@ as shown in @inlineraw{docbook, <xref linkend="figure-array-elements"/>}: @noindent Only the values are stored; the indices are implicit from the order of -the values. Here, 8 is the value at index zero, because 8 appears in the +the values. Here, eight is the value at index zero, because eight appears in the position with zero elements before it. -@c STARTOFRANGE arrin @cindex arrays, indexing -@c STARTOFRANGE inarr @cindex indexing arrays @cindex associative arrays @cindex arrays, associative Arrays in @command{awk} are different---they are @dfn{associative}. This means -that each array is a collection of pairs: an index and its corresponding +that each array is a collection of pairs---an index and its corresponding array element value: @ifnotdocbook -@example -@r{Index} 3 @r{Value} 30 -@r{Index} 1 @r{Value} "foo" -@r{Index} 0 @r{Value} 8 -@r{Index} 2 @r{Value} "" -@end example +@c extra empty column to indent it right +@multitable @columnfractions .1 .1 .1 +@headitem @tab Index @tab Value +@item @tab @code{3} @tab @code{30} +@item @tab @code{1} @tab @code{"foo"} +@item @tab @code{0} @tab @code{8} +@item @tab @code{2} @tab @code{""} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -14808,20 +15041,22 @@ at any time. For example, suppose a tenth element is added to the array whose value is @w{@code{"number ten"}}. The result is: @ifnotdocbook -@example -@r{Index} 10 @r{Value} "number ten" -@r{Index} 3 @r{Value} 30 -@r{Index} 1 @r{Value} "foo" -@r{Index} 0 @r{Value} 8 -@r{Index} 2 @r{Value} "" -@end example +@c extra empty column to indent it right +@multitable @columnfractions .1 .1 .2 +@headitem @tab Index @tab Value +@item @tab @code{10} @tab @code{"number ten"} +@item @tab @code{3} @tab @code{30} +@item @tab @code{1} @tab @code{"foo"} +@item @tab @code{0} @tab @code{8} +@item @tab @code{2} @tab @code{""} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -14868,24 +15103,25 @@ Now the array is @dfn{sparse}, which just means some indices are missing. It has elements 0--3 and 10, but doesn't have elements 4, 5, 6, 7, 8, or 9. Another consequence of associative arrays is that the indices don't -have to be positive integers. Any number, or even a string, can be +have to be nonnegative integers. Any number, or even a string, can be an index. For example, the following is an array that translates words from English to French: @ifnotdocbook -@example -@r{Index} "dog" @r{Value} "chien" -@r{Index} "cat" @r{Value} "chat" -@r{Index} "one" @r{Value} "un" -@r{Index} 1 @r{Value} "un" -@end example +@multitable @columnfractions .1 .1 .1 +@headitem @tab Index @tab Value +@item @tab @code{"dog"} @tab @code{"chien"} +@item @tab @code{"cat"} @tab @code{"chat"} +@item @tab @code{"one"} @tab @code{"un"} +@item @tab @code{1} @tab @code{"un"} +@end multitable @end ifnotdocbook @docbook <informaltable> <tgroup cols="2"> -<colspec colname="1" align="center"/> -<colspec colname="2" align="center"/> +<colspec colname="1" align="left"/> +<colspec colname="2" align="left"/> <thead> <row> <entry>Index</entry> @@ -14927,7 +15163,7 @@ numbers and strings as indices. There are some subtleties to how numbers work when used as array subscripts; this is discussed in more detail in @ref{Numeric Array Subscripts}.) -Here, the number @code{1} isn't double-quoted, since @command{awk} +Here, the number @code{1} isn't double-quoted, because @command{awk} automatically converts it to a string. @cindex @command{gawk}, @code{IGNORECASE} variable in @@ -14944,8 +15180,6 @@ that array's indices are consecutive integers starting at one. @command{awk}'s arrays are efficient---the time to access an element is independent of the number of elements in the array. -@c ENDOFRANGE arrin -@c ENDOFRANGE inarr @node Reference to Elements @subsection Referring to an Array Element @@ -14954,7 +15188,7 @@ is independent of the number of elements in the array. @cindex elements of arrays The principal way to use an array is to refer to one of its elements. -An array reference is an expression as follows: +An @dfn{array reference} is an expression as follows: @example @var{array}[@var{index-expression}] @@ -14964,8 +15198,11 @@ An array reference is an expression as follows: Here, @var{array} is the name of an array. The expression @var{index-expression} is the index of the desired element of the array. +@c 1/2015: Having the 4.3 in @samp is a little iffy. It's essentially +@c an expression though, so leave be. It's to early in the discussion +@c to mention that it's really a string. The value of the array reference is the current value of that array -element. For example, @code{foo[4.3]} is an expression for the element +element. For example, @code{foo[4.3]} is an expression referencing the element of array @code{foo} at index @samp{4.3}. @cindex arrays, unassigned elements @@ -15012,7 +15249,7 @@ This expression tests whether the particular index @var{indx} exists, without the side effect of creating that element if it is not present. The expression has the value one (true) if @code{@var{array}[@var{indx}]} exists and zero (false) if it does not exist. -(We use @var{indx} here, since @samp{index} is the name of a built-in +(We use @var{indx} here, because @samp{index} is the name of a built-in function.) For example, this statement tests whether the array @code{frequencies} contains the index @samp{2}: @@ -15057,7 +15294,7 @@ assign to that element of the array. The following program takes a list of lines, each beginning with a line number, and prints them out in order of line number. The line numbers -are not in order when they are first read---instead they +are not in order when they are first read---instead, they are scrambled. This program sorts the lines by making an array using the line numbers as subscripts. The program then prints out the lines in sorted order of their numbers. It is a very simple program and gets @@ -15129,7 +15366,7 @@ END @{ In programs that use arrays, it is often necessary to use a loop that executes once for each element of an array. In other languages, where -arrays are contiguous and indices are limited to positive integers, +arrays are contiguous and indices are limited to nonnegative integers, this is easy: all the valid indices can be found by counting from the lowest index up to the highest. This technique won't do the job in @command{awk}, because any number or string can be an array index. @@ -15151,11 +15388,11 @@ program has previously used, with the variable @var{var} set to that index. The following program uses this form of the @code{for} statement. The first rule scans the input records and notes which words appear (at least once) in the input, by storing a one into the array @code{used} with -the word as index. The second rule scans the elements of @code{used} to +the word as the index. The second rule scans the elements of @code{used} to find all the distinct words that appear in the input. It prints each word that is more than 10 characters long and also prints the number of such words. -@xref{String Functions}, +@DBXREF{String Functions} for more information on the built-in function @code{length()}. @example @@ -15178,7 +15415,7 @@ END @{ @end example @noindent -@xref{Word Sorting}, +@DBXREF{Word Sorting} for a more detailed example of this type. @cindex arrays, elements, order of access by @code{in} operator @@ -15233,7 +15470,7 @@ $ @kbd{nawk -f loopcheck.awk} @end example @node Controlling Scanning -@subsection Using Predefined Array Scanning Orders With @command{gawk} +@subsection Using Predefined Array Scanning Orders with @command{gawk} This @value{SUBSECTION} describes a feature that is specific to @command{gawk}. @@ -15248,7 +15485,7 @@ and will vary from one version of @command{awk} to the next. Often, though, you may wish to do something simple, such as ``traverse the array by comparing the indices in ascending order,'' or ``traverse the array by comparing the values in descending order.'' -@command{gawk} provides two mechanisms which give you this control. +@command{gawk} provides two mechanisms that give you this control: @itemize @value{BULLET} @item @@ -15258,7 +15495,7 @@ We describe this now. @item Set @code{PROCINFO["sorted_in"]} to the name of a user-defined function to use for comparison of array elements. This advanced feature -is described later, in @ref{Array Sorting}. +is described later in @ref{Array Sorting}. @end itemize @cindex @code{PROCINFO}, values of @code{sorted_in} @@ -15276,7 +15513,7 @@ the index is @code{"10"} rather than numeric 10.) @item "@@ind_num_asc" Order by indices in ascending order but force them to be treated as numbers in the process. -Any index with a non-numeric value will end up positioned as if it were zero. +Any index with a non-numeric value will end up positioned as if it were zero. @item "@@val_type_asc" Order by element values in ascending order (rather than by indices). @@ -15288,11 +15525,11 @@ which in turn come before all subarrays. @pxref{Arrays of Arrays}.) @item "@@val_str_asc" -Order by element values in ascending order (rather than by indices). Scalar values are +Order by element values in ascending order (rather than by indices). Scalar values are compared as strings. Subarrays, if present, come out last. @item "@@val_num_asc" -Order by element values in ascending order (rather than by indices). Scalar values are +Order by element values in ascending order (rather than by indices). Scalar values are compared as numbers. Subarrays, if present, come out last. When numeric values are equal, the string values are used to provide an ordering: this guarantees consistent results across different @@ -15305,21 +15542,26 @@ across different environments.} which @command{gawk} uses internally to perform the sorting. @item "@@ind_str_desc" -String indices ordered from high to low. +Like @code{"@@ind_str_asc"}, but the +string indices are ordered from high to low. @item "@@ind_num_desc" -Numeric indices ordered from high to low. +Like @code{"@@ind_num_asc"}, but the +numeric indices are ordered from high to low. @item "@@val_type_desc" -Element values, based on type, ordered from high to low. +Like @code{"@@val_type_asc"}, but the +element values, based on type, are ordered from high to low. Subarrays, if present, come out first. @item "@@val_str_desc" -Element values, treated as strings, ordered from high to low. +Like @code{"@@val_str_asc"}, but the +element values, treated as strings, are ordered from high to low. Subarrays, if present, come out first. @item "@@val_num_desc" -Element values, treated as numbers, ordered from high to low. +Like @code{"@@val_num_asc"}, but the +element values, treated as numbers, are ordered from high to low. Subarrays, if present, come out first. @end table @@ -15353,11 +15595,11 @@ $ @kbd{gawk '} When sorting an array by element values, if a value happens to be a subarray then it is considered to be greater than any string or numeric value, regardless of what the subarray itself contains, -and all subarrays are treated as being equal to each other. Their +and all subarrays are treated as being equal to each other. Their order relative to each other is determined by their index strings. Here are some additional things to bear in mind about sorted -array traversal. +array traversal: @itemize @value{BULLET} @item @@ -15377,7 +15619,7 @@ if (save_sorted) @end example @item -As mentioned, the default array traversal order is represented by +As already mentioned, the default array traversal order is represented by @code{"@@unsorted"}. You can also get the default behavior by assigning the null string to @code{PROCINFO["sorted_in"]} or by just deleting the @code{"sorted_in"} element from the @code{PROCINFO} array with @@ -15422,7 +15664,7 @@ The program then changes the value of @code{CONVFMT}. The test @samp{(xyz in data)} generates a new string value from @code{xyz}---this time @code{"12.15"}---because the value of @code{CONVFMT} only allows two significant digits. This test fails, -since @code{"12.15"} is different from @code{"12.153"}. +because @code{"12.15"} is different from @code{"12.153"}. @cindex converting integer array subscripts @cindex integer array indices @@ -15440,19 +15682,19 @@ for (i = 1; i <= maxsub; i++) The ``integer values always convert to strings as integers'' rule has an additional consequence for array indexing. Octal and hexadecimal constants +@ifnotdocbook (@pxref{Nondecimal-numbers}) +@end ifnotdocbook +@ifdocbook +(covered in @ref{Nondecimal-numbers}) +@end ifdocbook are converted internally into numbers, and their original form -is forgotten. -This means, for example, that -@code{array[17]}, -@code{array[021]}, -and -@code{array[0x11]} -all refer to the same element! +is forgotten. This means, for example, that @code{array[17]}, +@code{array[021]}, and @code{array[0x11]} all refer to the same element! As with many things in @command{awk}, the majority of the time things work as you would expect them to. But it is useful to have a precise -knowledge of the actual rules since they can sometimes have a subtle +knowledge of the actual rules, as they can sometimes have a subtle effect on your programs. @node Uninitialized Subscripts @@ -15542,7 +15784,7 @@ for (i in frequencies) @noindent This example removes all the elements from the array @code{frequencies}. Once an element is deleted, a subsequent @code{for} statement to scan the array -does not report that element and the @code{in} operator to check for +does not report that element and using the @code{in} operator to check for the presence of that element returns zero (i.e., false): @example @@ -15595,7 +15837,7 @@ by a number of other implementations. @cindex Brian Kernighan's @command{awk} @quotation NOTE For many years, using @code{delete} without a subscript was a common -extension. In September, 2012, it was accepted for inclusion into the +extension. In September 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=544, the Austin Group website}. @end quotation @@ -15637,7 +15879,7 @@ a = 3 @cindex subscripts in arrays, multidimensional @cindex arrays, multidimensional -A multidimensional array is an array in which an element is identified +A @dfn{multidimensional array} is an array in which an element is identified by a sequence of indices instead of a single index. For example, a two-dimensional array requires two indices. The usual way (in many languages, including @command{awk}) to refer to an element of a @@ -15679,7 +15921,7 @@ stored as @samp{foo["a@@b@@c"]}. @cindex @code{in} operator, index existence in multidimensional arrays To test whether a particular index sequence exists in a multidimensional array, use the same operator (@code{in}) that is -used for single dimensional arrays. Write the whole sequence of indices +used for single-dimensional arrays. Write the whole sequence of indices in parentheses, separated by commas, as the left operand: @example @@ -15802,8 +16044,8 @@ a[1][2] = 2 This simulates a true two-dimensional array. Each subarray element can contain another subarray as a value, which in turn can hold other arrays as well. In this way, you can create arrays of three or more dimensions. -The indices can be any @command{awk} expression, including scalars -separated by commas (that is, a regular @command{awk} simulated +The indices can be any @command{awk} expressions, including scalars +separated by commas (i.e., a regular @command{awk} simulated multidimensional subscript). So the following is valid in @command{gawk}: @@ -15814,15 +16056,15 @@ a[1][3][1, "name"] = "barney" Each subarray and the main array can be of different length. In fact, the elements of an array or its subarray do not all have to have the same type. This means that the main array and any of its subarrays can be -non-rectangular, or jagged in structure. You can assign a scalar value to +nonrectangular, or jagged in structure. You can assign a scalar value to the index @code{4} of the main array @code{a}, even though @code{a[1]} is itself an array and not a scalar: @example a[4] = "An element in a jagged array" @end example - -The terms @dfn{dimension}, @dfn{row} and @dfn{column} are + +The terms @dfn{dimension}, @dfn{row}, and @dfn{column} are meaningless when applied to such an array, but we will use ``dimension'' henceforth to imply the maximum number of indices needed to refer to an existing element. The @@ -15838,7 +16080,8 @@ a[4][5][6][7] = "An element in a four-dimensional array" @noindent This removes the scalar value from index @code{4} and then inserts a -subarray of subarray of subarray containing a scalar. You can also +three-level nested subarray +containing a scalar. You can also delete an entire subarray or subarray of subarrays: @example @@ -15849,7 +16092,7 @@ a[4][5] = "An element in subarray a[4]" But recall that you can not delete the main array @code{a} and then use it as a scalar. -The built-in functions which take array arguments can also be used +The built-in functions that take array arguments can also be used with subarrays. For example, the following code fragment uses @code{length()} (@pxref{String Functions}) to determine the number of elements in the main array @code{a} and @@ -15878,14 +16121,14 @@ The @samp{for (item in array)} statement (@pxref{Scanning an Array}) can be nested to scan all the elements of an array of arrays if it is rectangular in structure. In order to print the contents (scalar values) of a two-dimensional array of arrays -(i.e., in which each first-level element is itself an -array, not necessarily of the same length) +(i.e., in which each first-level element is itself an +array, not necessarily of the same length), you could use the following code: @example for (i in array) for (j in array[i]) - print array[i][j] + print array[i][j] @end example The @code{isarray()} function (@pxref{Type Functions}) @@ -15895,7 +16138,7 @@ lets you test if an array element is itself an array: for (i in array) @{ if (isarray(array[i]) @{ for (j in array[i]) @{ - print array[i][j] + print array[i][j] @} @} else @@ -15905,7 +16148,7 @@ for (i in array) @{ If the structure of a jagged array of arrays is known in advance, you can often devise workarounds using control statements. For example, -the following code prints the elements of our main array @code{a}: +the following code prints the elements of our main array @code{a}: @example for (i in a) @{ @@ -15915,13 +16158,13 @@ for (i in a) @{ print a[i][j][k] @} else print a[i][j] - @} + @} @} @end example @noindent -@xref{Walking Arrays}, for a user-defined function that ``walks'' an -arbitrarily-dimensioned array of arrays. +@DBXREF{Walking Arrays} for a user-defined function that ``walks'' an +arbitrarily dimensioned array of arrays. Recall that a reference to an uninitialized array element yields a value of @code{""}, the null string. This has one important implication when you @@ -15971,16 +16214,17 @@ special predefined values to @code{PROCINFO["sorted_in"]}. @item Use @samp{delete @var{array}[@var{indx}]} to delete an individual element. -You may also use @samp{delete @var{array}} to delete all of the elements -in the array. This latter feature has been a common extension for many +To delete all of the elements in an array, +use @samp{delete @var{array}}. +This latter feature has been a common extension for many years and is now standard, but may not be supported by all commercial versions of @command{awk}. @item Standard @command{awk} simulates multidimensional arrays by separating -subscript values with a comma. The values are concatenated into a +subscript values with commas. The values are concatenated into a single string, separated by the value of @code{SUBSEP}. The fact -that such a subscript was created in this way is not retained; thus +that such a subscript was created in this way is not retained; thus, changing @code{SUBSEP} may have unexpected consequences. You can use @samp{(@var{sub1}, @var{sub2}, @dots{}) in @var{array}} to see if such a multidimensional subscript exists in @var{array}. @@ -15989,7 +16233,7 @@ a multidimensional subscript exists in @var{array}. @command{gawk} provides true arrays of arrays. You use a separate set of square brackets for each dimension in such an array: @code{data[row][col]}, for example. Array elements may thus be either -scalar values (number or string) or another array. +scalar values (number or string) or other arrays. @item Use the @code{isarray()} built-in function to determine if an array @@ -15997,25 +16241,26 @@ element is itself a subarray. @end itemize -@c ENDOFRANGE arrs @node Functions @chapter Functions -@c STARTOFRANGE funcbi @cindex functions, built-in -@c STARTOFRANGE bifunc @cindex built-in functions This @value{CHAPTER} describes @command{awk}'s built-in functions, which fall into three categories: numeric, string, and I/O. @command{gawk} provides additional groups of functions to work with values that represent time, do -bit manipulation, sort arrays, and internationalize and localize programs. +bit manipulation, sort arrays, +provide type information, and internationalize and localize programs. Besides the built-in functions, @command{awk} has provisions for writing new functions that the rest of a program can use. The second half of this @value{CHAPTER} describes these @dfn{user-defined} functions. +Finally, we explore indirect function calls, a @command{gawk}-specific +extension that lets you determine at runtime what function is to +be called. @menu * Built-in:: Summarizes the built-in functions. @@ -16058,7 +16303,7 @@ is a call to the function @code{atan2()} and has two arguments. @cindex programming conventions, functions, calling @cindex whitespace, functions@comma{} calling Whitespace is ignored between the built-in function name and the -open parenthesis, but nonetheless it is good practice to avoid using whitespace +opening parenthesis, but nonetheless it is good practice to avoid using whitespace there. User-defined functions do not permit whitespace in this way, and it is easier to avoid mistakes by following a simple convention that always works---no whitespace after a function name. @@ -16095,11 +16340,11 @@ right to left. For example: @example i = 5 -j = atan2(i++, i *= 2) +j = atan2(++i, i *= 2) @end example If the order of evaluation is left to right, then @code{i} first becomes -6, and then 12, and @code{atan2()} is called with the two arguments 6 +six, and then 12, and @code{atan2()} is called with the two arguments six and 12. But if the order of evaluation is right to left, @code{i} first becomes 10, then 11, and @code{atan2()} is called with the two arguments 11 and 10. @@ -16126,23 +16371,6 @@ You can use @samp{pi = atan2(0, -1)} to retrieve the value of @cindex cosine Return the cosine of @var{x}, with @var{x} in radians. -@item @code{div(@var{numerator}, @var{denominator}, @var{result})} -@cindexawkfunc{div} -@cindex div -Perform integer division, similar to the standard C function of the -same name. First, truncate @code{numerator} and @code{denominator} -towards zero, creating integer values. Clear the @code{result} -array, and then set @code{result["quotient"]} to the result of -@samp{numerator / denominator}, truncated towards zero to an integer, -and set @code{result["remainder"]} to the result of @samp{numerator % -denominator}, truncated towards zero to an integer. This function is -primarily intended for use with arbitrary length integers; it avoids -creating MPFR arbitrary precision floating-point values (@pxref{Arbitrary -Precision Integers}). - -This function is a @code{gawk} extension. It is not available in -compatibility mode (@pxref{Options}). - @item @code{exp(@var{x})} @cindexawkfunc{exp} @cindex exponent @@ -16155,10 +16383,26 @@ depends on your machine's floating-point representation. @cindex round to nearest integer Return the nearest integer to @var{x}, located between @var{x} and zero and truncated toward zero. - For example, @code{int(3)} is 3, @code{int(3.9)} is 3, @code{int(-3.9)} is @minus{}3, and @code{int(-3)} is @minus{}3 as well. +@item @code{intdiv(@var{numerator}, @var{denominator}, @var{result})} +@cindexawkfunc{intdiv} +@cindex intdiv +Perform integer division, similar to the standard C function of the +same name. First, truncate @code{numerator} and @code{denominator} +towards zero, creating integer values. Clear the @code{result} +array, and then set @code{result["quotient"]} to the result of +@samp{numerator / denominator}, truncated towards zero to an integer, +and set @code{result["remainder"]} to the result of @samp{numerator % +denominator}, truncated towards zero to an integer. This function is +primarily intended for use with arbitrary length integers; it avoids +creating MPFR arbitrary precision floating-point values (@pxref{Arbitrary +Precision Integers}). + +This function is a @code{gawk} extension. It is not available in +compatibility mode (@pxref{Options}). + @item @code{log(@var{x})} @cindexawkfunc{log} @cindex logarithm @@ -16181,7 +16425,7 @@ In fact, @command{gawk} uses the BSD @code{random()} function, which is considerably better than @code{rand()}, to produce random numbers.} Often random integers are needed instead. Following is a user-defined function -that can be used to obtain a random non-negative integer less than @var{n}: +that can be used to obtain a random nonnegative integer less than @var{n}: @example function randint(n) @@ -16191,8 +16435,8 @@ function randint(n) @end example @noindent -The multiplication produces a random number greater than zero and less -than @code{n}. Using @code{int()}, this result is made into +The multiplication produces a random number greater than or equal to +zero and less than @code{n}. Using @code{int()}, this result is made into an integer between zero and @code{n} @minus{} 1, inclusive. The following example uses a similar function to produce random integers @@ -16244,8 +16488,8 @@ for generating random numbers to the value @var{x}. Each seed value leads to a particular sequence of random numbers.@footnote{Computer-generated random numbers really are not truly -random. They are technically known as ``pseudorandom.'' This means -that while the numbers in a sequence appear to be random, you can in +random. They are technically known as @dfn{pseudorandom}. This means +that although the numbers in a sequence appear to be random, you can in fact generate the same sequence of random numbers over and over again.} Thus, if the seed is set to the same value a second time, the same sequence of random numbers is produced again. @@ -16276,7 +16520,7 @@ implementations. The functions in this @value{SECTION} look at or change the text of one or more strings. -@code{gawk} understands locales (@pxref{Locales}), and does all +@command{gawk} understands locales (@pxref{Locales}) and does all string processing in terms of @emph{characters}, not @emph{bytes}. This distinction is particularly important to understand for locales where one character may be represented by multiple bytes. Thus, for @@ -16295,7 +16539,7 @@ doing index calculations, particularly if you are used to C. In the following list, optional parameters are enclosed in square brackets@w{ ([ ]).} Several functions perform string substitution; the full discussion is provided in the description of the @code{sub()} function, which comes -towards the end since the list is presented alphabetically. +toward the end, because the list is presented alphabetically. Those functions that are specific to @command{gawk} are marked with a pound sign (@samp{#}). They are not available in compatibility mode @@ -16321,10 +16565,10 @@ These two functions are similar in behavior, so they are described together. @quotation NOTE -The following description ignores the third argument, @var{how}, since it +The following description ignores the third argument, @var{how}, as it requires understanding features that we have not discussed yet. Thus, the discussion here is a deliberate simplification. (We do provide all -the details later on: @xref{Array Sorting Functions}, for the full story.) +the details later on; see @DBREF{Array Sorting Functions} for the full story.) @end quotation Both functions return the number of elements in the array @var{source}. @@ -16365,7 +16609,7 @@ a[2] = "de" a[3] = "sac" @end example -The @code{asorti()} function works similarly to @code{asort()}, however, +The @code{asorti()} function works similarly to @code{asort()}; however, the @emph{indices} are sorted, instead of the values. Thus, in the previous example, starting with the same initial set of indices and values in @code{a}, calling @samp{asorti(a)} would yield: @@ -16476,10 +16720,11 @@ $ @kbd{awk 'BEGIN @{ print index("peanut", "an") @}'} @noindent If @var{find} is not found, @code{index()} returns zero. +@cindex dark corner, regexp as second argument to @code{index()} With BWK @command{awk} and @command{gawk}, it is a fatal error to use a regexp constant for @var{find}. Other implementations allow it, simply treating the regexp -constant as an expression meaning @samp{$0 ~ /regexp/}. @value{DARKCORNER}. +constant as an expression meaning @samp{$0 ~ /regexp/}. @value{DARKCORNER} @item @code{length(}[@var{string}]@code{)} @cindexawkfunc{length} @@ -16562,7 +16807,7 @@ If @option{--posix} is supplied, using an array argument is a fatal error @cindex string, regular expression match @cindex match regexp in string Search @var{string} for the -longest, leftmost substring matched by the regular expression, +longest, leftmost substring matched by the regular expression @var{regexp} and return the character position (index) at which that substring begins (one, if it starts at the beginning of @var{string}). If no match is found, return zero. @@ -16570,11 +16815,11 @@ at which that substring begins (one, if it starts at the beginning of The @var{regexp} argument may be either a regexp constant (@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}). In the latter case, the string is treated as a regexp to be matched. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between the two forms, and the implications for writing your program correctly. -The order of the first two arguments is backwards from most other string +The order of the first two arguments is the opposite of most other string functions that work with regular expressions, such as @code{sub()} and @code{gsub()}. It might help to remember that for @code{match()}, the order is the same as for the @samp{~} operator: @@ -16663,7 +16908,7 @@ $ @kbd{echo foooobazbarrrrr |} @end example There may not be subscripts for the start and index for every parenthesized -subexpression, since they may not all have matched text; thus they +subexpression, because they may not all have matched text; thus, they should be tested for with the @code{in} operator (@pxref{Reference to Elements}). @@ -16710,15 +16955,15 @@ a regexp describing where to split @var{string} (much as @code{FS} can be a regexp describing where to split input records). If @var{fieldsep} is omitted, the value of @code{FS} is used. @code{split()} returns the number of elements created. -@var{seps} is a @command{gawk} extension with @code{@var{seps}[@var{i}]} +@var{seps} is a @command{gawk} extension, with @code{@var{seps}[@var{i}]} being the separator string -between @code{@var{array}[@var{i}]} and @code{@var{array}[@var{i}+1]}. +between @code{@var{array}[@var{i}]} and @code{@var{array}[@var{i}+1]}. If @var{fieldsep} is a single -space then any leading whitespace goes into @code{@var{seps}[0]} and +space, then any leading whitespace goes into @code{@var{seps}[0]} and any trailing -whitespace goes into @code{@var{seps}[@var{n}]} where @var{n} is the -return value of -@code{split()} (that is, the number of elements in @var{array}). +whitespace goes into @code{@var{seps}[@var{n}]}, where @var{n} is the +return value of +@code{split()} (i.e., the number of elements in @var{array}). The @code{split()} function splits strings into pieces in a manner similar to the way input lines are split into fields. For example: @@ -16729,7 +16974,7 @@ split("cul-de-sac", a, "-", seps) @noindent @cindex strings splitting, example -splits the string @samp{cul-de-sac} into three fields using @samp{-} as the +splits the string @code{"cul-de-sac"} into three fields using @samp{-} as the separator. It sets the contents of the array @code{a} as follows: @example @@ -16754,21 +16999,20 @@ As with input field-splitting, when the value of @var{fieldsep} is the elements of @var{array} but not in @var{seps}, and the elements are separated by runs of whitespace. -Also as with input field-splitting, if @var{fieldsep} is the null string, each +Also, as with input field splitting, if @var{fieldsep} is the null string, each individual character in the string is split into its own array element. @value{COMMONEXT} Note, however, that @code{RS} has no effect on the way @code{split()} -works. Even though @samp{RS = ""} causes newline to also be an input +works. Even though @samp{RS = ""} causes the newline character to also be an input field separator, this does not affect how @code{split()} splits strings. @cindex dark corner, @code{split()} function Modern implementations of @command{awk}, including @command{gawk}, allow -the third argument to be a regexp constant (@code{/abc/}) as well as a -string. -@value{DARKCORNER} +the third argument to be a regexp constant (@w{@code{/}@dots{}@code{/}}) +as well as a string. @value{DARKCORNER} The POSIX standard allows this as well. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between using a string constant or a regexp constant, and the implications for writing your program correctly. @@ -16819,7 +17063,7 @@ Using the @code{strtonum()} function is @emph{not} the same as adding zero to a string value; the automatic coercion of strings to numbers works only for decimal data, not for octal or hexadecimal.@footnote{Unless you use the @option{--non-decimal-data} option, which isn't recommended. -@xref{Nondecimal Data}, for more information.} +@DBXREF{Nondecimal Data} for more information.} Note also that @code{strtonum()} uses the current locale's decimal point for recognizing numbers (@pxref{Locales}). @@ -16837,7 +17081,7 @@ Return the number of substitutions made (zero or one). The @var{regexp} argument may be either a regexp constant (@code{/}@dots{}@code{/}) or a string constant (@code{"}@dots{}@code{"}). In the latter case, the string is treated as a regexp to be matched. -@xref{Computed Regexps}, for a +@DBXREF{Computed Regexps} for a discussion of the difference between the two forms, and the implications for writing your program correctly. @@ -16903,7 +17147,7 @@ an @samp{&}: @cindex @code{sub()} function, arguments of @cindex @code{gsub()} function, arguments of As mentioned, the third argument to @code{sub()} must -be a variable, field or array element. +be a variable, field, or array element. Some versions of @command{awk} allow the third argument to be an expression that is not an lvalue. In such a case, @code{sub()} still searches for the pattern and returns zero or one, but the result of @@ -17023,7 +17267,7 @@ Although this makes a certain amount of sense, it can be surprising. @node Gory Details -@subsubsection More About @samp{\} and @samp{&} with @code{sub()}, @code{gsub()}, and @code{gensub()} +@subsubsection More about @samp{\} and @samp{&} with @code{sub()}, @code{gsub()}, and @code{gensub()} @cindex escape processing, @code{gsub()}/@code{gensub()}/@code{sub()} functions @cindex @code{sub()} function, escape processing @@ -17062,15 +17306,15 @@ example, @code{"a\qb"} is treated as @code{"aqb"}. At the runtime level, the various functions handle sequences of @samp{\} and @samp{&} differently. The situation is (sadly) somewhat complex. -Historically, the @code{sub()} and @code{gsub()} functions treated the two -character sequence @samp{\&} specially; this sequence was replaced in +Historically, the @code{sub()} and @code{gsub()} functions treated the +two-character sequence @samp{\&} specially; this sequence was replaced in the generated text with a single @samp{&}. Any other @samp{\} within the @var{replacement} string that did not precede an @samp{&} was passed through unchanged. This is illustrated in @ref{table-sub-escapes}. @c Thank to Karl Berry for help with the TeX stuff. @float Table,table-sub-escapes -@caption{Historical Escape Sequence Processing for @code{sub()} and @code{gsub()}} +@caption{Historical escape sequence processing for @code{sub()} and @code{gsub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17121,7 +17365,7 @@ _bigskip} @end float @noindent -This table shows both the lexical-level processing, where +This table shows the lexical-level processing, where an odd number of backslashes becomes an even number at the runtime level, as well as the runtime processing done by @code{sub()}. (For the sake of simplicity, the rest of the following tables only show the @@ -17142,7 +17386,7 @@ This is shown in @ref{table-sub-proposed}. @float Table,table-sub-proposed -@caption{GNU @command{awk} Rules For @code{sub()} And Backslash} +@caption{@command{gawk} rules for @code{sub()} and backslash} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17187,7 +17431,7 @@ _bigskip} @end float In a nutshell, at the runtime level, there are now three special sequences -of characters (@samp{\\\&}, @samp{\\&} and @samp{\&}) whereas historically +of characters (@samp{\\\&}, @samp{\\&}, and @samp{\&}) whereas historically there was only one. However, as in the historical case, any @samp{\} that is not part of one of these three sequences is not special and appears in the output literally. @@ -17205,7 +17449,7 @@ by anything else is not special; the @samp{\} is placed straight into the output These rules are presented in @ref{table-posix-sub}. @float Table,table-posix-sub -@caption{POSIX Rules For @code{sub()} And @code{gsub()}} +@caption{POSIX rules for @code{sub()} and @code{gsub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17253,13 +17497,13 @@ The only case where the difference is noticeable is the last one: @samp{\\\\} is seen as @samp{\\} and produces @samp{\} instead of @samp{\\}. Starting with @value{PVERSION} 3.1.4, @command{gawk} followed the POSIX rules -when @option{--posix} is specified (@pxref{Options}). Otherwise, -it continued to follow the proposed rules, since +when @option{--posix} was specified (@pxref{Options}). Otherwise, +it continued to follow the proposed rules, as that had been its behavior for many years. When @value{PVERSION} 4.0.0 was released, the @command{gawk} maintainer made the POSIX rules the default, breaking well over a decade's worth -of backwards compatibility.@footnote{This was rather naive of him, despite +of backward compatibility.@footnote{This was rather naive of him, despite there being a note in this section indicating that the next major version would move to the POSIX rules.} Needless to say, this was a bad idea, and as of @value{PVERSION} 4.0.1, @command{gawk} resumed its historical @@ -17274,7 +17518,7 @@ appears in the generated text and the @samp{\} does not, as shown in @ref{table-gensub-escapes}. @float Table,table-gensub-escapes -@caption{Escape Sequence Processing For @code{gensub()}} +@caption{Escape sequence processing for @code{gensub()}} @tex \vbox{\bigskip % We need more characters for escape and tab ... @@ -17321,7 +17565,7 @@ _bigskip} @end ifnottex @end float -Because of the complexity of the lexical and runtime level processing +Because of the complexity of the lexical- and runtime-level processing and the special cases for @code{sub()} and @code{gsub()}, we recommend the use of @command{gawk} and @code{gensub()} when you have to do substitutions. @@ -17341,12 +17585,13 @@ Optional parameters are enclosed in square brackets ([ ]): Close the file @var{filename} for input or output. Alternatively, the argument may be a shell command that was used for creating a coprocess, or for redirecting to or from a pipe; then the coprocess or pipe is closed. -@xref{Close Files And Pipes}, +@DBXREF{Close Files And Pipes} for more information. When closing a coprocess, it is occasionally useful to first close one end of the two-way pipe and then to close the other. This is done by providing a second argument to @code{close()}. This second argument +(@var{how}) should be one of the two string values @code{"to"} or @code{"from"}, indicating which end of the pipe to close. Case in the string does not matter. @@ -17365,25 +17610,25 @@ a pipe or coprocess. @cindex buffers, flushing @cindex output, buffering -Many utility programs @dfn{buffer} their output; i.e., they save information +Many utility programs @dfn{buffer} their output (i.e., they save information to write to a disk file or the screen in memory until there is enough -for it to be worthwhile to send the data to the output device. +for it to be worthwhile to send the data to the output device). This is often more efficient than writing every little bit of information as soon as it is ready. However, sometimes -it is necessary to force a program to @dfn{flush} its buffers; that is, -write the information to its destination, even if a buffer is not full. +it is necessary to force a program to @dfn{flush} its buffers (i.e., +write the information to its destination, even if a buffer is not full). This is the purpose of the @code{fflush()} function---@command{gawk} also -buffers its output and the @code{fflush()} function forces +buffers its output, and the @code{fflush()} function forces @command{gawk} to flush its buffers. @cindex extensions, common@comma{} @code{fflush()} function @cindex Brian Kernighan's @command{awk} Brian Kernighan added @code{fflush()} to his @command{awk} in April -of 1992. For two decades, it was a common extension. In December, +1992. For two decades, it was a common extension. In December 2012, it was accepted for inclusion into the POSIX standard. See @uref{http://austingroupbugs.net/view.php?id=634, the Austin Group website}. -POSIX standardizes @code{fflush()} as follows: If there +POSIX standardizes @code{fflush()} as follows: if there is no argument, or if the argument is the null string (@w{@code{""}}), then @command{awk} flushes the buffers for @emph{all} open output files and pipes. @@ -17394,7 +17639,7 @@ would flush only the standard output if there was no argument, and flush all output files and pipes if the argument was the null string. This was changed in order to be compatible with Brian Kernighan's @command{awk}, in the hope that standardizing this -feature in POSIX would then be easier (which indeed helped). +feature in POSIX would then be easier (which indeed proved to be the case). With @command{gawk}, you can use @samp{fflush("/dev/stdout")} if you wish to flush @@ -17405,7 +17650,7 @@ only the standard output. @c @cindex warnings, automatic @cindex troubleshooting, @code{fflush()} function @code{fflush()} returns zero if the buffer is successfully flushed; -otherwise, it returns non-zero. (@command{gawk} returns @minus{}1.) +otherwise, it returns a nonzero value. (@command{gawk} returns @minus{}1.) In the case where all buffers are flushed, the return value is zero only if all buffers were flushed successfully. Otherwise, it is @minus{}1, and @command{gawk} warns about the problem @var{filename}. @@ -17415,11 +17660,54 @@ a file or pipe that was opened for reading (such as with @code{getline}), or if @var{filename} is not an open file, pipe, or coprocess. In such a case, @code{fflush()} returns @minus{}1, as well. +@sidebar Interactive Versus Noninteractive Buffering +@cindex buffering, interactive vs.@: noninteractive + +As a side point, buffering issues can be even more confusing if +your program is @dfn{interactive} (i.e., communicating +with a user sitting at a keyboard).@footnote{A program is interactive +if the standard output is connected to a terminal device. On modern +systems, this means your keyboard and screen.} + +@c Thanks to Walter.Mecky@dresdnerbank.de for this example, and for +@c motivating me to write this section. +Interactive programs generally @dfn{line buffer} their output (i.e., they +write out every line). Noninteractive programs wait until they have +a full buffer, which may be many lines of output. +Here is an example of the difference: + +@example +$ @kbd{awk '@{ print $1 + $2 @}'} +@kbd{1 1} +@print{} 2 +@kbd{2 3} +@print{} 5 +@kbd{Ctrl-d} +@end example + +@noindent +Each line of output is printed immediately. Compare that behavior +with this example: + +@example +$ @kbd{awk '@{ print $1 + $2 @}' | cat} +@kbd{1 1} +@kbd{2 3} +@kbd{Ctrl-d} +@print{} 2 +@print{} 5 +@end example + +@noindent +Here, no output is printed until after the @kbd{Ctrl-d} is typed, because +it is all buffered and sent down the pipe to @command{cat} in one shot. +@end sidebar + @item @code{system(@var{command})} @cindexawkfunc{system} @cindex invoke shell command @cindex interacting with other programs -Execute the operating-system +Execute the operating system command @var{command} and then return to the @command{awk} program. Return @var{command}'s exit status. @@ -17462,49 +17750,6 @@ When @option{--sandbox} is specified, the @code{system()} function is disabled @end table -@sidebar Interactive Versus Noninteractive Buffering -@cindex buffering, interactive vs.@: noninteractive - -As a side point, buffering issues can be even more confusing, depending -upon whether your program is @dfn{interactive}, i.e., communicating -with a user sitting at a keyboard.@footnote{A program is interactive -if the standard output is connected to a terminal device. On modern -systems, this means your keyboard and screen.} - -@c Thanks to Walter.Mecky@dresdnerbank.de for this example, and for -@c motivating me to write this section. -Interactive programs generally @dfn{line buffer} their output; i.e., they -write out every line. Noninteractive programs wait until they have -a full buffer, which may be many lines of output. -Here is an example of the difference: - -@example -$ @kbd{awk '@{ print $1 + $2 @}'} -@kbd{1 1} -@print{} 2 -@kbd{2 3} -@print{} 5 -@kbd{Ctrl-d} -@end example - -@noindent -Each line of output is printed immediately. Compare that behavior -with this example: - -@example -$ @kbd{awk '@{ print $1 + $2 @}' | cat} -@kbd{1 1} -@kbd{2 3} -@kbd{Ctrl-d} -@print{} 2 -@print{} 5 -@end example - -@noindent -Here, no output is printed until after the @kbd{Ctrl-d} is typed, because -it is all buffered and sent down the pipe to @command{cat} in one shot. -@end sidebar - @sidebar Controlling Output Buffering with @code{system()} @cindex buffers, flushing @cindex buffering, input/output @@ -17523,7 +17768,7 @@ system("") # flush output @command{gawk} treats this use of the @code{system()} function as a special case and is smart enough not to run a shell (or other command interpreter) with the empty command. Therefore, with @command{gawk}, this -idiom is not only useful, it is also efficient. While this method should work +idiom is not only useful, it is also efficient. Although this method should work with other @command{awk} implementations, it does not necessarily avoid starting an unnecessary shell. (Other implementations may only flush the buffer associated with the standard output and not necessarily @@ -17566,18 +17811,14 @@ you would see the latter (undesirable) output. @subsection Time Functions @cindex time functions -@c STARTOFRANGE tst @cindex timestamps -@c STARTOFRANGE logftst @cindex log files, timestamps in -@c STARTOFRANGE filogtst @cindex files, log@comma{} timestamps in -@c STARTOFRANGE gawtst @cindex @command{gawk}, timestamps @cindex POSIX @command{awk}, timestamps and -@code{awk} programs are commonly used to process log files +@command{awk} programs are commonly used to process log files containing timestamp information, indicating when a -particular log record was written. Many programs log their timestamp +particular log record was written. Many programs log their timestamps in the form returned by the @code{time()} system call, which is the number of seconds since a particular epoch. On POSIX-compliant systems, it is the number of seconds since @@ -17604,6 +17845,7 @@ which is sufficient to represent times through 2038-01-19 03:14:07 UTC. Many systems support a wider range of timestamps, including negative timestamps that represent times before the epoch. +@c FIXME: Use @sup here for superscript @cindex @command{date} utility, GNU @cindex time, retrieving @@ -17638,7 +17880,7 @@ The values of these numbers need not be within the ranges specified; for example, an hour of @minus{}1 means 1 hour before midnight. The origin-zero Gregorian calendar is assumed, with year 0 preceding year 1 and year @minus{}1 preceding year 0. -The time is assumed to be in the local timezone. +The time is assumed to be in the local time zone. If the daylight-savings flag is positive, the time is assumed to be daylight savings time; if zero, the time is assumed to be standard time; and if negative (the default), @code{mktime()} attempts to determine @@ -17650,7 +17892,6 @@ is out of range, @code{mktime()} returns @minus{}1. @cindex @command{gawk}, @code{PROCINFO} array in @cindex @code{PROCINFO} array @item @code{strftime(}[@var{format} [@code{,} @var{timestamp} [@code{,} @var{utc-flag}] ] ]@code{)} -@c STARTOFRANGE strf @cindexgawkfunc{strftime} @cindex format time string Format the time specified by @var{timestamp} @@ -17662,14 +17903,14 @@ Mean Time). Otherwise, the value is formatted for the local time zone. The @var{timestamp} is in the same format as the value returned by the @code{systime()} function. If no @var{timestamp} argument is supplied, @command{gawk} uses the current time of day as the timestamp. -If no @var{format} argument is supplied, @code{strftime()} uses +Without a @var{format} argument, @code{strftime()} uses the value of @code{PROCINFO["strftime"]} as the format string (@pxref{Built-in Variables}). The default string value is @code{@w{"%a %b %e %H:%M:%S %Z %Y"}}. This format string produces output that is equivalent to that of the @command{date} utility. You can assign a new value to @code{PROCINFO["strftime"]} to -change the default format; see below for the various format directives. +change the default format; see the following list for the various format directives. @item @code{systime()} @cindexgawkfunc{systime} @@ -17746,16 +17987,16 @@ This is the ISO 8601 date format. @item %g The year modulo 100 of the ISO 8601 week number, as a decimal number (00--99). -For example, January 1, 2012 is in week 53 of 2011. Thus, the year +For example, January 1, 2012, is in week 53 of 2011. Thus, the year of its ISO 8601 week number is 2011, even though its year is 2012. -Similarly, December 31, 2012 is in week 1 of 2013. Thus, the year +Similarly, December 31, 2012, is in week 1 of 2013. Thus, the year of its ISO week number is 2013, even though its year is 2012. @item %G The full year of the ISO week number, as a decimal number. @item %h -Equivalent to @code{%b}. +Equivalent to @samp{%b}. @item %H The hour (24-hour clock) as a decimal number (00--23). @@ -17799,12 +18040,12 @@ Equivalent to specifying @samp{%H:%M:%S}. The weekday as a decimal number (1--7). Monday is day one. @item %U -The week number of the year (the first Sunday as the first day of week one) +The week number of the year (with the first Sunday as the first day of week one) as a decimal number (00--53). @c @cindex ISO 8601 @item %V -The week number of the year (the first Monday as the first +The week number of the year (with the first Monday as the first day of week one) as a decimal number (01--53). The method for determining the week number is as specified by ISO 8601. (To wit: if the week containing January 1 has four or more days in the @@ -17815,7 +18056,7 @@ and the next week is week one.) The weekday as a decimal number (0--6). Sunday is day zero. @item %W -The week number of the year (the first Monday as the first day of week one) +The week number of the year (with the first Monday as the first day of week one) as a decimal number (00--53). @item %x @@ -17824,7 +18065,7 @@ The locale's ``appropriate'' date representation. @item %X The locale's ``appropriate'' time representation. -(This is @code{%T} in the @code{"C"} locale.) +(This is @samp{%T} in the @code{"C"} locale.) @item %y The year modulo 100 as a decimal number (00--99). @@ -17835,8 +18076,8 @@ The full year as a decimal number (e.g., 2015). @c @cindex RFC 822 @c @cindex RFC 1036 @item %z -The timezone offset in a +HHMM format (e.g., the format necessary to -produce RFC 822/RFC 1036 date headers). +The time zone offset in a @samp{+@var{HHMM}} format (e.g., the format +necessary to produce RFC 822/RFC 1036 date headers). @item %Z The time zone name or abbreviation; no characters if @@ -17844,8 +18085,8 @@ no time zone is determinable. @item %Ec %EC %Ex %EX %Ey %EY %Od %Oe %OH @itemx %OI %Om %OM %OS %Ou %OU %OV %Ow %OW %Oy -``Alternate representations'' for the specifications -that use only the second letter (@code{%c}, @code{%C}, +``Alternative representations'' for the specifications +that use only the second letter (@samp{%c}, @samp{%C}, and so on).@footnote{If you don't understand any of this, don't worry about it; these facilities are meant to make it easier to ``internationalize'' programs. @@ -17857,7 +18098,7 @@ Other internationalization features are described in A literal @samp{%}. @end table -If a conversion specifier is not one of the above, the behavior is +If a conversion specifier is not one of those just listed, the behavior is undefined.@footnote{This is because ISO C leaves the behavior of the C version of @code{strftime()} undefined and @command{gawk} uses the system's version of @code{strftime()} if it's there. @@ -17884,11 +18125,11 @@ Single-digit numbers are padded with a space. @ignore @item %N The ``Emperor/Era'' name. -Equivalent to @code{%C}. +Equivalent to @samp{%C}. @item %o The ``Emperor/Era'' year. -Equivalent to @code{%y}. +Equivalent to @samp{%y}. @end ignore @item %s @@ -17899,9 +18140,8 @@ The time as a decimal timestamp in seconds since the epoch. The date in VMS format (e.g., @samp{20-JUN-1991}). @end ignore @end table -@c ENDOFRANGE strf -Additionally, the alternate representations are recognized but their +Additionally, the alternative representations are recognized but their normal representations are used. @cindex @code{date} utility, POSIX @@ -17915,7 +18155,7 @@ interprets the current time according to the format specifiers in the string. For example: @example -$ date '+Today is %A, %B %d, %Y.' +$ @kbd{date '+Today is %A, %B %d, %Y.'} @print{} Today is Monday, September 22, 2014. @end example @@ -17950,23 +18190,14 @@ gawk 'BEGIN @{ exit exitval @}' "$@@" @end example -@c ENDOFRANGE tst -@c ENDOFRANGE logftst -@c ENDOFRANGE filogtst -@c ENDOFRANGE gawtst @node Bitwise Functions @subsection Bit-Manipulation Functions @cindex bit-manipulation functions -@c STARTOFRANGE bit @cindex bitwise, operations -@c STARTOFRANGE and @cindex AND bitwise operation -@c STARTOFRANGE oro @cindex OR bitwise operation -@c STARTOFRANGE xor @cindex XOR bitwise operation -@c STARTOFRANGE opbit @cindex operations, bitwise @quotation @i{I can explain it for you, but I can't understand it for you.} @@ -17979,12 +18210,14 @@ each successive pair of bits in the operands. Three common operations are bitwise AND, OR, and XOR. The operations are described in @ref{table-bitwise-ops}. +@c 11/2014: Postprocessing turns the docbook informaltable +@c into a table. Hurray for scripting! @float Table,table-bitwise-ops -@caption{Bitwise Operations} +@caption{Bitwise operations} @ifnottex @ifnotdocbook @display - Bit Operator + Bit operator | AND | OR | XOR |---+---+---+---+---+--- Operands | 0 | 1 | 0 | 1 | 0 | 1 @@ -18042,7 +18275,7 @@ Operands | 0 | 1 | 0 | 1 | 0 | 1 <tbody> <row> <entry colsep="0"></entry> -<entry spanname="optitle"><emphasis role="bold">Bit Operator</emphasis></entry> +<entry spanname="optitle"><emphasis role="bold">Bit operator</emphasis></entry> </row> <row rowsep="1"> @@ -18106,10 +18339,9 @@ of a given value. Finally, two other common operations are to shift the bits left or right. For example, if you have a bit string @samp{10111001} and you shift it right by three bits, you end up with @samp{00010111}.@footnote{This example -shows that 0's come in on the left side. For @command{gawk}, this is +shows that zeros come in on the left side. For @command{gawk}, this is always true, but in some languages, it's possible to have the left side -fill with 1's.} -@c Purposely decided to use 0's and 1's here. 2/2001. +fill with ones.} If you start over again with @samp{10111001} and shift it left by three bits, you end up with @samp{11001000}. The following list describes @command{gawk}'s built-in functions that implement the bitwise operations. @@ -18148,7 +18380,7 @@ Return the value of @var{val}, shifted right by @var{count} bits. Return the bitwise XOR of the arguments. There must be at least two. @end table -For all of these functions, first the double precision floating-point value is +For all of these functions, first the double-precision floating-point value is converted to the widest C unsigned integer type, then the bitwise operation is performed. If the result cannot be represented exactly as a C @code{double}, leading nonzero bits are removed one by one until it can be represented @@ -18163,7 +18395,7 @@ that illustrates the use of these functions: @example @group @c file eg/lib/bits2str.awk -# bits2str --- turn a byte into readable 1's and 0's +# bits2str --- turn a byte into readable ones and zeros function bits2str(bits, data, mask) @{ @@ -18237,17 +18469,18 @@ $ @kbd{gawk -f testbits.awk} @cindex converting, numbers to strings @cindex number as string of bits The @code{bits2str()} function turns a binary number into a string. -The number @code{1} represents a binary value where the rightmost bit -is set to 1. Using this mask, +Initializing @code{mask} to one creates +a binary value where the rightmost bit +is set to one. Using this mask, the function repeatedly checks the rightmost bit. ANDing the mask with the value indicates whether the -rightmost bit is 1 or not. If so, a @code{"1"} is concatenated onto the front +rightmost bit is one or not. If so, a @code{"1"} is concatenated onto the front of the string. Otherwise, a @code{"0"} is added. The value is then shifted right by one bit and the loop continues -until there are no more 1 bits. +until there are no more one bits. -If the initial value is zero it returns a simple @code{"0"}. +If the initial value is zero, it returns a simple @code{"0"}. Otherwise, at the end, it pads the value with zeros to represent multiples of 8-bit quantities. This is typical in modern computers. @@ -18256,11 +18489,6 @@ decimal and octal values for the same numbers (@pxref{Nondecimal-numbers}), and then demonstrates the results of the @code{compl()}, @code{lshift()}, and @code{rshift()} functions. -@c ENDOFRANGE bit -@c ENDOFRANGE and -@c ENDOFRANGE oro -@c ENDOFRANGE xor -@c ENDOFRANGE opbit @node Type Functions @subsection Getting Type Information @@ -18274,7 +18502,7 @@ that traverses every element of an array of arrays @cindexgawkfunc{isarray} @cindex scalar or array @item isarray(@var{x}) -Return a true value if @var{x} is an array. Otherwise return false. +Return a true value if @var{x} is an array. Otherwise, return false. @end table @code{isarray()} is meant for use in two circumstances. The first is when @@ -18284,8 +18512,8 @@ an array or not. The second is inside the body of a user-defined function array or not. @quotation NOTE -Using @code{isarray()} at the global level to test -variables makes no sense. Since you are the one writing the program, you +Using @code{isarray()} at the global level to test +variables makes no sense. Because you are the one writing the program, you are supposed to know if your variables are arrays or not. And in fact, due to the way @command{gawk} works, if you pass the name of a variable that has not been previously used to @code{isarray()}, @command{gawk} @@ -18335,25 +18563,21 @@ The default value for @var{category} is @code{"LC_MESSAGES"}. Return the plural form used for @var{number} of the translation of @var{string1} and @var{string2} in text domain @var{domain} for locale category @var{category}. @var{string1} is the -English singular variant of a message, and @var{string2} the English plural +English singular variant of a message, and @var{string2} is the English plural variant of the same message. The default value for @var{domain} is the current value of @code{TEXTDOMAIN}. The default value for @var{category} is @code{"LC_MESSAGES"}. @end table -@c ENDOFRANGE funcbi -@c ENDOFRANGE bifunc @node User-defined @section User-Defined Functions -@c STARTOFRANGE udfunc @cindex user-defined functions -@c STARTOFRANGE funcud @cindex functions, user-defined Complicated @command{awk} programs can often be simplified by defining your own functions. User-defined functions can be called just like built-in ones (@pxref{Function Calls}), but it is up to you to define -them, i.e., to tell @command{awk} what they should do. +them (i.e., to tell @command{awk} what they should do). @menu * Definition Syntax:: How to write definitions and what they mean. @@ -18368,12 +18592,11 @@ them, i.e., to tell @command{awk} what they should do. @subsection Function Definition Syntax @quotation -@i{It's entirely fair to say that the @command{awk} syntax for local +@i{It's entirely fair to say that the awk syntax for local variable definitions is appallingly awful.} @author Brian Kernighan @end quotation -@c STARTOFRANGE fdef @cindex functions, defining Definitions of functions can appear anywhere between the rules of an @command{awk} program. Thus, the general form of an @command{awk} program is @@ -18411,14 +18634,23 @@ the call. A function cannot have two parameters with the same name, nor may it have a parameter with the same name as the function itself. -In addition, according to the POSIX standard, function parameters + +@quotation CAUTION +According to the POSIX standard, function parameters cannot have the same name as one of the special predefined variables -(@pxref{Built-in Variables}). Not all versions of @command{awk} enforce -this restriction. +(@pxref{Built-in Variables}), nor may a function parameter have the +same name as another function. + +Not all versions of @command{awk} enforce +these restrictions. +@command{gawk} always enforces the first restriction. +With @option{--posix} (@pxref{Options}), +it also enforces the second restriction. +@end quotation Local variables act like the empty string if referenced where a string value is required, and like zero if referenced where a numeric value -is required. This is the same as regular variables that have never been +is required. This is the same as the behavior of regular variables that have never been assigned a value. (There is more to understand about local variables; @pxref{Dynamic Typing}.) @@ -18452,7 +18684,7 @@ During execution of the function body, the arguments and local variable values hide, or @dfn{shadow}, any variables of the same names used in the rest of the program. The shadowed variables are not accessible in the function definition, because there is no way to name them while their -names have been taken away for the local variables. All other variables +names have been taken away for the arguments and local variables. All other variables used in the @command{awk} program can be referenced or set normally in the function's body. @@ -18492,13 +18724,13 @@ func foo() @{ a = sqrt($1) ; print a @} @end example @noindent -Instead it defines a rule that, for each record, concatenates the value +Instead, it defines a rule that, for each record, concatenates the value of the variable @samp{func} with the return value of the function @samp{foo}. If the resulting string is non-null, the action is executed. This is probably not what is desired. (@command{awk} accepts this input as syntactically valid, because functions may be used before they are defined in @command{awk} programs.@footnote{This program won't actually run, -since @code{foo()} is undefined.}) +because @code{foo()} is undefined.}) @cindex portability, functions@comma{} defining To ensure that your @command{awk} programs are portable, always use the @@ -18519,7 +18751,7 @@ function myprint(num) @end example @noindent -To illustrate, here is an @command{awk} rule that uses our @code{myprint} +To illustrate, here is an @command{awk} rule that uses our @code{myprint()} function: @example @@ -18560,16 +18792,16 @@ in an array and start over with a new list of elements (@pxref{Delete}). Instead of having to repeat this loop everywhere that you need to clear out -an array, your program can just call @code{delarray}. +an array, your program can just call @code{delarray()}. (This guarantees portability. The use of @samp{delete @var{array}} to delete the contents of an entire array is a relatively recent@footnote{Late in 2012.} addition to the POSIX standard.) The following is an example of a recursive function. It takes a string -as an input parameter and returns the string in backwards order. +as an input parameter and returns the string in reverse order. Recursive functions must always have a test that stops the recursion. In this case, the recursion terminates when the input string is -already empty. +already empty: @c 8/2014: Thanks to Mike Brennan for the improved formulation @cindex @code{rev()} user-defined function @@ -18617,15 +18849,13 @@ function ctime(ts, format) @end example You might think that @code{ctime()} could use @code{PROCINFO["strftime"]} -for its format string. That would be a mistake, since @code{ctime()} is +for its format string. That would be a mistake, because @code{ctime()} is supposed to return the time formatted in a standard fashion, and user-level code could have changed @code{PROCINFO["strftime"]}. -@c ENDOFRANGE fdef @node Function Caveats @subsection Calling User-Defined Functions -@c STARTOFRANGE fudc @cindex functions, user-defined, calling @dfn{Calling a function} means causing the function to run and do its job. A function call is an expression and its value is the value returned by @@ -18638,7 +18868,7 @@ the function. @end menu @node Calling A Function -@subsubsection Writing A Function Call +@subsubsection Writing a Function Call A function call consists of the function name followed by the arguments in parentheses. @command{awk} expressions are what you write in the @@ -18653,7 +18883,7 @@ foo(x y, "lose", 4 * z) @quotation CAUTION Whitespace characters (spaces and TABs) are not allowed -between the function name and the open-parenthesis of the argument list. +between the function name and the opening parenthesis of the argument list. If you write whitespace by mistake, @command{awk} might think that you mean to concatenate a variable with an expression in parentheses. However, it notices that you used a function name and not a variable name, and reports @@ -18665,7 +18895,7 @@ an error. @cindex local variables, in a function @cindex variables, local to a function -Unlike many languages, +Unlike in many languages, there is no way to make a variable local to a @code{@{} @dots{} @code{@}} block in @command{awk}, but you can make a variable local to a function. It is good practice to do so whenever a variable is needed only in that @@ -18674,7 +18904,7 @@ function. To make a variable local to a function, simply declare the variable as an argument after the actual function arguments (@pxref{Definition Syntax}). -Look at the following example where variable +Look at the following example, where variable @code{i} is a global variable used by both functions @code{foo()} and @code{bar()}: @@ -18693,7 +18923,7 @@ function foo(j) print "foo's i=" i @} -BEGIN @{ +BEGIN @{ i = 10 print "top's i=" i foo(0) @@ -18715,14 +18945,14 @@ foo's i=3 top's i=3 @end example -If you want @code{i} to be local to both @code{foo()} and @code{bar()} do as -follows (the extra-space before @code{i} is a coding convention to +If you want @code{i} to be local to both @code{foo()} and @code{bar()}, do as +follows (the extra space before @code{i} is a coding convention to indicate that @code{i} is a local variable, not an argument): @example function bar( i) @{ - for (i = 0; i < 3; i++) + for (i = 0; i < 3; i++) print "bar's i=" i @} @@ -18734,10 +18964,10 @@ function foo(j, i) print "foo's i=" i @} -BEGIN @{ +BEGIN @{ i = 10 print "top's i=" i - foo(0) + foo(0) print "top's i=" i @} @end example @@ -18796,21 +19026,16 @@ At level 2, index 2 is found in a @end example @node Pass By Value/Reference -@subsubsection Passing Function Arguments By Value Or By Reference +@subsubsection Passing Function Arguments by Value Or by Reference In @command{awk}, when you declare a function, there is no way to declare explicitly whether the arguments are passed @dfn{by value} or @dfn{by reference}. -Instead the passing convention is determined at runtime when -the function is called according to the following rule: - -@itemize -@item -If the argument is an array variable, then it is passed by reference, -@item -Otherwise the argument is passed by value. -@end itemize +Instead, the passing convention is determined at runtime when +the function is called, according to the following rule: +if the argument is an array variable, then it is passed by reference. +Otherwise, the argument is passed by value. @cindex call by value Passing an argument by value means that when a function is called, it @@ -18885,7 +19110,7 @@ prints @samp{a[1] = 1, a[2] = two, a[3] = 3}, because @cindex undefined functions @cindex functions, undefined Some @command{awk} implementations allow you to call a function that -has not been defined. They only report a problem at runtime when the +has not been defined. They only report a problem at runtime, when the program actually tries to call the function. For example: @example @@ -18913,10 +19138,15 @@ If @option{--lint} is specified Some @command{awk} implementations generate a runtime error if you use either the @code{next} statement or the @code{nextfile} statement -(@pxref{Next Statement}, also @pxref{Nextfile Statement}) +(@pxref{Next Statement}, and +@ifdocbook +@ref{Nextfile Statement}) +@end ifdocbook +@ifnotdocbook +@pxref{Nextfile Statement}) +@end ifnotdocbook inside a user-defined function. @command{gawk} does not have this limitation. -@c ENDOFRANGE fudc @node Return Statement @subsection The @code{return} Statement @@ -18939,15 +19169,15 @@ makes the returned value undefined, and therefore, unpredictable. In practice, though, all versions of @command{awk} simply return the null string, which acts like zero if used in a numeric context. -A @code{return} statement with no value expression is assumed at the end of -every function definition. So if control reaches the end of the function -body, then technically, the function returns an unpredictable value. +A @code{return} statement without an @var{expression} is assumed at the end of +every function definition. So, if control reaches the end of the function +body, then technically the function returns an unpredictable value. In practice, it returns the empty string. @command{awk} does @emph{not} warn you if you use the return value of such a function. Sometimes, you want to write a function for what it does, not for what it returns. Such a function corresponds to a @code{void} function -in C, C++ or Java, or to a @code{procedure} in Ada. Thus, it may be appropriate to not +in C, C++, or Java, or to a @code{procedure} in Ada. Thus, it may be appropriate to not return any value; simply bear in mind that you should not be using the return value of such a function. @@ -18969,8 +19199,8 @@ function maxelt(vec, i, ret) @noindent You call @code{maxelt()} with one argument, which is an array name. The local variables @code{i} and @code{ret} are not intended to be arguments; -while there is nothing to stop you from passing more than one argument -to @code{maxelt()}, the results would be strange. The extra space before +there is nothing to stop you from passing more than one argument +to @code{maxelt()} but the results would be strange. The extra space before @code{i} in the function parameter list indicates that @code{i} and @code{ret} are local variables. You should follow this convention when defining functions. @@ -19044,7 +19274,6 @@ does report the second error. Usually, such things aren't a big issue, but it's worth being aware of them. -@c ENDOFRANGE udfunc @node Indirect Calls @section Indirect Function Calls @@ -19067,13 +19296,15 @@ function calls, you can specify the name of the function to call as a string variable, and then call the function. Let's look at an example. Suppose you have a file with your test scores for the classes you -are taking. The first field is the class name. The following fields +are taking, and +you wish to get the sum and the average of +your test scores. +The first field is the class name. The following fields are the functions to call to process the data, up to a ``marker'' field @samp{data:}. Following the marker, to the end of the record, are the various numeric test scores. -Here is the initial file; you wish to get the sum and the average of -your test scores: +Here is the initial file: @example @c file eg/data/class_data1 @@ -19107,8 +19338,8 @@ variable as the @emph{name} of the function to call. @cindex indirect function calls, @code{@@}-notation @cindex function calls, indirect, @code{@@}-notation for The syntax is similar to that of a regular function call: an identifier -immediately followed by a left parenthesis, any arguments, and then -a closing right parenthesis, with the addition of a leading @samp{@@} +immediately followed by an opening parenthesis, any arguments, and then +a closing parenthesis, with the addition of a leading @samp{@@} character: @example @@ -19117,7 +19348,7 @@ result = @@the_func() # calls the sum() function @end example Here is a full program that processes the previously shown data, -using indirect function calls. +using indirect function calls: @example @c file eg/prog/indirectcall.awk @@ -19156,9 +19387,9 @@ function sum(first, last, ret, i) @c endfile @end example -These two functions expect to work on fields; thus the parameters +These two functions expect to work on fields; thus, the parameters @code{first} and @code{last} indicate where in the fields to start and end. -Otherwise they perform the expected computations and are not unusual. +Otherwise, they perform the expected computations and are not unusual: @example @c file eg/prog/indirectcall.awk @@ -19191,7 +19422,7 @@ saving it in @code{start}. The last part of the code loops through each function name (from @code{$2} up to the marker, @samp{data:}), calling the function named by the field. The indirect function call itself occurs as a parameter in the call to @code{printf}. -(The @code{printf} format string uses @code{%s} as the format specifier so that we +(The @code{printf} format string uses @samp{%s} as the format specifier so that we can use functions that return strings, as well as numbers. Note that the result from the indirect call is concatenated with the empty string, in order to force it to be a string value.) @@ -19203,11 +19434,11 @@ $ @kbd{gawk -f indirectcall.awk class_data1} @print{} Biology 101: @print{} sum: <352.8> @print{} average: <88.2> -@print{} +@print{} @print{} Chemistry 305: @print{} sum: <356.4> @print{} average: <89.1> -@print{} +@print{} @print{} English 401: @print{} sum: <376.1> @print{} average: <94.025> @@ -19217,8 +19448,8 @@ The ability to use indirect function calls is more powerful than you may think at first. The C and C++ languages provide ``function pointers,'' which are a mechanism for calling a function chosen at runtime. One of the most well-known uses of this ability is the C @code{qsort()} function, which sorts -an array using the famous ``quick sort'' algorithm -(see @uref{http://en.wikipedia.org/wiki/Quick_sort, the Wikipedia article} +an array using the famous ``quicksort'' algorithm +(see @uref{http://en.wikipedia.org/wiki/Quicksort, the Wikipedia article} for more information). To use this function, you supply a pointer to a comparison function. This mechanism allows you to sort arbitrary data in an arbitrary fashion. @@ -19237,11 +19468,11 @@ We can do something similar using @command{gawk}, like this: # January 2009 @c endfile - @end ignore @c file eg/lib/quicksort.awk -# quicksort --- C.A.R. Hoare's quick sort algorithm. See Wikipedia -# or almost any algorithms or computer science text + +# quicksort --- C.A.R. Hoare's quicksort algorithm. See Wikipedia +# or almost any algorithms or computer science text. @c endfile @ignore @c file eg/lib/quicksort.awk @@ -19279,7 +19510,7 @@ function quicksort_swap(data, i, j, temp) The @code{quicksort()} function receives the @code{data} array, the starting and ending indices to sort (@code{left} and @code{right}), and the name of a function that -performs a ``less than'' comparison. It then implements the quick sort algorithm. +performs a ``less than'' comparison. It then implements the quicksort algorithm. To make use of the sorting function, we return to our previous example. The first thing to do is write some comparison functions: @@ -19329,7 +19560,7 @@ function do_sort(first, last, compare, data, i, retval) retval = data[1] for (i = 2; i in data; i++) retval = retval " " data[i] - + return retval @} @c endfile @@ -19375,13 +19606,13 @@ $ @kbd{gawk -f quicksort.awk -f indirectcall.awk class_data2} @print{} average: <88.2> @print{} sort: <78.5 87.0 92.4 94.9> @print{} rsort: <94.9 92.4 87.0 78.5> -@print{} +@print{} @print{} Chemistry 305: @print{} sum: <356.4> @print{} average: <89.1> @print{} sort: <75.2 88.2 94.7 98.3> @print{} rsort: <98.3 94.7 88.2 75.2> -@print{} +@print{} @print{} English 401: @print{} sum: <376.1> @print{} average: <94.025> @@ -19390,76 +19621,29 @@ $ @kbd{gawk -f quicksort.awk -f indirectcall.awk class_data2} @end example Another example where indirect functions calls are useful can be found in -processing arrays. @DBREF{Walking Arrays} presented a simple function -for ``walking'' an array of arrays. That function simply printed the -name and value of each scalar array element. However, it is easy to -generalize that function, by passing in the name of a function to call -when walking an array. The modified function looks like this: - -@example -@c file eg/lib/processarray.awk -function process_array(arr, name, process, do_arrays, i, new_name) -@{ - for (i in arr) @{ - new_name = (name "[" i "]") - if (isarray(arr[i])) @{ - if (do_arrays) - @@process(new_name, arr[i]) - process_array(arr[i], new_name, process, do_arrays) - @} else - @@process(new_name, arr[i]) - @} -@} -@c endfile -@end example - -The arguments are as follows: - -@table @code -@item arr -The array. - -@item name -The name of the array (a string). - -@item process -The name of the function to call. - -@item do_arrays -If this is true, the function can handle elements that are subarrays. -@end table - -If subarrays are to be processed, that is done before walking them further. - -When run with the following scaffolding, the function produces the same -results as does the earlier @code{walk_array()} function: - -@example -BEGIN @{ - a[1] = 1 - a[2][1] = 21 - a[2][2] = 22 - a[3] = 3 - a[4][1][1] = 411 - a[4][2] = 42 - - process_array(a, "a", "do_print", 0) -@} - -function do_print(name, element) -@{ - printf "%s = %s\n", name, element -@} -@end example +processing arrays. This is described in @ref{Walking Arrays}. Remember that you must supply a leading @samp{@@} in front of an indirect function call. Starting with @value{PVERSION} 4.1.2 of @command{gawk}, indirect function calls may also be used with built-in functions and with extension functions -(@pxref{Dynamic Extensions}). The only thing you cannot do is pass a regular -expression constant to a built-in function through an indirect function -call.@footnote{This may change in a future version; recheck the documentation that -comes with your version of @command{gawk} to see if it has.} +(@pxref{Dynamic Extensions}). There are some limitations when calling +built-in functions indirectly, as follows. + +@itemize @value{BULLET} +@item +You cannot pass a regular expression constant to a built-in function +through an indirect function call.@footnote{This may change in a future +version; recheck the documentation that comes with your version of +@command{gawk} to see if it has.} This applies to the @code{sub()}, +@code{gsub()}, @code{gensub()}, @code{match()}, @code{split()} and +@code{patsplit()} functions. + +@item +If calling @code{sub()} or @code{gsub()}, you may only pass two arguments, +since those functions are unusual in that they update their third argument. +This means that @code{$0} will be updated. +@end itemize @command{gawk} does its best to make indirect function calls efficient. For example, in the following case: @@ -19470,7 +19654,7 @@ for (i = 1; i <= n; i++) @end example @noindent -@code{gawk} looks up the actual function to call only once. +@command{gawk} looks up the actual function to call only once. @node Functions Summary @section Summary @@ -19484,7 +19668,7 @@ functions. POSIX @command{awk} provides three kinds of built-in functions: numeric, string, and I/O. @command{gawk} provides functions that sort arrays, work with values representing time, do bit manipulation, determine variable -type (array vs.@: scalar), and internationalize and localize programs. +type (array versus scalar), and internationalize and localize programs. @command{gawk} also provides several extensions to some of standard functions, typically in the form of additional arguments. @@ -19537,10 +19721,9 @@ program. This is equivalent to function pointers in C and C++. @end itemize -@c ENDOFRANGE funcud @ifnotinfo -@part @value{PART2}Problem Solving With @command{awk} +@part @value{PART2}Problem Solving with @command{awk} @end ifnotinfo @ifdocbook @@ -19550,27 +19733,24 @@ It contains the following chapters: @itemize @value{BULLET} @item -@ref{Library Functions}. +@ref{Library Functions} @item -@ref{Sample Programs}. +@ref{Sample Programs} @end itemize @end ifdocbook @node Library Functions @chapter A Library of @command{awk} Functions -@c STARTOFRANGE libf @cindex libraries of @command{awk} functions -@c STARTOFRANGE flib @cindex functions, library -@c STARTOFRANGE fudlib @cindex functions, user-defined, library of @DBREF{User-defined} describes how to write your own @command{awk} functions. Writing functions is important, because it allows you to encapsulate algorithms and program tasks in a single place. It simplifies programming, making program development more -manageable, and making programs more readable. +manageable and making programs more readable. @cindex Kernighan, Brian @cindex Plauger, P.J.@: @@ -19614,9 +19794,9 @@ and would like to contribute them to the @command{awk} user community, see @cindex portability, example programs The programs in this @value{CHAPTER} and in @ref{Sample Programs}, -freely use features that are @command{gawk}-specific. +freely use @command{gawk}-specific features. Rewriting these programs for different implementations of @command{awk} -is pretty straightforward. +is pretty straightforward: @itemize @value{BULLET} @item @@ -19686,7 +19866,7 @@ Library functions often need to have global variables that they can use to preserve state information between calls to the function---for example, @code{getopt()}'s variable @code{_opti} (@pxref{Getopt Function}). -Such variables are called @dfn{private}, since the only functions that need to +Such variables are called @dfn{private}, as the only functions that need to use them are the ones in the library. When writing a library function, you should try to choose names for your @@ -19699,7 +19879,7 @@ often use variable names like these for their own purposes. The example programs shown in this @value{CHAPTER} all start the names of their private variables with an underscore (@samp{_}). Users generally don't use leading underscores in their variable names, so this convention immediately -decreases the chances that the variable name will be accidentally shared +decreases the chances that the variable names will be accidentally shared with the user's program. @cindex @code{_} (underscore), in names of private variables @@ -19708,17 +19888,17 @@ In addition, several of the library functions use a prefix that helps indicate what function or set of functions use the variables---for example, @code{_pw_byname()} in the user database routines (@pxref{Passwd Functions}). -This convention is recommended, since it even further decreases the +This convention is recommended, as it even further decreases the chance of inadvertent conflict among variable names. Note that this convention is used equally well for variable names and for private -function names.@footnote{While all the library routines could have +function names.@footnote{Although all the library routines could have been rewritten to use this convention, this was not done, in order to show how our own @command{awk} programming style has evolved and to provide some basis for this discussion.} As a final note on variable naming, if a function makes global variables -available for use by a main program, it is a good convention to start that -variable's name with a capital letter---for +available for use by a main program, it is a good convention to start those +variables' names with a capital letter---for example, @code{getopt()}'s @code{Opterr} and @code{Optind} variables (@pxref{Getopt Function}). The leading capital letter indicates that it is global, while the fact that @@ -19729,7 +19909,7 @@ not one of @command{awk}'s predefined variables, such as @code{FS}. It is also important that @emph{all} variables in library functions that do not need to save state are, in fact, declared local.@footnote{@command{gawk}'s @option{--dump-variables} command-line -option is useful for verifying this.} If this is not done, the variable +option is useful for verifying this.} If this is not done, the variables could accidentally be used in the user's program, leading to bugs that are very difficult to track down: @@ -19784,7 +19964,7 @@ programming use. @end menu @node Strtonum Function -@subsection Converting Strings To Numbers +@subsection Converting Strings to Numbers The @code{strtonum()} function (@pxref{String Functions}) is a @command{gawk} extension. The following function @@ -19852,7 +20032,7 @@ function mystrtonum(str, ret, n, i, k, c) # a[6] = "1.e3" # a[7] = "1.32" # a[8] = "1.32E2" -# +# # for (i = 1; i in a; i++) # print a[i], strtonum(a[i]), mystrtonum(a[i]) # @} @@ -19866,7 +20046,7 @@ string. It sets @code{k} to the index in @code{"1234567"} of the current octal digit. The return value will either be the same number as the digit, or zero if the character is not there, which will be true for a @samp{0}. -This is safe, since the regexp test in the @code{if} ensures that +This is safe, because the regexp test in the @code{if} ensures that only octal values are converted. Similar logic applies to the code that checks for and converts a @@ -19886,13 +20066,9 @@ be tested with @command{gawk} and the results compared to the built-in @node Assert Function @subsection Assertions -@c STARTOFRANGE asse @cindex assertions -@c STARTOFRANGE assef @cindex @code{assert()} function (C library) -@c STARTOFRANGE libfass @cindex libraries of @command{awk} functions, assertions -@c STARTOFRANGE flibass @cindex functions, library, assertions @cindex @command{awk} programs, lengthy, assertions When writing large programs, it is often useful to know @@ -19931,7 +20107,7 @@ Following is the function: @example @c file eg/lib/assert.awk -# assert --- assert that a condition is true. Otherwise exit. +# assert --- assert that a condition is true. Otherwise, exit. @c endfile @ignore @@ -19967,7 +20143,7 @@ is false, it prints a message to standard error, using the @code{string} parameter to describe the failed condition. It then sets the variable @code{_assert_exit} to one and executes the @code{exit} statement. The @code{exit} statement jumps to the @code{END} rule. If the @code{END} -rules finds @code{_assert_exit} to be true, it exits immediately. +rule finds @code{_assert_exit} to be true, it exits immediately. The purpose of the test in the @code{END} rule is to keep any other @code{END} rules from running. When an assertion fails, the @@ -20008,10 +20184,6 @@ most likely causing the program to hang as it waits for input. There is a simple workaround to this: make sure that such a @code{BEGIN} rule always ends with an @code{exit} statement. -@c ENDOFRANGE asse -@c ENDOFRANGE assef -@c ENDOFRANGE flibass -@c ENDOFRANGE libfass @node Round Function @subsection Rounding Numbers @@ -20213,7 +20385,7 @@ is always 1. This means that on those systems, characters have numeric values from 128 to 255. Finally, large mainframe systems use the EBCDIC character set, which uses all 256 values. -While there are other character sets in use on some older systems, +There are other character sets in use on some older systems, but they are not really worth worrying about: @example @@ -20263,11 +20435,11 @@ all the strings in an array into one long string. The following function, the application programs (@pxref{Sample Programs}). -Good function design is important; this function needs to be general but it +Good function design is important; this function needs to be general, but it should also have a reasonable default behavior. It is called with an array as well as the beginning and ending indices of the elements in the array to be merged. This assumes that the array indices are numeric---a reasonable -assumption since the array was likely created with @code{split()} +assumption, as the array was likely created with @code{split()} (@pxref{String Functions}): @cindex @code{join()} user-defined function @@ -20320,7 +20492,7 @@ more difficult than they really need to be.} The @code{systime()} and @code{strftime()} functions described in @DBREF{Time Functions} provide the minimum functionality necessary for dealing with the time of day -in human readable form. While @code{strftime()} is extensive, the control +in human-readable form. Although @code{strftime()} is extensive, the control formats are not necessarily easy to remember or intuitively obvious when reading a program. @@ -20411,7 +20583,7 @@ allowed the user to supply an optional timestamp value to use instead of the current time. @node Readfile Function -@subsection Reading A Whole File At Once +@subsection Reading a Whole File at Once Often, it is convenient to have the entire contents of a file available in memory as a single string. A straightforward but naive way to @@ -20468,13 +20640,13 @@ function readfile(file, tmp, save_rs) It works by setting @code{RS} to @samp{^$}, a regular expression that will never match if the file has contents. @command{gawk} reads data from -the file into @code{tmp} attempting to match @code{RS}. The match fails +the file into @code{tmp}, attempting to match @code{RS}. The match fails after each read, but fails quickly, such that @command{gawk} fills @code{tmp} with the entire contents of the file. -(@xref{Records}, for information on @code{RT} and @code{RS}.) +(@DBXREF{Records} for information on @code{RT} and @code{RS}.) In the case that @code{file} is empty, the return value is the null -string. Thus calling code may use something like: +string. Thus, calling code may use something like: @example contents = readfile("/some/path") @@ -20485,11 +20657,11 @@ if (length(contents) == 0) This tests the result to see if it is empty or not. An equivalent test would be @samp{contents == ""}. -@xref{Extension Sample Readfile}, for an extension function that +@DBXREF{Extension Sample Readfile} for an extension function that also reads an entire file into memory. @node Shell Quoting -@subsection Quoting Strings to Pass to The Shell +@subsection Quoting Strings to Pass to the Shell @c included by permission @ignore @@ -20531,7 +20703,7 @@ chmod -w file.flac Note the need for shell quoting. The function @code{shell_quote()} does it. @code{SINGLE} is the one-character string @code{"'"} and -@code{QSINGLE} is the three-character string @code{"\"'\""}. +@code{QSINGLE} is the three-character string @code{"\"'\""}: @example @c file eg/lib/shellquote.awk @@ -20569,11 +20741,8 @@ function shell_quote(s, # parameter @node Data File Management @section @value{DDF} Management -@c STARTOFRANGE dataf @cindex files, managing -@c STARTOFRANGE libfdataf @cindex libraries of @command{awk} functions, managing, data files -@c STARTOFRANGE flibdataf @cindex functions, library, managing data files This @value{SECTION} presents functions that are useful for managing command-line @value{DF}s. @@ -20591,12 +20760,12 @@ command-line @value{DF}s. @cindex files, managing, data file boundaries @cindex files, initialization and cleanup -The @code{BEGIN} and @code{END} rules are each executed exactly once at +The @code{BEGIN} and @code{END} rules are each executed exactly once, at the beginning and end of your @command{awk} program, respectively (@pxref{BEGIN/END}). We (the @command{gawk} authors) once had a user who mistakenly thought that the -@code{BEGIN} rule is executed at the beginning of each @value{DF} and the -@code{END} rule is executed at the end of each @value{DF}. +@code{BEGIN} rules were executed at the beginning of each @value{DF} and the +@code{END} rules were executed at the end of each @value{DF}. When informed that this was not the case, the user requested that we add new special @@ -20636,7 +20805,7 @@ END @{ endfile(FILENAME) @} This file must be loaded before the user's ``main'' program, so that the rule it supplies is executed first. -This rule relies on @command{awk}'s @code{FILENAME} variable that +This rule relies on @command{awk}'s @code{FILENAME} variable, which automatically changes for each new @value{DF}. The current @value{FN} is saved in a private variable, @code{_oldfilename}. If @code{FILENAME} does not equal @code{_oldfilename}, then a new @value{DF} is being processed and @@ -20652,7 +20821,7 @@ first @value{DF}. The program also supplies an @code{END} rule to do the final processing for the last file. Because this @code{END} rule comes before any @code{END} rules supplied in the ``main'' program, @code{endfile()} is called first. Once -again the value of multiple @code{BEGIN} and @code{END} rules should be clear. +again, the value of multiple @code{BEGIN} and @code{END} rules should be clear. @cindex @code{beginfile()} user-defined function @cindex @code{endfile()} user-defined function @@ -20663,7 +20832,7 @@ The following version solves the problem: @example @c file eg/lib/ftrans.awk -# ftrans.awk --- handle data file transitions +# ftrans.awk --- handle datafile transitions # # user supplies beginfile() and endfile() functions @c endfile @@ -20691,26 +20860,27 @@ END @{ endfile(_filename_) @} shows how this library function can be used and how it simplifies writing the main program. -@sidebar So Why Does @command{gawk} have @code{BEGINFILE} and @code{ENDFILE}? +@sidebar So Why Does @command{gawk} Have @code{BEGINFILE} and @code{ENDFILE}? You are probably wondering, if @code{beginfile()} and @code{endfile()} functions can do the job, why does @command{gawk} have -@code{BEGINFILE} and @code{ENDFILE} patterns (@pxref{BEGINFILE/ENDFILE})? +@code{BEGINFILE} and @code{ENDFILE} patterns? Good question. Normally, if @command{awk} cannot open a file, this causes an immediate fatal error. In this case, there is no way for a -user-defined function to deal with the problem, since the mechanism for +user-defined function to deal with the problem, as the mechanism for calling it relies on the file being open and at the first record. Thus, the main reason for @code{BEGINFILE} is to give you a ``hook'' to catch files that cannot be processed. @code{ENDFILE} exists for symmetry, and because it provides an easy way to do per-file cleanup processing. +For more information, refer to @ref{BEGINFILE/ENDFILE}. @end sidebar @node Rewind Function @subsection Rereading the Current File @cindex files, reading -Another request for a new built-in function was for a @code{rewind()} +Another request for a new built-in function was for a function that would make it possible to reread the current file. The requesting user didn't want to have to use @code{getline} (@pxref{Getline}) @@ -20719,7 +20889,7 @@ inside a loop. However, as long as you are not in the @code{END} rule, it is quite easy to arrange to immediately close the current input file and then start over with it from the top. -For lack of a better name, we'll call it @code{rewind()}: +For lack of a better name, we'll call the function @code{rewind()}: @cindex @code{rewind()} user-defined function @example @@ -20757,8 +20927,8 @@ The @code{rewind()} function relies on the @code{ARGIND} variable (@pxref{Auto-set}), which is specific to @command{gawk}. It also relies on the @code{nextfile} keyword (@pxref{Nextfile Statement}). Because of this, you should not call it from an @code{ENDFILE} rule. -(This isn't necessary anyway, since as soon as an @code{ENDFILE} rule -finishes @command{gawk} goes to the next file!) +(This isn't necessary anyway, because @command{gawk} goes to the next +file as soon as an @code{ENDFILE} rule finishes!) @node File Checking @subsection Checking for Readable @value{DDF}s @@ -20806,22 +20976,22 @@ BEGIN @{ @cindex troubleshooting, @code{getline} function This works, because the @code{getline} won't be fatal. Removing the element from @code{ARGV} with @code{delete} -skips the file (since it's no longer in the list). +skips the file (because it's no longer in the list). See also @ref{ARGC and ARGV}. -The regular expression check purposely does not use character classes +Because @command{awk} variable names only allow the English letters, +the regular expression check purposely does not use character classes such as @samp{[:alpha:]} and @samp{[:alnum:]} -(@pxref{Bracket Expressions}) -since @command{awk} variable names only allow the English letters. +(@pxref{Bracket Expressions}). @node Empty Files -@subsection Checking for Zero-length Files +@subsection Checking for Zero-Length Files All known @command{awk} implementations silently skip over zero-length files. -This is a by-product of @command{awk}'s implicit +This is a by-product of @command{awk}'s implicit read-a-record-and-match-against-the-rules loop: when @command{awk} tries to read a record from an empty file, it immediately receives an -end of file indication, closes the file, and proceeds on to the next +end-of-file indication, closes the file, and proceeds on to the next command-line @value{DF}, @emph{without} executing any user-level @command{awk} program code. @@ -20886,7 +21056,7 @@ Occasionally, you might not want @command{awk} to process command-line variable assignments (@pxref{Assignment Options}). In particular, if you have a @value{FN} that contains an @samp{=} character, -@command{awk} treats the @value{FN} as an assignment, and does not process it. +@command{awk} treats the @value{FN} as an assignment and does not process it. Some users have suggested an additional command-line option for @command{gawk} to disable command-line assignments. However, some simple programming with @@ -20936,30 +21106,22 @@ The use of @code{No_command_assign} allows you to disable command-line assignments at invocation time, by giving the variable a true value. When not set, it is initially zero (i.e., false), so the command-line arguments are left alone. -@c ENDOFRANGE dataf -@c ENDOFRANGE flibdataf -@c ENDOFRANGE libfdataf @node Getopt Function @section Processing Command-Line Options -@c STARTOFRANGE libfclo @cindex libraries of @command{awk} functions, command-line options -@c STARTOFRANGE flibclo @cindex functions, library, command-line options -@c STARTOFRANGE clop @cindex command-line options, processing -@c STARTOFRANGE oclp @cindex options, command-line, processing -@c STARTOFRANGE clibf @cindex functions, library, C library @cindex arguments, processing -Most utilities on POSIX compatible systems take options on +Most utilities on POSIX-compatible systems take options on the command line that can be used to change the way a program behaves. @command{awk} is an example of such a program (@pxref{Options}). -Often, options take @dfn{arguments}; i.e., data that the program needs to -correctly obey the command-line option. For example, @command{awk}'s +Often, options take @dfn{arguments} (i.e., data that the program needs to +correctly obey the command-line option). For example, @command{awk}'s @option{-F} option requires a string to use as the field separator. The first occurrence on the command line of either @option{--} or a string that does not begin with @samp{-} ends the options. @@ -21063,7 +21225,7 @@ necessary for accessing individual characters (@pxref{String Functions}).@footnote{This function was written before @command{gawk} acquired the ability to split strings into single characters using @code{""} as the separator. -We have left it alone, since using @code{substr()} is more portable.} +We have left it alone, as using @code{substr()} is more portable.} The discussion that follows walks through the code a bit at a time: @@ -21231,9 +21393,9 @@ next element in @code{argv}. If neither condition is true, then only on the next call to @code{getopt()}. The @code{BEGIN} rule initializes both @code{Opterr} and @code{Optind} to one. -@code{Opterr} is set to one, since the default behavior is for @code{getopt()} +@code{Opterr} is set to one, because the default behavior is for @code{getopt()} to print a diagnostic message upon seeing an invalid option. @code{Optind} -is set to one, since there's no reason to look at the program name, which is +is set to one, because there's no reason to look at the program name, which is in @code{ARGV[0]}: @example @@ -21256,8 +21418,8 @@ BEGIN @{ @c endfile @end example -The rest of the @code{BEGIN} rule is a simple test program. Here is the -result of two sample runs of the test program: +The rest of the @code{BEGIN} rule is a simple test program. Here are the +results of two sample runs of the test program: @example $ @kbd{awk -f getopt.awk -v _getopt_test=1 -- -a -cbARG bax -x} @@ -21283,46 +21445,44 @@ etc., as its own options. @quotation NOTE After @code{getopt()} is through, -user level code must clear out all the elements of @code{ARGV} from 1 +user-level code must clear out all the elements of @code{ARGV} from 1 to @code{Optind}, so that @command{awk} does not try to process the command-line options as @value{FN}s. @end quotation Using @samp{#!} with the @option{-E} option may help avoid conflicts between your program's options and @command{gawk}'s options, -since @option{-E} causes @command{gawk} to abandon processing of +as @option{-E} causes @command{gawk} to abandon processing of further options -(@pxref{Executable Scripts}, and @pxref{Options}). +(@DBPXREF{Executable Scripts} and +@ifnotdocbook +@pxref{Options}). +@end ifnotdocbook +@ifdocbook +@ref{Options}). +@end ifdocbook Several of the sample programs presented in @ref{Sample Programs}, use @code{getopt()} to process their arguments. -@c ENDOFRANGE libfclo -@c ENDOFRANGE flibclo -@c ENDOFRANGE clop -@c ENDOFRANGE oclp @node Passwd Functions @section Reading the User Database -@c STARTOFRANGE libfudata @cindex libraries of @command{awk} functions, user database, reading -@c STARTOFRANGE flibudata @cindex functions, library, user database@comma{} reading -@c STARTOFRANGE udatar @cindex user database@comma{} reading -@c STARTOFRANGE dataur @cindex database, users@comma{} reading @cindex @code{PROCINFO} array The @code{PROCINFO} array (@pxref{Built-in Variables}) provides access to the current user's real and effective user and group ID -numbers, and if available, the user's supplementary group set. +numbers, and, if available, the user's supplementary group set. However, because these are numbers, they do not provide very useful information to the average user. There needs to be some way to find the user information associated with the user and group ID numbers. This @value{SECTION} presents a suite of functions for retrieving information from the -user database. @xref{Group Functions}, +user database. @DBXREF{Group Functions} for a similar suite that retrieves information from the group database. @cindex @code{getpwent()} function (C library) @@ -21337,11 +21497,11 @@ kept. Instead, it provides the @code{<pwd.h>} header file and several C language subroutines for obtaining user information. The primary function is @code{getpwent()}, for ``get password entry.'' The ``password'' comes from the original user database file, -@file{/etc/passwd}, which stores user information, along with the +@file{/etc/passwd}, which stores user information along with the encrypted passwords (hence the name). @cindex @command{pwcat} program -While an @command{awk} program could simply read @file{/etc/passwd} +Although an @command{awk} program could simply read @file{/etc/passwd} directly, this file may not contain complete information about the system's set of users.@footnote{It is often the case that password information is stored in a network database.} To be sure you are able to @@ -21436,12 +21596,12 @@ The user's encrypted password. This may not be available on some systems. @item User-ID The user's numeric user ID number. -(On some systems it's a C @code{long}, and not an @code{int}. Thus +(On some systems, it's a C @code{long}, and not an @code{int}. Thus, we cast it to @code{long} for all cases.) @item Group-ID The user's numeric group ID number. -(Similar comments about @code{long} vs.@: @code{int} apply here.) +(Similar comments about @code{long} versus @code{int} apply here.) @item Full name The user's full name, and perhaps other information associated with the @@ -21463,7 +21623,7 @@ A few lines representative of @command{pwcat}'s output are as follows: @cindex Robbins, Miriam @example $ @kbd{pwcat} -@print{} root:3Ov02d5VaUPB6:0:1:Operator:/:/bin/sh +@print{} root:x:0:1:Operator:/:/bin/sh @print{} nobody:*:65534:65534::/: @print{} daemon:*:1:1::/: @print{} sys:*:2:2::/:/bin/csh @@ -21542,7 +21702,7 @@ The function @code{_pw_init()} fills three copies of the user information into three associative arrays. The arrays are indexed by username (@code{_pw_byname}), by user ID number (@code{_pw_byuid}), and by order of occurrence (@code{_pw_bycount}). -The variable @code{_pw_inited} is used for efficiency, since @code{_pw_init()} +The variable @code{_pw_inited} is used for efficiency, as @code{_pw_init()} needs to be called only once. @cindex @code{PROCINFO} array, testing the field splitting @@ -21551,7 +21711,7 @@ Because this function uses @code{getline} to read information from @command{pwcat}, it first saves the values of @code{FS}, @code{RS}, and @code{$0}. It notes in the variable @code{using_fw} whether field splitting with @code{FIELDWIDTHS} is in effect or not. -Doing so is necessary, since these functions could be called +Doing so is necessary, as these functions could be called from anywhere within a user's program, and the user may have his or her own way of splitting records and fields. This makes it possible to restore the correct @@ -21563,7 +21723,7 @@ The code that checks for using @code{FPAT}, using @code{using_fpat} and @code{PROCINFO["FS"]}, is similar. The main part of the function uses a loop to read database lines, split -the line into fields, and then store the line into each array as necessary. +the lines into fields, and then store the lines into each array as necessary. When the loop is done, @code{@w{_pw_init()}} cleans up by closing the pipeline, setting @code{@w{_pw_inited}} to one, and restoring @code{FS} (and @code{FIELDWIDTHS} or @code{FPAT} @@ -21653,26 +21813,18 @@ In turn, calling @code{_pw_init()} is not too expensive, because the once. If you are worried about squeezing every last cycle out of your @command{awk} program, the check of @code{_pw_inited} could be moved out of @code{_pw_init()} and duplicated in all the other functions. In practice, -this is not necessary, since most @command{awk} programs are I/O-bound, +this is not necessary, as most @command{awk} programs are I/O-bound, and such a change would clutter up the code. The @command{id} program in @DBREF{Id Program} uses these functions. -@c ENDOFRANGE libfudata -@c ENDOFRANGE flibudata -@c ENDOFRANGE udatar -@c ENDOFRANGE dataur @node Group Functions @section Reading the Group Database -@c STARTOFRANGE libfgdata @cindex libraries of @command{awk} functions, group database, reading -@c STARTOFRANGE flibgdata @cindex functions, library, group database@comma{} reading -@c STARTOFRANGE gdatar @cindex group database, reading -@c STARTOFRANGE datagr @cindex database, group, reading @cindex @code{PROCINFO} array, and group membership @cindex @code{getgrent()} function (C library) @@ -21788,11 +21940,11 @@ it is usually empty or set to @samp{*}. @item Group ID Number The group's numeric group ID number; the association of name to number must be unique within the file. -(On some systems it's a C @code{long}, and not an @code{int}. Thus +(On some systems it's a C @code{long}, and not an @code{int}. Thus, we cast it to @code{long} for all cases.) @item Group Member List -A comma-separated list of user names. These users are members of the group. +A comma-separated list of usernames. These users are members of the group. Modern Unix systems allow users to be members of several groups simultaneously. If your system does, then there are elements @code{"group1"} through @code{"group@var{N}"} in @code{PROCINFO} @@ -21902,32 +22054,32 @@ The @code{@w{_gr_init()}} function first saves @code{FS}, @code{$0}, and then sets @code{FS} and @code{RS} to the correct values for scanning the group information. It also takes care to note whether @code{FIELDWIDTHS} or @code{FPAT} -is being used, and to restore the appropriate field splitting mechanism. +is being used, and to restore the appropriate field-splitting mechanism. -The group information is stored is several associative arrays. +The group information is stored in several associative arrays. The arrays are indexed by group name (@code{@w{_gr_byname}}), by group ID number (@code{@w{_gr_bygid}}), and by position in the database (@code{@w{_gr_bycount}}). -There is an additional array indexed by user name (@code{@w{_gr_groupsbyuser}}), +There is an additional array indexed by username (@code{@w{_gr_groupsbyuser}}), which is a space-separated list of groups to which each user belongs. -Unlike the user database, it is possible to have multiple records in the +Unlike in the user database, it is possible to have multiple records in the database for the same group. This is common when a group has a large number of members. A pair of such entries might look like the following: @example -tvpeople:*:101:johny,jay,arsenio +tvpeople:*:101:johnny,jay,arsenio tvpeople:*:101:david,conan,tom,joan @end example For this reason, @code{_gr_init()} looks to see if a group name or -group ID number is already seen. If it is, then the user names are -simply concatenated onto the previous list of users.@footnote{There is actually a +group ID number is already seen. If so, the usernames are +simply concatenated onto the previous list of users.@footnote{There is a subtle problem with the code just presented. Suppose that the first time there were no names. This code adds the names with a leading comma. It also doesn't check that there is a @code{$4}.} Finally, @code{_gr_init()} closes the pipeline to @command{grcat}, restores -@code{FS} (and @code{FIELDWIDTHS} or @code{FPAT} if necessary), @code{RS}, and @code{$0}, +@code{FS} (and @code{FIELDWIDTHS} or @code{FPAT}, if necessary), @code{RS}, and @code{$0}, initializes @code{_gr_count} to zero (it is used later), and makes @code{_gr_inited} nonzero. @@ -21966,7 +22118,7 @@ function getgrgid(gid) @cindex @code{getgruser()} function (C library) The @code{getgruser()} function does not have a C counterpart. It takes a -user name and returns the list of groups that have the user as a member: +username and returns the list of groups that have the user as a member: @cindex @code{getgruser()} function, user-defined @example @@ -21995,7 +22147,6 @@ function getgrent() @} @c endfile @end example -@c ENDOFRANGE clibf @cindex @code{endgrent()} function (C library) The @code{endgrent()} function resets @code{_gr_count} to zero so that @code{getgrent()} can @@ -22028,12 +22179,12 @@ uses these functions. @DBREF{Arrays of Arrays} described how @command{gawk} provides arrays of arrays. In particular, any element of -an array may be either a scalar, or another array. The +an array may be either a scalar or another array. The @code{isarray()} function (@pxref{Type Functions}) lets you distinguish an array from a scalar. The following function, @code{walk_array()}, recursively traverses -an array, printing each element's indices and value. +an array, printing the element indices and values. You call it with the array and a string representing the name of the array: @@ -22084,10 +22235,66 @@ $ @kbd{gawk -f walk_array.awk} @print{} a[4][2] = 42 @end example -@c ENDOFRANGE libfgdata -@c ENDOFRANGE flibgdata -@c ENDOFRANGE gdatar -@c ENDOFRANGE libf +The function just presented simply prints the +name and value of each scalar array element. However, it is easy to +generalize it, by passing in the name of a function to call +when walking an array. The modified function looks like this: + +@example +@c file eg/lib/processarray.awk +function process_array(arr, name, process, do_arrays, i, new_name) +@{ + for (i in arr) @{ + new_name = (name "[" i "]") + if (isarray(arr[i])) @{ + if (do_arrays) + @@process(new_name, arr[i]) + process_array(arr[i], new_name, process, do_arrays) + @} else + @@process(new_name, arr[i]) + @} +@} +@c endfile +@end example + +The arguments are as follows: + +@table @code +@item arr +The array. + +@item name +The name of the array (a string). + +@item process +The name of the function to call. + +@item do_arrays +If this is true, the function can handle elements that are subarrays. +@end table + +If subarrays are to be processed, that is done before walking them further. + +When run with the following scaffolding, the function produces the same +results as does the earlier version of @code{walk_array()}: + +@example +BEGIN @{ + a[1] = 1 + a[2][1] = 21 + a[2][2] = 22 + a[3] = 3 + a[4][1][1] = 411 + a[4][2] = 42 + + process_array(a, "a", "do_print", 0) +@} + +function do_print(name, element) +@{ + printf "%s = %s\n", name, element +@} +@end example @node Library Functions Summary @section Summary @@ -22109,24 +22316,24 @@ The functions presented here fit into the following categories: @c nested list @table @asis @item General problems -Number to string conversion, assertions, rounding, random number +Number-to-string conversion, testing assertions, rounding, random number generation, converting characters to numbers, joining strings, getting easily usable time-of-day information, and reading a whole file in -one shot. +one shot @item Managing @value{DF}s Noting @value{DF} boundaries, rereading the current file, checking for readable files, checking for zero-length files, and treating assignments -as @value{FN}s. +as @value{FN}s @item Processing command-line options -An @command{awk} version of the standard C @code{getopt()} function. +An @command{awk} version of the standard C @code{getopt()} function @item Reading the user and group databases -Two sets of routines that parallel the C library versions. +Two sets of routines that parallel the C library versions @item Traversing arrays of arrays -A simple function to traverse an array of arrays to any depth. +Two functions that traverse an array of arrays to any depth @end table @c end nested list @@ -22169,7 +22376,7 @@ ARGIND != Argind @{ @} END @{ if (ARGIND < ARGC - 1) - ARGIND = ARGC - 1 + ARGIND = ARGC - 1 if (ARGIND > Argind) for (Argind++; Argind <= ARGIND; Argind++) zerofile(ARGV[Argind], Argind) @@ -22201,13 +22408,9 @@ output identical to that of the original version. @end enumerate @c EXCLUDE END -@c ENDOFRANGE flib -@c ENDOFRANGE fudlib -@c ENDOFRANGE datagr @node Sample Programs @chapter Practical @command{awk} Programs -@c STARTOFRANGE awkpex @cindex @command{awk} programs, examples of @c FULLXREF ON @@ -22225,10 +22428,10 @@ in this @value{CHAPTER}. The second presents @command{awk} versions of several common POSIX utilities. These are programs that you are hopefully already familiar with, -and therefore, whose problems are understood. +and therefore whose problems are understood. By reimplementing these programs in @command{awk}, you can focus on the @command{awk}-related aspects of solving -the programming problem. +the programming problems. The third is a grab bag of interesting programs. These solve a number of different data-manipulation and management @@ -22277,7 +22480,6 @@ cut.awk -- -c1-8 myfiles > results @node Clones @section Reinventing Wheels for Fun and Profit -@c STARTOFRANGE posimawk @cindex POSIX, programs@comma{} implementing in @command{awk} This @value{SECTION} presents a number of POSIX utilities implemented in @@ -22289,7 +22491,7 @@ It should be noted that these programs are not necessarily intended to replace the installed versions on your system. Nor may all of these programs be fully compliant with the most recent POSIX standard. This is not a problem; their -purpose is to illustrate @command{awk} language programming for ``real world'' +purpose is to illustrate @command{awk} language programming for ``real-world'' tasks. The programs are presented in alphabetical order. @@ -22305,14 +22507,11 @@ The programs are presented in alphabetical order. @end menu @node Cut Program -@subsection Cutting out Fields and Columns +@subsection Cutting Out Fields and Columns @cindex @command{cut} utility -@c STARTOFRANGE cut @cindex @command{cut} utility -@c STARTOFRANGE ficut @cindex fields, cutting -@c STARTOFRANGE colcut @cindex columns, cutting The @command{cut} utility selects, or ``cuts,'' characters or fields from its standard input and sends them to its standard output. @@ -22321,7 +22520,7 @@ but you may supply a command-line option to change the field @dfn{delimiter} (i.e., the field-separator character). @command{cut}'s definition of fields is less general than @command{awk}'s. -A common use of @command{cut} might be to pull out just the login name of +A common use of @command{cut} might be to pull out just the login names of logged-on users from the output of @command{who}. For example, the following pipeline generates a sorted, unique list of the logged-on users: @@ -22582,7 +22781,7 @@ function set_charlist( field, i, j, f, g, n, m, t, @c endfile @end example -Next is the rule that actually processes the data. If the @option{-s} option +Next is the rule that processes the data. If the @option{-s} option is given, then @code{suppress} is true. The first @code{if} statement makes sure that the input record does have the field separator. If @command{cut} is processing fields, @code{suppress} is true, and the field @@ -22614,27 +22813,20 @@ written out between the fields: @end example This version of @command{cut} relies on @command{gawk}'s @code{FIELDWIDTHS} -variable to do the character-based cutting. While it is possible in +variable to do the character-based cutting. It is possible in other @command{awk} implementations to use @code{substr()} -(@pxref{String Functions}), +(@pxref{String Functions}), but it is also extremely painful. The @code{FIELDWIDTHS} variable supplies an elegant solution to the problem of picking the input line apart by characters. -@c ENDOFRANGE cut -@c ENDOFRANGE ficut -@c ENDOFRANGE colcut @node Egrep Program @subsection Searching for Regular Expressions in Files -@c STARTOFRANGE regexps @cindex regular expressions, searching for -@c STARTOFRANGE sfregexp @cindex searching, files for regular expressions -@c STARTOFRANGE fsregexp @cindex files, searching for regular expressions -@c STARTOFRANGE egrep @cindex @command{egrep} utility The @command{egrep} utility searches files for patterns. It uses regular expressions that are almost identical to those available in @command{awk} @@ -22761,7 +22953,7 @@ matched lines in the output: @c endfile @end example -The last two lines are commented out, since they are not needed in +The last two lines are commented out, as they are not needed in @command{gawk}. They should be uncommented if you have to use another version of @command{awk}. @@ -22771,7 +22963,7 @@ into lowercase if the @option{-i} option is specified.@footnote{It also introduces a subtle bug; if a match happens, we output the translated line, not the original.} The rule is -commented out since it is not necessary with @command{gawk}: +commented out as it is not necessary with @command{gawk}: @example @c file eg/prog/egrep.awk @@ -22837,7 +23029,7 @@ successful or unsuccessful match. If the line does not match, the @code{next} statement just moves on to the next record. A number of additional tests are made, but they are only done if we -are not counting lines. First, if the user only wants exit status +are not counting lines. First, if the user only wants the exit status (@code{no_print} is true), then it is enough to know that @emph{one} line in this file matched, and we can skip on to the next file with @code{nextfile}. Similarly, if we are only printing @value{FN}s, we can @@ -22878,7 +23070,7 @@ if necessary: @end example The @code{END} rule takes care of producing the correct exit status. If -there are no matches, the exit status is one; otherwise it is zero: +there are no matches, the exit status is one; otherwise, it is zero: @example @c file eg/prog/egrep.awk @@ -22902,17 +23094,12 @@ function usage() @c endfile @end example -@c ENDOFRANGE regexps -@c ENDOFRANGE sfregexp -@c ENDOFRANGE fsregexp -@c ENDOFRANGE egrep @node Id Program -@subsection Printing out User Information +@subsection Printing Out User Information @cindex printing, user information @cindex users, information about, printing -@c STARTOFRANGE id @cindex @command{id} utility The @command{id} utility lists a user's real and effective user ID numbers, real and effective group ID numbers, and the user's group set, if any. @@ -22935,7 +23122,8 @@ Here is a simple version of @command{id} written in @command{awk}. It uses the user database library functions (@pxref{Passwd Functions}) and the group database library functions -(@pxref{Group Functions}): +(@pxref{Group Functions}) +from @ref{Library Functions}. The program is fairly straightforward. All the work is done in the @code{BEGIN} rule. The user and group ID numbers are obtained from @@ -23023,7 +23211,7 @@ function pr_first_field(str, a) The test in the @code{for} loop is worth noting. Any supplementary groups in the @code{PROCINFO} array have the indices @code{"group1"} through @code{"group@var{N}"} for some -@var{N}, i.e., the total number of supplementary groups. +@var{N} (i.e., the total number of supplementary groups). However, we don't know in advance how many of these groups there are. @@ -23041,16 +23229,13 @@ code that is used repeatedly, making the whole program shorter and cleaner. In particular, moving the check for the empty string into this function saves several lines of code. -@c ENDOFRANGE id @node Split Program @subsection Splitting a Large File into Pieces @c FIXME: One day, update to current POSIX version of split -@c STARTOFRANGE filspl @cindex files, splitting -@c STARTOFRANGE split @cindex @code{split} utility The @command{split} program splits large text files into smaller pieces. Usage is as follows:@footnote{This is the traditional usage. The @@ -23063,10 +23248,10 @@ aims to demonstrate.} By default, the output files are named @file{xaa}, @file{xab}, and so on. Each file has -1000 lines in it, with the likely exception of the last file. To change the +1,000 lines in it, with the likely exception of the last file. To change the number of lines in each file, supply a number on the command line -preceded with a minus; e.g., @samp{-500} for files with 500 lines in them -instead of 1000. To change the name of the output files to something like +preceded with a minus sign (e.g., @samp{-500} for files with 500 lines in them +instead of 1,000). To change the names of the output files to something like @file{myfileaa}, @file{myfileab}, and so on, supply an additional argument that specifies the @value{FN} prefix. @@ -23114,7 +23299,7 @@ BEGIN @{ @} # test argv in case reading from stdin instead of file if (i in ARGV) - i++ # skip data file name + i++ # skip datafile name if (i in ARGV) @{ outfile = ARGV[i] ARGV[i] = "" @@ -23185,15 +23370,12 @@ You might want to consider how to eliminate the use of way as to solve the EBCDIC issue as well. @end ifset -@c ENDOFRANGE filspl -@c ENDOFRANGE split @node Tee Program @subsection Duplicating Output into Multiple Files @cindex files, multiple@comma{} duplicating output into @cindex output, duplicating into files -@c STARTOFRANGE tee @cindex @code{tee} utility The @code{tee} program is known as a ``pipe fitting.'' @code{tee} copies its standard input to its standard output and also duplicates it to the @@ -23208,8 +23390,8 @@ truncating them and starting over. The @code{BEGIN} rule first makes a copy of all the command-line arguments into an array named @code{copy}. -@code{ARGV[0]} is not copied, since it is not needed. -@code{tee} cannot use @code{ARGV} directly, since @command{awk} attempts to +@code{ARGV[0]} is not needed, so it is not copied. +@code{tee} cannot use @code{ARGV} directly, because @command{awk} attempts to process each @value{FN} in @code{ARGV} as input data. @cindex flag variables @@ -23258,7 +23440,7 @@ BEGIN @{ @c endfile @end example -The following single rule does all the work. Since there is no pattern, it is +The following single rule does all the work. Because there is no pattern, it is executed for each line of input. The body of the rule simply prints the line into each file on the command line, and then to the standard output: @@ -23289,7 +23471,7 @@ for (i in copy) @end example @noindent -This is more concise but it is also less efficient. The @samp{if} is +This is more concise, but it is also less efficient. The @samp{if} is tested for each record and for each output file. By duplicating the loop body, the @samp{if} is only tested once for each input record. If there are @var{N} input records and @var{M} output files, the first method only @@ -23306,18 +23488,14 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE tee @node Uniq Program @subsection Printing Nonduplicated Lines of Text @c FIXME: One day, update to current POSIX version of uniq -@c STARTOFRANGE prunt @cindex printing, unduplicated lines of text -@c STARTOFRANGE tpul @cindex text@comma{} printing, unduplicated lines of -@c STARTOFRANGE uniq @cindex @command{uniq} utility The @command{uniq} utility reads sorted lines of data on its standard input, and by default removes duplicate lines. In other words, it only @@ -23509,10 +23687,10 @@ The second rule does the work. The variable @code{equal} is one or zero, depending upon the results of @code{are_equal()}'s comparison. If @command{uniq} is counting repeated lines, and the lines are equal, then it increments the @code{count} variable. Otherwise, it prints the line and resets @code{count}, -since the two lines are not equal. +because the two lines are not equal. If @command{uniq} is not counting, and if the lines are equal, @code{count} is incremented. -Nothing is printed, since the point is to remove duplicates. +Nothing is printed, as the point is to remove duplicates. Otherwise, if @command{uniq} is counting repeated lines and more than one line is seen, or if @command{uniq} is counting nonrepeated lines and only one line is seen, then the line is printed, and @code{count} @@ -23581,31 +23759,22 @@ Brian Kernighan suggests that ``an alternative approach to state machines is to just read the input into an array, then use indexing. It's almost always easier code, and for most inputs where you would use this, just -as fast.'' Consider how to rewrite the logic to follow this +as fast.'' Consider how to rewrite the logic to follow this suggestion. @end ifset -@c ENDOFRANGE prunt -@c ENDOFRANGE tpul -@c ENDOFRANGE uniq @node Wc Program @subsection Counting Things @c FIXME: One day, update to current POSIX version of wc -@c STARTOFRANGE count @cindex counting -@c STARTOFRANGE infco @cindex input files, counting elements in -@c STARTOFRANGE woco @cindex words, counting -@c STARTOFRANGE chco @cindex characters, counting -@c STARTOFRANGE lico @cindex lines, counting -@c STARTOFRANGE wc @cindex @command{wc} utility The @command{wc} (word count) utility counts lines, words, and characters in one or more input files. Its usage is as follows: @@ -23633,7 +23802,7 @@ Count only characters. @end table Implementing @command{wc} in @command{awk} is particularly elegant, -since @command{awk} does a lot of the work for us; it splits lines into +because @command{awk} does a lot of the work for us; it splits lines into words (i.e., fields) and counts them, it counts lines (i.e., records), and it can easily tell us how long a line is. @@ -23738,7 +23907,7 @@ function endfile(file) @end example There is one rule that is executed for each line. It adds the length of -the record, plus one, to @code{chars}.@footnote{Since @command{gawk} +the record, plus one, to @code{chars}.@footnote{Because @command{gawk} understands multibyte locales, this code counts characters, not bytes.} Adding one plus the record length is needed because the newline character separating records (the value @@ -23775,13 +23944,6 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE count -@c ENDOFRANGE infco -@c ENDOFRANGE lico -@c ENDOFRANGE woco -@c ENDOFRANGE chco -@c ENDOFRANGE wc -@c ENDOFRANGE posimawk @node Miscellaneous Programs @section A Grab Bag of @command{awk} Programs @@ -23912,9 +24074,7 @@ Aharon Robbins <arnold@skeeve.com> wrote: @author Erik Quanstrom @end quotation -@c STARTOFRANGE tialarm @cindex time, alarm clock example program -@c STARTOFRANGE alaex @cindex alarm clock example program The following program is a simple ``alarm clock'' program. You give it a time of day and an optional message. At the specified time, @@ -23930,7 +24090,7 @@ checking and setting of defaults: the delay, the count, and the message to print. If the user supplied a message without the ASCII BEL character (known as the ``alert'' character, @code{"\a"}), then it is added to the message. (On many systems, printing the ASCII BEL generates an -audible alert. Thus when the alarm goes off, the system calls attention +audible alert. Thus, when the alarm goes off, the system calls attention to itself in case the user is not looking at the computer.) Just for a change, this program uses a @code{switch} statement (@pxref{Switch Statement}), but the processing could be done with a series of @@ -24066,15 +24226,11 @@ seconds are necessary: @} @c endfile @end example -@c ENDOFRANGE tialarm -@c ENDOFRANGE alaex @node Translate Program @subsection Transliterating Characters -@c STARTOFRANGE chtra @cindex characters, transliterating -@c STARTOFRANGE tr @cindex @command{tr} utility The system @command{tr} utility transliterates characters. For example, it is often used to map uppercase letters into lowercase for further processing: @@ -24086,8 +24242,8 @@ often used to map uppercase letters into lowercase for further processing: @command{tr} requires two lists of characters.@footnote{On some older systems, including Solaris, the system version of @command{tr} may require that the lists be written as range expressions enclosed in square brackets -(@samp{[a-z]}) and quoted, to prevent the shell from attempting a file -name expansion. This is not a feature.} When processing the input, the +(@samp{[a-z]}) and quoted, to prevent the shell from attempting a +@value{FN} expansion. This is not a feature.} When processing the input, the first character in the first list is replaced with the first character in the second list, the second character in the first list is replaced with the second character in the second list, and so on. If there are @@ -24103,7 +24259,7 @@ to @command{gawk}. @c at least theoretically The following program was written to prove that character transliteration could be done with a user-level -function. This program is not as complete as the system @command{tr} utility +function. This program is not as complete as the system @command{tr} utility, but it does most of the job. The @command{translate} program was written long before @command{gawk} @@ -24115,13 +24271,13 @@ takes three arguments: @table @code @item from -A list of characters from which to translate. +A list of characters from which to translate @item to -A list of characters to which to translate. +A list of characters to which to translate @item target -The string on which to do the translation. +The string on which to do the translation @end table Associative arrays make the translation part fairly easy. @code{t_ar} holds @@ -24130,7 +24286,7 @@ loop goes through @code{from}, one character at a time. For each character in @code{from}, if the character appears in @code{target}, it is replaced with the corresponding @code{to} character. -The @code{translate()} function calls @code{stranslate()} using @code{$0} +The @code{translate()} function calls @code{stranslate()}, using @code{$0} as the target. The main program sets two global variables, @code{FROM} and @code{TO}, from the command line, and then changes @code{ARGV} so that @command{awk} reads from the standard input. @@ -24152,7 +24308,7 @@ Finally, the processing rule simply calls @code{translate()} for each record: @c endfile @end ignore @c file eg/prog/translate.awk -# Bugs: does not handle things like: tr A-Z a-z, it has +# Bugs: does not handle things like tr A-Z a-z; it has # to be spelled out. However, if `to' is shorter than `from', # the last character in `to' is used for the rest of `from'. @@ -24202,9 +24358,9 @@ BEGIN @{ @c endfile @end example -While it is possible to do character transliteration in a user-level -function, it is not necessarily efficient, and we (the @command{gawk} -authors) started to consider adding a built-in function. However, +It is possible to do character transliteration in a user-level +function, but it is not necessarily efficient, and we (the @command{gawk} +developers) started to consider adding a built-in function. However, shortly after writing this program, we learned that Brian Kernighan had added the @code{toupper()} and @code{tolower()} functions to his @command{awk} (@pxref{String Functions}). These functions handle the @@ -24222,17 +24378,13 @@ such as @samp{a-z}, as allowed by the @command{tr} utility. Look at the code for @file{cut.awk} (@pxref{Cut Program}) for inspiration. -@c ENDOFRANGE chtra -@c ENDOFRANGE tr @node Labels Program @subsection Printing Mailing Labels -@c STARTOFRANGE prml @cindex printing, mailing labels -@c STARTOFRANGE mlprint @cindex mailing labels@comma{} printing -Here is a ``real world''@footnote{``Real world'' is defined as +Here is a ``real-world''@footnote{``Real world'' is defined as ``a program actually used to get something done.''} program. This script reads lists of names and @@ -24241,14 +24393,14 @@ on it, two across and 10 down. The addresses are guaranteed to be no more than five lines of data. Each address is separated from the next by a blank line. -The basic idea is to read 20 labels worth of data. Each line of each label +The basic idea is to read 20 labels' worth of data. Each line of each label is stored in the @code{line} array. The single rule takes care of filling the @code{line} array and printing the page when 20 labels have been read. The @code{BEGIN} rule simply sets @code{RS} to the empty string, so that @command{awk} splits records at blank lines (@pxref{Records}). -It sets @code{MAXLINES} to 100, since 100 is the maximum number +It sets @code{MAXLINES} to 100, because 100 is the maximum number of lines on the page @iftex (@math{20 @cdot 5 = 100}). @@ -24264,12 +24416,12 @@ of lines on the page Most of the work is done in the @code{printpage()} function. The label lines are stored sequentially in the @code{line} array. But they -have to print horizontally; @code{line[1]} next to @code{line[6]}, +have to print horizontally: @code{line[1]} next to @code{line[6]}, @code{line[2]} next to @code{line[7]}, and so on. Two loops accomplish this. The outer loop, controlled by @code{i}, steps through every 10 lines of data; this is each row of labels. The inner loop, controlled by @code{j}, goes through the lines within the row. -As @code{j} goes from 0 to 4, @samp{i+j} is the @code{j}-th line in +As @code{j} goes from 0 to 4, @samp{i+j} is the @code{j}th line in the row, and @samp{i+j+5} is the entry next to it. The output ends up looking something like this: @@ -24294,7 +24446,6 @@ that there are two blank lines at the top and two blank lines at the bottom. The @code{END} rule arranges to flush the final page of labels; there may not have been an even multiple of 20 labels in the data: -@c STARTOFRANGE labels @cindex @code{labels.awk} program @example @c file eg/prog/labels.awk @@ -24359,14 +24510,10 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE prml -@c ENDOFRANGE mlprint -@c ENDOFRANGE labels @node Word Sorting @subsection Generating Word-Usage Counts -@c STARTOFRANGE worus @cindex words, usage counts@comma{} generating When working with large amounts of text, it can be interesting to know @@ -24392,8 +24539,8 @@ END @{ @} @end example -The program relies on @command{awk}'s default field splitting -mechanism to break each line up into ``words,'' and uses an +The program relies on @command{awk}'s default field-splitting +mechanism to break each line up into ``words'' and uses an associative array named @code{freq}, indexed by each word, to count the number of times the word occurs. In the @code{END} rule, it prints the counts. @@ -24405,9 +24552,9 @@ useful on real text files: @item The @command{awk} language considers upper- and lowercase characters to be distinct. Therefore, ``bartender'' and ``Bartender'' are not treated -as the same word. This is undesirable, since in normal text, words -are capitalized if they begin sentences, and a frequency analyzer should not -be sensitive to capitalization. +as the same word. This is undesirable, because words are capitalized +if they begin sentences in normal text, and a frequency analyzer should +not be sensitive to capitalization. @item Words are detected using the @command{awk} convention that fields are @@ -24428,7 +24575,6 @@ to remove punctuation characters. Finally, we solve the third problem by using the system @command{sort} utility to process the output of the @command{awk} script. Here is the new version of the program: -@c STARTOFRANGE wordfreq @cindex @code{wordfreq.awk} program @example @c file eg/prog/wordfreq.awk @@ -24449,8 +24595,8 @@ END @{ @} @end example -The regexp @samp{/[^[:alnum:]_[:blank:]]/} might have been written -@samp{/[[:punct:]]/}, but then underscores would also be removed, +The regexp @code{/[^[:alnum:]_[:blank:]]/} might have been written +@code{/[[:punct:]]/}, but then underscores would also be removed, and we want to keep them. Assuming we have saved this program in a file named @file{wordfreq.awk}, @@ -24493,16 +24639,13 @@ This way of sorting must be used on systems that do not have true pipes at the command-line (or batch-file) level. See the general operating system documentation for more information on how to use the @command{sort} program. -@c ENDOFRANGE worus -@c ENDOFRANGE wordfreq @node History Sorting @subsection Removing Duplicates from Unsorted Text -@c STARTOFRANGE lidu @cindex lines, duplicate@comma{} removing The @command{uniq} program -(@pxref{Uniq Program}), +(@pxref{Uniq Program}) removes duplicate lines from @emph{sorted} data. Suppose, however, you need to remove duplicate lines from a @value{DF} but @@ -24524,7 +24667,6 @@ Each element of @code{lines} is a unique command, and the indices of The @code{END} rule simply prints out the lines, in order: @cindex Rakitzis, Byron -@c STARTOFRANGE histsort @cindex @code{histsort.awk} program @example @c file eg/prog/histsort.awk @@ -24567,15 +24709,11 @@ print data[lines[i]], lines[i] @noindent This works because @code{data[$0]} is incremented each time a line is seen. -@c ENDOFRANGE lidu -@c ENDOFRANGE histsort @node Extract Program @subsection Extracting Programs from Texinfo Source Files -@c STARTOFRANGE texse @cindex Texinfo, extracting programs from source files -@c STARTOFRANGE fitex @cindex files, Texinfo@comma{} extracting programs from @ifnotinfo Both this chapter and the previous chapter @@ -24588,13 +24726,13 @@ The nodes and @ref{Sample Programs}, are the top level nodes for a large number of @command{awk} programs. @end ifinfo -If you want to experiment with these programs, it is tedious to have to type +If you want to experiment with these programs, it is tedious to type them in by hand. Here we present a program that can extract parts of a Texinfo input file into separate files. @cindex Texinfo This @value{DOCUMENT} is written in @uref{http://www.gnu.org/software/texinfo/, Texinfo}, -the GNU project's document formatting language. +the GNU Project's document formatting language. A single Texinfo source file can be used to produce both printed documentation, with @TeX{}, and online documentation. @ifnotinfo @@ -24653,7 +24791,7 @@ The Texinfo file looks something like this: @example @dots{} -This program has a @@code@{BEGIN@} rule, +This program has a @@code@{BEGIN@} rule that prints a nice message: @@example @@ -24666,7 +24804,7 @@ It also prints some final advice: @@example @@c file examples/messages.awk -END @@@{ print "Always avoid bored archeologists!" @@@} +END @@@{ print "Always avoid bored archaeologists!" @@@} @@c end file @@end example @dots{} @@ -24679,11 +24817,10 @@ The first rule handles calling @code{system()}, checking that a command is given (@code{NF} is at least three) and also checking that the command exits with a zero exit status, signifying OK: -@c STARTOFRANGE extract @cindex @code{extract.awk} program @example @c file eg/prog/extract.awk -# extract.awk --- extract files and run programs from texinfo files +# extract.awk --- extract files and run programs from Texinfo files @c endfile @ignore @c file eg/prog/extract.awk @@ -24724,12 +24861,12 @@ The second rule handles moving data into files. It verifies that a @value{FN} is given in the directive. If the file named is not the current file, then the current file is closed. Keeping the current file open until a new file is encountered allows the use of the @samp{>} -redirection for printing the contents, keeping open file management +redirection for printing the contents, keeping open-file management simple. The @code{for} loop does the work. It reads lines using @code{getline} (@pxref{Getline}). -For an unexpected end of file, it calls the @code{@w{unexpected_eof()}} +For an unexpected end-of-file, it calls the @code{@w{unexpected_eof()}} function. If the line is an ``endfile'' line, then it breaks out of the loop. If the line is an @samp{@@group} or @samp{@@end group} line, then it @@ -24825,20 +24962,17 @@ END @{ @} @c endfile @end example -@c ENDOFRANGE texse -@c ENDOFRANGE fitex -@c ENDOFRANGE extract @node Simple Sed @subsection A Simple Stream Editor @cindex @command{sed} utility @cindex stream editors -The @command{sed} utility is a stream editor, a program that reads a +The @command{sed} utility is a @dfn{stream editor}, a program that reads a stream of data, makes changes to it, and passes it on. It is often used to make global changes to a large file or to a stream of data generated by a pipeline of commands. -While @command{sed} is a complicated program in its own right, its most common +Although @command{sed} is a complicated program in its own right, its most common use is to perform global substitutions in the middle of a pipeline: @example @@ -24847,7 +24981,7 @@ use is to perform global substitutions in the middle of a pipeline: Here, @samp{s/old/new/g} tells @command{sed} to look for the regexp @samp{old} on each input line and globally replace it with the text -@samp{new}, i.e., all the occurrences on a line. This is similar to +@samp{new} (i.e., all the occurrences on a line). This is similar to @command{awk}'s @code{gsub()} function (@pxref{String Functions}). @@ -24857,7 +24991,6 @@ additional arguments are treated as @value{DF} names to process. If none are provided, the standard input is used: @cindex Brennan, Michael -@c STARTOFRANGE awksed @cindex @command{awksed.awk} program @c @cindex simple stream editor @c @cindex stream editor, simple @@ -24931,17 +25064,14 @@ not treated as @value{FN}s (@pxref{ARGC and ARGV}). The @code{usage()} function prints an error message and exits. -Finally, the single rule handles the printing scheme outlined above, +Finally, the single rule handles the printing scheme outlined earlier, using @code{print} or @code{printf} as appropriate, depending upon the value of @code{RT}. -@c ENDOFRANGE awksed @node Igawk Program @subsection An Easy Way to Use Library Functions -@c STARTOFRANGE libfex @cindex libraries of @command{awk} functions, example program for using -@c STARTOFRANGE flibex @cindex functions, library, example program for using In @ref{Include Files}, we saw how @command{gawk} provides a built-in file-inclusion capability. However, this is a @command{gawk} extension. @@ -24975,15 +25105,15 @@ BEGIN @{ The following program, @file{igawk.sh}, provides this service. It simulates @command{gawk}'s searching of the @env{AWKPATH} variable -and also allows @dfn{nested} includes; i.e., a file that is included -with @code{@@include} can contain further @code{@@include} statements. +and also allows @dfn{nested} includes (i.e., a file that is included +with @code{@@include} can contain further @code{@@include} statements). @command{igawk} makes an effort to only include files once, so that nested includes don't accidentally include a library function twice. @command{igawk} should behave just like @command{gawk} externally. This means it should accept all of @command{gawk}'s command-line arguments, including the ability to have multiple source files specified via -@option{-f}, and the ability to mix command-line and library source files. +@option{-f} and the ability to mix command-line and library source files. The program is written using the POSIX Shell (@command{sh}) command language.@footnote{Fully explaining the @command{sh} language is beyond @@ -25006,10 +25136,10 @@ Literal text, provided with @option{-e} or @option{--source}. This text is just appended directly. @item -Source @value{FN}s, provided with @option{-f}. We use a neat trick and append -@samp{@@include @var{filename}} to the shell variable's contents. Since the file-inclusion -program works the way @command{gawk} does, this gets the text -of the file included into the program at the correct point. +Source @value{FN}s, provided with @option{-f}. We use a neat trick and +append @samp{@@include @var{filename}} to the shell variable's contents. +Because the file-inclusion program works the way @command{gawk} does, this +gets the text of the file included in the program at the correct point. @end enumerate @item @@ -25022,7 +25152,7 @@ Run the expanded program with @command{gawk} and any other original command-line arguments that the user supplied (such as the @value{DF} names). @end enumerate -This program uses shell variables extensively: for storing command-line arguments, +This program uses shell variables extensively: for storing command-line arguments and the text of the @command{awk} program that will expand the user's program, for the user's original program, and for the expanded program. Doing so removes some potential problems that might arise were we to use temporary files instead, @@ -25080,7 +25210,6 @@ program. The program is as follows: -@c STARTOFRANGE igawk @cindex @code{igawk.sh} program @example @c file eg/prog/igawk.sh @@ -25308,9 +25437,10 @@ EOF @c endfile @end example -The shell construct @samp{@var{command} << @var{marker}} is called a @dfn{here document}. -Everything in the shell script up to the @var{marker} is fed to @var{command} as input. -The shell processes the contents of the here document for variable and command substitution +The shell construct @samp{@var{command} << @var{marker}} is called +a @dfn{here document}. Everything in the shell script up to the +@var{marker} is fed to @var{command} as input. The shell processes +the contents of the here document for variable and command substitution (and possibly other things as well, depending upon the shell). The shell construct @samp{$(@dots{})} is called @dfn{command substitution}. @@ -25325,34 +25455,21 @@ It's done in these steps: @enumerate @item Run @command{gawk} with the @code{@@include}-processing program (the -value of the @code{expand_prog} shell variable) on standard input. +value of the @code{expand_prog} shell variable) reading standard input. @item -Standard input is the contents of the user's program, from the shell variable @code{program}. -Its contents are fed to @command{gawk} via a here document. +Standard input is the contents of the user's program, +from the shell variable @code{program}. +Feed its contents to @command{gawk} via a here document. @item -The results of this processing are saved in the shell variable @code{processed_program} by using command substitution. +Save the results of this processing in the shell variable +@code{processed_program} by using command substitution. @end enumerate The last step is to call @command{gawk} with the expanded program, along with the original -options and command-line arguments that the user supplied. - -@c this causes more problems than it solves, so leave it out. -@ignore -The special file @file{/dev/null} is passed as a @value{DF} to @command{gawk} -to handle an interesting case. Suppose that the user's program only has -a @code{BEGIN} rule and there are no @value{DF}s to read. -The program should exit without reading any @value{DF}s. -However, suppose that an included library file defines an @code{END} -rule of its own. In this case, @command{gawk} will hang, reading standard -input. In order to avoid this, @file{/dev/null} is explicitly added to the -command line. Reading from @file{/dev/null} always returns an immediate -end of file indication. - -@c Hmm. Add /dev/null if $# is 0? Still messes up ARGV. Sigh. -@end ignore +options and command-line arguments that the user supplied: @example @c file eg/prog/igawk.sh @@ -25402,13 +25519,9 @@ features to a program; they can often be layered on top.@footnote{@command{gawk} does @code{@@include} processing itself in order to support the use of @command{awk} programs as Web CGI scripts.} -@c ENDOFRANGE libfex -@c ENDOFRANGE flibex -@c ENDOFRANGE awkpex -@c ENDOFRANGE igawk @node Anagram Program -@subsection Finding Anagrams From A Dictionary +@subsection Finding Anagrams from a Dictionary @cindex anagrams, finding An interesting programming challenge is to @@ -25417,24 +25530,23 @@ word list (such as @file{/usr/share/dict/words} on many GNU/Linux systems). One word is an anagram of another if both words contain the same letters -(for example, ``babbling'' and ``blabbing''). +(e.g., ``babbling'' and ``blabbing''). -Column 2, Problem C of Jon Bentley's @cite{Programming Pearls}, second -edition, presents an elegant algorithm. The idea is to give words that +Column 2, Problem C, of Jon Bentley's @cite{Programming Pearls}, Second +Edition, presents an elegant algorithm. The idea is to give words that are anagrams a common signature, sort all the words together by their -signature, and then print them. Dr.@: Bentley observes that taking the -letters in each word and sorting them produces that common signature. +signatures, and then print them. Dr.@: Bentley observes that taking the +letters in each word and sorting them produces those common signatures. The following program uses arrays of arrays to bring together words with the same signature and array sorting to print the words -in sorted order. +in sorted order: -@c STARTOFRANGE anagram @cindex @code{anagram.awk} program @example @c file eg/prog/anagram.awk -# anagram.awk --- An implementation of the anagram finding algorithm -# from Jon Bentley's "Programming Pearls", 2nd edition. +# anagram.awk --- An implementation of the anagram-finding algorithm +# from Jon Bentley's "Programming Pearls," 2nd edition. # Addison Wesley, 2000, ISBN 0-201-65788-0. # Column 2, Problem C, section 2.8, pp 18-20. @c endfile @@ -25482,7 +25594,7 @@ sorts the letters, and then joins them back together: @example @c file eg/prog/anagram.awk -# word2key --- split word apart into letters, sort, joining back together +# word2key --- split word apart into letters, sort, and join back together function word2key(word, a, i, n, result) @{ @@ -25499,7 +25611,7 @@ function word2key(word, a, i, n, result) Finally, the @code{END} rule traverses the array and prints out the anagram lists. It sends the output -to the system @command{sort} command, since otherwise +to the system @command{sort} command because otherwise the anagrams would appear in arbitrary order: @example @@ -25527,21 +25639,20 @@ Here is some partial output when the program is run: @example $ @kbd{gawk -f anagram.awk /usr/share/dict/words | grep '^b'} @dots{} -babbled blabbed -babbler blabber brabble -babblers blabbers brabbles -babbling blabbing -babbly blabby -babel bable -babels beslab -babery yabber +babbled blabbed +babbler blabber brabble +babblers blabbers brabbles +babbling blabbing +babbly blabby +babel bable +babels beslab +babery yabber @dots{} @end example -@c ENDOFRANGE anagram @node Signature Program -@subsection And Now For Something Completely Different +@subsection And Now for Something Completely Different @cindex signature program @cindex Brini, Davide @@ -25581,28 +25692,28 @@ Subject: The GNU Awk User's Guide, Section 13.3.11 From: "Chris Johansen" <johansen@main.nc.us> Message-ID: <op.v0iw6wlv7finx3@asusodin.thrudvang.lan> -Arnold, you don't know me, but we have a tenuous connection. My wife is +Arnold, you don't know me, but we have a tenuous connection. My wife is Barbara A. Field, FAIA, GIT '65 (B. Arch.). -I have had a couple of paper copies of "Effective Awk Programming" for -years, and now I'm going through a Kindle version of "The GNU Awk User's -Guide" again. When I got to section 13.3.11, I reformatted and lightly +I have had a couple of paper copies of "Effective Awk Programming" for +years, and now I'm going through a Kindle version of "The GNU Awk User's +Guide" again. When I got to section 13.3.11, I reformatted and lightly commented Davide Brin's signature script to understand its workings. -It occurs to me that this might have pedagogical value as an example -(although imperfect) of the value of whitespace and comments, and a -starting point for that discussion. It certainly helped _me_ understand -what's going on. You are welcome to it, as-is or modified (subject to +It occurs to me that this might have pedagogical value as an example +(although imperfect) of the value of whitespace and comments, and a +starting point for that discussion. It certainly helped _me_ understand +what's going on. You are welcome to it, as-is or modified (subject to Davide's constraints, of course, which I think I have met). -If I were to include it in a future edition, I would put it at some -distance from section 13.3.11, say, as a note or an appendix, so as not to +If I were to include it in a future edition, I would put it at some +distance from section 13.3.11, say, as a note or an appendix, so as not to be a "spoiler" to the puzzle. Best regards, --- +-- Chris Johansen {johansen at main dot nc dot us} - . . . collapsing the probability wave function, sending ripples of + . . . collapsing the probability wave function, sending ripples of certainty through the space-time continuum. @@ -25611,7 +25722,7 @@ certainty through the space-time continuum. # From "13.3.11 And Now For Something Completely Different" # http://www.gnu.org/software/gawk/manual/html_node/Signature-Program.html#Signature-Program -# Copyright © 2008 Davide Brini +# Copyright © 2008 Davide Brini # Copying and distribution of the code published in this page, with # or without modification, are permitted in any medium without @@ -25678,12 +25789,13 @@ characters. The ability to use @code{split()} with the empty string as the separator can considerably simplify such tasks. @item -The library functions from @ref{Library Functions}, proved their -usefulness for a number of real (if small) programs. +The examples here demonstrate the usefulness of the library +functions from @DBREF{Library Functions} +for a number of real (if small) programs. @item Besides reinventing POSIX wheels, other programs solved a selection of -interesting problems, such as finding duplicates words in text, printing +interesting problems, such as finding duplicate words in text, printing mailing labels, and finding anagrams. @end itemize @@ -25730,7 +25842,7 @@ Brian Kernighan suggests that ``an alternative approach to state machines is to just read the input into an array, then use indexing. It's almost always easier code, and for most inputs where you would use this, just -as fast.'' Rewrite the logic to follow this +as fast.'' Rewrite the logic to follow this suggestion. @@ -25831,7 +25943,7 @@ the use of the external @command{sort} utility. @c EXCLUDE END @ifnotinfo -@part @value{PART3}Moving Beyond Standard @command{awk} With @command{gawk} +@part @value{PART3}Moving Beyond Standard @command{awk} with @command{gawk} @end ifnotinfo @ifdocbook @@ -25840,27 +25952,25 @@ It contains the following chapters: @itemize @value{BULLET} @item -@ref{Advanced Features}. +@ref{Advanced Features} @item -@ref{Internationalization}. +@ref{Internationalization} @item -@ref{Debugger}. +@ref{Debugger} @item -@ref{Arbitrary Precision Arithmetic}. +@ref{Arbitrary Precision Arithmetic} @item -@ref{Dynamic Extensions}. +@ref{Dynamic Extensions} @end itemize @end ifdocbook @node Advanced Features @chapter Advanced Features of @command{gawk} -@c STARTOFRANGE gawadv @cindex @command{gawk}, features, advanced -@c STARTOFRANGE advgaw @cindex advanced features, @command{gawk} @ignore Contributed by: Peter Langston <pud!psl@bellcore.bellcore.com> @@ -25881,18 +25991,18 @@ a violent psychopath who knows where you live.} This @value{CHAPTER} discusses advanced features in @command{gawk}. It's a bit of a ``grab bag'' of items that are otherwise unrelated to each other. -First, a command-line option allows @command{gawk} to recognize +First, we look at a command-line option that allows @command{gawk} to recognize nondecimal numbers in input data, not just in @command{awk} programs. Then, @command{gawk}'s special features for sorting arrays are presented. Next, two-way I/O, discussed briefly in earlier parts of this @value{DOCUMENT}, is described in full detail, along with the basics -of TCP/IP networking. Finally, @command{gawk} +of TCP/IP networking. Finally, we see how @command{gawk} can @dfn{profile} an @command{awk} program, making it possible to tune it for performance. @c FULLXREF ON -A number of advanced features require separate @value{CHAPTER}s of their +Additional advanced features are discussed in separate @value{CHAPTER}s of their own: @itemize @value{BULLET} @@ -25986,7 +26096,8 @@ This option may disappear in a future version of @command{gawk}. @node Array Sorting @section Controlling Array Traversal and Array Sorting -@command{gawk} lets you control the order in which a @samp{for (i in array)} +@command{gawk} lets you control the order in which a +@samp{for (@var{indx} in @var{array})} loop traverses an array. In addition, two built-in functions, @code{asort()} and @code{asorti()}, @@ -26002,7 +26113,7 @@ to order the elements during sorting. @node Controlling Array Traversal @subsection Controlling Array Traversal -By default, the order in which a @samp{for (i in array)} loop +By default, the order in which a @samp{for (@var{indx} in @var{array})} loop scans an array is not defined; it is generally based upon the internal implementation of arrays inside @command{awk}. @@ -26011,7 +26122,7 @@ in a particular order that you, the programmer, choose. @command{gawk} lets you do this. @DBREF{Controlling Scanning} describes how you can assign special, -pre-defined values to @code{PROCINFO["sorted_in"]} in order to +predefined values to @code{PROCINFO["sorted_in"]} in order to control the order in which @command{gawk} traverses an array during a @code{for} loop. @@ -26031,23 +26142,23 @@ function comp_func(i1, v1, i2, v2) @} @end example -Here, @var{i1} and @var{i2} are the indices, and @var{v1} and @var{v2} +Here, @code{i1} and @code{i2} are the indices, and @code{v1} and @code{v2} are the corresponding values of the two elements being compared. -Either @var{v1} or @var{v2}, or both, can be arrays if the array being +Either @code{v1} or @code{v2}, or both, can be arrays if the array being traversed contains subarrays as values. -(@xref{Arrays of Arrays}, for more information about subarrays.) +(@DBXREF{Arrays of Arrays} for more information about subarrays.) The three possible return values are interpreted as follows: @table @code @item comp_func(i1, v1, i2, v2) < 0 -Index @var{i1} comes before index @var{i2} during loop traversal. +Index @code{i1} comes before index @code{i2} during loop traversal. @item comp_func(i1, v1, i2, v2) == 0 -Indices @var{i1} and @var{i2} -come together but the relative order with respect to each other is undefined. +Indices @code{i1} and @code{i2} +come together, but the relative order with respect to each other is undefined. @item comp_func(i1, v1, i2, v2) > 0 -Index @var{i1} comes after index @var{i2} during loop traversal. +Index @code{i1} comes after index @code{i2} during loop traversal. @end table Our first comparison function can be used to scan an array in @@ -26078,7 +26189,7 @@ function cmp_str_val(i1, v1, i2, v2) The third comparison function makes all numbers, and numeric strings without -any leading or trailing spaces, come out first during loop traversal: +any leading or trailing spaces, come out first during loop traversal: @example function cmp_num_str_val(i1, v1, i2, v2, n1, n2) @@ -26086,10 +26197,10 @@ function cmp_num_str_val(i1, v1, i2, v2, n1, n2) # numbers before string value comparison, ascending order n1 = v1 + 0 n2 = v2 + 0 - if (n1 == v1) + if (n1 == v1) return (n2 == v2) ? (n1 - n2) : -1 else if (n2 == v2) - return 1 + return 1 return (v1 < v2) ? -1 : (v1 != v2) @} @end example @@ -26104,7 +26215,7 @@ BEGIN @{ data[10] = "one" data[100] = 100 data[20] = "two" - + f[1] = "cmp_num_idx" f[2] = "cmp_str_val" f[3] = "cmp_num_str_val" @@ -26128,14 +26239,14 @@ $ @kbd{gawk -f compdemo.awk} @print{} data[10] = one @print{} data[20] = two @print{} data[100] = 100 -@print{} +@print{} @print{} Sort function: cmp_str_val @ii{Sort by element values as strings} @print{} data[one] = 10 @print{} data[100] = 100 @ii{String 100 is less than string 20} @print{} data[two] = 20 @print{} data[10] = one @print{} data[20] = two -@print{} +@print{} @print{} Sort function: cmp_num_str_val @ii{Sort all numeric values before all strings} @print{} data[one] = 10 @print{} data[two] = 20 @@ -26146,7 +26257,7 @@ $ @kbd{gawk -f compdemo.awk} Consider sorting the entries of a GNU/Linux system password file according to login name. The following program sorts records -by a specific field position and can be used for this purpose: +by a specific field position and can be used for this purpose: @example # passwd-sort.awk --- simple program to sort by field position @@ -26192,7 +26303,7 @@ $ @kbd{gawk -v POS=1 -F: -f sort.awk /etc/passwd} The comparison should normally always return the same value when given a specific pair of array elements as its arguments. If inconsistent -results are returned then the order is undefined. This behavior can be +results are returned, then the order is undefined. This behavior can be exploited to introduce random order into otherwise seemingly ordered data: @@ -26204,11 +26315,11 @@ function cmp_randomize(i1, v1, i2, v2) @} @end example -As mentioned above, the order of the indices is arbitrary if two +As already mentioned, the order of the indices is arbitrary if two elements compare equal. This is usually not a problem, but letting the tied elements come out in arbitrary order can be an issue, especially when comparing item values. The partial ordering of the equal elements -may change the next time the array is traversed, if other elements are added or +may change the next time the array is traversed, if other elements are added to or removed from the array. One way to resolve ties when comparing elements with otherwise equal values is to include the indices in the comparison rules. Note that doing this may make the loop traversal less efficient, @@ -26245,21 +26356,21 @@ When string comparisons are made during a sort, either for element values where one or both aren't numbers, or for element indices handled as strings, the value of @code{IGNORECASE} (@pxref{Built-in Variables}) controls whether -the comparisons treat corresponding uppercase and lowercase letters as +the comparisons treat corresponding upper- and lowercase letters as equivalent or distinct. -Another point to keep in mind is that in the case of subarrays +Another point to keep in mind is that in the case of subarrays, the element values can themselves be arrays; a production comparison function should use the @code{isarray()} function -(@pxref{Type Functions}), +(@pxref{Type Functions}) to check for this, and choose a defined sorting order for subarrays. All sorting based on @code{PROCINFO["sorted_in"]} is disabled in POSIX mode, -since the @code{PROCINFO} array is not special in that case. +because the @code{PROCINFO} array is not special in that case. As a side note, sorting the array indices before traversing -the array has been reported to add 15% to 20% overhead to the +the array has been reported to add a 15% to 20% overhead to the execution time of @command{awk} programs. For this reason, sorted array traversal is not the default. @@ -26277,8 +26388,8 @@ sorted array traversal is not the default. @cindex @code{asorti()} function (@command{gawk}), arrays@comma{} sorting @cindex sort function, arrays, sorting In most @command{awk} implementations, sorting an array requires writing -a @code{sort()} function. While this can be educational for exploring -different sorting algorithms, usually that's not the point of the program. +a @code{sort()} function. This can be educational for exploring +different sorting algorithms, but usually that's not the point of the program. @command{gawk} provides the built-in @code{asort()} and @code{asorti()} functions (@pxref{String Functions}) for sorting arrays. For example: @@ -26318,7 +26429,7 @@ However, the @code{source} array is not affected. Often, what's needed is to sort on the values of the @emph{indices} instead of the values of the elements. To do that, use the @code{asorti()} function. The interface and behavior are identical to -that of @code{asort()}, except that the index values are used for sorting, +that of @code{asort()}, except that the index values are used for sorting and become the values of the result array: @example @@ -26353,8 +26464,8 @@ it chooses}, taking into account just the indices, just the values, or both. This is extremely powerful. Once the array is sorted, @code{asort()} takes the @emph{values} in -their final order, and uses them to fill in the result array, whereas -@code{asorti()} takes the @emph{indices} in their final order, and uses +their final order and uses them to fill in the result array, whereas +@code{asorti()} takes the @emph{indices} in their final order and uses them to fill in the result array. @cindex reference counting, sorting arrays @@ -26374,8 +26485,8 @@ Because @code{IGNORECASE} affects string comparisons, the value of @code{IGNORECASE} also affects sorting for both @code{asort()} and @code{asorti()}. Note also that the locale's sorting order does @emph{not} come into play; comparisons are based on character values only.@footnote{This -is true because locale-based comparison occurs only when in POSIX -compatibility mode, and since @code{asort()} and @code{asorti()} are +is true because locale-based comparison occurs only when in +POSIX-compatibility mode, and because @code{asort()} and @code{asorti()} are @command{gawk} extensions, they are not available in that case.} @node Two-way I/O @@ -26451,7 +26562,7 @@ remain more difficult to use than two-way pipes.} @c 8/2014 @cindex @command{csh} utility, @code{|&} operator, comparison with However, with @command{gawk}, it is possible to open a @emph{two-way} pipe to another process. The second process is -termed a @dfn{coprocess}, since it runs in parallel with @command{gawk}. +termed a @dfn{coprocess}, as it runs in parallel with @command{gawk}. The two-way connection is created using the @samp{|&} operator (borrowed from the Korn shell, @command{ksh}):@footnote{This is very different from the same operator in the C shell and in Bash.} @@ -26556,7 +26667,7 @@ like so: @example command = "sort -nr" # command, save in convenience variable PROCINFO[command, "pty"] = 1 # update PROCINFO -print @dots{} |& command # start two-way pipe +print @dots{} |& command # start two-way pipe @dots{} @end example @@ -26570,7 +26681,6 @@ using regular pipes. @section Using @command{gawk} for Network Programming @cindex advanced features, network programming @cindex networks, programming -@c STARTOFRANGE tcpip @cindex TCP/IP @cindex @code{/inet/@dots{}} special files (@command{gawk}) @cindex files, @code{/inet/@dots{}} (@command{gawk}) @@ -26611,7 +26721,7 @@ You can think of this as just a @emph{very long} two-way pipeline to a coprocess. The way @command{gawk} decides that you want to use TCP/IP networking is by recognizing special @value{FN}s that begin with one of @samp{/inet/}, -@samp{/inet4/} or @samp{/inet6/}. +@samp{/inet4/}, or @samp{/inet6/}. The full syntax of the special @value{FN} is @file{/@var{net-type}/@var{protocol}/@var{local-port}/@var{remote-host}/@var{remote-port}}. @@ -26640,7 +26750,7 @@ or @samp{http}, in which case @command{gawk} attempts to determine the predefined port number using the C @code{getaddrinfo()} function. @item remote-host -The IP address or fully-qualified domain name of the Internet +The IP address or fully qualified domain name of the Internet host to which you want to connect. @item remote-port @@ -26652,7 +26762,7 @@ service name. @cindex @command{gawk}, @code{ERRNO} variable in @cindex @code{ERRNO} variable @quotation NOTE -Failure in opening a two-way socket will result in a non-fatal error +Failure in opening a two-way socket will result in a nonfatal error being returned to the calling code. The value of @code{ERRNO} indicates the error (@pxref{Auto-set}). @end quotation @@ -26669,31 +26779,28 @@ BEGIN @{ @end example This program reads the current date and time from the local system's -TCP @samp{daytime} server. +TCP @code{daytime} server. It then prints the results and closes the connection. Because this topic is extensive, the use of @command{gawk} for TCP/IP programming is documented separately. @ifinfo See -@inforef{Top, , General Introduction, gawkinet, TCP/IP Internetworking with @command{gawk}}, +@inforef{Top, , General Introduction, gawkinet, @value{GAWKINETTITLE}}, @end ifinfo @ifnotinfo See @uref{http://www.gnu.org/software/gawk/manual/gawkinet/, -@cite{TCP/IP Internetworking with @command{gawk}}}, +@cite{@value{GAWKINETTITLE}}}, which comes as part of the @command{gawk} distribution, @end ifnotinfo for a much more complete introduction and discussion, as well as extensive examples. -@c ENDOFRANGE tcpip @node Profiling @section Profiling Your @command{awk} Programs -@c STARTOFRANGE awkp @cindex @command{awk} programs, profiling -@c STARTOFRANGE proawk @cindex profiling @command{awk} programs @cindex @code{awkprof.out} file @cindex files, @code{awkprof.out} @@ -26714,12 +26821,12 @@ gawk --profile=myprog.prof -f myprog.awk data1 data2 @end example @noindent -In the above example, @command{gawk} places the profile in +In the preceding example, @command{gawk} places the profile in @file{myprog.prof} instead of in @file{awkprof.out}. -Here is a sample session showing a simple @command{awk} program, its input data, and the -results from running @command{gawk} with the @option{--profile} option. -First, the @command{awk} program: +Here is a sample session showing a simple @command{awk} program, +its input data, and the results from running @command{gawk} with the +@option{--profile} option. First, the @command{awk} program: @example BEGIN @{ print "First BEGIN rule" @} @@ -26760,9 +26867,9 @@ junk @end example Here is the @file{awkprof.out} that results from running the -@command{gawk} profiler on this program and data. (This example also +@command{gawk} profiler on this program and data (this example also illustrates that @command{awk} programmers sometimes get up very early -in the morning to work.) +in the morning to work): @cindex @code{BEGIN} pattern, and profiling @cindex @code{END} pattern, and profiling @@ -26822,8 +26929,8 @@ They are as follows: @item The program is printed in the order @code{BEGIN} rules, @code{BEGINFILE} rules, -pattern/action rules, -@code{ENDFILE} rules, @code{END} rules and functions, listed +pattern--action rules, +@code{ENDFILE} rules, @code{END} rules, and functions, listed alphabetically. Multiple @code{BEGIN} and @code{END} rules retain their separate identities, as do @@ -26831,7 +26938,7 @@ multiple @code{BEGINFILE} and @code{ENDFILE} rules. @cindex patterns, counts, in a profile @item -Pattern-action rules have two counts. +Pattern--action rules have two counts. The first count, to the left of the rule, shows how many times the rule's pattern was @emph{tested}. The second count, to the right of the rule's opening left brace @@ -26877,7 +26984,7 @@ the body of an @code{if}, @code{else}, or loop is only a single statement. @item Parentheses are used only where needed, as indicated by the structure of the program and the precedence rules. -For example, @samp{(3 + 5) * 4} means add three plus five, then multiply +For example, @samp{(3 + 5) * 4} means add three and five, then multiply the total by four. However, @samp{3 + 5 * 4} has no parentheses, and means @samp{3 + (5 * 4)}. @@ -26898,13 +27005,13 @@ the target of a redirection isn't a scalar, it gets parenthesized. @command{gawk} supplies leading comments in front of the @code{BEGIN} and @code{END} rules, the @code{BEGINFILE} and @code{ENDFILE} rules, -the pattern/action rules, and the functions. +the pattern--action rules, and the functions. @end itemize The profiled version of your program may not look exactly like what you typed when you wrote it. This is because @command{gawk} creates the -profiled version by ``pretty printing'' its internal representation of +profiled version by ``pretty-printing'' its internal representation of the program. The advantage to this is that @command{gawk} can produce a standard representation. Also, things such as: @@ -26987,16 +27094,16 @@ If you use the @code{HUP} signal instead of the @code{USR1} signal, @cindex @code{SIGQUIT} signal (MS-Windows) @cindex signals, @code{QUIT}/@code{SIGQUIT} (MS-Windows) When @command{gawk} runs on MS-Windows systems, it uses the -@code{INT} and @code{QUIT} signals for producing the profile and, in +@code{INT} and @code{QUIT} signals for producing the profile, and in the case of the @code{INT} signal, @command{gawk} exits. This is because these systems don't support the @command{kill} command, so the only signals you can deliver to a program are those generated by the keyboard. The @code{INT} signal is generated by the -@kbd{Ctrl-@key{C}} or @kbd{Ctrl-@key{BREAK}} key, while the -@code{QUIT} signal is generated by the @kbd{Ctrl-@key{\}} key. +@kbd{Ctrl-c} or @kbd{Ctrl-BREAK} key, while the +@code{QUIT} signal is generated by the @kbd{Ctrl-\} key. Finally, @command{gawk} also accepts another option, @option{--pretty-print}. -When called this way, @command{gawk} ``pretty prints'' the program into +When called this way, @command{gawk} ``pretty-prints'' the program into @file{awkprof.out}, without any execution counts. @quotation NOTE @@ -27020,9 +27127,6 @@ that the profiling output does. This makes it easy to pretty-print your code once development is completed, and then use the result as the final version of your program. -@c ENDOFRANGE awkp -@c ENDOFRANGE proawk - @node Advanced Features Summary @section Summary @@ -27053,7 +27157,7 @@ optionally, close off one side of the two-way communications. @item By using special @value{FN}s with the @samp{|&} operator, you can open a -TCP/IP (or UDP/IP) connection to remote hosts in the Internet. @command{gawk} +TCP/IP (or UDP/IP) connection to remote hosts on the Internet. @command{gawk} supports both IPv4 and IPv6. @item @@ -27063,13 +27167,11 @@ you tune them more easily. Sending the @code{USR1} signal while profiling cause @command{gawk} to dump the profile and keep going, including a function call stack. @item -You can also just ``pretty print'' the program. This currently also runs +You can also just ``pretty-print'' the program. This currently also runs the program, but that will change in the next major release. @end itemize -@c ENDOFRANGE advgaw -@c ENDOFRANGE gawadv @node Internationalization @chapter Internationalization with @command{gawk} @@ -27082,7 +27184,6 @@ countries, they were able to sell more systems. As a result, internationalization and localization of programs and software systems became a common practice. -@c STARTOFRANGE inloc @cindex internationalization, localization @cindex @command{gawk}, internationalization and, See internationalization @cindex internationalization, localization, @command{gawk} and @@ -27115,7 +27216,7 @@ a requirement. @cindex localization @dfn{Internationalization} means writing (or modifying) a program once, in such a way that it can use multiple languages without requiring -further source-code changes. +further source code changes. @dfn{Localization} means providing the data necessary for an internationalized program to work in a particular language. Most typically, these terms refer to features such as the language @@ -27127,11 +27228,10 @@ monetary values are printed and read. @section GNU @command{gettext} @cindex internationalizing a program -@c STARTOFRANGE gettex @cindex @command{gettext} library @command{gawk} uses GNU @command{gettext} to provide its internationalization features. -The facilities in GNU @command{gettext} focus on messages; strings printed +The facilities in GNU @command{gettext} focus on messages: strings printed by a program, either directly or via formatting with @code{printf} or @code{sprintf()}.@footnote{For some operating systems, the @command{gawk} port doesn't support GNU @command{gettext}. @@ -27152,8 +27252,7 @@ following steps, in this order: @enumerate @item -The programmer goes -through the source for all of @command{guide}'s components +The programmer reviews the source for all of @command{guide}'s components and marks each string that is a candidate for translation. For example, @code{"`-F': option required"} is a good candidate for translation. A table with strings of option names is not (e.g., @command{gawk}'s @@ -27180,7 +27279,6 @@ lookup of the translations. @cindex @code{.po} files @cindex files, @code{.po} -@c STARTOFRANGE portobfi @cindex portable object files @cindex files, portable object @item @@ -27192,7 +27290,6 @@ For example, there might be a @file{fr.po} for a French translation. @cindex @code{.gmo} files @cindex files, @code{.gmo} @cindex message object files -@c STARTOFRANGE portmsgfi @cindex files, message object @item Each language's @file{.po} file is converted into a binary @@ -27273,8 +27370,8 @@ if necessary. (It is almost never necessary to supply a different category.) @cindex sorting characters in different languages @cindex @code{LC_COLLATE} locale category @item LC_COLLATE -Text-collation information; i.e., how different characters -and/or groups of characters sort in a given language. +Text-collation information (i.e., how different characters +and/or groups of characters sort in a given language). @cindex @code{LC_CTYPE} locale category @item LC_CTYPE @@ -27320,14 +27417,12 @@ before or after the day in a date, local month abbreviations, and so on. @item LC_ALL All of the above. (Not too useful in the context of @command{gettext}.) @end table -@c ENDOFRANGE gettex @node Programmer i18n @section Internationalizing @command{awk} Programs -@c STARTOFRANGE inap @cindex @command{awk} programs, internationalizing -@command{gawk} provides the following variables and functions for +@command{gawk} provides the following variables for internationalization: @table @code @@ -27343,7 +27438,12 @@ value is @code{"messages"}. String constants marked with a leading underscore are candidates for translation at runtime. String constants without a leading underscore are not translated. +@end table +@command{gawk} provides the following functions for +internationalization: + +@table @code @cindexgawkfunc{dcgettext} @item @code{dcgettext(@var{string}} [@code{,} @var{domain} [@code{,} @var{category}]]@code{)} Return the translation of @var{string} in @@ -27400,15 +27500,7 @@ If @var{directory} is the null string (@code{""}), then given @var{domain}. @end table -To use these facilities in your @command{awk} program, follow the steps -outlined in -@ifnotinfo -the previous @value{SECTION}, -@end ifnotinfo -@ifinfo -@ref{Explaining gettext}, -@end ifinfo -like so: +To use these facilities in your @command{awk} program, follow these steps: @enumerate @cindex @code{BEGIN} pattern, @code{TEXTDOMAIN} variable and @@ -27493,7 +27585,7 @@ BEGIN @{ @end enumerate -@xref{I18N Example}, +@DBXREF{I18N Example} for an example program showing the steps to create and use translations from @command{awk}. @@ -27554,11 +27646,9 @@ second argument to @code{dcngettext()}.@footnote{The You should distribute the generated @file{.pot} file with your @command{awk} program; translators will eventually use it to provide you translations that you can also then distribute. -@xref{I18N Example}, +@DBXREF{I18N Example} for the full list of steps to go through to create and test translations for @command{guide}. -@c ENDOFRANGE portobfi -@c ENDOFRANGE portmsgfi @node Printf Ordering @subsection Rearranging @code{printf} Arguments @@ -27682,7 +27772,7 @@ change: @cindex @code{TEXTDOMAIN} variable, portability and @item Assignments to @code{TEXTDOMAIN} won't have any effect, -since @code{TEXTDOMAIN} is not special in other @command{awk} implementations. +because @code{TEXTDOMAIN} is not special in other @command{awk} implementations. @item Non-GNU versions of @command{awk} treat marked strings @@ -27693,7 +27783,7 @@ the null string (@code{""}) as its value, leaving the original string constant a the result. @item -By defining ``dummy'' functions to replace @code{dcgettext()}, @code{dcngettext()} +By defining ``dummy'' functions to replace @code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()}, the @command{awk} program can be made to run, but all the messages are output in the original language. For example: @@ -27730,11 +27820,10 @@ enough arguments are supplied in the function call. Many versions of underlying C library version of @code{sprintf()}, but only one format and argument at a time. What happens if a positional specification is used is anybody's guess. -However, since the positional specifications are primarily for use in -@emph{translated} format strings, and since non-GNU @command{awk}s never +However, because the positional specifications are primarily for use in +@emph{translated} format strings, and because non-GNU @command{awk}s never retrieve the translated string, this should not be a problem in practice. @end itemize -@c ENDOFRANGE inap @node I18N Example @section A Simple Internationalization Example @@ -27794,7 +27883,7 @@ called ``Hippy.'' Ah, well.} @example @group -$ cp guide.pot guide-mellow.po +$ @kbd{cp guide.pot guide-mellow.po} @var{Add translations to} guide-mellow.po @dots{} @end group @end example @@ -27820,7 +27909,7 @@ msgstr "Like, the scoop is" The next step is to make the directory to hold the binary message object file and then to create the @file{guide.mo} file. We pretend that our file is to be used in the @code{en_US.UTF-8} locale, -since we have to use a locale name known to the C @command{gettext} routines. +because we have to use a locale name known to the C @command{gettext} routines. The directory layout shown here is standard for GNU @command{gettext} on GNU/Linux systems. Other versions of @command{gettext} may use a different layout: @@ -27857,7 +27946,7 @@ $ @kbd{gawk -f guide.awk} @print{} Pardon me, Zaphod who? @end example -If the three replacement functions for @code{dcgettext()}, @code{dcngettext()} +If the three replacement functions for @code{dcgettext()}, @code{dcngettext()}, and @code{bindtextdomain()} (@pxref{I18N Portability}) are in a file named @file{libintl.awk}, @@ -27878,15 +27967,15 @@ using the GNU @command{gettext} package. (GNU @command{gettext} is described in complete detail in @ifinfo -@inforef{Top, , GNU @command{gettext} utilities, gettext, GNU gettext tools}.) +@inforef{Top, , GNU @command{gettext} utilities, gettext, GNU @command{gettext} utilities}.) @end ifinfo @ifnotinfo @uref{http://www.gnu.org/software/gettext/manual/, -@cite{GNU gettext tools}}.) +@cite{GNU @command{gettext} utilities}}.) @end ifnotinfo As of this writing, the latest version of GNU @command{gettext} is -@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.2.tar.gz, -@value{PVERSION} 0.19.2}. +@uref{ftp://ftp.gnu.org/gnu/gettext/gettext-0.19.4.tar.gz, +@value{PVERSION} 0.19.4}. If a translation of @command{gawk}'s messages exists, then @command{gawk} produces usage messages, warnings, @@ -27898,7 +27987,7 @@ and fatal errors in the local language. @itemize @value{BULLET} @item Internationalization means writing a program such that it can use multiple -languages without requiring source-code changes. Localization means +languages without requiring source code changes. Localization means providing the data necessary for an internationalized program to work in a particular language. @@ -27915,9 +28004,9 @@ file, and the @file{.po} files are compiled into @file{.gmo} files for use at runtime. @item -You can use position specifications with @code{sprintf()} and +You can use positional specifications with @code{sprintf()} and @code{printf} to rearrange the placement of argument values in formatted -strings and output. This is useful for the translations of format +strings and output. This is useful for the translation of format control strings. @item @@ -27930,7 +28019,6 @@ a number of translations for its messages. @end itemize -@c ENDOFRANGE inloc @node Debugger @chapter Debugging @command{awk} Programs @@ -27959,7 +28047,7 @@ how to use @command{gawk} for debugging your program is easy. @end menu @node Debugging -@section Introduction to The @command{gawk} Debugger +@section Introduction to the @command{gawk} Debugger This @value{SECTION} introduces debugging in general and begins the discussion of debugging in @command{gawk}. @@ -27974,11 +28062,10 @@ the discussion of debugging in @command{gawk}. @subsection Debugging in General (If you have used debuggers in other languages, you may want to skip -ahead to the next section on the specific features of the @command{gawk} -debugger.) +ahead to @ref{Awk Debugging}.) -Of course, a debugging program cannot remove bugs for you, since it has -no way of knowing what you or your users consider a ``bug'' and what is a +Of course, a debugging program cannot remove bugs for you, because it has +no way of knowing what you or your users consider a ``bug'' versus a ``feature.'' (Sometimes, we humans have a hard time with this ourselves.) In that case, what can you expect from such a tool? The answer to that depends on the language being debugged, but in general, you can expect at @@ -27999,7 +28086,7 @@ having to change your source files. @item The chance to see the values of data in the program at any point in execution, and also to change that data on the fly, to see how that -affects what happens afterwards. (This often includes the ability +affects what happens afterward. (This often includes the ability to look at internal data structures besides the variables you actually defined in your code.) @@ -28019,11 +28106,11 @@ functional program that you or someone else wrote). Before diving in to the details, we need to introduce several important concepts that apply to just about all debuggers. The following list defines terms used throughout the rest of -this @value{CHAPTER}. +this @value{CHAPTER}: @table @dfn @cindex stack frame -@item Stack Frame +@item Stack frame Programs generally call functions during the course of their execution. One function can call another, or a function can call itself (recursion). You can view the chain of called functions (main program calls A, which @@ -28058,7 +28145,7 @@ as many breakpoints as you like. A watchpoint is similar to a breakpoint. The difference is that breakpoints are oriented around the code: stop when a certain point in the code is reached. A watchpoint, however, specifies that program execution -should stop when a @emph{data value} is changed. This is useful, since +should stop when a @emph{data value} is changed. This is useful, as sometimes it happens that a variable receives an erroneous value, and it's hard to track down where this happens just by looking at the code. By using a watchpoint, you can stop whenever a variable is assigned to, @@ -28066,26 +28153,26 @@ and usually find the errant code quite quickly. @end table @node Awk Debugging -@subsection Awk Debugging +@subsection @command{awk} Debugging Debugging an @command{awk} program has some specific aspects that are -not shared with other programming languages. +not shared with programs written in other languages. First of all, the fact that @command{awk} programs usually take input -line-by-line from a file or files and operate on those lines using specific +line by line from a file or files and operate on those lines using specific rules makes it especially useful to organize viewing the execution of the program in terms of these rules. As we will see, each @command{awk} rule is treated almost like a function call, with its own specific block of instructions. -In addition, since @command{awk} is by design a very concise language, +In addition, because @command{awk} is by design a very concise language, it is easy to lose sight of everything that is going on ``inside'' each line of @command{awk} code. The debugger provides the opportunity to look at the individual primitive instructions carried out by the higher-level @command{awk} commands. @node Sample Debugging Session -@section Sample Debugging Session +@section Sample @command{gawk} Debugging Session @cindex sample debugging session In order to illustrate the use of @command{gawk} as a debugger, let's look at a sample @@ -28104,8 +28191,8 @@ as our example. @cindex debugger, how to start Starting the debugger is almost exactly like running @command{gawk} normally, -except you have to pass an additional option @option{--debug}, or the -corresponding short option @option{-D}. The file(s) containing the +except you have to pass an additional option, @option{--debug}, or the +corresponding short option, @option{-D}. The file(s) containing the program and any supporting code are given on the command line as arguments to one or more @option{-f} options. (@command{gawk} is not designed to debug command-line programs, only programs contained in files.) @@ -28118,7 +28205,7 @@ $ @kbd{gawk -D -f getopt.awk -f join.awk -f uniq.awk -1 inputfile} @noindent where both @file{getopt.awk} and @file{uniq.awk} are in @env{$AWKPATH}. (Experienced users of GDB or similar debuggers should note that -this syntax is slightly different from what they are used to. +this syntax is slightly different from what you are used to. With the @command{gawk} debugger, you give the arguments for running the program in the command line to the debugger rather than as part of the @code{run} command at the debugger prompt.) @@ -28205,7 +28292,7 @@ gawk> @kbd{bt} @end example This tells us that @code{are_equal()} was called by the main program at -line 88 of @file{uniq.awk}. (This is not a big surprise, since this +line 88 of @file{uniq.awk}. (This is not a big surprise, because this is the only call to @code{are_equal()} in the program, but in more complex programs, knowing who called a function and with what parameters can be the key to finding the source of the problem.) @@ -28222,7 +28309,7 @@ gawk> @kbd{p n} @end example @noindent -In this case, @code{n} is an uninitialized local variable, since the +In this case, @code{n} is an uninitialized local variable, because the function was called without arguments (@pxref{Function Calls}). A more useful variable to display might be the current record: @@ -28233,8 +28320,8 @@ gawk> @kbd{p $0} @end example @noindent -This might be a bit puzzling at first since this is the second line of -our test input above. Let's look at @code{NR}: +This might be a bit puzzling at first, as this is the second line of +our test input. Let's look at @code{NR}: @example gawk> @kbd{p NR} @@ -28272,10 +28359,10 @@ gawk> @kbd{n} @end example This tells us that @command{gawk} is now ready to execute line 66, which -decides whether to give the lines the special ``field skipping'' treatment +decides whether to give the lines the special ``field-skipping'' treatment indicated by the @option{-1} command-line option. (Notice that we skipped -from where we were before at line 63 to here, since the condition in line 63 -@samp{if (fcount == 0 && charcount == 0)} was false.) +from where we were before, at line 63, to here, because the condition +in line 63, @samp{if (fcount == 0 && charcount == 0)}, was false.) Continuing to step, we now get to the splitting of the current and last records: @@ -28304,7 +28391,7 @@ gawk> @kbd{p n m alast aline} This is kind of disappointing, though. All we found out is that there are five elements in @code{alast}; @code{m} and @code{aline} don't have -values since we are at line 68 but haven't executed it yet. +values because we are at line 68 but haven't executed it yet. This information is useful enough (we now know that none of the words were accidentally left out), but what if we want to see inside the array? @@ -28349,7 +28436,7 @@ gawk> @kbd{n} Well, here we are at our error (sorry to spoil the suspense). What we had in mind was to join the fields starting from the second one to make -the virtual record to compare, and if the first field was numbered zero, +the virtual record to compare, and if the first field were numbered zero, this would work. Let's look at what we've got: @example @@ -28358,7 +28445,7 @@ gawk> @kbd{p cline clast} @print{} clast = "awk is a wonderful program!" @end example -Hey, those look pretty familiar! They're just our original, unaltered, +Hey, those look pretty familiar! They're just our original, unaltered input records. A little thinking (the human brain is still the best debugging tool), and we realize that we were off by one! @@ -28408,13 +28495,14 @@ Miscellaneous @end itemize Each of these are discussed in the following subsections. -In the following descriptions, commands which may be abbreviated +In the following descriptions, commands that may be abbreviated show the abbreviation on a second description line. A debugger command name may also be truncated if that partial name is unambiguous. The debugger has the built-in capability to -automatically repeat the previous command just by hitting @key{Enter}. -This works for the commands @code{list}, @code{next}, @code{nexti}, @code{step}, @code{stepi} -and @code{continue} executed without any argument. +automatically repeat the previous command just by hitting @kbd{Enter}. +This works for the commands @code{list}, @code{next}, @code{nexti}, +@code{step}, @code{stepi}, and @code{continue} executed without any +argument. @menu * Breakpoint Control:: Control of Breakpoints. @@ -28429,9 +28517,9 @@ and @code{continue} executed without any argument. @node Breakpoint Control @subsection Control of Breakpoints -As we saw above, the first thing you probably want to do in a debugging -session is to get your breakpoints set up, since otherwise your program -will just run as if it was not under the debugger. The commands for +As we saw earlier, the first thing you probably want to do in a debugging +session is to get your breakpoints set up, because your program +will otherwise just run as if it was not under the debugger. The commands for controlling breakpoints are: @table @asis @@ -28461,7 +28549,7 @@ Set a breakpoint at entry to (the first instruction of) function @var{function}. @end table -Each breakpoint is assigned a number which can be used to delete it from +Each breakpoint is assigned a number that can be used to delete it from the breakpoint list using the @code{delete} command. With a breakpoint, you may also supply a condition. This is an @@ -28502,8 +28590,8 @@ that the debugger evaluates whenever the breakpoint or watchpoint is reached. If the condition is true, then the debugger stops execution and prompts for a command. Otherwise, the debugger continues executing the program. If the condition expression is -not specified, any existing condition is removed; i.e., the breakpoint or -watchpoint is made unconditional. +not specified, any existing condition is removed (i.e., the breakpoint or +watchpoint is made unconditional). @cindex debugger commands, @code{d} (@code{delete}) @cindex debugger commands, @code{delete} @@ -28513,7 +28601,7 @@ watchpoint is made unconditional. @cindex breakpoint, delete by number @item @code{delete} [@var{n1 n2} @dots{}] [@var{n}--@var{m}] @itemx @code{d} [@var{n1 n2} @dots{}] [@var{n}--@var{m}] -Delete specified breakpoints or a range of breakpoints. Deletes +Delete specified breakpoints or a range of breakpoints. Delete all defined breakpoints if no argument is supplied. @cindex debugger commands, @code{disable} @@ -28522,7 +28610,7 @@ all defined breakpoints if no argument is supplied. @cindex breakpoint, how to disable or enable @item @code{disable} [@var{n1 n2} @dots{} | @var{n}--@var{m}] Disable specified breakpoints or a range of breakpoints. Without -any argument, disables all breakpoints. +any argument, disable all breakpoints. @cindex debugger commands, @code{e} (@code{enable}) @cindex debugger commands, @code{enable} @@ -28532,18 +28620,18 @@ any argument, disables all breakpoints. @item @code{enable} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}] @itemx @code{e} [@code{del} | @code{once}] [@var{n1 n2} @dots{}] [@var{n}--@var{m}] Enable specified breakpoints or a range of breakpoints. Without -any argument, enables all breakpoints. -Optionally, you can specify how to enable the breakpoint: +any argument, enable all breakpoints. +Optionally, you can specify how to enable the breakpoints: @c nested table @table @code @item del -Enable the breakpoint(s) temporarily, then delete it when -the program stops at the breakpoint. +Enable the breakpoints temporarily, then delete each one when +the program stops at it. @item once -Enable the breakpoint(s) temporarily, then disable it when -the program stops at the breakpoint. +Enable the breakpoints temporarily, then disable each one when +the program stops at it. @end table @cindex debugger commands, @code{ignore} @@ -28611,7 +28699,7 @@ gawk> @item @code{continue} [@var{count}] @itemx @code{c} [@var{count}] Resume program execution. If continued from a breakpoint and @var{count} is -specified, ignores the breakpoint at that location the next @var{count} times +specified, ignore the breakpoint at that location the next @var{count} times before stopping. @cindex debugger commands, @code{finish} @@ -28644,7 +28732,7 @@ Execute one (or @var{count}) instruction(s), stepping over function calls. @item @code{return} [@var{value}] Cancel execution of a function call. If @var{value} (either a string or a number) is specified, it is used as the function's return value. If used in a -frame other than the innermost one (the currently executing function, i.e., +frame other than the innermost one (the currently executing function; i.e., frame number 0), discard all inner frames in addition to the selected one, and the caller of that frame becomes the innermost frame. @@ -28665,7 +28753,7 @@ automatic display variables, and debugger options. @item @code{step} [@var{count}] @itemx @code{s} [@var{count}] Continue execution until control reaches a different source line in the -current stack frame. @code{step} steps inside any function called within +current stack frame, stepping inside any function called within the line. If the argument @var{count} is supplied, steps that many times before stopping, unless it encounters a breakpoint or watchpoint. @@ -28710,7 +28798,7 @@ gawk> @kbd{display x} @end example @noindent -displays the assigned item number, the variable name and its current value. +This displays the assigned item number, the variable name, and its current value. If the display variable refers to a function parameter, it is silently deleted from the list as soon as the execution reaches a context where no such variable of the given name exists. @@ -28778,7 +28866,7 @@ or field. String values must be enclosed between double quotes (@code{"}@dots{}@code{"}). You can also set special @command{awk} variables, such as @code{FS}, -@code{NF}, @code{NR}, etc. +@code{NF}, @code{NR}, and so on. @cindex debugger commands, @code{w} (@code{watch}) @cindex debugger commands, @code{watch} @@ -28790,7 +28878,7 @@ You can also set special @command{awk} variables, such as @code{FS}, Add variable @var{var} (or field @code{$@var{n}}) to the watch list. The debugger then stops whenever the value of the variable or field changes. Each watched item is assigned a -number which can be used to delete it from the watch list using the +number that can be used to delete it from the watch list using the @code{unwatch} command. With a watchpoint, you may also supply a condition. This is an @@ -28818,11 +28906,11 @@ watch list. @node Execution Stack @subsection Working with the Stack -Whenever you run a program which contains any function calls, +Whenever you run a program that contains any function calls, @command{gawk} maintains a stack of all of the function calls leading up to where the program is right now. You can see how you got to where you are, and also move around in the stack to see what the state of things was in the -functions which called the one you are in. The commands for doing this are: +functions that called the one you are in. The commands for doing this are: @table @asis @cindex debugger commands, @code{bt} (@code{backtrace}) @@ -28841,7 +28929,7 @@ Print a backtrace of all function calls (stack frames), or innermost @var{count} frames if @var{count} > 0. Print the outermost @var{count} frames if @var{count} < 0. The backtrace displays the name and arguments to each function, the source @value{FN}, and the line number. -The alias @code{where} for @code{backtrace} is provided for long-time +The alias @code{where} for @code{backtrace} is provided for longtime GDB users who may be used to that command. @cindex debugger commands, @code{down} @@ -28857,8 +28945,8 @@ Then select and print the frame. @item @code{frame} [@var{n}] @itemx @code{f} [@var{n}] Select and print stack frame @var{n}. Frame 0 is the currently executing, -or @dfn{innermost}, frame (function call), frame 1 is the frame that -called the innermost one. The highest numbered frame is the one for the +or @dfn{innermost}, frame (function call); frame 1 is the frame that +called the innermost one. The highest-numbered frame is the one for the main program. The printed information consists of the frame number, function and argument names, source file, and the source line. @@ -28870,11 +28958,11 @@ Then select and print the frame. @end table @node Debugger Info -@subsection Obtaining Information about the Program and the Debugger State +@subsection Obtaining Information About the Program and the Debugger State Besides looking at the values of variables, there is often a need to get other sorts of information about the state of your program and of the -debugging environment itself. The @command{gawk} debugger has one command which +debugging environment itself. The @command{gawk} debugger has one command that provides this information, appropriately called @code{info}. @code{info} is used with one of a number of arguments that tell it exactly what you want to know: @@ -28962,12 +29050,12 @@ The available options are: @table @asis @item @code{history_size} @cindex debugger history size -The maximum number of lines to keep in the history file @file{./.gawk_history}. -The default is 100. +Set the maximum number of lines to keep in the history file +@file{./.gawk_history}. The default is 100. @item @code{listsize} @cindex debugger default list amount -The number of lines that @code{list} prints. The default is 15. +Specify the number of lines that @code{list} prints. The default is 15. @item @code{outfile} @cindex redirect @command{gawk} output, in debugger @@ -28977,7 +29065,7 @@ standard output. @item @code{prompt} @cindex debugger prompt -The debugger prompt. The default is @samp{@w{gawk> }}. +Change the debugger prompt. The default is @samp{@w{gawk> }}. @item @code{save_history} [@code{on} | @code{off}] @cindex debugger history file @@ -28988,7 +29076,7 @@ The default is @code{on}. @cindex save debugger options Save current options to file @file{./.gawkrc} upon exit. The default is @code{on}. -Options are read back in to the next session upon startup. +Options are read back into the next session upon startup. @item @code{trace} [@code{on} | @code{off}] @cindex instruction tracing, in debugger @@ -29011,7 +29099,7 @@ command in the file. Also, the list of commands may include additional @code{source} commands; however, the @command{gawk} debugger will not source the same file more than once in order to avoid infinite recursion. -In addition to, or instead of the @code{source} command, you can use +In addition to, or instead of, the @code{source} command, you can use the @option{-D @var{file}} or @option{--debug=@var{file}} command-line options to execute commands from a file non-interactively (@pxref{Options}). @@ -29020,16 +29108,16 @@ options to execute commands from a file non-interactively @node Miscellaneous Debugger Commands @subsection Miscellaneous Commands -There are a few more commands which do not fit into the +There are a few more commands that do not fit into the previous categories, as follows: @table @asis @cindex debugger commands, @code{dump} @cindex @code{dump} debugger command @item @code{dump} [@var{filename}] -Dump bytecode of the program to standard output or to the file +Dump byte code of the program to standard output or to the file named in @var{filename}. This prints a representation of the internal -instructions which @command{gawk} executes to implement the @command{awk} +instructions that @command{gawk} executes to implement the @command{awk} commands in a program. This can be very enlightening, as the following partial dump of Davide Brini's obfuscated code (@pxref{Signature Program}) demonstrates: @@ -29038,51 +29126,51 @@ partial dump of Davide Brini's obfuscated code @smallexample gawk> @kbd{dump} @print{} # BEGIN -@print{} -@print{} [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] -@print{} [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc280] Op_match : -@print{} [ 1:0xfcc1e0] Op_store_var : O -@print{} [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] -@print{} [ 1:0xfcc320] Op_equal : -@print{} [ 1:0xfcc200] Op_store_var : o -@print{} [ 1:0xfcc380] Op_push : o -@print{} [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] -@print{} [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] -@print{} [ 1:0xfcc300] Op_assign_plus : -@print{} [ :0xfcc2c0] Op_pop : -@print{} [ 1:0xfcc400] Op_push : O -@print{} [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] -@print{} [ :0xfcc4a0] Op_no_op : -@print{} [ 1:0xfcc480] Op_push : O -@print{} [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] -@print{} [ 1:0xfcc3c0] Op_store_var : x -@print{} [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] -@print{} [ 1:0xfcc3a0] Op_postincrement : -@print{} [ 1:0xfcc4e0] Op_push : x -@print{} [ 1:0xfcc540] Op_push : o -@print{} [ 1:0xfcc500] Op_plus : -@print{} [ 1:0xfcc580] Op_push : o -@print{} [ 1:0xfcc560] Op_plus : -@print{} [ 1:0xfcc460] Op_leq : -@print{} [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] -@print{} [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] -@print{} [ :0xfcc660] Op_no_op : -@print{} [ 1:0xfcc520] Op_assign_concat : c -@print{} [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] -@print{} -@dots{} -@print{} +@print{} +@print{} [ 1:0xfcd340] Op_rule : [in_rule = BEGIN] [source_file = brini.awk] +@print{} [ 1:0xfcc240] Op_push_i : "~" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc2a0] Op_push_i : "~" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc280] Op_match : +@print{} [ 1:0xfcc1e0] Op_store_var : O +@print{} [ 1:0xfcc2e0] Op_push_i : "==" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc340] Op_push_i : "==" [MALLOC|STRING|STRCUR] +@print{} [ 1:0xfcc320] Op_equal : +@print{} [ 1:0xfcc200] Op_store_var : o +@print{} [ 1:0xfcc380] Op_push : o +@print{} [ 1:0xfcc360] Op_plus_i : 0 [MALLOC|NUMCUR|NUMBER] +@print{} [ 1:0xfcc220] Op_push_lhs : o [do_reference = true] +@print{} [ 1:0xfcc300] Op_assign_plus : +@print{} [ :0xfcc2c0] Op_pop : +@print{} [ 1:0xfcc400] Op_push : O +@print{} [ 1:0xfcc420] Op_push_i : "" [MALLOC|STRING|STRCUR] +@print{} [ :0xfcc4a0] Op_no_op : +@print{} [ 1:0xfcc480] Op_push : O +@print{} [ :0xfcc4c0] Op_concat : [expr_count = 3] [concat_flag = 0] +@print{} [ 1:0xfcc3c0] Op_store_var : x +@print{} [ 1:0xfcc440] Op_push_lhs : X [do_reference = true] +@print{} [ 1:0xfcc3a0] Op_postincrement : +@print{} [ 1:0xfcc4e0] Op_push : x +@print{} [ 1:0xfcc540] Op_push : o +@print{} [ 1:0xfcc500] Op_plus : +@print{} [ 1:0xfcc580] Op_push : o +@print{} [ 1:0xfcc560] Op_plus : +@print{} [ 1:0xfcc460] Op_leq : +@print{} [ :0xfcc5c0] Op_jmp_false : [target_jmp = 0xfcc5e0] +@print{} [ 1:0xfcc600] Op_push_i : "%c" [MALLOC|STRING|STRCUR] +@print{} [ :0xfcc660] Op_no_op : +@print{} [ 1:0xfcc520] Op_assign_concat : c +@print{} [ :0xfcc620] Op_jmp : [target_jmp = 0xfcc440] +@print{} +@dots{} +@print{} @print{} [ 2:0xfcc5a0] Op_K_printf : [expr_count = 17] [redir_type = ""] -@print{} [ :0xfcc140] Op_no_op : -@print{} [ :0xfcc1c0] Op_atexit : -@print{} [ :0xfcc640] Op_stop : -@print{} [ :0xfcc180] Op_no_op : -@print{} [ :0xfcd150] Op_after_beginfile : -@print{} [ :0xfcc160] Op_no_op : -@print{} [ :0xfcc1a0] Op_after_endfile : +@print{} [ :0xfcc140] Op_no_op : +@print{} [ :0xfcc1c0] Op_atexit : +@print{} [ :0xfcc640] Op_stop : +@print{} [ :0xfcc180] Op_no_op : +@print{} [ :0xfcd150] Op_after_beginfile : +@print{} [ :0xfcc160] Op_no_op : +@print{} [ :0xfcc1a0] Op_after_endfile : gawk> @end smallexample @@ -29126,7 +29214,7 @@ Print lines centered around line number @var{n} in source file @var{filename}. This command may change the current source file. @item @var{function} -Print lines centered around beginning of the +Print lines centered around the beginning of the function @var{function}. This command may change the current source file. @end table @@ -29138,16 +29226,16 @@ function @var{function}. This command may change the current source file. @item @code{quit} @itemx @code{q} Exit the debugger. Debugging is great fun, but sometimes we all have -to tend to other obligations in life, and sometimes we find the bug, -and are free to go on to the next one! As we saw above, if you are -running a program, the debugger warns you if you accidentally type +to tend to other obligations in life, and sometimes we find the bug +and are free to go on to the next one! As we saw earlier, if you are +running a program, the debugger warns you when you type @samp{q} or @samp{quit}, to make sure you really want to quit. @cindex debugger commands, @code{trace} @cindex @code{trace} debugger command @item @code{trace} [@code{on} | @code{off}] -Turn on or off a continuous printing of instructions which are about to -be executed, along with printing the @command{awk} line which they +Turn on or off continuous printing of the instructions that are about to +be executed, along with the @command{awk} lines they implement. The default is @code{off}. It is to be hoped that most of the ``opcodes'' in these instructions are @@ -29163,7 +29251,7 @@ fairly self-explanatory, and using @code{stepi} and @code{nexti} while If @command{gawk} is compiled with @uref{http://cnswww.cns.cwru.edu/php/chet/readline/readline.html, -the @code{readline} library}, you can take advantage of that library's +the GNU Readline library}, you can take advantage of that library's command completion and history expansion features. The following types of completion are available: @@ -29200,7 +29288,7 @@ and We hope you find the @command{gawk} debugger useful and enjoyable to work with, but as with any program, especially in its early releases, it still has -some limitations. A few which are worth being aware of are: +some limitations. A few that it's worth being aware of are: @itemize @value{BULLET} @item @@ -29216,13 +29304,13 @@ If you perused the dump of opcodes in @ref{Miscellaneous Debugger Commands} (or if you are already familiar with @command{gawk} internals), you will realize that much of the internal manipulation of data in @command{gawk}, as in many interpreters, is done on a stack. -@code{Op_push}, @code{Op_pop}, etc., are the ``bread and butter'' of +@code{Op_push}, @code{Op_pop}, and the like are the ``bread and butter'' of most @command{gawk} code. Unfortunately, as of now, the @command{gawk} debugger does not allow you to examine the stack's contents. That is, the intermediate results of expression evaluation are on the -stack, but cannot be printed. Rather, only variables which are defined +stack, but cannot be printed. Rather, only variables that are defined in the program can be printed. Of course, a workaround for this is to use more explicit variables at the debugging stage and then change back to obscure, perhaps more optimal code later. @@ -29230,18 +29318,18 @@ change back to obscure, perhaps more optimal code later. @item There is no way to look ``inside'' the process of compiling regular expressions to see if you got it right. As an @command{awk} -programmer, you are expected to know what @code{/[^[:alnum:][:blank:]]/} -means. +programmer, you are expected to know the meaning of +@code{/[^[:alnum:][:blank:]]/}. @item The @command{gawk} debugger is designed to be used by running a program (with all its parameters) on the command line, as described in @ref{Debugger Invocation}. -There is no way (as of now) to attach or ``break in'' to a running program. -This seems reasonable for a language which is used mainly for quickly +There is no way (as of now) to attach or ``break into'' a running program. +This seems reasonable for a language that is used mainly for quickly executing, short programs. @item -The @command{gawk} debugger only accepts source supplied with the @option{-f} option. +The @command{gawk} debugger only accepts source code supplied with the @option{-f} option. @end itemize @ignore @@ -29255,8 +29343,8 @@ be added, and of course feel free to try to add them yourself! @itemize @value{BULLET} @item Programs rarely work correctly the first time. Finding bugs -is @dfn{debugging} and a program that helps you find bugs is a -@dfn{debugger}. @command{gawk} has a built-in debugger that works very +is called debugging, and a program that helps you find bugs is a +debugger. @command{gawk} has a built-in debugger that works very similarly to the GNU Debugger, GDB. @item @@ -29276,14 +29364,14 @@ breakpoints, execution, viewing and changing data, working with the stack, getting information, and other tasks. @item -If the @code{readline} library is available when @command{gawk} is +If the GNU Readline library is available when @command{gawk} is compiled, it is used by the debugger to provide command-line history and editing. @end itemize @node Arbitrary Precision Arithmetic -@chapter Arithmetic and Arbitrary Precision Arithmetic with @command{gawk} +@chapter Arithmetic and Arbitrary-Precision Arithmetic with @command{gawk} @cindex arbitrary precision @cindex multiple precision @cindex infinite precision @@ -29293,9 +29381,9 @@ This @value{CHAPTER} introduces some basic concepts relating to how computers do arithmetic and defines some important terms. It then proceeds to describe floating-point arithmetic, which is what @command{awk} uses for all its computations, including a -discussion of arbitrary precision floating point arithmetic, which is +discussion of arbitrary-precision floating-point arithmetic, which is a feature available only in @command{gawk}. It continues on to present -arbitrary precision integers, and concludes with a description of some +arbitrary-precision integers, and concludes with a description of some points where @command{gawk} and the POSIX standard are not quite in agreement. @@ -29340,7 +29428,7 @@ paper and pencil (and/or a calculator). In theory, numbers can have an arbitrary number of digits on either side (or both sides) of the decimal point, and the results of a computation are always exact. -Some modern system can do decimal arithmetic in hardware, but usually you +Some modern systems can do decimal arithmetic in hardware, but usually you need a special software library to provide access to these instructions. There are also libraries that do decimal arithmetic entirely in software. @@ -29358,55 +29446,82 @@ The disadvantage is that their range is limited. @cindex integers, unsigned In computers, integer values come in two flavors: @dfn{signed} and @dfn{unsigned}. Signed values may be negative or positive, whereas -unsigned values are always positive (that is, greater than or equal -to zero). +unsigned values are always greater than or equal +to zero. In computer systems, integer arithmetic is exact, but the possible range of values is limited. Integer arithmetic is generally faster than -floating point arithmetic. +floating-point arithmetic. -@item Floating point arithmetic +@item Floating-point arithmetic Floating-point numbers represent what were called in school ``real'' -numbers; i.e., those that have a fractional part, such as 3.1415927. +numbers (i.e., those that have a fractional part, such as 3.1415927). The advantage to floating-point numbers is that they can represent a much larger range of values than can integers. The disadvantage is that there are numbers that they cannot represent exactly. -Modern systems support floating point arithmetic in hardware, with a +Modern systems support floating-point arithmetic in hardware, with a limited range of values. There are software libraries that allow -the use of arbitrary precision floating point calculations. +the use of arbitrary-precision floating-point calculations. -POSIX @command{awk} uses @dfn{double precision} floating-point numbers, which -can hold more digits than @dfn{single precision} floating-point numbers. -@command{gawk} has facilities for performing arbitrary precision floating -point arithmetic, which we describe in more detail shortly. +POSIX @command{awk} uses @dfn{double-precision} floating-point numbers, which +can hold more digits than @dfn{single-precision} floating-point numbers. +@command{gawk} has facilities for performing arbitrary-precision +floating-point arithmetic, which we describe in more detail shortly. @end table -Computers work with integer and floating point values of different -ranges. Integer values are usually either 32 or 64 bits in size. Single -precision floating point values occupy 32 bits, whereas double precision -floating point values occupy 64 bits. Floating point values are always +Computers work with integer and floating-point values of different +ranges. Integer values are usually either 32 or 64 bits in size. +Single-precision floating-point values occupy 32 bits, whereas double-precision +floating-point values occupy 64 bits. Floating-point values are always signed. The possible ranges of values are shown in @ref{table-numeric-ranges}. @float Table,table-numeric-ranges -@caption{Value Ranges for Different Numeric Representations} +@caption{Value ranges for different numeric representations} @multitable @columnfractions .34 .33 .33 @headitem Numeric representation @tab Minimum value @tab Maximum value @item 32-bit signed integer @tab @minus{}2,147,483,648 @tab 2,147,483,647 @item 32-bit unsigned integer @tab 0 @tab 4,294,967,295 @item 64-bit signed integer @tab @minus{}9,223,372,036,854,775,808 @tab 9,223,372,036,854,775,807 @item 64-bit unsigned integer @tab 0 @tab 18,446,744,073,709,551,615 -@item Single precision floating point (approximate) @tab @code{1.175494e-38} @tab @code{3.402823e+38} -@item Double precision floating point (approximate) @tab @code{2.225074e-308} @tab @code{1.797693e+308} +@iftex +@item Single-precision floating point (approximate) @tab @math{1.175494^{-38}} @tab @math{3.402823^{38}} +@item Double-precision floating point (approximate) @tab @math{2.225074^{-308}} @tab @math{1.797693^{308}} +@end iftex +@ifnottex +@ifnotdocbook +@item Single-precision floating point (approximate) @tab 1.175494e-38 @tab 3.402823e38 +@item Double-precision floating point (approximate) @tab 2.225074e-308 @tab 1.797693e308 +@end ifnotdocbook +@end ifnottex +@ifdocbook +@item Single-precision floating point (approximate) @tab +@c FIXME: Use @sup here for superscript +@docbook +1.175494<superscript>-38</superscript> +@end docbook +@tab +@docbook +3.402823<superscript>38</superscript> +@end docbook +@item Double-precision floating point (approximate) @tab +@docbook +2.225074<superscript>-308</superscript> +@end docbook +@tab +@docbook +1.797693<superscript>308</superscript> +@end docbook +@end ifdocbook @end multitable @end float @node Math Definitions -@section Other Stuff To Know +@section Other Stuff to Know The rest of this @value{CHAPTER} uses a number of terms. Here are some informal definitions that should help you work your way through the material -here. +here: @table @dfn @item Accuracy @@ -29427,7 +29542,7 @@ A special value representing infinity. Operations involving another number and infinity produce infinity. @item NaN -``Not A Number.''@footnote{Thanks to Michael Brennan for this description, +``Not a number.''@footnote{Thanks to Michael Brennan for this description, which we have paraphrased, and for the examples.} A special value that results from attempting a calculation that has no answer as a real number. In such a case, programs can either receive a floating-point exception, @@ -29470,8 +29585,8 @@ formula: @end display @noindent -Here, @var{prec} denotes the binary precision -(measured in bits) and @var{dps} (short for decimal places) +Here, @emph{prec} denotes the binary precision +(measured in bits) and @emph{dps} (short for decimal places) is the decimal digits. @item Rounding mode @@ -29479,7 +29594,7 @@ How numbers are rounded up or down when necessary. More details are provided later. @item Significand -A floating point value consists the significand multiplied by 10 +A floating-point value consists of the significand multiplied by 10 to the power of the exponent. For example, in @code{1.2345e67}, the significand is @code{1.2345}. @@ -29497,19 +29612,19 @@ on some of those terms. On modern systems, floating-point hardware uses the representation and operations defined by the IEEE 754 standard. Three of the standard IEEE 754 types are 32-bit single precision, -64-bit double precision and 128-bit quadruple precision. +64-bit double precision, and 128-bit quadruple precision. The standard also specifies extended precision formats to allow greater precisions and larger exponent ranges. -(@command{awk} uses only the 64-bit double precision format.) +(@command{awk} uses only the 64-bit double-precision format.) @ref{table-ieee-formats} lists the precision and exponent -field values for the basic IEEE 754 binary formats: +field values for the basic IEEE 754 binary formats. @float Table,table-ieee-formats -@caption{Basic IEEE Format Values} +@caption{Basic IEEE format values} @multitable @columnfractions .20 .20 .20 .20 .20 @headitem Name @tab Total bits @tab Precision @tab Minimum exponent @tab Maximum exponent -@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127 +@item Single @tab 32 @tab 24 @tab @minus{}126 @tab +127 @item Double @tab 64 @tab 53 @tab @minus{}1022 @tab +1023 @item Quadruple @tab 128 @tab 113 @tab @minus{}16382 @tab +16383 @end multitable @@ -29521,19 +29636,19 @@ one extra bit of significand. @end quotation @node MPFR features -@section Arbitrary Precision Arithmetic Features In @command{gawk} +@section Arbitrary-Precision Arithmetic Features in @command{gawk} -By default, @command{gawk} uses the double precision floating-point values +By default, @command{gawk} uses the double-precision floating-point values supplied by the hardware of the system it runs on. However, if it was -compiled to do so, @command{gawk} uses the @uref{http://www.mpfr.org -GNU MPFR} and @uref{http://gmplib.org, GNU MP} (GMP) libraries for arbitrary -precision arithmetic on numbers. You can see if MPFR support is available -like so: +compiled to do so, @command{gawk} uses the @uref{http://www.mpfr.org, +GNU MPFR} and @uref{http://gmplib.org, GNU MP} (GMP) libraries for +arbitrary-precision arithmetic on numbers. You can see if MPFR support +is available like so: @example $ @kbd{gawk --version} @print{} GNU Awk 4.1.2, API: 1.1 (GNU MPFR 3.1.0-p3, GNU MP 5.0.2) -@print{} Copyright (C) 1989, 1991-2014 Free Software Foundation. +@print{} Copyright (C) 1989, 1991-2015 Free Software Foundation. @dots{} @end example @@ -29556,23 +29671,23 @@ Two predefined variables, @code{PREC} and @code{ROUNDMODE}, provide control over the working precision and the rounding mode. The precision and the rounding mode are set globally for every operation to follow. -@xref{Setting precision}, and @ref{Setting the rounding mode}, +@DBXREF{Setting precision} and @DBREF{Setting the rounding mode} for more information. @node FP Math Caution -@section Floating Point Arithmetic: Caveat Emptor! +@section Floating-Point Arithmetic: Caveat Emptor! @quotation @i{Math class is tough!} @author Teen Talk Barbie, July 1992 @end quotation -This @value{SECTION} provides a high level overview of the issues +This @value{SECTION} provides a high-level overview of the issues involved when doing lots of floating-point arithmetic.@footnote{There is a very nice @uref{http://www.validlab.com/goldberg/paper.pdf, paper on floating-point arithmetic} by David Goldberg, ``What Every -Computer Scientist Should Know About Floating-point Arithmetic,'' -@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03), 5-48. This is +Computer Scientist Should Know About Floating-Point Arithmetic,'' +@cite{ACM Computing Surveys} @strong{23}, 1 (1991-03): 5-48. This is worth reading if you are interested in the details, but it does require a background in computer science.} The discussion applies to both hardware and arbitrary-precision @@ -29593,17 +29708,17 @@ rely just on what we tell you. @end menu @node Inexactness of computations -@subsection Floating Point Arithmetic Is Not Exact +@subsection Floating-Point Arithmetic Is Not Exact Binary floating-point representations and arithmetic are inexact. Simple values like 0.1 cannot be precisely represented using binary floating-point numbers, and the limited precision of floating-point numbers means that slight changes in the order of operations or the precision of intermediate storage -can change the result. To make matters worse, with arbitrary precision -floating-point, you can set the precision before starting a computation, -but then you cannot be sure of the number of significant decimal places -in the final result. +can change the result. To make matters worse, with arbitrary-precision +floating-point arithmetic, you can set the precision before starting a +computation, but then you cannot be sure of the number of significant +decimal places in the final result. @menu * Inexact representation:: Numbers are not exactly represented. @@ -29625,7 +29740,7 @@ y = 0.425 Unlike the number in @code{y}, the number stored in @code{x} is exactly representable -in binary since it can be written as a finite sum of one or +in binary because it can be written as a finite sum of one or more fractions whose denominators are all powers of two. When @command{gawk} reads a floating-point number from program source, it automatically rounds that number to whatever @@ -29641,7 +29756,7 @@ $ @kbd{gawk 'BEGIN @{ x = 0.875; y = 0.425} Often the error is so small you do not even notice it, and if you do, you can always specify how much precision you would like in your output. -Usually this is a format string like @code{"%.15g"}, which when +Usually this is a format string like @code{"%.15g"}, which, when used in the previous example, produces an output identical to the input. @node Comparing FP Values @@ -29651,7 +29766,7 @@ Because the underlying representation can be a little bit off from the exact val comparing floating-point values to see if they are exactly equal is generally a bad idea. Here is an example where it does not work like you would expect: -@example +@example $ @kbd{gawk 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 0 @end example @@ -29660,7 +29775,7 @@ The general wisdom when comparing floating-point values is to see if they are within some small range of each other (called a @dfn{delta}, or @dfn{tolerance}). You have to decide how small a delta is important to you. Code to do -this looks something like this: +this looks something like the following: @example delta = 0.00001 # for example @@ -29680,7 +29795,7 @@ else The loss of accuracy during a single computation with floating-point numbers usually isn't enough to worry about. However, if you compute a -value which is the result of a sequence of floating point operations, +value that is the result of a sequence of floating-point operations, the error can accumulate and greatly affect the computation itself. Here is an attempt to compute the value of @value{PI} using one of its many series representations: @@ -29726,23 +29841,23 @@ $ @kbd{gawk 'BEGIN @{} @end example @node Getting Accuracy -@subsection Getting The Accuracy You Need +@subsection Getting the Accuracy You Need -Can arbitrary precision arithmetic give exact results? There are +Can arbitrary-precision arithmetic give exact results? There are no easy answers. The standard rules of algebra often do not apply when using floating-point arithmetic. Among other things, the distributive and associative laws do not hold completely, and order of operation may be important -for your computation. Rounding error, cumulative precision loss +for your computation. Rounding error, cumulative precision loss, and underflow are often troublesome. When @command{gawk} tests the expressions @samp{0.1 + 12.2} and -@samp{12.3} for equality using the machine double precision arithmetic, +@samp{12.3} for equality using the machine double-precision arithmetic, it decides that they are not equal! (@xref{Comparing FP Values}.) You can get the result you want by increasing the precision; 56 bits in this case does the job: -@example +@example $ @kbd{gawk -M -v PREC=56 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 1 @end example @@ -29751,7 +29866,7 @@ If adding more bits is good, perhaps adding even more bits of precision is better? Here is what happens if we use an even larger value of @code{PREC}: -@example +@example $ @kbd{gawk -M -v PREC=201 'BEGIN @{ print (0.1 + 12.2 == 12.3) @}'} @print{} 0 @end example @@ -29760,20 +29875,21 @@ This is not a bug in @command{gawk} or in the MPFR library. It is easy to forget that the finite number of bits used to store the value is often just an approximation after proper rounding. The test for equality succeeds if and only if @emph{all} bits in the two operands -are exactly the same. Since this is not necessarily true after floating-point +are exactly the same. Because this is not necessarily true after floating-point computations with a particular precision and effective rounding mode, a straight test for equality may not work. Instead, compare the two numbers to see if they are within the desirable delta of each other. In applications where 15 or fewer decimal places suffice, -hardware double precision arithmetic can be adequate, and is usually much faster. +hardware double-precision arithmetic can be adequate, and is usually much faster. But you need to keep in mind that every floating-point operation -can suffer a new rounding error with catastrophic consequences as illustrated +can suffer a new rounding error with catastrophic consequences, as illustrated by our earlier attempt to compute the value of @value{PI}. Extra precision can greatly enhance the stability and the accuracy of your computation in such cases. -Repeated addition is not necessarily equivalent to multiplication +Additionally, you should understand that +repeated addition is not necessarily equivalent to multiplication in floating-point arithmetic. In the example in @ref{Errors accumulate}: @@ -29792,9 +29908,9 @@ an arbitrarily large value for @code{PREC}. Reformulation of the problem at hand is often the correct approach in such situations. @node Try To Round -@subsection Try A Few Extra Bits of Precision and Rounding +@subsection Try a Few Extra Bits of Precision and Rounding -Instead of arbitrary precision floating-point arithmetic, +Instead of arbitrary-precision floating-point arithmetic, often all you need is an adjustment of your logic or a different order for the operations in your calculation. The stability and the accuracy of the computation of @value{PI} @@ -29806,7 +29922,7 @@ simple algebraic transformation: @end example @noindent -After making this, change the program converges to +After making this change, the program converges to @value{PI} in under 30 iterations: @example @@ -29822,7 +29938,7 @@ $ @kbd{gawk -f pi2.awk} @end example @node Setting precision -@subsection Setting The Precision +@subsection Setting the Precision @command{gawk} uses a global working precision; it does not keep track of the precision or accuracy of individual numbers. Performing an arithmetic @@ -29834,14 +29950,14 @@ shown in @ref{table-predefined-precision-strings}, to emulate an IEEE 754 binary format. @float Table,table-predefined-precision-strings -@caption{Predefined Precision Strings For @code{PREC}} +@caption{Predefined precision strings for @code{PREC}} @multitable {@code{"double"}} {12345678901234567890123456789012345} -@headitem @code{PREC} @tab IEEE 754 Binary Format -@item @code{"half"} @tab 16-bit half-precision. -@item @code{"single"} @tab Basic 32-bit single precision. -@item @code{"double"} @tab Basic 64-bit double precision. -@item @code{"quad"} @tab Basic 128-bit quadruple precision. -@item @code{"oct"} @tab 256-bit octuple precision. +@headitem @code{PREC} @tab IEEE 754 binary format +@item @code{"half"} @tab 16-bit half-precision +@item @code{"single"} @tab Basic 32-bit single precision +@item @code{"double"} @tab Basic 64-bit double precision +@item @code{"quad"} @tab Basic 128-bit quadruple precision +@item @code{"oct"} @tab 256-bit octuple precision @end multitable @end float @@ -29868,11 +29984,10 @@ than the default and cannot use a command-line assignment to @code{PREC}, you should either specify the constant as a string, or as a rational number, whenever possible. The following example illustrates the differences among various ways to print a floating-point constant: -@end quotation @example $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 0.1) @}'} -@print{} 0.1000000000000000055511151 +@print{} 0.1000000000000000055511151 $ @kbd{gawk -M -v PREC=113 'BEGIN @{ printf("%0.25f\n", 0.1) @}'} @print{} 0.1000000000000000000000000 $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'} @@ -29880,22 +29995,23 @@ $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", "0.1") @}'} $ @kbd{gawk -M 'BEGIN @{ PREC = 113; printf("%0.25f\n", 1/10) @}'} @print{} 0.1000000000000000000000000 @end example +@end quotation @node Setting the rounding mode -@subsection Setting The Rounding Mode +@subsection Setting the Rounding Mode The @code{ROUNDMODE} variable provides -program level control over the rounding mode. +program-level control over the rounding mode. The correspondence between @code{ROUNDMODE} and the IEEE rounding modes is shown in @ref{table-gawk-rounding-modes}. @float Table,table-gawk-rounding-modes -@caption{@command{gawk} Rounding Modes} +@caption{@command{gawk} rounding modes} @multitable @columnfractions .45 .30 .25 -@headitem Rounding Mode @tab IEEE Name @tab @code{ROUNDMODE} +@headitem Rounding mode @tab IEEE name @tab @code{ROUNDMODE} @item Round to nearest, ties to even @tab @code{roundTiesToEven} @tab @code{"N"} or @code{"n"} -@item Round toward plus Infinity @tab @code{roundTowardPositive} @tab @code{"U"} or @code{"u"} -@item Round toward negative Infinity @tab @code{roundTowardNegative} @tab @code{"D"} or @code{"d"} +@item Round toward positive infinity @tab @code{roundTowardPositive} @tab @code{"U"} or @code{"u"} +@item Round toward negative infinity @tab @code{roundTowardNegative} @tab @code{"D"} or @code{"d"} @item Round toward zero @tab @code{roundTowardZero} @tab @code{"Z"} or @code{"z"} @item Round to nearest, ties away from zero @tab @code{roundTiesToAway} @tab @code{"A"} or @code{"a"} @end multitable @@ -29905,7 +30021,7 @@ rounding modes is shown in @ref{table-gawk-rounding-modes}. selects the IEEE 754 rounding mode @code{roundTiesToEven}. In @ref{table-gawk-rounding-modes}, the value @code{"A"} selects @code{roundTiesToAway}. This is only available if your version of the -MPFR library supports it; otherwise setting @code{ROUNDMODE} to @code{"A"} +MPFR library supports it; otherwise, setting @code{ROUNDMODE} to @code{"A"} has no effect. The default mode @code{roundTiesToEven} is the most preferred, @@ -29956,8 +30072,8 @@ distributes upward and downward rounds of exact halves, which might cause any accumulating round-off error to cancel itself out. This is the default rounding mode for IEEE 754 computing functions and operators. -The other rounding modes are rarely used. Round toward positive infinity -(@code{roundTowardPositive}) and round toward negative infinity +The other rounding modes are rarely used. Rounding toward positive infinity +(@code{roundTowardPositive}) and toward negative infinity (@code{roundTowardNegative}) are often used to implement interval arithmetic, where you adjust the rounding mode to calculate upper and lower bounds for the range of output. The @code{roundTowardZero} mode can @@ -29976,14 +30092,14 @@ accumulation of round-off error, look for a significant difference in output when you change the rounding mode to be sure. @node Arbitrary Precision Integers -@section Arbitrary Precision Integer Arithmetic with @command{gawk} +@section Arbitrary-Precision Integer Arithmetic with @command{gawk} @cindex integers, arbitrary precision @cindex arbitrary precision integers When given the @option{-M} option, -@command{gawk} performs all integer arithmetic using GMP arbitrary -precision integers. Any number that looks like an integer in a source -or @value{DF} is stored as an arbitrary precision integer. The size +@command{gawk} performs all integer arithmetic using GMP arbitrary-precision +integers. Any number that looks like an integer in a source +or @value{DF} is stored as an arbitrary-precision integer. The size of the integer is limited only by the available memory. For example, the following computes @iftex @@ -29998,7 +30114,8 @@ the following computes 5<superscript>4<superscript>3<superscript>2</superscript></superscript></superscript>, @c @end docbook the result of which is beyond the -limits of ordinary hardware double precision floating point values: +limits of ordinary hardware double-precision floating-point values: +@c FIXME: Use @sup here for superscript @example $ @kbd{gawk -M 'BEGIN @{} @@ -30010,21 +30127,21 @@ $ @kbd{gawk -M 'BEGIN @{} @print{} 62060698786608744707 ... 92256259918212890625 @end example -If instead you were to compute the same value using arbitrary precision +If instead you were to compute the same value using arbitrary-precision floating-point values, the precision needed for correct output (using the formula @iftex -@math{prec = 3.322 @cdot dps}), +@math{prec = 3.322 @cdot dps}) would be @math{3.322 @cdot 183231}, @end iftex @ifnottex @ifnotdocbook -@samp{prec = 3.322 * dps}), +@samp{prec = 3.322 * dps}) would be 3.322 x 183231, @end ifnotdocbook @end ifnottex @docbook -<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis>), +<emphasis>prec</emphasis> = 3.322 ⋅ <emphasis>dps</emphasis>) would be <emphasis>prec</emphasis> = 3.322 ⋅ 183231, @c @end docbook @@ -30055,14 +30172,14 @@ floating-point results exactly. You can either increase the precision @samp{2.0} with an integer, to perform all computations using integer arithmetic to get the correct output. -Sometimes @command{gawk} must implicitly convert an arbitrary precision -integer into an arbitrary precision floating-point value. This is +Sometimes @command{gawk} must implicitly convert an arbitrary-precision +integer into an arbitrary-precision floating-point value. This is primarily because the MPFR library does not always provide the relevant -interface to process arbitrary precision integers or mixed-mode numbers +interface to process arbitrary-precision integers or mixed-mode numbers as needed by an operation or function. In such a case, the precision is set to the minimum value necessary for exact conversion, and the working precision is not used for this purpose. If this is not what you need or -want, you can employ a subterfuge, and convert the integer to floating +want, you can employ a subterfuge and convert the integer to floating point first, like this: @example @@ -30076,7 +30193,7 @@ to begin with: gawk -M 'BEGIN @{ n = 13.0; print n % 2.0 @}' @end example -Note that for the particular example above, it is likely best +Note that for this particular example, it is likely best to just use the following: @example @@ -30088,27 +30205,30 @@ When dividing two arbitrary precision integers with either precision floating point value (unless the denominator evenly divides into the numerator). In order to do integer division or remainder with arbitrary precision integers, use the built-in -@code{div()} function (@pxref{Numeric Functions}). +@code{intdiv()} function (@pxref{Numeric Functions}). -You can simulate the @code{div()} function in standard @command{awk} +You can simulate the @code{intdiv()} function in standard @command{awk} using this user-defined function: @example -@c file eg/lib/div.awk -# div --- do integer division +@c file eg/lib/intdiv.awk +# intdiv --- do integer division @c endfile @ignore -@c file eg/lib/div.awk +@c file eg/lib/intdiv.awk # # Arnold Robbins, arnold@@skeeve.com, Public Domain # July, 2014 +# +# Name changed from div() to intdiv() +# April, 2015 @c endfile @end ignore -@c file eg/lib/div.awk -function div(numerator, denominator, result) +@c file eg/lib/intdiv.awk +function intdiv(numerator, denominator, result) @{ split("", result) @@ -30123,7 +30243,7 @@ function div(numerator, denominator, result) @end example The following example program, contributed by Katie Wasserman, -uses @code{div()} to +uses @code{intdiv()} to compute the digits of @value{PI} to as many places as you choose to set: @@ -30148,7 +30268,7 @@ BEGIN @{ for (m = digits * 4; m > 0; --m) @{ d = m * 2 + 1 x = pi * m - div(x, d, result) + intdiv(x, d, result) pi = result["quotient"] pi = pi + two @} @@ -30187,7 +30307,7 @@ When asked about the algorithm used, Katie replied: @quotation It's not that well known but it's not that obscure either. It's Euler's modification to Newton's method for calculating pi. -Take a look at lines (23) - (25) here: @uref{http://mathworld.wolfram.com/PiFormulas.htm}. +Take a look at lines (23) - (25) here: @uref{http://mathworld.wolfram.com/PiFormulas.html}. The algorithm I wrote simply expands the multiply by 2 and works from the innermost expression outwards. I used this to program HP calculators @@ -30199,7 +30319,7 @@ word sizes. See @node POSIX Floating Point Problems @section Standards Versus Existing Practice -Historically, @command{awk} has converted any non-numeric looking string +Historically, @command{awk} has converted any nonnumeric-looking string to the numeric value zero, when required. Furthermore, the original definition of the language and the original POSIX standards specified that @command{awk} only understands decimal numbers (base 10), and not octal @@ -30211,13 +30331,13 @@ should support additional features. These features are: @itemize @value{BULLET} @item -Interpretation of floating point data values specified in hexadecimal +Interpretation of floating-point data values specified in hexadecimal notation (e.g., @code{0xDEADBEEF}). (Note: data values, @emph{not} source code constants.) @item -Support for the special IEEE 754 floating point values ``Not A Number'' -(NaN), positive Infinity (``inf'') and negative Infinity (``@minus{}inf''). +Support for the special IEEE 754 floating-point values ``not a number'' +(NaN), positive infinity (``inf''), and negative infinity (``@minus{}inf''). In particular, the format for these values is as specified by the ISO 1999 C standard, which ignores case and can allow implementation-dependent additional characters after the @samp{nan} and allow either @samp{inf} or @samp{infinity}. @@ -30228,8 +30348,8 @@ practice: @itemize @value{BULLET} @item -The @command{gawk} maintainer feels that supporting hexadecimal floating -point values, in particular, is ugly, and was never intended by the +The @command{gawk} maintainer feels that supporting hexadecimal +floating-point values, in particular, is ugly, and was never intended by the original designers to be part of the language. @item @@ -30237,22 +30357,22 @@ Allowing completely alphabetic strings to have valid numeric values is also a very severe departure from historical practice. @end itemize -The second problem is that the @code{gawk} maintainer feels that this -interpretation of the standard, which requires a certain amount of +The second problem is that the @command{gawk} maintainer feels that this +interpretation of the standard, which required a certain amount of ``language lawyering'' to arrive at in the first place, was not even -intended by the standard developers. In other words, ``we see how you +intended by the standard developers. In other words, ``We see how you got where you are, but we don't think that that's where you want to be.'' -Recognizing the above issues, but attempting to provide compatibility +Recognizing these issues, but attempting to provide compatibility with the earlier versions of the standard, the 2008 POSIX standard added explicit wording to allow, but not require, -that @command{awk} support hexadecimal floating point values and -special values for ``Not A Number'' and infinity. +that @command{awk} support hexadecimal floating-point values and +special values for ``not a number'' and infinity. Although the @command{gawk} maintainer continues to feel that providing those features is inadvisable, nevertheless, on systems that support IEEE floating point, it seems -reasonable to provide @emph{some} way to support NaN and Infinity values. +reasonable to provide @emph{some} way to support NaN and infinity values. The solution implemented in @command{gawk} is as follows: @itemize @value{BULLET} @@ -30272,7 +30392,7 @@ $ @kbd{echo 0xDeadBeef | gawk --posix '@{ print $1 + 0 @}'} @end example @item -Without @option{--posix}, @command{gawk} interprets the four strings +Without @option{--posix}, @command{gawk} interprets the four string values @samp{+inf}, @samp{-inf}, @samp{+nan}, @@ -30294,7 +30414,7 @@ $ @kbd{echo 0xDeadBeef | gawk '@{ print $1 + 0 @}'} @end example @command{gawk} ignores case in the four special values. -Thus @samp{+nan} and @samp{+NaN} are the same. +Thus, @samp{+nan} and @samp{+NaN} are the same. @end itemize @node Floating point summary @@ -30303,13 +30423,13 @@ Thus @samp{+nan} and @samp{+NaN} are the same. @itemize @value{BULLET} @item Most computer arithmetic is done using either integers or floating-point -values. Standard @command{awk} uses double precision +values. Standard @command{awk} uses double-precision floating-point values. @item -In the early 1990's, Barbie mistakenly said ``Math class is tough!'' -While math isn't tough, floating-point arithmetic isn't the same -as pencil and paper math, and care must be taken: +In the early 1990s Barbie mistakenly said, ``Math class is tough!'' +Although math isn't tough, floating-point arithmetic isn't the same +as pencil-and-paper math, and care must be taken: @c nested list @itemize @value{MINUS} @@ -30341,12 +30461,12 @@ arithmetic. Use @code{PREC} to set the precision in bits, and @item With @option{-M}, @command{gawk} performs -arbitrary precision integer arithmetic using the GMP library. -This is faster and more space efficient than using MPFR for +arbitrary-precision integer arithmetic using the GMP library. +This is faster and more space-efficient than using MPFR for the same calculations. @item -There are several ``dark corners'' with respect to floating-point +There are several areas with respect to floating-point numbers where @command{gawk} disagrees with the POSIX standard. It pays to be aware of them. @@ -30354,7 +30474,7 @@ It pays to be aware of them. Overall, there is no need to be unduly suspicious about the results from floating-point arithmetic. The lesson to remember is that floating-point arithmetic is always more complex than arithmetic using pencil and -paper. In order to take advantage of the power of computer floating-point, +paper. In order to take advantage of the power of floating-point arithmetic, you need to know its limitations and work within them. For most casual use of floating-point arithmetic, you will often get the expected result if you simply round the display of your final results to the correct number @@ -30396,7 +30516,7 @@ When @option{--sandbox} is specified, extensions are disabled * Finding Extensions:: How @command{gawk} finds compiled extensions. * Extension Example:: Example C code for an extension. * Extension Samples:: The sample extensions that ship with - @code{gawk}. + @command{gawk}. * gawkextlib:: The @code{gawkextlib} project. * Extension summary:: Extension summary. * Extension Exercises:: Exercises. @@ -30415,15 +30535,15 @@ Extensions are useful because they allow you (of course) to extend @command{gawk}'s functionality. For example, they can provide access to system calls (such as @code{chdir()} to change directory) and to other C library routines that could be of use. As with most software, -``the sky is the limit;'' if you can imagine something that you might +``the sky is the limit''; if you can imagine something that you might want to do and can write in C or C++, you can write an extension to do it! -Extensions are written in C or C++, using the @dfn{Application Programming -Interface} (API) defined for this purpose by the @command{gawk} +Extensions are written in C or C++, using the @dfn{application programming +interface} (API) defined for this purpose by the @command{gawk} developers. The rest of this @value{CHAPTER} explains the facilities that the API provides and how to use them, and presents a small example extension. In addition, it documents -the sample extensions included in the @command{gawk} distribution, +the sample extensions included in the @command{gawk} distribution and describes the @code{gawkextlib} project. @ifclear FOR_PRINT @xref{Extension Design}, for a discussion of the extension mechanism @@ -30456,7 +30576,7 @@ int plugin_is_GPL_compatible; @end example @node Extension Mechanism Outline -@section At A High Level How It Works +@section How It Works at a High Level Communication between @command{gawk} and an extension is two-way. First, when an extension @@ -30471,22 +30591,22 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-load-extension"/>}. @ifnotdocbook @float Figure,figure-load-extension -@caption{Loading The Extension} +@caption{Loading the extension} @c FIXME: One day, it should not be necessary to have two cases, @c but rather just the one without the "txt" final argument. @c This applies to the other figures as well. @ifinfo -@center @image{api-figure1, , , Loading The Extension, txt} +@center @image{api-figure1, , , Loading the extension, txt} @end ifinfo @ifnotinfo -@center @image{api-figure1, , , Loading The Extension} +@center @image{api-figure1, , , Loading the extension} @end ifnotinfo @end float @end ifnotdocbook @docbook <figure id="figure-load-extension" float="0"> -<title>Loading The Extension</title> +<title>Loading the extension</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure1.png" format="PNG"/></imageobject> </mediaobject> @@ -30506,19 +30626,19 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-register-new-function @ifnotdocbook @float Figure,figure-register-new-function -@caption{Registering A New Function} +@caption{Registering a new function} @ifinfo -@center @image{api-figure2, , , Registering A New Function, txt} +@center @image{api-figure2, , , Registering a new Function, txt} @end ifinfo @ifnotinfo -@center @image{api-figure2, , , Registering A New Function} +@center @image{api-figure2, , , Registering a new Function} @end ifnotinfo @end float @end ifnotdocbook @docbook <figure id="figure-register-new-function" float="0"> -<title>Registering A New Function</title> +<title>Registering a new function</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure2.png" format="PNG"/></imageobject> </mediaobject> @@ -30539,7 +30659,7 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-call-new-function"/>} @ifnotdocbook @float Figure,figure-call-new-function -@caption{Calling The New Function} +@caption{Calling the new function} @ifinfo @center @image{api-figure3, , , Calling the new function, txt} @end ifinfo @@ -30551,7 +30671,7 @@ This is shown in @inlineraw{docbook, <xref linkend="figure-call-new-function"/>} @docbook <figure id="figure-call-new-function" float="0"> -<title>Calling The New Function</title> +<title>Calling the new function</title> <mediaobject> <imageobject role="web"><imagedata fileref="api-figure3.png" format="PNG"/></imageobject> </mediaobject> @@ -30576,7 +30696,7 @@ Some other bits and pieces: @itemize @value{BULLET} @item The API provides access to @command{gawk}'s @code{do_@var{xxx}} values, -reflecting command-line options, like @code{do_lint}, @code{do_profiling} +reflecting command-line options, like @code{do_lint}, @code{do_profiling}, and so on (@pxref{Extension API Variables}). These are informational: an extension cannot affect their values inside @command{gawk}. In addition, attempting to assign to them @@ -30587,10 +30707,9 @@ The API also provides major and minor version numbers, so that an extension can check if the @command{gawk} it is loaded with supports the facilities it was compiled with. (Version mismatches ``shouldn't'' happen, but we all know how @emph{that} goes.) -@xref{Extension Versioning}, for details. +@DBXREF{Extension Versioning} for details. @end itemize - @node Extension API Description @section API Description @cindex extension API @@ -30614,6 +30733,7 @@ This (rather large) @value{SECTION} describes the API in detail. * Symbol Table Access:: Functions for accessing global variables. * Array Manipulation:: Functions for working with arrays. +* Redirection API:: How to access and manipulate redirections. * Extension API Variables:: Variables provided by the API. * Extension API Boilerplate:: Boilerplate code for using the API. @end menu @@ -30621,7 +30741,7 @@ This (rather large) @value{SECTION} describes the API in detail. @node Extension API Functions Introduction @subsection Introduction -Access to facilities within @command{gawk} are made available +Access to facilities within @command{gawk} is achieved by calling through function pointers passed into your extension. API function pointers are provided for the following kinds of operations: @@ -30632,21 +30752,24 @@ Allocating, reallocating, and releasing memory. @item Registration functions. You may register: + +@c nested list @itemize @value{MINUS} @item -extension functions, +Extension functions @item -exit callbacks, +Exit callbacks @item -a version string, +A version string @item -input parsers, +Input parsers @item -output wrappers, +Output wrappers @item -and two-way processors. +Two-way processors @end itemize -All of these are discussed in detail, later in this @value{CHAPTER}. + +All of these are discussed in detail later in this @value{CHAPTER}. @item Printing fatal, warning, and ``lint'' warning messages. @@ -30684,20 +30807,25 @@ Creating a new array Clearing an array @item -Flattening an array for easy C style looping over all its indices and elements +Flattening an array for easy C-style looping over all its indices and elements @end itemize + +@item +Accessing and manipulating redirections. + @end itemize Some points about using the API: @itemize @value{BULLET} @item -The following types and/or macros and/or functions are referenced +The following types, macros, and/or functions are referenced in @file{gawkapi.h}. For correct use, you must therefore include the corresponding standard header file @emph{before} including @file{gawkapi.h}: +@c FIXME: Make this is a float at some point. @multitable {@code{memset()}, @code{memcpy()}} {@code{<sys/types.h>}} -@headitem C Entity @tab Header File +@headitem C entity @tab Header file @item @code{EOF} @tab @code{<stdio.h>} @item Values for @code{errno} @tab @code{<errno.h>} @item @code{FILE} @tab @code{<stdio.h>} @@ -30706,7 +30834,7 @@ corresponding standard header file @emph{before} including @file{gawkapi.h}: @item @code{memset()} @tab @code{<string.h>} @item @code{size_t} @tab @code{<sys/types.h>} @item @code{struct stat} @tab @code{<sys/stat.h>} -@end multitable +@end multitable Due to portability concerns, especially to systems that are not fully standards-compliant, it is your responsibility @@ -30723,7 +30851,7 @@ Doing so, however, is poor coding practice. Although the API only uses ISO C 90 features, there is an exception; the ``constructor'' functions use the @code{inline} keyword. If your compiler does not support this keyword, you should either place -@samp{-Dinline=''} on your command line, or use the GNU Autotools and include a +@samp{-Dinline=''} on your command line or use the GNU Autotools and include a @file{config.h} file in your extensions. @item @@ -30731,21 +30859,21 @@ All pointers filled in by @command{gawk} point to memory managed by @command{gawk} and should be treated by the extension as read-only. Memory for @emph{all} strings passed into @command{gawk} from the extension @emph{must} come from calling one of -@code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}, +@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}, and is managed by @command{gawk} from then on. @item The API defines several simple @code{struct}s that map values as seen from @command{awk}. A value can be a @code{double}, a string, or an array (as in multidimensional arrays, or when creating a new array). -String values maintain both pointer and length since embedded @value{NUL} +String values maintain both pointer and length, because embedded @sc{nul} characters are allowed. @quotation NOTE By intent, strings are maintained using the current multibyte encoding (as defined by @env{LC_@var{xxx}} environment variables) and not using wide characters. This matches how @command{gawk} stores strings internally -and also how characters are likely to be input and output from files. +and also how characters are likely to be input into and output from files. @end quotation @item @@ -30763,14 +30891,14 @@ so that the extension can, e.g., print an error message @c The table there should be presented here @end itemize -While you may call the API functions by using the function pointers -directly, the interface is not so pretty. To make extension code look +You may call the API functions by using the function pointers +directly, but the interface is not so pretty. To make extension code look more like regular code, the @file{gawkapi.h} header file defines several macros that you should use in your code. This @value{SECTION} presents the macros as if they were functions. @node General Data Types -@subsection General Purpose Data Types +@subsection General-Purpose Data Types @cindex Robbins, Arnold @cindex Ramey, Chet @@ -30785,9 +30913,12 @@ can accommodate both love and hate.} @author Chet Ramey @end quotation -The extension API defines a number of simple types and structures for general -purpose use. Additional, more specialized, data structures are introduced -in subsequent @value{SECTION}s, together with the functions that use them. +The extension API defines a number of simple types and structures for +general-purpose use. Additional, more specialized, data structures are +introduced in subsequent @value{SECTION}s, together with the functions +that use them. + +The general-purpose types and structures are as follows: @table @code @item typedef void *awk_ext_id_t; @@ -30805,7 +30936,7 @@ while allowing @command{gawk} to use them as it needs to. @itemx @ @ @ @ awk_false = 0, @itemx @ @ @ @ awk_true @itemx @} awk_bool_t; -A simple boolean type. +A simple Boolean type. @item typedef struct awk_string @{ @itemx @ @ @ @ char *str;@ @ @ @ @ @ /* data */ @@ -30814,8 +30945,9 @@ A simple boolean type. This represents a mutable string. @command{gawk} owns the memory pointed to if it supplied the value. Otherwise, it takes ownership of the memory pointed to. -@strong{Such memory must come from calling one of the -@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()} functions!} +@emph{Such memory must come from calling one of the +@code{gawk_malloc()}, @code{gawk_calloc()}, or +@code{gawk_realloc()} functions!} As mentioned earlier, strings are maintained using the current multibyte encoding. @@ -30841,7 +30973,7 @@ It is used in the following @code{struct}. @itemx @ @ @ @ @ @ @ @ awk_value_cookie_t@ vc; @itemx @ @ @ @ @} u; @itemx @} awk_value_t; -An ``@command{awk} value.'' +An ``@command{awk} value.'' The @code{val_type} member indicates what kind of value the @code{union} holds, and each member is of the appropriate type. @@ -30850,17 +30982,18 @@ The @code{val_type} member indicates what kind of value the @itemx #define array_cookie@ @ @ u.a @itemx #define scalar_cookie@ @ u.scl @itemx #define value_cookie@ @ @ u.vc -These macros make accessing the fields of the @code{awk_value_t} more +Using these macros makes accessing the fields of the @code{awk_value_t} more readable. @item typedef void *awk_scalar_t; -Scalars can be represented as an opaque type. These values are obtained from -@command{gawk} and then passed back into it. This is discussed in a general fashion below, -and in more detail in @ref{Symbol table by cookie}. +Scalars can be represented as an opaque type. These values are obtained +from @command{gawk} and then passed back into it. This is discussed +in a general fashion in the text following this list, and in more detail in +@ref{Symbol table by cookie}. @item typedef void *awk_value_cookie_t; A ``value cookie'' is an opaque type representing a cached value. -This is also discussed in a general fashion below, +This is also discussed in a general fashion in the text following this list, and in more detail in @ref{Cached values}. @end table @@ -30870,9 +31003,9 @@ Scalar values in @command{awk} are either numbers or strings. The indicates what is in the @code{union}. Representing numbers is easy---the API uses a C @code{double}. Strings -require more work. Since @command{gawk} allows embedded @value{NUL} bytes +require more work. Because @command{gawk} allows embedded @sc{nul} bytes in string values, a string must be represented as a pair containing a -data-pointer and length. This is the @code{awk_string_t} type. +data pointer and length. This is the @code{awk_string_t} type. Identifiers (i.e., the names of global variables) can be associated with either scalar values or with arrays. In addition, @command{gawk} @@ -30883,14 +31016,14 @@ itself be an array. Discussion of arrays is delayed until The various macros listed earlier make it easier to use the elements of the @code{union} as if they were fields in a @code{struct}; this is a common coding practice in C. Such code is easier to write and to -read, however it remains @emph{your} responsibility to make sure that +read, but it remains @emph{your} responsibility to make sure that the @code{val_type} member correctly reflects the type of the value in -the @code{awk_value_t}. +the @code{awk_value_t} struct. Conceptually, the first three members of the @code{union} (number, string, and array) are all that is needed for working with @command{awk} values. -However, since the API provides routines for accessing and changing -the value of global scalar variables only by using the variable's name, +However, because the API provides routines for accessing and changing +the value of a global scalar variable only by using the variable's name, there is a performance penalty: @command{gawk} must find the variable each time it is accessed and changed. This turns out to be a real issue, not just a theoretical one. @@ -30908,7 +31041,9 @@ See also the entry for ``Cookie'' in the @ref{Glossary}. object for that variable, and then use the cookie for getting the variable's value or for changing the variable's value. -This is the @code{awk_scalar_t} type and @code{scalar_cookie} macro. +The @code{awk_scalar_t} type holds a scalar cookie, and the +@code{scalar_cookie} macro provides access to the value of that type +in the @code{awk_value_t} struct. Given a scalar cookie, @command{gawk} can directly retrieve or modify the value, as required, without having to find it first. @@ -30917,8 +31052,8 @@ If you know that you wish to use the same numeric or string @emph{value} for one or more variables, you can create the value once, retaining a @dfn{value cookie} for it, and then pass in that value cookie whenever you wish to set the value of a -variable. This saves both storage space within the running @command{gawk} -process as well as the time needed to create the value. +variable. This saves storage space within the running @command{gawk} +process and reduces the time needed to create the value. @node Memory Allocation Functions @subsection Memory Allocation Functions and Convenience Macros @@ -30929,7 +31064,7 @@ The API provides a number of @dfn{memory allocation} functions for allocating memory that can be passed to @command{gawk}, as well as a number of convenience macros. This @value{SUBSECTION} presents them all as function prototypes, in -the way that extension code would use them. +the way that extension code would use them: @table @code @item void *gawk_malloc(size_t size); @@ -30946,13 +31081,13 @@ be passed to @command{gawk}. @item void gawk_free(void *ptr); Call the correct version of @code{free()} to release storage that was -allocated with @code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}. +allocated with @code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. @end table The API has to provide these functions because it is possible for an extension to be compiled and linked against a different version of the C library than was used for the @command{gawk} -executable.@footnote{This is more common on MS-Windows systems, but +executable.@footnote{This is more common on MS-Windows systems, but it can happen on Unix-like systems as well.} If @command{gawk} were to use its version of @code{free()} when the memory came from an unrelated version of @code{malloc()}, unexpected behavior would @@ -30962,7 +31097,7 @@ Two convenience macros may be used for allocating storage from @code{gawk_malloc()} and @code{gawk_realloc()}. If the allocation fails, they cause @command{gawk} to exit with a fatal error message. They should be used as if they were -procedure calls that do not return a value. +procedure calls that do not return a value: @table @code @item #define emalloc(pointer, type, size, message) @dots{} @@ -30974,7 +31109,8 @@ The arguments to this macro are as follows: The pointer variable to point at the allocated storage. @item type -The type of the pointer variable, used to create a cast for the call to @code{gawk_malloc()}. +The type of the pointer variable. This is used to create a cast for +the call to @code{gawk_malloc()}. @item size The total number of bytes to be allocated. @@ -30998,7 +31134,7 @@ make_malloced_string(message, strlen(message), & result); @end example @item #define erealloc(pointer, type, size, message) @dots{} -This is like @code{emalloc()}, but it calls @code{gawk_realloc()}, +This is like @code{emalloc()}, but it calls @code{gawk_realloc()} instead of @code{gawk_malloc()}. The arguments are the same as for the @code{emalloc()} macro. @end table @@ -31009,32 +31145,32 @@ The arguments are the same as for the @code{emalloc()} macro. The API provides a number of @dfn{constructor} functions for creating string and numeric values, as well as a number of convenience macros. This @value{SUBSECTION} presents them all as function prototypes, in -the way that extension code would use them. +the way that extension code would use them: @table @code @item static inline awk_value_t * -@itemx make_const_string(const char *string, size_t length, awk_value_t *result) +@itemx make_const_string(const char *string, size_t length, awk_value_t *result); This function creates a string value in the @code{awk_value_t} variable pointed to by @code{result}. It expects @code{string} to be a C string constant (or other string data), and automatically creates a @emph{copy} of the data for storage in @code{result}. It returns @code{result}. @item static inline awk_value_t * -@itemx make_malloced_string(const char *string, size_t length, awk_value_t *result) +@itemx make_malloced_string(const char *string, size_t length, awk_value_t *result); This function creates a string value in the @code{awk_value_t} variable pointed to by @code{result}. It expects @code{string} to be a @samp{char *} -value pointing to data previously obtained from @code{gawk_malloc()}, @code{gawk_calloc()} or @code{gawk_realloc()}. The idea here +value pointing to data previously obtained from @code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. The idea here is that the data is passed directly to @command{gawk}, which assumes responsibility for it. It returns @code{result}. @item static inline awk_value_t * -@itemx make_null_string(awk_value_t *result) +@itemx make_null_string(awk_value_t *result); This specialized function creates a null string (the ``undefined'' value) in the @code{awk_value_t} variable pointed to by @code{result}. It returns @code{result}. @item static inline awk_value_t * -@itemx make_number(double num, awk_value_t *result) +@itemx make_number(double num, awk_value_t *result); This function simply creates a numeric value in the @code{awk_value_t} variable pointed to by @code{result}. @end table @@ -31074,7 +31210,7 @@ The fields are: @table @code @item const char *name; The name of the new function. -@command{awk} level code calls the function by this name. +@command{awk}-level code calls the function by this name. This is a regular C string. Function names must obey the rules for @command{awk} @@ -31088,8 +31224,8 @@ This is a pointer to the C function that provides the extension's functionality. The function must fill in @code{*result} with either a number or a string. @command{gawk} takes ownership of any string memory. -As mentioned earlier, string memory @strong{must} come from one of @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}. +As mentioned earlier, string memory @emph{must} come from one of +@code{gawk_malloc()}, @code{gawk_calloc()}, or @code{gawk_realloc()}. The @code{num_actual_args} argument tells the C function how many actual parameters were passed from the calling @command{awk} code. @@ -31124,7 +31260,7 @@ Such functions are useful if you have general ``cleanup'' tasks that should be performed in your extension (such as closing database connections or other resource deallocations). You can register such -a function with @command{gawk} using the following function. +a function with @command{gawk} using the following function: @table @code @item void awk_atexit(void (*funcp)(void *data, int exit_status), @@ -31140,24 +31276,25 @@ The @code{exit_status} parameter is the exit status value that @command{gawk} intends to pass to the @code{exit()} system call. @item arg0 -A pointer to private data which @command{gawk} saves in order to pass to +A pointer to private data that @command{gawk} saves in order to pass to the function pointed to by @code{funcp}. @end table @end table -Exit callback functions are called in Last-In-First-Out (LIFO) order---that is, in -the reverse order in which they are registered with @command{gawk}. +Exit callback functions are called in last-in, first-out (LIFO) +order---that is, in the reverse order in which they are registered with +@command{gawk}. @node Extension Version String @subsubsection Registering An Extension Version String -You can register a version string which indicates the name and -version of your extension, with @command{gawk}, as follows: +You can register a version string that indicates the name and +version of your extension with @command{gawk}, as follows: @table @code @item void register_ext_version(const char *version); Register the string pointed to by @code{version} with @command{gawk}. -@command{gawk} does @emph{not} copy the @code{version} string, so +Note that @command{gawk} does @emph{not} copy the @code{version} string, so it should not be changed. @end table @@ -31174,7 +31311,7 @@ of @code{RS} to find the end of the record, and then uses @code{FS} Additionally, it sets the value of @code{RT} (@pxref{Built-in Variables}). If you want, you can provide your own custom input parser. An input -parser's job is to return a record to the @command{gawk} record processing +parser's job is to return a record to the @command{gawk} record-processing code, along with indicators for the value and length of the data to be used for @code{RT}, if any. @@ -31182,19 +31319,19 @@ To provide an input parser, you must first provide two functions (where @var{XXX} is a prefix name for your extension): @table @code -@item awk_bool_t @var{XXX}_can_take_file(const awk_input_buf_t *iobuf) +@item awk_bool_t @var{XXX}_can_take_file(const awk_input_buf_t *iobuf); This function examines the information available in @code{iobuf} (which we discuss shortly). Based on the information there, it decides if the input parser should be used for this file. If so, it should return true. Otherwise, it should return false. It should not change any state (variable values, etc.) within @command{gawk}. -@item awk_bool_t @var{XXX}_take_control_of(awk_input_buf_t *iobuf) +@item awk_bool_t @var{XXX}_take_control_of(awk_input_buf_t *iobuf); When @command{gawk} decides to hand control of the file over to the input parser, it calls this function. This function in turn must fill -in certain fields in the @code{awk_input_buf_t} structure, and ensure +in certain fields in the @code{awk_input_buf_t} structure and ensure that certain conditions are true. It should then return true. If an -error of some kind occurs, it should not fill in any fields, and should +error of some kind occurs, it should not fill in any fields and should return false; then @command{gawk} will not use the input parser. The details are presented shortly. @end table @@ -31239,7 +31376,7 @@ appropriately. @item When your extension is loaded, register your input parser with @command{gawk} using the @code{register_input_parser()} API function -(described below). +(described next). @end enumerate An @code{awk_input_buf_t} looks like this: @@ -31269,7 +31406,7 @@ The name of the file. @item int fd; A file descriptor for the file. If @command{gawk} was able to -open the file, then @code{fd} will @emph{not} be equal to +open the file, then @code{fd} will @emph{not} be equal to @code{INVALID_HANDLE}. Otherwise, it will. @item struct stat sbuf; @@ -31283,15 +31420,15 @@ The decision can be made based upon @command{gawk} state (the value of a variable defined previously by the extension and set by @command{awk} code), the name of the file, whether or not the file descriptor is valid, the information -in the @code{struct stat}, or any combination of the above. +in the @code{struct stat}, or any combination of these factors. Once @code{@var{XXX}_can_take_file()} has returned true, and @command{gawk} has decided to use your input parser, it calls -@code{@var{XXX}_take_control_of()}. That function then fills one of +@code{@var{XXX}_take_control_of()}. That function then fills either the @code{get_record} field or the @code{read_func} field in the @code{awk_input_buf_t}. It must also ensure that @code{fd} is @emph{not} -set to @code{INVALID_HANDLE}. All of the fields that may be filled by -@code{@var{XXX}_take_control_of()} are as follows: +set to @code{INVALID_HANDLE}. The following list describes the fields that +may be filled by @code{@var{XXX}_take_control_of()}: @table @code @item void *opaque; @@ -31306,24 +31443,24 @@ is not required to use this pointer. @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ size_t *rt_len); This function pointer should point to a function that creates the input records. Said function is the core of the input parser. Its behavior -is described below. +is described in the text following this list. @item ssize_t (*read_func)(); -This function pointer should point to function that has the +This function pointer should point to a function that has the same behavior as the standard POSIX @code{read()} system call. It is an alternative to the @code{get_record} pointer. Its behavior -is also described below. +is also described in the text following this list. @item void (*close_func)(struct awk_input *iobuf); This function pointer should point to a function that does -the ``tear down.'' It should release any resources allocated by +the ``teardown.'' It should release any resources allocated by @code{@var{XXX}_take_control_of()}. It may also close the file. If it does so, it should set the @code{fd} field to @code{INVALID_HANDLE}. If @code{fd} is still not @code{INVALID_HANDLE} after the call to this function, @command{gawk} calls the regular @code{close()} system call. -Having a ``tear down'' function is optional. If your input parser does +Having a ``teardown'' function is optional. If your input parser does not need it, do not set this field. Then, @command{gawk} calls the regular @code{close()} system call on the file descriptor, so it should be valid. @@ -31334,7 +31471,7 @@ input records. The parameters are as follows: @table @code @item char **out -This is a pointer to a @code{char *} variable which is set to point +This is a pointer to a @code{char *} variable that is set to point to the record. @command{gawk} makes its own copy of the data, so the extension must manage this storage. @@ -31353,7 +31490,7 @@ If the concept of a ``record terminator'' makes sense, then @code{*rt_start} should be set to point to the data to be used for @code{RT}, and @code{*rt_len} should be set to the length of the data. Otherwise, @code{*rt_len} should be set to zero. -@code{gawk} makes its own copy of this data, so the +@command{gawk} makes its own copy of this data, so the extension must manage this storage. @end table @@ -31387,19 +31524,19 @@ set this field explicitly. You must choose one method or the other: either a function that returns a record, or one that returns raw data. In particular, if you supply a function to get a record, @command{gawk} will -call it, and never call the raw read function. +call it, and will never call the raw read function. @end quotation @command{gawk} ships with a sample extension that reads directories, -returning records for each entry in the directory (@pxref{Extension +returning records for each entry in a directory (@pxref{Extension Sample Readdir}). You may wish to use that code as a guide for writing your own input parser. When writing an input parser, you should think about (and document) how it is expected to interact with @command{awk} code. You may want -it to always be called, and take effect as appropriate (as the +it to always be called, and to take effect as appropriate (as the @code{readdir} extension does). Or you may want it to take effect -based upon the value of an @code{awk} variable, as the XML extension +based upon the value of an @command{awk} variable, as the XML extension from the @code{gawkextlib} project does (@pxref{gawkextlib}). In the latter case, code in a @code{BEGINFILE} section can look at @code{FILENAME} and @code{ERRNO} to decide whether or @@ -31450,7 +31587,7 @@ values, etc.) within @command{gawk}. The function pointed to by this field is called when @command{gawk} decides to let the output wrapper take control of the file. It should fill in appropriate members of the @code{awk_output_buf_t} structure, -as described below, and return true if successful, false otherwise. +as described next, and return true if successful, false otherwise. @item awk_const struct output_wrapper *awk_const next; This is for use by @command{gawk}; @@ -31507,7 +31644,7 @@ a pointer to any private data associated with the file. These pointers should be set to point to functions that perform the equivalent function as the @code{<stdio.h>} functions do, if appropriate. @command{gawk} uses these function pointers for all output. -@command{gawk} initializes the pointers to point to internal, ``pass through'' +@command{gawk} initializes the pointers to point to internal ``pass-through'' functions that just call the regular @code{<stdio.h>} functions, so an extension only needs to redefine those functions that are appropriate for what it does. @@ -31518,7 +31655,7 @@ upon the @code{name} and @code{mode} fields, and any additional state (such as @command{awk} variable values) that is appropriate. When @command{gawk} calls @code{@var{XXX}_take_control_of()}, that function should fill -in the other fields, as appropriate, except for @code{fp}, which it should just +in the other fields as appropriate, except for @code{fp}, which it should just use normally. You register your output wrapper with the following function: @@ -31558,14 +31695,14 @@ The fields are as follows: The name of the two-way processor. @item awk_bool_t (*can_take_two_way)(const char *name); -This function returns true if it wants to take over two-way I/O for this @value{FN}. +The function pointed to by this field should return true if it wants to take over two-way I/O for this @value{FN}. It should not change any state (variable values, etc.) within @command{gawk}. @item awk_bool_t (*take_control_of)(const char *name, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_input_buf_t *inbuf, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_output_buf_t *outbuf); -This function should fill in the @code{awk_input_buf_t} and +The function pointed to by this field should fill in the @code{awk_input_buf_t} and @code{awk_outut_buf_t} structures pointed to by @code{inbuf} and @code{outbuf}, respectively. These structures were described earlier. @@ -31593,9 +31730,9 @@ Register the two-way processor pointed to by @code{two_way_processor} with @cindex messages from extensions You can print different kinds of warning messages from your -extension, as described below. Note that for these functions, -you must pass in the extension id received from @command{gawk} -when the extension was loaded.@footnote{Because the API uses only ISO C 90 +extension, as described here. Note that for these functions, +you must pass in the extension ID received from @command{gawk} +when the extension was loaded:@footnote{Because the API uses only ISO C 90 features, it cannot make use of the ISO C 99 variadic macro feature to hide that parameter. More's the pity.} @@ -31647,12 +31784,12 @@ matches what you requested, the function returns true and fills in the @code{awk_value_t} result. Otherwise, the function returns false, and the @code{val_type} member indicates the type of the actual value. You may then -print an error message, or reissue the request for the actual +print an error message or reissue the request for the actual value type, as appropriate. This behavior is summarized in @ref{table-value-types-returned}. @float Table,table-value-types-returned -@caption{API Value Types Returned} +@caption{API value types returned} @docbook <informaltable> <tgroup cols="6"> @@ -31664,7 +31801,7 @@ value type, as appropriate. This behavior is summarized in <colspec colwidth="16.6*" colname="c6"/> <spanspec spanname="hspan" namest="c3" nameend="c6" align="center"/> <thead> - <row><entry></entry><entry spanname="hspan"><para>Type of Actual Value:</para></entry></row> + <row><entry></entry><entry spanname="hspan"><para>Type of Actual Value</para></entry></row> <row> <entry></entry> <entry></entry> @@ -31680,32 +31817,32 @@ value type, as appropriate. This behavior is summarized in <entry><para><emphasis role="bold">String</emphasis></para></entry> <entry><para>String</para></entry> <entry><para>String</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry></entry> <entry><para><emphasis role="bold">Number</emphasis></para></entry> <entry><para>Number if can be converted, else false</para></entry> <entry><para>Number</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry><para><emphasis role="bold">Type</emphasis></para></entry> <entry><para><emphasis role="bold">Array</emphasis></para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> <entry><para>Array</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> </row> <row> - <entry><para><emphasis role="bold">Requested:</emphasis></para></entry> + <entry><para><emphasis role="bold">Requested</emphasis></para></entry> <entry><para><emphasis role="bold">Scalar</emphasis></para></entry> <entry><para>Scalar</para></entry> <entry><para>Scalar</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> </row> <row> <entry></entry> @@ -31717,11 +31854,11 @@ value type, as appropriate. This behavior is summarized in </row> <row> <entry></entry> - <entry><para><emphasis role="bold">Value Cookie</emphasis></para></entry> - <entry><para>false</para></entry> - <entry><para>false</para></entry> - <entry><para>false</para> - </entry><entry><para>false</para></entry> + <entry><para><emphasis role="bold">Value cookie</emphasis></para></entry> + <entry><para>False</para></entry> + <entry><para>False</para></entry> + <entry><para>False</para> + </entry><entry><para>False</para></entry> </row> </tbody> </tgroup> @@ -31731,7 +31868,7 @@ value type, as appropriate. This behavior is summarized in @ifnotplaintext @ifnotdocbook @multitable @columnfractions .50 .50 -@headitem @tab Type of Actual Value: +@headitem @tab Type of Actual Value @end multitable @c 10/2014: Thanks to Karl Berry for this bit to reduce the space: @tex @@ -31739,12 +31876,12 @@ value type, as appropriate. This behavior is summarized in @end tex @multitable @columnfractions .166 .166 .198 .15 .15 .166 @headitem @tab @tab String @tab Number @tab Array @tab Undefined -@item @tab @b{String} @tab String @tab String @tab false @tab false -@item @tab @b{Number} @tab Number if can be converted, else false @tab Number @tab false @tab false -@item @b{Type} @tab @b{Array} @tab false @tab false @tab Array @tab false -@item @b{Requested:} @tab @b{Scalar} @tab Scalar @tab Scalar @tab false @tab false +@item @tab @b{String} @tab String @tab String @tab False @tab False +@item @tab @b{Number} @tab Number if can be converted, else false @tab Number @tab False @tab False +@item @b{Type} @tab @b{Array} @tab False @tab False @tab Array @tab False +@item @b{Requested} @tab @b{Scalar} @tab Scalar @tab Scalar @tab False @tab False @item @tab @b{Undefined} @tab String @tab Number @tab Array @tab Undefined -@item @tab @b{Value Cookie} @tab false @tab false @tab false @tab false +@item @tab @b{Value cookie} @tab False @tab False @tab False @tab False @end multitable @end ifnotdocbook @end ifnotplaintext @@ -31755,21 +31892,21 @@ value type, as appropriate. This behavior is summarized in +------------+------------+-----------+-----------+ | String | Number | Array | Undefined | +-----------+-----------+------------+------------+-----------+-----------+ -| | String | String | String | false | false | +| | String | String | String | False | False | | |-----------+------------+------------+-----------+-----------+ -| | Number | Number if | Number | false | false | +| | Number | Number if | Number | False | False | | | | can be | | | | | | | converted, | | | | | | | else false | | | | | |-----------+------------+------------+-----------+-----------+ -| Type | Array | false | false | Array | false | +| Type | Array | False | False | Array | False | | Requested |-----------+------------+------------+-----------+-----------+ -| | Scalar | Scalar | Scalar | false | false | +| | Scalar | Scalar | Scalar | False | False | | |-----------+------------+------------+-----------+-----------+ | | Undefined | String | Number | Array | Undefined | | |-----------+------------+------------+-----------+-----------+ -| | Value | false | false | false | false | -| | Cookie | | | | | +| | Value | False | False | False | False | +| | cookie | | | | | +-----------+-----------+------------+------------+-----------+-----------+ @end example @end ifplaintext @@ -31786,17 +31923,17 @@ passed to your extension function. They are: @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_valtype_t wanted, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_value_t *result); Fill in the @code{awk_value_t} structure pointed to by @code{result} -with the @code{count}'th argument. Return true if the actual -type matches @code{wanted}, false otherwise. In the latter +with the @code{count}th argument. Return true if the actual +type matches @code{wanted}, and false otherwise. In the latter case, @code{result@w{->}val_type} indicates the actual type -(@pxref{table-value-types-returned}). Counts are zero based---the first +(@pxref{table-value-types-returned}). Counts are zero-based---the first argument is numbered zero, the second one, and so on. @code{wanted} indicates the type of value expected. @item awk_bool_t set_argument(size_t count, awk_array_t array); Convert a parameter that was undefined into an array; this provides -call-by-reference for arrays. Return false if @code{count} is too big, -or if the argument's type is not undefined. @xref{Array Manipulation}, +call by reference for arrays. Return false if @code{count} is too big, +or if the argument's type is not undefined. @DBXREF{Array Manipulation} for more information on creating arrays. @end table @@ -31819,8 +31956,9 @@ allows you to create and release cached values. The following routines provide the ability to access and update global @command{awk}-level variables by name. In compiler terminology, identifiers of different kinds are termed @dfn{symbols}, thus the ``sym'' -in the routines' names. The data structure which stores information +in the routines' names. The data structure that stores information about symbols is termed a @dfn{symbol table}. +The functions are as follows: @table @code @item awk_bool_t sym_lookup(const char *name, @@ -31829,14 +31967,14 @@ about symbols is termed a @dfn{symbol table}. Fill in the @code{awk_value_t} structure pointed to by @code{result} with the value of the variable named by the string @code{name}, which is a regular C string. @code{wanted} indicates the type of value expected. -Return true if the actual type matches @code{wanted}, false otherwise. +Return true if the actual type matches @code{wanted}, and false otherwise. In the latter case, @code{result->val_type} indicates the actual type (@pxref{table-value-types-returned}). @item awk_bool_t sym_update(const char *name, awk_value_t *value); Update the variable named by the string @code{name}, which is a regular C string. The variable is added to @command{gawk}'s symbol table -if it is not there. Return true if everything worked, false otherwise. +if it is not there. Return true if everything worked, and false otherwise. Changing types (scalar to array or vice versa) of an existing variable is @emph{not} allowed, nor may this routine be used to update an array. @@ -31851,7 +31989,7 @@ cannot change any of those variables. @quotation CAUTION It is possible for the lookup of @code{PROCINFO} to fail. This happens if the @command{awk} program being run does not reference @code{PROCINFO}; -in this case @command{gawk} doesn't bother to create the array and +in this case, @command{gawk} doesn't bother to create the array and populate it. @end quotation @@ -31863,7 +32001,7 @@ to a global variable or array. It is an optimization that avoids looking up variables in @command{gawk}'s symbol table every time access is needed. This was discussed earlier, in @ref{General Data Types}. -The following functions let you work with scalar cookies. +The following functions let you work with scalar cookies: @table @code @item awk_bool_t sym_lookup_scalar(awk_scalar_t cookie, @@ -31908,18 +32046,21 @@ do_magic(int nargs, awk_value_t *result) @noindent This code looks (and is) simple and straightforward. So what's the problem? -Consider what happens if @command{awk}-level code associated with your -extension calls the @code{magic()} function (implemented in C by @code{do_magic()}), -once per record, while processing hundreds of thousands or millions of records. -The @code{MAGIC_VAR} variable is looked up in the symbol table once or twice per function call! +Well, consider what happens if @command{awk}-level code associated +with your extension calls the @code{magic()} function (implemented in +C by @code{do_magic()}), once per record, while processing hundreds +of thousands or millions of records. The @code{MAGIC_VAR} variable is +looked up in the symbol table once or twice per function call! -The symbol table lookup is really pure overhead; it is considerably more efficient -to get a cookie that represents the variable, and use that to get the variable's -value and update it as needed.@footnote{The difference is measurable and quite real. Trust us.} +The symbol table lookup is really pure overhead; it is considerably +more efficient to get a cookie that represents the variable, and use +that to get the variable's value and update it as needed.@footnote{The +difference is measurable and quite real. Trust us.} -Thus, the way to use cookies is as follows. First, install your extension's variable -in @command{gawk}'s symbol table using @code{sym_update()}, as usual. Then get a -scalar cookie for the variable using @code{sym_lookup()}: +Thus, the way to use cookies is as follows. First, install +your extension's variable in @command{gawk}'s symbol table using +@code{sym_update()}, as usual. Then get a scalar cookie for the variable +using @code{sym_lookup()}: @example static awk_scalar_t magic_var_cookie; /* cookie for MAGIC_VAR */ @@ -31974,7 +32115,7 @@ and carefully check the return values from the API functions. @subsubsection Creating and Using Cached Values The routines in this section allow you to create and release -cached values. As with scalar cookies, in theory, cached values +cached values. Like scalar cookies, in theory, cached values are not necessary. You can create numbers and strings using the functions in @ref{Constructor Functions}. You can then assign those values to variables using @code{sym_update()} @@ -31982,7 +32123,7 @@ or @code{sym_update_scalar()}, as you like. However, you can understand the point of cached values if you remember that @emph{every} string value's storage @emph{must} come from @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}. +@code{gawk_calloc()}, or @code{gawk_realloc()}. If you have 20 variables, all of which have the same string value, you must create 20 identical copies of the string.@footnote{Numeric values are clearly less problematic, requiring only a C @code{double} to store.} @@ -31993,11 +32134,11 @@ is what the routines in this section let you do. The functions are as follows: @table @code @item awk_bool_t create_value(awk_value_t *value, awk_value_cookie_t *result); -Create a cached string or numeric value from @code{value} for efficient later -assignment. -Only values of type @code{AWK_NUMBER} and @code{AWK_STRING} are allowed. Any other type -is rejected. While @code{AWK_UNDEFINED} could be allowed, doing so would -result in inferior performance. +Create a cached string or numeric value from @code{value} for +efficient later assignment. Only values of type @code{AWK_NUMBER} +and @code{AWK_STRING} are allowed. Any other type is rejected. +@code{AWK_UNDEFINED} could be allowed, but doing so would result in +inferior performance. @item awk_bool_t release_value(awk_value_cookie_t vc); Release the memory associated with a value cookie obtained @@ -32018,7 +32159,7 @@ my_extension_init() size_t long_string_len; /* code from earlier */ - @dots{} + @dots{} /* @dots{} fill in long_string and long_string_len @dots{} */ make_malloced_string(long_string, long_string_len, & value); create_value(& value, & answer_cookie); /* create cookie */ @@ -32048,11 +32189,11 @@ do_magic(int nargs, awk_value_t *result) @end example @noindent -Using value cookies in this way saves considerable storage, since all of +Using value cookies in this way saves considerable storage, as all of @code{VAR1} through @code{VAR100} share the same value. You might be wondering, ``Is this sharing problematic? -What happens if @command{awk} code assigns a new value to @code{VAR1}, +What happens if @command{awk} code assigns a new value to @code{VAR1}; are all the others changed too?'' That's a great question. The answer is that no, it's not a problem. @@ -32070,7 +32211,7 @@ you should release any cached values that you created, using @subsection Array Manipulation @cindex array manipulation in extensions -The primary data structure@footnote{Okay, the only data structure.} in @command{awk} +The primary data structure@footnote{OK, the only data structure.} in @command{awk} is the associative array (@pxref{Arrays}). Extensions need to be able to manipulate @command{awk} arrays. The API provides a number of data structures for working with arrays, @@ -32091,7 +32232,7 @@ both work with and create true arrays of arrays (@pxref{General Data Types}). @node Array Data Types @subsubsection Array Data Types -The data types associated with arrays are listed below. +The data types associated with arrays are as follows: @table @code @item typedef void *awk_array_t; @@ -32156,7 +32297,7 @@ modify them. @node Array Functions @subsubsection Array Functions -The following functions relate to individual array elements. +The following functions relate to individual array elements: @table @code @item awk_bool_t get_element_count(awk_array_t a_cookie, size_t *count); @@ -32175,13 +32316,13 @@ Return false if @code{wanted} does not match the actual type or if @code{index} is not in the array (@pxref{table-value-types-returned}). The value for @code{index} can be numeric, in which case @command{gawk} -converts it to a string. Using non-integral values is possible, but +converts it to a string. Using nonintegral values is possible, but requires that you understand how such values are converted to strings -(@pxref{Conversion}); thus using integral values is safest. +(@pxref{Conversion}); thus, using integral values is safest. -As with @emph{all} strings passed into @code{gawk} from an extension, +As with @emph{all} strings passed into @command{gawk} from an extension, the string value of @code{index} must come from @code{gawk_malloc()}, -@code{gawk_calloc()} or @code{gawk_realloc()}, and +@code{gawk_calloc()}, or @code{gawk_realloc()}, and @command{gawk} releases the storage. @item awk_bool_t set_array_element(awk_array_t a_cookie, @@ -32210,7 +32351,7 @@ The following functions relate to arrays as a whole: @table @code @item awk_array_t create_array(void); Create a new array to which elements may be added. -@xref{Creating Arrays}, for a discussion of how to +@DBXREF{Creating Arrays} for a discussion of how to create a new array and add elements to it. @item awk_bool_t clear_array(awk_array_t a_cookie); @@ -32237,7 +32378,7 @@ flatten an array and work with it. @item awk_bool_t release_flattened_array(awk_array_t a_cookie, @itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ awk_flat_array_t *data); When done with a flattened array, release the storage using this function. -You must pass in both the original array cookie, and the address of +You must pass in both the original array cookie and the address of the created @code{awk_flat_array_t} structure. The function returns true upon success, false otherwise. @end table @@ -32245,9 +32386,9 @@ The function returns true upon success, false otherwise. @node Flattening Arrays @subsubsection Working With All The Elements of an Array -To @dfn{flatten} an array is create a structure that +To @dfn{flatten} an array is to create a structure that represents the full array in a fashion that makes it easy -for C code to traverse the entire array. Test code +for C code to traverse the entire array. Some of the code in @file{extension/testext.c} does this, and also serves as a nice example showing how to use the APIs. @@ -32304,9 +32445,9 @@ dump_array_and_delete(int nargs, awk_value_t *result) @end example The function then proceeds in steps, as follows. First, retrieve -the name of the array, passed as the first argument. Then -retrieve the array itself. If either operation fails, print -error messages and return: +the name of the array, passed as the first argument, followed by +the array itself. If either operation fails, print an +error message and return: @example /* get argument named array as flat array and print it */ @@ -32342,7 +32483,7 @@ and print it: @end example The third step is to actually flatten the array, and then -to double check that the count in the @code{awk_flat_array_t} +to double-check that the count in the @code{awk_flat_array_t} is the same as the count just retrieved: @example @@ -32363,7 +32504,7 @@ is the same as the count just retrieved: The fourth step is to retrieve the index of the element to be deleted, which was passed as the second argument. Remember that argument counts passed to @code{get_argument()} -are zero-based, thus the second argument is numbered one: +are zero-based, and thus the second argument is numbered one: @example if (! get_argument(1, AWK_STRING, & value3)) @{ @@ -32378,7 +32519,7 @@ element values. In addition, upon finding the element with the index that is supposed to be deleted, the function sets the @code{AWK_ELEMENT_DELETE} bit in the @code{flags} field of the element. When the array is released, @command{gawk} -traverses the flattened array, and deletes any elements which +traverses the flattened array, and deletes any elements that have this flag bit set: @example @@ -32413,7 +32554,7 @@ code) once you have called @code{release_flattened_array()}: @} @end example -Finally, since everything was successful, the function sets the +Finally, because everything was successful, the function sets the return value to success, and returns: @example @@ -32448,7 +32589,7 @@ code can access them and manipulate them. There are two important points about creating arrays from extension code: -@enumerate 1 +@itemize @value{BULLET} @item You must install a new array into @command{gawk}'s symbol table immediately upon creating it. Once you have done so, @@ -32470,7 +32611,7 @@ using @code{sym_update()}, or install it as an element in a previously existing array using @code{set_array_element()}. We show example code shortly. @item -Due to gawk internals, after using @code{sym_update()} to install an array +Due to @command{gawk} internals, after using @code{sym_update()} to install an array into @command{gawk}, you have to retrieve the array cookie from the value passed in to @command{sym_update()} before doing anything else with it, like so: @@ -32490,7 +32631,7 @@ new_array = val.array_cookie; /* YOU MUST DO THIS */ If installing an array as a subarray, you must also retrieve the value of the array cookie after the call to @code{set_element()}. -@end enumerate +@end itemize The following C code is a simple test extension to create an array with two regular elements and with a subarray. The leading @code{#include} @@ -32609,7 +32750,7 @@ dl_load_func(func_table, testarray, "") @end ignore @end example -Here is sample script that loads the extension +Here is a sample script that loads the extension and then dumps the array: @example @@ -32639,9 +32780,78 @@ $ @kbd{AWKLIBPATH=$PWD ./gawk -f subarray.awk} @end example @noindent -(@xref{Finding Extensions}, for more information on the +(@DBXREF{Finding Extensions} for more information on the @env{AWKLIBPATH} environment variable.) +@node Redirection API +@subsection Accessing and Manipulating Redirections + +The following function allows extensions to access and manipulate redirections. + +@table @code +@item awk_bool_t get_file(const char *name, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ size_t name_len, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const char *filetype, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ int fd, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const awk_input_buf_t **ibufp, +@itemx @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ const awk_output_buf_t **obufp); +Look up a file in @command{gawk}'s internal redirection table. +If @code{name} is @code{NULL} or @code{name_len} is zero, return +data for the currently open input file corresponding to @code{FILENAME}. +(This does not access the @code{filetype} argument, so that may be undefined). +If the file is not already open, attempt to open it. +The @code{filetype} argument must be zero-terminated and should be one of: + +@table @code +@item ">" +A file opened for output. + +@item ">>" +A file opened for append. + +@item "<" +A file opened for input. + +@item "|>" +A pipe opened for output. + +@item "|<" +A pipe opened for input. + +@item "|&" +A two-way coprocess. +@end table + +On error, return a @code{false} value. Otherwise, return +@code{true}, and return additional information about the redirection +in the @code{ibufp} and @code{obufp} pointers. For input +redirections, the @code{*ibufp} value should be non-@code{NULL}, +and @code{*obufp} should be @code{NULL}. For output redirections, +the @code{*obufp} value should be non-@code{NULL}, and @code{*ibufp} +should be @code{NULL}. For two-way coprocesses, both values should +be non-@code{NULL}. + +In the usual case, the extension is interested in @code{(*ibufp)->fd} +and/or @code{fileno((*obufp)->fp)}. If the file is not already +open, and the @code{fd} argument is non-negative, @command{gawk} +will use that file descriptor instead of opening the file in the +usual way. If @code{fd} is non-negative, but the file exists already, +@command{gawk} ignores @code{fd} and returns the existing file. It is +the caller's responsibility to notice that neither the @code{fd} in +the returned @code{awk_input_buf_t} nor the @code{fd} in the returned +@code{awk_output_buf_t} matches the requested value. + +Note that supplying a file descriptor is currently @emph{not} supported +for pipes. However, supplying a file descriptor should work for input, +output, append, and two-way (coprocess) sockets. If @code{filetype} +is two-way, @command{gawk} assumes that it is a socket! Note that in +the two-way case, the input and output file descriptors may differ. +To check for success, you must check whether either matches. +@end table + +It is anticipated that this API function will be used to implement I/O +multiplexing and a socket library. + @node Extension API Variables @subsection API Variables @@ -32666,10 +32876,10 @@ The API versions are available at compile time as constants: @table @code @item GAWK_API_MAJOR_VERSION -The major version of the API. +The major version of the API @item GAWK_API_MINOR_VERSION -The minor version of the API. +The minor version of the API @end table The minor version increases when new functions are added to the API. Such @@ -32687,14 +32897,14 @@ constant integers: @table @code @item api->major_version -The major version of the running @command{gawk}. +The major version of the running @command{gawk} @item api->minor_version -The minor version of the running @command{gawk}. +The minor version of the running @command{gawk} @end table It is up to the extension to decide if there are API incompatibilities. -Typically a check like this is enough: +Typically, a check like this is enough: @example if (api->major_version != GAWK_API_MAJOR_VERSION @@ -32708,7 +32918,7 @@ if (api->major_version != GAWK_API_MAJOR_VERSION @end example Such code is included in the boilerplate @code{dl_load_func()} macro -provided in @file{gawkapi.h} (discussed later, in +provided in @file{gawkapi.h} (discussed in @ref{Extension API Boilerplate}). @node Extension API Informational Variables @@ -32725,8 +32935,7 @@ whether the corresponding command-line options were enabled when This variable is true if @command{gawk} was invoked with @option{--debug} option. @item do_lint -This variable is true if @command{gawk} was invoked with @option{--lint} option -(@pxref{Options}). +This variable is true if @command{gawk} was invoked with @option{--lint} option. @item do_mpfr This variable is true if @command{gawk} was invoked with @option{--bignum} option. @@ -32751,12 +32960,12 @@ The others should not change during execution. As mentioned earlier (@pxref{Extension Mechanism Outline}), the function definitions as presented are really macros. To use these macros, your extension must provide a small amount of boilerplate code (variables and -functions) towards the top of your source file, using pre-defined names -as described below. The boilerplate needed is also provided in comments +functions) toward the top of your source file, using predefined names +as described here. The boilerplate needed is also provided in comments in the @file{gawkapi.h} header file: @example -/* Boiler plate code: */ +/* Boilerplate code: */ int plugin_is_GPL_compatible; static gawk_api_t *const api; @@ -32815,7 +33024,7 @@ to @code{NULL}, or to point to a string giving the name and version of your extension. @item static awk_ext_func_t func_table[] = @{ @dots{} @}; -This is an array of one or more @code{awk_ext_func_t} structures +This is an array of one or more @code{awk_ext_func_t} structures, as described earlier (@pxref{Extension Functions}). It can then be looped over for multiple calls to @code{add_ext_func()}. @@ -32840,7 +33049,7 @@ This macro expands to a @code{dl_load()} function that performs all the necessary initializations. @end table -The point of the all the variables and arrays is to let the +The point of all the variables and arrays is to let the @code{dl_load()} function (from the @code{dl_load_func()} macro) do all the standard work. It does the following: @@ -32875,7 +33084,7 @@ Compiled extensions have to be installed in a directory where built in the default fashion, the directory in which to find extensions is @file{/usr/local/lib/gawk}. You can also specify a search path with a list of directories to search for compiled extensions. -@xref{AWKLIBPATH Variable}, for more information. +@DBXREF{AWKLIBPATH Variable} for more information. @node Extension Example @section Example: Some File Functions @@ -32883,7 +33092,7 @@ path with a list of directories to search for compiled extensions. @quotation @i{No matter where you go, there you are.} -@author Buckaroo Bonzai +@author Buckaroo Banzai @end quotation @c It's enough to show chdir and stat, no need for fts @@ -32946,7 +33155,7 @@ the @code{stat()} fails. It fills in the following elements: @table @code @item "name" -The name of the file that was @code{stat()}'ed. +The name of the file that was @code{stat()}ed. @item "dev" @itemx "ino" @@ -33002,7 +33211,7 @@ interprocess communications). The file is a directory. @item "fifo" -The file is a named-pipe (also known as a FIFO). +The file is a named pipe (also known as a FIFO). @item "file" The file is just a regular file. @@ -33025,7 +33234,7 @@ For some other systems, @dfn{a priori} knowledge is used to provide a value. Where no value can be determined, it defaults to 512. @end table -Several additional elements may be present depending upon the operating +Several additional elements may be present, depending upon the operating system and the type of the file. You can test for them in your @command{awk} program by using the @code{in} operator (@pxref{Reference to Elements}): @@ -33055,10 +33264,10 @@ edited slightly for presentation. See @file{extension/filefuncs.c} in the @command{gawk} distribution for the complete version.} The file includes a number of standard header files, and then includes -the @file{gawkapi.h} header file which provides the API definitions. -Those are followed by the necessary variable declarations +the @file{gawkapi.h} header file, which provides the API definitions. +Those are followed by the necessary variable declarations to make use of the API macros and boilerplate code -(@pxref{Extension API Boilerplate}). +(@pxref{Extension API Boilerplate}): @example #ifdef HAVE_CONFIG_H @@ -33096,10 +33305,10 @@ int plugin_is_GPL_compatible; @cindex programming conventions, @command{gawk} extensions By convention, for an @command{awk} function @code{foo()}, the C function that implements it is called @code{do_foo()}. The function should have -two arguments: the first is an @code{int} usually called @code{nargs}, +two arguments. The first is an @code{int}, usually called @code{nargs}, that represents the number of actual arguments for the function. -The second is a pointer to an @code{awk_value_t}, usually named -@code{result}. +The second is a pointer to an @code{awk_value_t} structure, usually named +@code{result}: @example /* do_chdir --- provide dynamically loaded chdir() function for gawk */ @@ -33119,13 +33328,13 @@ do_chdir(int nargs, awk_value_t *result) @end example The @code{newdir} -variable represents the new directory to change to, retrieved +variable represents the new directory to change to, which is retrieved with @code{get_argument()}. Note that the first argument is numbered zero. If the argument is retrieved successfully, the function calls the @code{chdir()} system call. If the @code{chdir()} fails, @code{ERRNO} -is updated. +is updated: @example if (get_argument(0, AWK_STRING, & newdir)) @{ @@ -33144,7 +33353,7 @@ Finally, the function returns the return value to the @command{awk} level: The @code{stat()} extension is more involved. First comes a function that turns a numeric mode into a printable representation -(e.g., 644 becomes @samp{-rw-r--r--}). This is omitted here for brevity: +(e.g., octal @code{0644} becomes @samp{-rw-r--r--}). This is omitted here for brevity: @example /* format_mode --- turn a stat mode field into something readable */ @@ -33200,9 +33409,9 @@ array_set_numeric(awk_array_t array, const char *sub, double num) The following function does most of the work to fill in the @code{awk_array_t} result array with values obtained -from a valid @code{struct stat}. It is done in a separate function +from a valid @code{struct stat}. This work is done in a separate function to support the @code{stat()} function for @command{gawk} and also -to support the @code{fts()} extension which is included in +to support the @code{fts()} extension, which is included in the same file but whose code is not shown here (@pxref{Extension Sample File Functions}). @@ -33323,10 +33532,10 @@ the @code{stat()} system call instead of the @code{lstat()} system call. This is done by using a function pointer: @code{statfunc}. @code{statfunc} is initialized to point to @code{lstat()} (instead of @code{stat()}) to get the file information, in case the file is a -symbolic link. However, if there were three arguments, @code{statfunc} -is set point to @code{stat()}, instead. +symbolic link. However, if the third argument is included, @code{statfunc} +is set to point to @code{stat()}, instead. -Here is the @code{do_stat()} function. It starts with +Here is the @code{do_stat()} function, which starts with variable declarations and argument checking: @ignore @@ -33380,7 +33589,7 @@ Next, it gets the information for the file. If the called function /* always empty out the array */ clear_array(array); - /* stat the file, if error, set ERRNO and return */ + /* stat the file; if error, set ERRNO and return */ ret = statfunc(name, & sbuf); if (ret < 0) @{ update_ERRNO_int(errno); @@ -33402,7 +33611,9 @@ Finally, it's necessary to provide the ``glue'' that loads the new function(s) into @command{gawk}. The @code{filefuncs} extension also provides an @code{fts()} -function, which we omit here. For its sake there is an initialization +function, which we omit here +(@pxref{Extension Sample File Functions}). +For its sake, there is an initialization function: @example @@ -33441,7 +33652,7 @@ dl_load_func(func_table, filefuncs, "") And that's it! @node Using Internal File Ops -@subsection Integrating The Extensions +@subsection Integrating the Extensions @cindex @command{gawk}, interpreter@comma{} adding code to Now that the code is written, it must be possible to add it at @@ -33450,9 +33661,9 @@ code must be compiled. Assuming that the functions are in a file named @file{filefuncs.c}, and @var{idir} is the location of the @file{gawkapi.h} header file, the following steps@footnote{In practice, you would probably want to -use the GNU Autotools---Automake, Autoconf, Libtool, and @command{gettext}---to +use the GNU Autotools (Automake, Autoconf, Libtool, and @command{gettext}) to configure and build your libraries. Instructions for doing so are beyond -the scope of this @value{DOCUMENT}. @xref{gawkextlib}, for Internet links to +the scope of this @value{DOCUMENT}. @DBXREF{gawkextlib} for Internet links to the tools.} create a GNU/Linux shared library: @example @@ -33460,7 +33671,7 @@ $ @kbd{gcc -fPIC -shared -DHAVE_CONFIG_H -c -O -g -I@var{idir} filefuncs.c} $ @kbd{gcc -o filefuncs.so -shared filefuncs.o} @end example -Once the library exists, it is loaded by using the @code{@@load} keyword. +Once the library exists, it is loaded by using the @code{@@load} keyword: @example # file testff.awk @@ -33524,13 +33735,14 @@ $ @kbd{AWKLIBPATH=$PWD gawk -f testff.awk} @end example @node Extension Samples -@section The Sample Extensions In The @command{gawk} Distribution +@section The Sample Extensions in the @command{gawk} Distribution @cindex extensions distributed with @command{gawk} -This @value{SECTION} provides brief overviews of the sample extensions +This @value{SECTION} provides a brief overview of the sample extensions that come in the @command{gawk} distribution. Some of them are intended -for production use, such the @code{filefuncs}, @code{readdir} and @code{inplace} extensions. -Others mainly provide example code that shows how to use the extension API. +for production use (e.g., the @code{filefuncs}, @code{readdir}, and +@code{inplace} extensions). Others mainly provide example code that +shows how to use the extension API. @menu * Extension Sample File Functions:: The file functions sample. @@ -33551,9 +33763,9 @@ Others mainly provide example code that shows how to use the extension API. @end menu @node Extension Sample File Functions -@subsection File Related Functions +@subsection File-Related Functions -The @code{filefuncs} extension provides three different functions, as follows: +The @code{filefuncs} extension provides three different functions, as follows. The usage is: @table @asis @@ -33564,15 +33776,15 @@ This is how you load the extension. @item @code{result = chdir("/some/directory")} The @code{chdir()} function is a direct hook to the @code{chdir()} system call to change the current directory. It returns zero -upon success or less than zero upon error. In the latter case it updates -@code{ERRNO}. +upon success or a value less than zero upon error. +In the latter case, it updates @code{ERRNO}. @cindex @code{stat()} extension function @item @code{result = stat("/some/path", statdata} [@code{, follow}]@code{)} The @code{stat()} function provides a hook into the @code{stat()} system call. -It returns zero upon success or less than zero upon error. -In the latter case it updates @code{ERRNO}. +It returns zero upon success or a value less than zero upon error. +In the latter case, it updates @code{ERRNO}. By default, it uses the @code{lstat()} system call. However, if passed a third argument, it uses @code{stat()} instead. @@ -33598,10 +33810,10 @@ array with information retrieved from the filesystem, as follows: @item @code{"major"} @tab @code{st_major} @tab Device files @item @code{"minor"} @tab @code{st_minor} @tab Device files @item @code{"blksize"} @tab @code{st_blksize} @tab All -@item @code{"pmode"} @tab A human-readable version of the mode value, such as printed by -@command{ls}. For example, @code{"-rwxr-xr-x"} @tab All +@item @code{"pmode"} @tab A human-readable version of the mode value, like that printed by +@command{ls} (for example, @code{"-rwxr-xr-x"}) @tab All @item @code{"linkval"} @tab The value of the symbolic link @tab Symbolic links -@item @code{"type"} @tab The type of the file as a string. One of +@item @code{"type"} @tab The type of the file as a string---one of @code{"file"}, @code{"blockdev"}, @code{"chardev"}, @@ -33611,16 +33823,16 @@ array with information retrieved from the filesystem, as follows: @code{"symlink"}, @code{"door"}, or -@code{"unknown"}. -Not all systems support all file types. @tab All +@code{"unknown"} +(not all systems support all file types) @tab All @end multitable @cindex @code{fts()} extension function @item @code{flags = or(FTS_PHYSICAL, ...)} @itemx @code{result = fts(pathlist, flags, filedata)} Walk the file trees provided in @code{pathlist} and fill in the -@code{filedata} array as described below. @code{flags} is the bitwise -OR of several predefined values, also described below. +@code{filedata} array, as described next. @code{flags} is the bitwise +OR of several predefined values, also described in a moment. Return zero if there were no errors, otherwise return @minus{}1. @end table @@ -33668,17 +33880,18 @@ whether or not @code{FTS_LOGICAL} is set. By default, the C library @code{fts()} routines do not return entries for @file{.} (dot) and @file{..} (dot-dot). This option causes entries for dot-dot to also be included. (The extension always includes an entry -for dot, see below.) +for dot; more on this in a moment.) @item FTS_XDEV During a traversal, do not cross onto a different mounted filesystem. @end table @item filedata -The @code{filedata} array is first cleared. Then, @code{fts()} creates +The @code{filedata} array holds the results. +@code{fts()} first clears it. Then it creates an element in @code{filedata} for every element in @code{pathlist}. The index is the name of the directory or file given in @code{pathlist}. -The element for this index is itself an array. There are two cases. +The element for this index is itself an array. There are two cases: @c nested table @table @emph @@ -33704,8 +33917,8 @@ contain an element named @code{"error"}, which is a string describing the error. @item The path is a directory In this case, the array contains one element for each entry in the -directory. If an entry is a file, that element is as for files, just -described. If the entry is a directory, that element is (recursively), +directory. If an entry is a file, that element is the same as for files, just +described. If the entry is a directory, that element is (recursively) an array describing the subdirectory. If @code{FTS_SEEDOT} was provided in the flags, then there will also be an element named @code{".."}. This element will be an array containing the data as provided by @code{stat()}. @@ -33717,15 +33930,15 @@ for a file: @code{"path"}, @code{"stat"}, and @code{"error"}. @end table The @code{fts()} function returns zero if there were no errors. -Otherwise it returns @minus{}1. +Otherwise, it returns @minus{}1. @quotation NOTE The @code{fts()} extension does not exactly mimic the interface of the C library @code{fts()} routines, choosing instead to provide an interface that is based on associative arrays, which is more comfortable to use from an @command{awk} program. This includes the -lack of a comparison function, since @command{gawk} already provides -powerful array sorting facilities. While an @code{fts_read()}-like +lack of a comparison function, because @command{gawk} already provides +powerful array sorting facilities. Although an @code{fts_read()}-like interface could have been provided, this felt less natural than simply creating a multidimensional array to represent the file hierarchy and its information. @@ -33735,7 +33948,7 @@ See @file{test/fts.awk} in the @command{gawk} distribution for an example use of the @code{fts()} extension function. @node Extension Sample Fnmatch -@subsection Interface To @code{fnmatch()} +@subsection Interface to @code{fnmatch()} This extension provides an interface to the C library @code{fnmatch()} function. The usage is: @@ -33748,10 +33961,10 @@ This is how you load the extension. @item result = fnmatch(pattern, string, flags) The return value is zero on success, @code{FNM_NOMATCH} if the string did not match the pattern, or -a different non-zero value if an error occurred. +a different nonzero value if an error occurred. @end table -Besides the @code{fnmatch()} function, the @code{fnmatch} extension +In addition to the @code{fnmatch()} function, the @code{fnmatch} extension adds one constant (@code{FNM_NOMATCH}), and an array of flag values named @code{FNM}. @@ -33759,17 +33972,17 @@ The arguments to @code{fnmatch()} are: @table @code @item pattern -The @value{FN} wildcard to match. +The @value{FN} wildcard to match @item string -The @value{FN} string. +The @value{FN} string @item flag Either zero, or the bitwise OR of one or more of the -flags in the @code{FNM} array. +flags in the @code{FNM} array @end table -The flags are follows: +The flags are as follows: @multitable @columnfractions .25 .75 @headitem Array element @tab Corresponding flag defined by @code{fnmatch()} @@ -33792,9 +34005,9 @@ if (fnmatch("*.a", "foo.c", flags) == FNM_NOMATCH) @end example @node Extension Sample Fork -@subsection Interface To @code{fork()}, @code{wait()} and @code{waitpid()} +@subsection Interface to @code{fork()}, @code{wait()}, and @code{waitpid()} -The @code{fork} extension adds three functions, as follows. +The @code{fork} extension adds three functions, as follows: @table @code @item @@load "fork" @@ -33803,14 +34016,14 @@ This is how you load the extension. @cindex @code{fork()} extension function @item pid = fork() This function creates a new process. The return value is zero in the -child and the process-ID number of the child in the parent, or @minus{}1 +child and the process ID number of the child in the parent, or @minus{}1 upon error. In the latter case, @code{ERRNO} indicates the problem. In the child, @code{PROCINFO["pid"]} and @code{PROCINFO["ppid"]} are updated to reflect the correct values. @cindex @code{waitpid()} extension function @item ret = waitpid(pid) -This function takes a numeric argument, which is the process-ID to +This function takes a numeric argument, which is the process ID to wait for. The return value is that of the @code{waitpid()} system call. @@ -33838,8 +34051,8 @@ else @subsection Enabling In-Place File Editing @cindex @code{inplace} extension -The @code{inplace} extension emulates GNU @command{sed}'s @option{-i} option -which performs ``in place'' editing of each input file. +The @code{inplace} extension emulates GNU @command{sed}'s @option{-i} option, +which performs ``in-place'' editing of each input file. It uses the bundled @file{inplace.awk} include file to invoke the extension properly: @@ -33853,11 +34066,16 @@ properly: # Please set INPLACE_SUFFIX to make a backup copy. For example, you may # want to set INPLACE_SUFFIX to .bak on the command line or in a BEGIN rule. +# N.B. We call inplace_end() in the BEGINFILE and END rules so that any +# actions in an ENDFILE rule will be redirected as expected. + BEGINFILE @{ - inplace_begin(FILENAME, INPLACE_SUFFIX) + if (_inplace_filename != "") + inplace_end(_inplace_filename, INPLACE_SUFFIX) + inplace_begin(_inplace_filename = FILENAME, INPLACE_SUFFIX) @} -ENDFILE @{ +END @{ inplace_end(FILENAME, INPLACE_SUFFIX) @} @end group @@ -33872,6 +34090,10 @@ If @code{INPLACE_SUFFIX} is not an empty string, the original file is linked to a backup @value{FN} created by appending that suffix. Finally, the temporary file is renamed to the original @value{FN}. +The @code{_inplace_filename} variable serves to keep track of the +current filename so as to not invoke @code{inplace_end()} before +processing the first file. + If any error occurs, the extension issues a fatal error to terminate processing immediately without damaging the original file. @@ -33892,7 +34114,7 @@ $ @kbd{gawk -i inplace -v INPLACE_SUFFIX=.bak '@{ gsub(/foo/, "bar") @}} @subsection Character and Numeric values: @code{ord()} and @code{chr()} The @code{ordchr} extension adds two functions, named -@code{ord()} and @code{chr()}, as follows. +@code{ord()} and @code{chr()}, as follows: @table @code @item @@load "ordchr" @@ -33935,14 +34157,14 @@ they are read, with each entry returned as a record. The record consists of three fields. The first two are the inode number and the @value{FN}, separated by a forward slash character. On systems where the directory entry contains the file type, the record -has a third field (also separated by a slash) which is a single letter -indicating the type of the file. The letters are file types are shown -in @ref{table-readdir-file-types}. +has a third field (also separated by a slash), which is a single letter +indicating the type of the file. The letters and their corresponding file +types are shown in @ref{table-readdir-file-types}. @float Table,table-readdir-file-types -@caption{File Types Returned By The @code{readdir} Extension} +@caption{File types returned by the @code{readdir} extension} @multitable @columnfractions .1 .9 -@headitem Letter @tab File Type +@headitem Letter @tab File type @item @code{b} @tab Block device @item @code{c} @tab Character device @item @code{d} @tab Directory @@ -33970,14 +34192,14 @@ Here is an example: @@load "readdir" @dots{} BEGIN @{ FS = "/" @} -@{ print "file name is", $2 @} +@{ print "@value{FN} is", $2 @} @end example @node Extension Sample Revout @subsection Reversing Output The @code{revoutput} extension adds a simple output wrapper that reverses -the characters in each output line. It's main purpose is to show how to +the characters in each output line. Its main purpose is to show how to write an output wrapper, although it may be mildly amusing for the unwary. Here is an example: @@ -33991,15 +34213,14 @@ BEGIN @{ @} @end example -The output from this program is: -@samp{cinap t'nod}. +The output from this program is @samp{cinap t'nod}. @node Extension Sample Rev2way @subsection Two-Way I/O Example The @code{revtwoway} extension adds a simple two-way processor that reverses the characters in each line sent to it for reading back by -the @command{awk} program. It's main purpose is to show how to write +the @command{awk} program. Its main purpose is to show how to write a two-way processor, although it may also be mildly amusing. The following example shows how to use it: @@ -34026,7 +34247,7 @@ is: @samp{cinap t'nod}. @node Extension Sample Read write array -@subsection Dumping and Restoring An Array +@subsection Dumping and Restoring an Array The @code{rwarray} extension adds two functions, named @code{writea()} and @code{reada()}, as follows: @@ -34047,12 +34268,12 @@ success, or zero upon failure. @code{reada()} is the inverse of @code{writea()}; it reads the file named as its first argument, filling in the array named as the second argument. It clears the array first. -Here too, the return value is one on success and zero upon failure. +Here too, the return value is one on success, or zero upon failure. @end table The array created by @code{reada()} is identical to that written by @code{writea()} in the sense that the contents are the same. However, -due to implementation issues, the array traversal order of the recreated +due to implementation issues, the array traversal order of the re-created array is likely to be different from that of the original array. As array traversal order in @command{awk} is by default undefined, this is (technically) not a problem. If you need to guarantee a particular traversal @@ -34060,7 +34281,7 @@ order, use the array sorting features in @command{gawk} to do so (@pxref{Array Sorting}). The file contains binary data. All integral values are written in network -byte order. However, double precision floating-point values are written +byte order. However, double-precision floating-point values are written as native binary data. Thus, arrays containing only string data can theoretically be dumped on systems with one byte order and restored on systems with a different one, but this has not been tried. @@ -34076,7 +34297,7 @@ ret = reada("arraydump.bin", array) @end example @node Extension Sample Readfile -@subsection Reading An Entire File +@subsection Reading an Entire File The @code{readfile} extension adds a single function named @code{readfile()}, and an input parser: @@ -34123,7 +34344,7 @@ This is how you load the extension. @cindex @code{gettimeofday()} extension function @item the_time = gettimeofday() Return the time in seconds that has elapsed since 1970-01-01 UTC as a -floating point value. If the time is unavailable on this platform, return +floating-point value. If the time is unavailable on this platform, return @minus{}1 and set @code{ERRNO}. The returned time should have sub-second precision, but the actual precision may vary based on the platform. If the standard C @code{gettimeofday()} system call is available on this @@ -34135,7 +34356,7 @@ it tries to use @code{GetSystemTimeAsFileTime()}. Attempt to sleep for @var{seconds} seconds. If @var{seconds} is negative, or the attempt to sleep fails, return @minus{}1 and set @code{ERRNO}. Otherwise, return zero after sleeping for the indicated amount of time. -Note that @var{seconds} may be a floating-point (non-integral) value. +Note that @var{seconds} may be a floating-point (nonintegral) value. Implementation details: depending on platform availability, this function tries to use @code{nanosleep()} or @code{select()} to implement the delay. @end table @@ -34162,26 +34383,35 @@ project provides a number of @command{gawk} extensions, including one for processing XML files. This is the evolution of the original @command{xgawk} (XML @command{gawk}) project. -As of this writing, there are five extensions: +As of this writing, there are seven extensions: @itemize @value{BULLET} @item -GD graphics library extension. +@code{errno} extension @item -PDF extension. +GD graphics library extension @item -PostgreSQL extension. +MPFR library extension +(this provides access to a number of MPFR functions that @command{gawk}'s +native MPFR support does not) @item -MPFR library extension. -This provides access to a number of MPFR functions which @command{gawk}'s -native MPFR support does not. +PDF extension + +@item +PostgreSQL extension + +@item +Redis extension + +@item +Select extension @item XML parser extension, using the @uref{http://expat.sourceforge.net, Expat} -XML parsing library. +XML parsing library @end itemize @cindex @command{git} utility @@ -34227,14 +34457,14 @@ make install @ii{Install the extensions} If you have installed @command{gawk} in the standard way, then you will likely not need the @option{--with-gawk} option when configuring -@code{gawkextlib}. You may also need to use the @command{sudo} utility +@code{gawkextlib}. You may need to use the @command{sudo} utility to install both @command{gawk} and @code{gawkextlib}, depending upon how your system works. If you write an extension that you wish to share with other -@command{gawk} users, please consider doing so through the +@command{gawk} users, consider doing so through the @code{gawkextlib} project. -See the project's web site for more information. +See the project's website for more information. @node Extension summary @section Summary @@ -34242,7 +34472,7 @@ See the project's web site for more information. @itemize @value{BULLET} @item You can write extensions (sometimes called plug-ins) for @command{gawk} -in C or C++ using the Application Programming Interface (API) defined +in C or C++ using the application programming interface (API) defined by the @command{gawk} developers. @item @@ -34252,7 +34482,7 @@ named @code{plugin_is_GPL_compatible}. @item Communication between @command{gawk} and an extension is two-way. -@command{gawk} passes a @code{struct} to the extension which contains +@command{gawk} passes a @code{struct} to the extension that contains various data fields and function pointers. The extension can then call into @command{gawk} via the supplied function pointers to accomplish certain tasks. @@ -34265,7 +34495,7 @@ By convention, implementation functions are named @code{do_@var{XXXX}()} for some @command{awk}-level function @code{@var{XXXX}()}. @item -The API is defined in a header file named @file{gawkpi.h}. You must include +The API is defined in a header file named @file{gawkapi.h}. You must include a number of standard header files @emph{before} including it in your source file. @item @@ -34273,44 +34503,44 @@ API function pointers are provided for the following kinds of operations: @itemize @value{BULLET} @item -Allocating, reallocating, and releasing memory. +Allocating, reallocating, and releasing memory @item -Registration functions. You may register +Registration functions (you may register extension functions, exit callbacks, a version string, input parsers, output wrappers, -and two-way processors. +and two-way processors) @item -Printing fatal, warning, and ``lint'' warning messages. +Printing fatal, warning, and ``lint'' warning messages @item -Updating @code{ERRNO}, or unsetting it. +Updating @code{ERRNO}, or unsetting it @item Accessing parameters, including converting an undefined parameter into -an array. +an array @item -Symbol table access: retrieving a global variable, creating one, -or changing one. +Symbol table access (retrieving a global variable, creating one, +or changing one) @item Creating and releasing cached values; this provides an efficient way to use values for multiple variables and -can be a big performance win. +can be a big performance win @item -Manipulating arrays: -retrieving, adding, deleting, and modifying elements; +Manipulating arrays +(retrieving, adding, deleting, and modifying elements; getting the count of elements in an array; creating a new array; clearing an array; and -flattening an array for easy C style looping over all its indices and elements. +flattening an array for easy C-style looping over all its indices and elements) @end itemize @item @@ -34318,7 +34548,7 @@ The API defines a number of standard data types for representing @command{awk} values, array elements, and arrays. @item -The API provide convenience functions for constructing values. +The API provides convenience functions for constructing values. It also provides memory management functions to ensure compatibility between memory allocated by @command{gawk} and memory allocated by an extension. @@ -34339,13 +34569,13 @@ that loaded it. @item It is easiest to start a new extension by copying the boilerplate code -described in this @value{CHAPTER}. Macros in the @file{gawkapi.h} make -this easier to do. +described in this @value{CHAPTER}. Macros in the @file{gawkapi.h} header +file make this easier to do. @item The @command{gawk} distribution includes a number of small but useful -sample extensions. The @code{gawkextlib} project includes several more, -larger, extensions. If you wish to write an extension and contribute it +sample extensions. The @code{gawkextlib} project includes several more +(larger) extensions. If you wish to write an extension and contribute it to the community of @command{gawk} users, the @code{gawkextlib} project is the place to do so. @@ -34361,6 +34591,24 @@ Add functions to implement system calls such as @code{chown()}, @code{chmod()}, and @code{umask()} to the file operations extension presented in @ref{Internal File Ops}. +@c Idea from comp.lang.awk, February 2015 +@item +Write an input parser that prints a prompt if the input is +a from a ``terminal'' device. You can use the @code{isatty()} +function to tell if the input file is a terminal. (Hint: this function +is usually expensive to call; try to call it just once.) +The content of the prompt should come from a variable settable +by @command{awk}-level code. +You can write the prompt to stanard error. However, +for best results, open a new file descriptor (or file pointer) +on @file{/dev/tty} and print the prompt there, in case standard +error has been redirected. + +Why is standard error a better +choice than standard output for writing the prompt? +Which reading mechanism should you replace, the one to get +a record, or the one to read raw bytes? + @item (Hard.) How would you provide namespaces in @command{gawk}, so that the @@ -34389,34 +34637,34 @@ and the Glossary: @end ifclear @ifset FOR_PRINT -Part IV contains two appendices and the license that +Part IV contains three appendices, the last of which is the license that covers the @command{gawk} source code: @end ifset @itemize @value{BULLET} @item -@ref{Language History}. +@ref{Language History} @item -@ref{Installation}. +@ref{Installation} @ifclear FOR_PRINT @item -@ref{Notes}. +@ref{Notes} @item -@ref{Basic Concepts}. +@ref{Basic Concepts} @item -@ref{Glossary}. +@ref{Glossary} @end ifclear @item -@ref{Copying}. +@ref{Copying} @ifclear FOR_PRINT @item -@ref{GNU Free Documentation License}. +@ref{GNU Free Documentation License} @end ifclear @end itemize @end ifdocbook @@ -34425,7 +34673,7 @@ covers the @command{gawk} source code: @appendix The Evolution of the @command{awk} Language This @value{DOCUMENT} describes the GNU implementation of @command{awk}, -which follows the POSIX specification. Many long-time @command{awk} +which follows the POSIX specification. Many longtime @command{awk} users learned @command{awk} programming with the original @command{awk} implementation in Version 7 Unix. (This implementation was the basis for @command{awk} in Berkeley Unix, through 4.3-Reno. Subsequent versions @@ -34462,9 +34710,7 @@ online documentation}. @node V7/SVR3.1 @appendixsec Major Changes Between V7 and SVR3.1 -@c STARTOFRANGE gawkv @cindex @command{awk}, versions of -@c STARTOFRANGE gawkv1 @cindex @command{awk}, versions of, changes between V7 and SVR3.1 The @command{awk} language evolved considerably between the release of @@ -34475,83 +34721,82 @@ cross-references to further details: @itemize @value{BULLET} @item The requirement for @samp{;} to separate rules on a line -(@pxref{Statements/Lines}). +(@pxref{Statements/Lines}) @item User-defined functions and the @code{return} statement -(@pxref{User-defined}). +(@pxref{User-defined}) @item The @code{delete} statement (@pxref{Delete}). @item The @code{do}-@code{while} statement -(@pxref{Do Statement}). +(@pxref{Do Statement}) @item The built-in functions @code{atan2()}, @code{cos()}, @code{sin()}, @code{rand()}, and -@code{srand()} (@pxref{Numeric Functions}). +@code{srand()} (@pxref{Numeric Functions}) @item The built-in functions @code{gsub()}, @code{sub()}, and @code{match()} -(@pxref{String Functions}). +(@pxref{String Functions}) @item The built-in functions @code{close()} and @code{system()} -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @item The @code{ARGC}, @code{ARGV}, @code{FNR}, @code{RLENGTH}, @code{RSTART}, -and @code{SUBSEP} predefined variables (@pxref{Built-in Variables}). +and @code{SUBSEP} predefined variables (@pxref{Built-in Variables}) @item -Assignable @code{$0} (@pxref{Changing Fields}). +Assignable @code{$0} (@pxref{Changing Fields}) @item The conditional expression using the ternary operator @samp{?:} -(@pxref{Conditional Exp}). +(@pxref{Conditional Exp}) @item -The expression @samp{@var{index-variable} in @var{array}} outside of @code{for} -statements (@pxref{Reference to Elements}). +The expression @samp{@var{indx} in @var{array}} outside of @code{for} +statements (@pxref{Reference to Elements}) @item The exponentiation operator @samp{^} (@pxref{Arithmetic Ops}) and its assignment operator -form @samp{^=} (@pxref{Assignment Ops}). +form @samp{^=} (@pxref{Assignment Ops}) @item C-compatible operator precedence, which breaks some old @command{awk} -programs (@pxref{Precedence}). +programs (@pxref{Precedence}) @item Regexps as the value of @code{FS} (@pxref{Field Separators}) and as the third argument to the @code{split()} function (@pxref{String Functions}), rather than using only the first character -of @code{FS}. +of @code{FS} @item Dynamic regexps as operands of the @samp{~} and @samp{!~} operators -(@pxref{Computed Regexps}). +(@pxref{Computed Regexps}) @item The escape sequences @samp{\b}, @samp{\f}, and @samp{\r} -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @item Redirection of input for the @code{getline} function -(@pxref{Getline}). +(@pxref{Getline}) @item Multiple @code{BEGIN} and @code{END} rules -(@pxref{BEGIN/END}). +(@pxref{BEGIN/END}) @item Multidimensional arrays -(@pxref{Multidimensional}). +(@pxref{Multidimensional}) @end itemize -@c ENDOFRANGE gawkv1 @node SVR4 @appendixsec Changes Between SVR3.1 and SVR4 @@ -34562,54 +34807,54 @@ The System V Release 4 (1989) version of Unix @command{awk} added these features @itemize @value{BULLET} @item -The @code{ENVIRON} array (@pxref{Built-in Variables}). +The @code{ENVIRON} array (@pxref{Built-in Variables}) @c gawk and MKS awk @item Multiple @option{-f} options on the command line -(@pxref{Options}). +(@pxref{Options}) @c MKS awk @item The @option{-v} option for assigning variables before program execution begins -(@pxref{Options}). +(@pxref{Options}) @c GNU, Bell Laboratories & MKS together @item -The @option{--} signal for terminating command-line options. +The @option{--} signal for terminating command-line options @item The @samp{\a}, @samp{\v}, and @samp{\x} escape sequences -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @c GNU, for ANSI C compat @item A defined return value for the @code{srand()} built-in function -(@pxref{Numeric Functions}). +(@pxref{Numeric Functions}) @item The @code{toupper()} and @code{tolower()} built-in string functions for case translation -(@pxref{String Functions}). +(@pxref{String Functions}) @item -A cleaner specification for the @code{%c} format-control letter in the +A cleaner specification for the @samp{%c} format-control letter in the @code{printf} function -(@pxref{Control Letters}). +(@pxref{Control Letters}) @item The ability to dynamically pass the field width and precision (@code{"%*.*d"}) in the argument list of @code{printf} and @code{sprintf()} -(@pxref{Control Letters}). +(@pxref{Control Letters}) @item The use of regexp constants, such as @code{/foo/}, as expressions, where they are equivalent to using the matching operator, as in @samp{$0 ~ /foo/} -(@pxref{Using Constant Regexps}). +(@pxref{Using Constant Regexps}) @item Processing of escape sequences inside command-line variable assignments -(@pxref{Assignment Options}). +(@pxref{Assignment Options}) @end itemize @node POSIX @@ -34623,23 +34868,23 @@ introduced the following changes into the language: @itemize @value{BULLET} @item The use of @option{-W} for implementation-specific options -(@pxref{Options}). +(@pxref{Options}) @item The use of @code{CONVFMT} for controlling the conversion of numbers -to strings (@pxref{Conversion}). +to strings (@pxref{Conversion}) @item The concept of a numeric string and tighter comparison rules to go -with it (@pxref{Typing and Comparison}). +with it (@pxref{Typing and Comparison}) @item The use of predefined variables as function parameter names is forbidden -(@pxref{Definition Syntax}). +(@pxref{Definition Syntax}) @item More complete documentation of many of the previously undocumented -features of the language. +features of the language @end itemize In 2012, a number of extensions that had been commonly available for @@ -34648,25 +34893,24 @@ many years were finally added to POSIX. They are: @itemize @value{BULLET} @item The @code{fflush()} built-in function for flushing buffered output -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @item The @code{nextfile} statement -(@pxref{Nextfile Statement}). +(@pxref{Nextfile Statement}) @item The ability to delete all of an array at once with @samp{delete @var{array}} -(@pxref{Delete}). +(@pxref{Delete}) @end itemize -@xref{Common Extensions}, for a list of common extensions +@DBXREF{Common Extensions} for a list of common extensions not permitted by the POSIX standard. The 2008 POSIX standard can be found online at @url{http://www.opengroup.org/onlinepubs/9699919799/}. -@c ENDOFRANGE gawkv @node BTL @appendixsec Extensions in Brian Kernighan's @command{awk} @@ -34680,43 +34924,40 @@ has made his version available via his home page (@pxref{Other Versions}). This @value{SECTION} describes common extensions that -originally appeared in his version of @command{awk}. +originally appeared in his version of @command{awk}: @itemize @value{BULLET} @item The @samp{**} and @samp{**=} operators (@pxref{Arithmetic Ops} and -@ref{Assignment Ops}). +@ref{Assignment Ops}) @item The use of @code{func} as an abbreviation for @code{function} -(@pxref{Definition Syntax}). +(@pxref{Definition Syntax}) @item The @code{fflush()} built-in function for flushing buffered output -(@pxref{I/O Functions}). +(@pxref{I/O Functions}) @ignore @item The @code{SYMTAB} array, that allows access to @command{awk}'s internal symbol table. This feature was never documented for his @command{awk}, largely because it is somewhat shakily implemented. For instance, you cannot access arrays -or array elements through it. +or array elements through it @end ignore @end itemize -@xref{Common Extensions}, for a full list of the extensions +@DBXREF{Common Extensions} for a full list of the extensions available in his @command{awk}. @node POSIX/GNU @appendixsec Extensions in @command{gawk} Not in POSIX @command{awk} -@c STARTOFRANGE fripls @cindex compatibility mode (@command{gawk}), extensions -@c STARTOFRANGE exgnot @cindex extensions, in @command{gawk}, not in POSIX @command{awk} -@c STARTOFRANGE posnot @cindex POSIX, @command{gawk} extensions not included in The GNU implementation, @command{gawk}, adds a large number of features. They can all be disabled with either the @option{--traditional} or @@ -34735,7 +34976,7 @@ Additional predefined variables: @itemize @value{MINUS} @item The -@code{ARGIND} +@code{ARGIND}, @code{BINMODE}, @code{ERRNO}, @code{FIELDWIDTHS}, @@ -34747,7 +34988,7 @@ The and @code{TEXTDOMAIN} variables -(@pxref{Built-in Variables}). +(@pxref{Built-in Variables}) @end itemize @item @@ -34755,15 +34996,15 @@ Special files in I/O redirections: @itemize @value{MINUS} @item -The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr} and +The @file{/dev/stdin}, @file{/dev/stdout}, @file{/dev/stderr}, and @file{/dev/fd/@var{N}} special @value{FN}s -(@pxref{Special Files}). +(@pxref{Special Files}) @item The @file{/inet}, @file{/inet4}, and @samp{/inet6} special files for TCP/IP networking using @samp{|&} to specify which version of the -IP protocol to use. -(@pxref{TCP/IP Networking}). +IP protocol to use +(@pxref{TCP/IP Networking}) @end itemize @item @@ -34772,37 +35013,41 @@ Changes and/or additions to the language: @itemize @value{MINUS} @item The @samp{\x} escape sequence -(@pxref{Escape Sequences}). +(@pxref{Escape Sequences}) @item Full support for both POSIX and GNU regexps -(@pxref{Regexp}). +(@pxref{Regexp}) @item The ability for @code{FS} and for the third argument to @code{split()} to be null strings -(@pxref{Single Character Fields}). +(@pxref{Single Character Fields}) @item The ability for @code{RS} to be a regexp -(@pxref{Records}). +(@pxref{Records}) @item The ability to use octal and hexadecimal constants in @command{awk} program source code -(@pxref{Nondecimal-numbers}). +(@pxref{Nondecimal-numbers}) @item The @samp{|&} operator for two-way I/O to a coprocess -(@pxref{Two-way I/O}). +(@pxref{Two-way I/O}) @item Indirect function calls -(@pxref{Indirect Calls}). +(@pxref{Indirect Calls}) @item Directories on the command line produce a warning and are skipped -(@pxref{Command-line directories}). +(@pxref{Command-line directories}) + +@item +Output with @code{print} and @code{printf} need not be fatal +(@pxref{Nonfatal}) @end itemize @item @@ -34810,12 +35055,12 @@ New keywords: @itemize @value{MINUS} @item -The @code{BEGINFILE} and @code{ENDFILE} special patterns. -(@pxref{BEGINFILE/ENDFILE}). +The @code{BEGINFILE} and @code{ENDFILE} special patterns +(@pxref{BEGINFILE/ENDFILE}) @item The @code{switch} statement -(@pxref{Switch Statement}). +(@pxref{Switch Statement}) @end itemize @item @@ -34825,30 +35070,30 @@ Changes to standard @command{awk} functions: @item The optional second argument to @code{close()} that allows closing one end of a two-way pipe to a coprocess -(@pxref{Two-way I/O}). +(@pxref{Two-way I/O}) @item -POSIX compliance for @code{gsub()} and @code{sub()} with @option{--posix}. +POSIX compliance for @code{gsub()} and @code{sub()} with @option{--posix} @item The @code{length()} function accepts an array argument and returns the number of elements in the array -(@pxref{String Functions}). +(@pxref{String Functions}) @item The optional third argument to the @code{match()} function for capturing text-matching subexpressions within a regexp -(@pxref{String Functions}). +(@pxref{String Functions}) @item Positional specifiers in @code{printf} formats for making translations easier -(@pxref{Printf Ordering}). +(@pxref{Printf Ordering}) @item The @code{split()} function's additional optional fourth -argument which is an array to hold the text of the field separators. -(@pxref{String Functions}). +argument, which is an array to hold the text of the field separators +(@pxref{String Functions}) @end itemize @item @@ -34858,16 +35103,16 @@ Additional functions only in @command{gawk}: @item The @code{gensub()}, @code{patsplit()}, and @code{strtonum()} functions for more powerful text manipulation -(@pxref{String Functions}). +(@pxref{String Functions}) @item The @code{asort()} and @code{asorti()} functions for sorting arrays -(@pxref{Array Sorting}). +(@pxref{Array Sorting}) @item The @code{mktime()}, @code{systime()}, and @code{strftime()} functions for working with timestamps -(@pxref{Time Functions}). +(@pxref{Time Functions}) @item The @@ -34879,17 +35124,22 @@ The and @code{xor()} functions for bit manipulation -(@pxref{Bitwise Functions}). +(@pxref{Bitwise Functions}) @c In 4.1, and(), or() and xor() grew the ability to take > 2 arguments @item The @code{isarray()} function to check if a variable is an array or not -(@pxref{Type Functions}). +(@pxref{Type Functions}) @item -The @code{bindtextdomain()}, @code{dcgettext()} and @code{dcngettext()} +The @code{bindtextdomain()}, @code{dcgettext()}, and @code{dcngettext()} functions for internationalization -(@pxref{Programmer i18n}). +(@pxref{Programmer i18n}) + +@item +The @code{intdiv()} function for doing integer +division and remainder +(@pxref{Numeric Functions}) @end itemize @item @@ -34899,12 +35149,12 @@ Changes and/or additions in the command-line options: @item The @env{AWKPATH} environment variable for specifying a path search for the @option{-f} command-line option -(@pxref{Options}). +(@pxref{Options}) @item The @env{AWKLIBPATH} environment variable for specifying a path search for the @option{-l} command-line option -(@pxref{Options}). +(@pxref{Options}) @item The @@ -34933,7 +35183,7 @@ The and @option{-V} short options. Also, the -ability to use GNU-style long-named options that start with @option{--} +ability to use GNU-style long-named options that start with @option{--}, and the @option{--assign}, @option{--bignum}, @@ -35013,7 +35263,7 @@ GCC for VAX and Alpha has not been tested for a while. @end itemize @item -Support for the following obsolete systems was removed from the code +Support for the following obsolete system was removed from the code for @command{gawk} @value{PVERSION} 4.1: @c nested table @@ -35023,16 +35273,19 @@ Ultrix @end itemize @item -@c FIXME: Verify the version here. -Support for MirBSD was removed at @command{gawk} @value{PVERSION} 4.2. +Support for the following systems was removed from the code +for @command{gawk} @value{PVERSION} 4.2: + +@c nested table +@itemize @value{MINUS} +@item +MirBSD +@end itemize @end itemize @c XXX ADD MORE STUFF HERE -@c ENDOFRANGE fripls -@c ENDOFRANGE exgnot -@c ENDOFRANGE posnot @c This does not need to be in the formal book. @ifclear FOR_PRINT @@ -35614,11 +35867,11 @@ load @command{awk} library files. @item The @option{-l} and @option{--load} options load compiled dynamic extensions. -@item +@item The @option{-M} and @option{--bignum} options enable MPFR. @item -The @option{-o} only does pretty-printing. +The @option{-o} option only does pretty-printing. @item The @option{-p} option is used for profiling. @@ -35641,6 +35894,44 @@ with a minimum of two The dynamic extension interface was completely redone (@pxref{Dynamic Extensions}). +@item +Support for Ultrix was removed. + +@end itemize + +Version 4.2 introduced the following changes: + +@itemize @bullet +@item +Changes to @code{ENVIRON} are reflected into @command{gawk}'s +environment and that of programs that it runs. +@xref{Auto-set}. + +@item +The @option{--pretty-print} option no longer runs the @command{awk} +program too. +@xref{Options}. + +@item +The @command{igawk} program and its manual page are no longer +installed when @command{gawk} is built. +@xref{Igawk Program}. + +@item +The @code{intdiv()} function. +@xref{Numeric Functions}. + +@item +The maximum number of hexdecimal digits in @samp{\x} escapes +is now two. +@xref{Escape Sequences}. + +@item +Nonfatal output with @code{print} and @code{printf}. +@xref{Nonfatal}. + +@item +Support for MirBSD was removed. @end itemize @c XXX ADD MORE STUFF HERE @@ -35653,12 +35944,12 @@ The dynamic extension interface was completely redone @cindex extensions, @command{mawk} The following table summarizes the common extensions supported by @command{gawk}, Brian Kernighan's @command{awk}, and @command{mawk}, -the three most widely-used freely available versions of @command{awk} +the three most widely used freely available versions of @command{awk} (@pxref{Other Versions}). -@multitable {@file{/dev/stderr} special file} {BWK Awk} {Mawk} {GNU Awk} {Now standard} -@headitem Feature @tab BWK Awk @tab Mawk @tab GNU Awk @tab Now standard -@item @samp{\x} Escape sequence @tab X @tab X @tab X @tab +@multitable {@file{/dev/stderr} special file} {BWK @command{awk}} {@command{mawk}} {@command{gawk}} {Now standard} +@headitem Feature @tab BWK @command{awk} @tab @command{mawk} @tab @command{gawk} @tab Now standard +@item @samp{\x} escape sequence @tab X @tab X @tab X @tab @item @code{FS} as null string @tab X @tab X @tab X @tab @item @file{/dev/stdin} special file @tab X @tab X @tab X @tab @item @file{/dev/stdout} special file @tab X @tab X @tab X @tab @@ -35671,7 +35962,7 @@ the three most widely-used freely available versions of @command{awk} @item @code{func} keyword @tab X @tab @tab X @tab @item @code{BINMODE} variable @tab @tab X @tab X @tab @item @code{RS} as regexp @tab @tab X @tab X @tab -@item Time related functions @tab @tab X @tab X @tab +@item Time-related functions @tab @tab X @tab X @tab @end multitable @node Ranges and Locales @@ -35687,9 +35978,9 @@ the first character in the range and the last character in the range, inclusive. Ordering was based on the numeric value of each character in the machine's native character set. Thus, on ASCII-based systems, @samp{[a-z]} matched all the lowercase letters, and only the lowercase -letters, since the numeric values for the letters from @samp{a} through +letters, as the numeric values for the letters from @samp{a} through @samp{z} were contiguous. (On an EBCDIC system, the range @samp{[a-z]} -includes additional, non-alphabetic characters as well.) +includes additional nonalphabetic characters as well.) Almost all introductory Unix literature explained range expressions as working in this fashion, and in particular, would teach that the @@ -35698,8 +35989,8 @@ that @samp{[A-Z]} was the ``correct'' way to match uppercase letters. And indeed, this was true.@footnote{And Life was good.} The 1992 POSIX standard introduced the idea of locales (@pxref{Locales}). -Since many locales include other letters besides the plain twenty-six -letters of the American English alphabet, the POSIX standard added +Because many locales include other letters besides the plain 26 +letters of the English alphabet, the POSIX standard added character classes (@pxref{Bracket Expressions}) as a way to match different kinds of characters besides the traditional ones in the ASCII character set. @@ -35714,9 +36005,9 @@ What does that mean? In many locales, @samp{A} and @samp{a} are both less than @samp{B}. In other words, these locales sort characters in dictionary order, and @samp{[a-dx-z]} is typically not equivalent to @samp{[abcdxyz]}; -instead it might be equivalent to @samp{[ABCXYabcdxyz]}, for example. +instead, it might be equivalent to @samp{[ABCXYabcdxyz]}, for example. -This point needs to be emphasized: Much literature teaches that you should +This point needs to be emphasized: much literature teaches that you should use @samp{[a-z]} to match a lowercase character. But on systems with non-ASCII locales, this also matches all of the uppercase characters except @samp{A} or @samp{Z}! This was a continuous cause of confusion, even well @@ -35732,7 +36023,7 @@ $ @kbd{echo something1234abc | gawk-3.1.8 '@{ sub("[A-Z]*$", ""); print @}'} @end example @noindent -This output is unexpected, since the @samp{bc} at the end of +This output is unexpected, as the @samp{bc} at the end of @samp{something1234abc} should not normally match @samp{[A-Z]*}. This result is due to the locale setting (and thus you may not see it on your system). @@ -35743,13 +36034,13 @@ is perfectly valid in ASCII, but is not valid in many Unicode locales, such as @code{en_US.UTF-8}. Early versions of @command{gawk} used regexp matching code that was not -locale aware, so ranges had their traditional interpretation. +locale-aware, so ranges had their traditional interpretation. When @command{gawk} switched to using locale-aware regexp matchers, the problems began; especially as both GNU/Linux and commercial Unix vendors started implementing non-ASCII locales, @emph{and making them the default}. Perhaps the most frequently asked question became something -like ``why does @samp{[A-Z]} match lowercase letters?!?'' +like, ``Why does @samp{[A-Z]} match lowercase letters?!?'' @cindex Berry, Karl This situation existed for close to 10 years, if not more, and @@ -35759,7 +36050,7 @@ was in the user's locale. During the development of @value{PVERSION} 4.0, he modified @command{gawk} to always treat ranges in the original, pre-POSIX fashion, unless @option{--posix} was used (@pxref{Options}).@footnote{And thus was born the Campaign for Rational Range Interpretation (or -RRI). A number of GNU tools have either implemented this change, +RRI). A number of GNU tools have already implemented this change, or will soon. Thanks to Karl Berry for coining the phrase ``Rational Range Interpretation.''} @@ -35773,9 +36064,10 @@ and By using this lovely technical term, the standard gives license to implementors to implement ranges in whatever way they choose. -The @command{gawk} maintainer chose to apply the pre-POSIX meaning in all -cases: the default regexp matching; with @option{--traditional} and with -@option{--posix}; in all cases, @command{gawk} remains POSIX compliant. +The @command{gawk} maintainer chose to apply the pre-POSIX meaning +both with the default regexp matching and when @option{--traditional} or +@option{--posix} are used. +In all cases @command{gawk} remains POSIX-compliant. @node Contributors @appendixsec Major Contributors to @command{gawk} @@ -35821,7 +36113,7 @@ to around 90 pages. Richard Stallman helped finish the implementation and the initial draft of this @value{DOCUMENT}. -He is also the founder of the FSF and the GNU project. +He is also the founder of the FSF and the GNU Project. @item @cindex Woods, John @@ -35967,7 +36259,7 @@ Michael Benzinger contributed the initial code for @code{switch} statements. @cindex McPhee, Patrick Patrick T.J.@: McPhee contributed the code for dynamic loading in Windows32 environments. -(This is no longer supported) +(This is no longer supported.) @item @cindex Wallin, Anders @@ -35985,28 +36277,28 @@ John Haque made the following contributions: @itemize @value{MINUS} @item The modifications to convert @command{gawk} -into a byte-code interpreter, including the debugger. +into a byte-code interpreter, including the debugger @item -The addition of true arrays of arrays. +The addition of true arrays of arrays @item -The additional modifications for support of arbitrary precision arithmetic. +The additional modifications for support of arbitrary-precision arithmetic @item The initial text of -@ref{Arbitrary Precision Arithmetic}. +@ref{Arbitrary Precision Arithmetic} @item The work to merge the three versions of @command{gawk} -into one, for the 4.1 release. +into one, for the 4.1 release @item -Improved array internals for arrays indexed by integers. +Improved array internals for arrays indexed by integers @item -The improved array sorting features were driven by John together -with Pat Rankin. +The improved array sorting features were also driven by John, together +with Pat Rankin @end itemize @cindex Papadopoulos, Panos @@ -36047,10 +36339,10 @@ helping David Trueman, and as the primary maintainer since around 1994. @itemize @value{BULLET} @item The @command{awk} language has evolved over time. The first release -was with V7 Unix circa 1978. In 1987 for System V Release 3.1, +was with V7 Unix, circa 1978. In 1987, for System V Release 3.1, major additions, including user-defined functions, were made to the language. Additional changes were made for System V Release 4, in 1989. -Since then, further minor changes happen under the auspices of the +Since then, further minor changes have happened under the auspices of the POSIX standard. @item @@ -36066,7 +36358,7 @@ options. The interaction of POSIX locales and regexp matching in @command{gawk} has been confusing over the years. Today, @command{gawk} implements Rational Range Interpretation, where ranges of the form @samp{[a-z]} match @emph{only} the characters numerically between -@samp{a} through @samp{z} in the machine's native character set. Usually this is ASCII +@samp{a} through @samp{z} in the machine's native character set. Usually this is ASCII, but it can be EBCDIC on IBM S/390 systems. @item @@ -36081,16 +36373,14 @@ the appropriate credit where credit is due. @c last two commas are part of see also @cindex operating systems, See Also GNU/Linux@comma{} PC operating systems@comma{} Unix -@c STARTOFRANGE gligawk @cindex @command{gawk}, installing -@c STARTOFRANGE ingawk @cindex installing @command{gawk} This appendix provides instructions for installing @command{gawk} on the various platforms that are supported by the developers. The primary developer supports GNU/Linux (and Unix), whereas the other ports are contributed. -@xref{Bugs}, -for the electronic mail addresses of the people who maintain +@DBXREF{Bugs} +for the email addresses of the people who maintain the respective ports. @menu @@ -36144,7 +36434,7 @@ wget http://ftp.gnu.org/gnu/gawk/gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz The GNU software archive is mirrored around the world. The up-to-date list of mirror sites is available from -@uref{http://www.gnu.org/order/ftp.html, the main FSF web site}. +@uref{http://www.gnu.org/order/ftp.html, the main FSF website}. Try to use one of the mirrors; they will be less busy, and you can usually find one closer to your site. @@ -36153,9 +36443,9 @@ will be less busy, and you can usually find one closer to your site. @command{gawk} is distributed as several @code{tar} files compressed with different compression programs: @command{gzip}, @command{bzip2}, and @command{xz}. For simplicity, the rest of these instructions assume -you are using the one compressed with the GNU Zip program, @code{gzip}. +you are using the one compressed with the GNU Gzip program (@command{gzip}). -Once you have the distribution (for example, +Once you have the distribution (e.g., @file{gawk-@value{VERSION}.@value{PATCHLEVEL}.tar.gz}), use @code{gzip} to expand the file and then use @code{tar} to extract it. You can use the following @@ -36193,7 +36483,6 @@ a local expert. @node Distribution contents @appendixsubsec Contents of the @command{gawk} Distribution -@c STARTOFRANGE gawdis @cindex @command{gawk}, distribution The @command{gawk} distribution has a number of C source files, @@ -36205,12 +36494,12 @@ operating systems: @table @asis @item Various @samp{.c}, @samp{.y}, and @samp{.h} files -The actual @command{gawk} source code. +These files contain the actual @command{gawk} source code. @end table @table @file @item ABOUT-NLS -Information about GNU @command{gettext} and translations. +A file containing information about GNU @command{gettext} and translations. @item AUTHORS A file with some information about the authorship of @command{gawk}. @@ -36240,7 +36529,7 @@ An older list of changes to @command{gawk}. The GNU General Public License. @item POSIX.STD -A description of behaviors in the POSIX standard for @command{awk} which +A description of behaviors in the POSIX standard for @command{awk} that are left undefined, or where @command{gawk} may not comply fully, as well as a list of things that the POSIX standard should describe but does not. @@ -36248,7 +36537,7 @@ as a list of things that the POSIX standard should describe but does not. @item doc/awkforai.txt Pointers to the original draft of a short article describing why @command{gawk} is a good language for -Artificial Intelligence (AI) programming. +artificial intelligence (AI) programming. @item doc/bc_notes A brief description of @command{gawk}'s ``byte code'' internals. @@ -36291,10 +36580,10 @@ The generated Info file for this @value{DOCUMENT}. @item doc/gawkinet.texi The Texinfo source file for @ifinfo -@inforef{Top, , General Introduction, gawkinet, TCP/IP Internetworking with @command{gawk}}. +@inforef{Top, , General Introduction, gawkinet, @value{GAWKINETTITLE}}. @end ifinfo @ifnotinfo -@cite{TCP/IP Internetworking with @command{gawk}}. +@cite{@value{GAWKINETTITLE}}. @end ifnotinfo It should be processed with @TeX{} (via @command{texi2dvi} or @command{texi2pdf}) @@ -36303,7 +36592,7 @@ with @command{makeinfo} to produce an Info or HTML file. @item doc/gawkinet.info The generated Info file for -@cite{TCP/IP Internetworking with @command{gawk}}. +@cite{@value{GAWKINETTITLE}}. @item doc/igawk.1 The @command{troff} source for a manual page describing the @command{igawk} @@ -36365,6 +36654,12 @@ The source code, manual pages, and infrastructure files for the sample extensions included with @command{gawk}. @xref{Dynamic Extensions}, for more information. +@item extras/* +Additional non-essential files. Currently, this directory contains some shell +startup files to be installed in @file{/etc/profile.d} to aid in manipulating +the @env{AWKPATH} and @env{AWKLIBPATH} environment variables. +@xref{Shell Startup Files}, for more information. + @item posix/* Files needed for building @command{gawk} on POSIX-compliant systems. @@ -36373,11 +36668,11 @@ Files needed for building @command{gawk} under MS-Windows @ifclear FOR_PRINT and OS/2 @end ifclear -(@pxref{PC Installation}, for details). +(@DBPXREF{PC Installation} for details). @item vms/* Files needed for building @command{gawk} under Vax/VMS and OpenVMS -(@pxref{VMS Installation}, for details). +(@DBPXREF{VMS Installation} for details). @item test/* A test suite for @@ -36386,10 +36681,9 @@ directory to run your version of @command{gawk} against the test suite. If @command{gawk} successfully passes @samp{make check}, then you can be confident of a successful port. @end table -@c ENDOFRANGE gawdis @node Unix Installation -@appendixsec Compiling and Installing @command{gawk} on Unix-like Systems +@appendixsec Compiling and Installing @command{gawk} on Unix-Like Systems Usually, you can compile and install @command{gawk} by typing only two commands. However, if you use an unusual system, you may need @@ -36397,12 +36691,13 @@ to configure @command{gawk} for your system yourself. @menu * Quick Installation:: Compiling @command{gawk} under Unix. +* Shell Startup Files:: Shell convenience functions. * Additional Configuration Options:: Other compile-time options. * Configuration Philosophy:: How it's all supposed to work. @end menu @node Quick Installation -@appendixsubsec Compiling @command{gawk} for Unix-like Systems +@appendixsubsec Compiling @command{gawk} for Unix-Like Systems The normal installation steps should work on all modern commercial Unix-derived systems, GNU/Linux, BSD-based systems, and the Cygwin @@ -36419,7 +36714,7 @@ described fully in @cite{Autoconf---Generating Automatic Configuration Scripts}, which can be found online at @uref{http://www.gnu.org/software/autoconf/manual/index.html, -the Free Software Foundation's web site}.) +the Free Software Foundation's website}.) @end ifnotinfo @ifinfo (The Autoconf software is described fully starting with @@ -36466,7 +36761,7 @@ run @samp{make check}. All of the tests should succeed. If these steps do not work, or if any of the tests fail, check the files in the @file{README_d} directory to see if you've found a known problem. If the failure is not described there, -please send in a bug report (@pxref{Bugs}). +send in a bug report (@pxref{Bugs}). Of course, once you've built @command{gawk}, it is likely that you will wish to install it. To do so, you need to run the command @samp{make @@ -36477,6 +36772,44 @@ is likely that you will be asked for your password, and you will have to have been set up previously as a user who is allowed to run the @command{sudo} command. +@node Shell Startup Files +@appendixsubsec Shell Startup Files + +The distribution contains shell startup files @file{gawk.sh} and +@file{gawk.csh} containing functions to aid in manipulating +the @env{AWKPATH} and @env{AWKLIBPATH} environment variables. +On a Fedora system, these files should be installed in @file{/etc/profile.d}; +on other platforms, the appropriate location may be different. + +@table @command + +@cindex @command{gawkpath_default} shell function +@item gawkpath_default +Reset the @env{AWKPATH} environment variable to its default value. + +@cindex @command{gawkpath_prepend} shell function +@item gawkpath_prepend +Add the argument to the front of the @env{AWKPATH} environment variable. + +@cindex @command{gawkpath_append} shell function +@item gawkpath_append +Add the argument to the end of the @env{AWKPATH} environment variable. + +@cindex @command{gawklibpath_default} shell function +@item gawklibpath_default +Reset the @env{AWKLIBPATH} environment variable to its default value. + +@cindex @command{gawklibpath_prepend} shell function +@item gawklibpath_prepend +Add the argument to the front of the @env{AWKLIBPATH} environment variable. + +@cindex @command{gawklibpath_append} shell function +@item gawklibpath_append +Add the argument to the end of the @env{AWKLIBPATH} environment variable. + +@end table + + @node Additional Configuration Options @appendixsubsec Additional Configuration Options @cindex @command{gawk}, configuring, options @@ -36498,7 +36831,7 @@ can be configured and compiled. @cindex @option{--disable-lint} configuration option @cindex configuration option, @code{--disable-lint} @item --disable-lint -Disable all lint checking within @code{gawk}. The +Disable all lint checking within @command{gawk}. The @option{--lint} and @option{--lint-old} options (@pxref{Options}) are accepted, but silently do nothing. @@ -36506,14 +36839,17 @@ Similarly, setting the @code{LINT} variable (@pxref{User-modified}) has no effect on the running @command{awk} program. -When used with GCC's automatic dead-code-elimination, this option +When used with the GNU Compiler Collection's (GCC's) +automatic dead-code-elimination, this option cuts almost 23K bytes off the size of the @command{gawk} executable on GNU/Linux x86_64 systems. Results on other systems and with other compilers are likely to vary. Using this option may bring you some slight performance improvement. +@quotation CAUTION Using this option will cause some of the tests in the test suite to fail. This option may be removed at a later date. +@end quotation @cindex @option{--disable-nls} configuration option @cindex configuration option, @code{--disable-nls} @@ -36530,7 +36866,7 @@ function for deficient systems. @end table Use the command @samp{./configure --help} to see the full list of -options that @command{configure} supplies. +options supplied by @command{configure}. @node Configuration Philosophy @appendixsubsec The Configuration Process @@ -36564,19 +36900,19 @@ facts about your operating system. For example, there may not be an @cindex @code{custom.h} file It is possible for your C compiler to lie to @command{configure}. It may do so by not exiting with an error when a library function is not -available. To get around this, edit the file @file{custom.h}. +available. To get around this, edit the @file{custom.h} file. Use an @samp{#ifdef} that is appropriate for your system, and either @code{#define} any constants that @command{configure} should have defined but didn't, or @code{#undef} any constants that @command{configure} defined and -should not have. @file{custom.h} is automatically included by -@file{config.h}. +should not have. The @file{custom.h} file is automatically included by +the @file{config.h} file. It is also possible that the @command{configure} program generated by Autoconf will not work on your system in some other fashion. -If you do have a problem, the file @file{configure.ac} is the input for +If you do have a problem, the @file{configure.ac} file is the input for Autoconf. You may be able to change this file and generate a new version of @command{configure} that works on your system -(@pxref{Bugs}, +(@DBPXREF{Bugs} for information on how to report problems in configuring @command{gawk}). The same mechanism may be used to send in updates to @file{configure.ac} and/or @file{custom.h}. @@ -36610,13 +36946,13 @@ running MS-DOS, any version of MS-Windows, or OS/2. running MS-DOS and any version of MS-Windows. @end ifset In this @value{SECTION}, the term ``Windows32'' -refers to any of Microsoft Windows-95/98/ME/NT/2000/XP/Vista/7/8. +refers to any of Microsoft Windows 95/98/ME/NT/2000/XP/Vista/7/8. The limitations of MS-DOS (and MS-DOS shells under the other operating -systems) has meant that various ``DOS extenders'' are often used with +systems) have meant that various ``DOS extenders'' are often used with programs such as @command{gawk}. The varying capabilities of Microsoft Windows 3.1 and Windows32 can add to the confusion. For an overview -of the considerations, please refer to @file{README_d/README.pc} in +of the considerations, refer to @file{README_d/README.pc} in the distribution. @menu @@ -36779,7 +37115,7 @@ Ancient OS/2 ports of GNU @command{make} are not able to handle the Makefiles of this package. If you encounter any problems with @command{make}, try GNU Make 3.79.1 or later versions. You should find the latest version on -@uref{ftp://hobbes.nmsu.edu/pub/os2/}.@footnote{As of May, 2014, +@uref{ftp://hobbes.nmsu.edu/pub/os2/}.@footnote{As of November 2014, this site is still there, but the author could not find a package for GNU Make.} @end quotation @@ -36812,9 +37148,7 @@ multibyte functionality is not available. @node PC Using @appendixsubsubsec Using @command{gawk} on PC Operating Systems -@c STARTOFRANGE opgawx @cindex operating systems, PC, @command{gawk} on -@c STARTOFRANGE pcgawon @cindex PC operating systems, @command{gawk} on Under MS-DOS and MS-Windows, the Cygwin and MinGW environments support @@ -36828,8 +37162,8 @@ EMX (OS/2 only) supports at least the @samp{|&} operator. @cindex search paths, for source files @cindex @command{gawk}, MS-DOS version of @cindex @command{gawk}, MS-Windows version of -@cindex @code{;} (semicolon), @code{AWKPATH} variable and -@cindex semicolon (@code{;}), @code{AWKPATH} variable and +@cindex @code{;} (semicolon), @env{AWKPATH} variable and +@cindex semicolon (@code{;}), @env{AWKPATH} variable and @cindex @env{AWKPATH} environment variable The MS-DOS and MS-Windows versions of @command{gawk} search for program files as described in @ref{AWKPATH Variable}. However, @@ -36874,7 +37208,7 @@ Under MS-Windows, OS/2 and MS-DOS, Under MS-Windows and MS-DOS, @end ifset @command{gawk} (and many other text programs) silently -translate end-of-line @samp{\r\n} to @samp{\n} on input and @samp{\n} +translates end-of-line @samp{\r\n} to @samp{\n} on input and @samp{\n} to @samp{\r\n} on output. A special @code{BINMODE} variable @value{COMMONEXT} allows control over these translations and is interpreted as follows: @@ -36908,7 +37242,7 @@ Setting @code{BINMODE} for standard input or standard output is accomplished by using an appropriate @samp{-v BINMODE=@var{N}} option on the command line. @code{BINMODE} is set at the time a file or pipe is opened and cannot be -changed mid-stream. +changed midstream. The name @code{BINMODE} was chosen to match @command{mawk} (@pxref{Other Versions}). @@ -36964,8 +37298,8 @@ moved into the @code{BEGIN} rule. @command{gawk} can be built and used ``out of the box'' under MS-Windows if you are using the @uref{http://www.cygwin.com, Cygwin environment}. -This environment provides an excellent simulation of GNU/Linux, using the -GNU tools, such as Bash, the GNU Compiler Collection (GCC), GNU Make, +This environment provides an excellent simulation of GNU/Linux, using +Bash, GCC, GNU Make, and other GNU programs. Compilation and installation for Cygwin is the same as for a Unix system: @@ -36984,12 +37318,12 @@ and then the @samp{make} proceeds as usual. @appendixsubsubsec Using @command{gawk} In The MSYS Environment In the MSYS environment under MS-Windows, @command{gawk} automatically -uses binary mode for reading and writing files. Thus there is no +uses binary mode for reading and writing files. Thus, there is no need to use the @code{BINMODE} variable. This can cause problems with other Unix-like components that have been ported to MS-Windows that expect @command{gawk} to do automatic -translation of @code{"\r\n"}, since it won't. +translation of @code{"\r\n"}, because it won't. @node VMS Installation @appendixsubsec Compiling and Installing @command{gawk} on Vax/VMS and OpenVMS @@ -37048,19 +37382,19 @@ With ODS-5 volumes and extended parsing enabled, the case of the target parameter may need to be exact. @command{gawk} has been tested under VAX/VMS 7.3 and Alpha/VMS 7.3-1 -using Compaq C V6.4, and Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3. +using Compaq C V6.4, and under Alpha/VMS 7.3, Alpha/VMS 7.3-2, and IA64/VMS 8.3. The most recent builds used HP C V7.3 on Alpha VMS 8.3 and both Alpha and IA64 VMS 8.4 used HP C 7.3.@footnote{The IA64 architecture is also known as ``Itanium.''} -@xref{VMS GNV}, for information on building +@DBXREF{VMS GNV} for information on building @command{gawk} as a PCSI kit that is compatible with the GNV product. @node VMS Dynamic Extensions @appendixsubsubsec Compiling @command{gawk} Dynamic Extensions on VMS The extensions that have been ported to VMS can be built using one of -the following commands. +the following commands: @example $ @kbd{MMS/DESCRIPTION=[.vms]descrip.mms extensions} @@ -37077,7 +37411,7 @@ $ @kbd{MMK/DESCRIPTION=[.vms]descrip.mms extensions} or a logical name to find the dynamic extensions. Dynamic extensions need to be compiled with the same compiler options for -floating point, pointer size, and symbol name handling as were used +floating-point, pointer size, and symbol name handling as were used to compile @command{gawk} itself. Alpha and Itanium should use IEEE floating point. The pointer size is 32 bits, and the symbol name handling should be exact case with CRC shortening for @@ -37096,7 +37430,7 @@ For VAX: /name=(as_is,short) @end example -Compile time macros need to be defined before the first VMS-supplied +Compile-time macros need to be defined before the first VMS-supplied header file is included, as follows: @example @@ -37143,7 +37477,7 @@ If your @command{gawk} was installed by a PCSI kit into the @file{GNV$GNU:[vms_help]gawk.hlp}. The PCSI kit also installs a @file{GNV$GNU:[vms_bin]gawk_verb.cld} file -which can be used to add @command{gawk} and @command{awk} as DCL commands. +that can be used to add @command{gawk} and @command{awk} as DCL commands. For just the current process you can use: @@ -37152,7 +37486,7 @@ $ @kbd{set command gnv$gnu:[vms_bin]gawk_verb.cld} @end example Or the system manager can use @file{GNV$GNU:[vms_bin]gawk_verb.cld} to -add the @command{gawk} and @command{awk} to the system wide @samp{DCLTABLES}. +add the @command{gawk} and @command{awk} to the system-wide @samp{DCLTABLES}. The DCL syntax is documented in the @file{gawk.hlp} file. @@ -37207,7 +37541,7 @@ Note that uppercase and mixed-case text must be quoted. The VMS port of @command{gawk} includes a @code{DCL}-style interface in addition to the original shell-style interface (see the help entry for details). One side effect of dual command-line parsing is that if there is only a -single parameter (as in the quoted string program above), the command +single parameter (as in the quoted string program), the command becomes ambiguous. To work around this, the normally optional @option{--} flag is required to force Unix-style parsing rather than @code{DCL} parsing. If any other dash-type options (or multiple parameters such as @value{DF}s to @@ -37218,14 +37552,14 @@ The @code{exit} value is a Unix-style value and is encoded into a VMS exit status value when the program exits. The VMS severity bits will be set based on the @code{exit} value. -A failure is indicated by 1 and VMS sets the @code{ERROR} status. -A fatal error is indicated by 2 and VMS sets the @code{FATAL} status. +A failure is indicated by 1, and VMS sets the @code{ERROR} status. +A fatal error is indicated by 2, and VMS sets the @code{FATAL} status. All other values will have the @code{SUCCESS} status. The exit value is encoded to comply with VMS coding standards and will have the @code{C_FACILITY_NO} of @code{0x350000} with the constant @code{0xA000} added to the number shifted over by 3 bits to make room for the severity codes. -To extract the actual @command{gawk} exit code from the VMS status use: +To extract the actual @command{gawk} exit code from the VMS status, use: @example unix_status = (vms_status .and. &x7f8) / 8 @@ -37244,7 +37578,7 @@ VAX/VMS floating point uses unbiased rounding. @xref{Round Function}. VMS reports time values in GMT unless one of the @code{SYS$TIMEZONE_RULE} or @code{TZ} logical names is set. Older versions of VMS, such as VAX/VMS -7.3 do not set these logical names. +7.3, do not set these logical names. @c @cindex directory search @c @cindex path, search @@ -37262,7 +37596,7 @@ translation and not a multitranslation @code{RMS} searchlist. The VMS GNV package provides a build environment similar to POSIX with ports of a collection of open source tools. The @command{gawk} found in the GNV -base kit is an older port. Currently the GNV project is being reorganized +base kit is an older port. Currently, the GNV project is being reorganized to supply individual PCSI packages for each component. See @w{@uref{https://sourceforge.net/p/gnv/wiki/InstallingGNVPackages/}.} @@ -37322,40 +37656,36 @@ $ @kbd{gawk :== $sys$common:[syshlp.examples.tcpip.snmp]gawk.exe} This is apparently @value{PVERSION} 2.15.6, which is extremely old. We recommend compiling and using the current version. -@c ENDOFRANGE opgawx -@c ENDOFRANGE pcgawon @node Bugs @appendixsec Reporting Problems and Bugs -@cindex archeologists +@cindex archaeologists @quotation -@i{There is nothing more dangerous than a bored archeologist.} -@author The Hitchhiker's Guide to the Galaxy +@i{There is nothing more dangerous than a bored archaeologist.} +@author Douglas Adams, @cite{The Hitchhiker's Guide to the Galaxy} @end quotation @c the radio show, not the book. :-) -@c STARTOFRANGE dbugg @cindex debugging @command{gawk}, bug reports -@c STARTOFRANGE tblgawb @cindex troubleshooting, @command{gawk}, bug reports If you have problems with @command{gawk} or think that you have found a bug, -please report it to the developers; we cannot promise to do anything +report it to the developers; we cannot promise to do anything, but we might well want to fix it. -Before reporting a bug, please make sure you have really found a genuine bug. +Before reporting a bug, make sure you have really found a genuine bug. Carefully reread the documentation and see if it says you can do what you're trying to do. If it's not clear whether you should be able to do something or not, report that too; it's a bug in the documentation! Before reporting a bug or trying to fix it yourself, try to isolate it to the smallest possible @command{awk} program and input @value{DF} that -reproduces the problem. Then send us the program and @value{DF}, +reproduce the problem. Then send us the program and @value{DF}, some idea of what kind of Unix system you're using, the compiler you used to compile @command{gawk}, and the exact results @command{gawk} gave you. Also say what you expected to occur; this helps us decide whether the problem is really in the documentation. -Please include the version number of @command{gawk} you are using. +Make sure to include the version number of @command{gawk} you are using. You can get this information with the command @samp{gawk --version}. @cindex @code{bug-gawk@@gnu.org} bug reporting address @@ -37364,10 +37694,10 @@ You can get this information with the command @samp{gawk --version}. Once you have a precise problem description, send email to @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org}. -The @command{gawk} maintainers subscribe to this address and +The @command{gawk} maintainers subscribe to this address, and thus they will receive your bug report. Although you can send mail to the maintainers directly, -the bug reporting address is preferred since the +the bug reporting address is preferred because the email list is archived at the GNU Project. @emph{All email must be in English. This is the only language understood in common by all the maintainers.} @@ -37376,32 +37706,32 @@ understood in common by all the maintainers.} @quotation CAUTION Do @emph{not} try to report bugs in @command{gawk} by posting to the Usenet/Internet newsgroup @code{comp.lang.awk}. -While the @command{gawk} developers do occasionally read this newsgroup, -there is no guarantee that we will see your posting. The steps described -above are the only official recognized way for reporting bugs. +The @command{gawk} developers do occasionally read this newsgroup, +but there is no guarantee that we will see your posting. The steps described +here are the only officially recognized way for reporting bugs. Really. @end quotation @quotation NOTE Many distributions of GNU/Linux and the various BSD-based operating systems have their own bug reporting systems. If you report a bug using your distribution's -bug reporting system, @emph{please} also send a copy to +bug reporting system, you should also send a copy to @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org}. -This is for two reasons. First, while some distributions forward +This is for two reasons. First, although some distributions forward bug reports ``upstream'' to the GNU mailing list, many don't, so there is a good chance that the @command{gawk} maintainers won't even see the bug report! Second, -mail to the GNU list is archived, and having everything at the GNU project -keeps things self-contained and not dependant on other organizations. +mail to the GNU list is archived, and having everything at the GNU Project +keeps things self-contained and not dependent on other organizations. @end quotation Non-bug suggestions are always welcome as well. If you have questions about things that are unclear in the documentation or are just obscure features, ask on the bug list; we will try to help you out if we can. -If you find bugs in one of the non-Unix ports of @command{gawk}, please -send an electronic mail message to the bug list, with a copy to the -person who maintains that port. They are named in the following list, +If you find bugs in one of the non-Unix ports of @command{gawk}, +send an email to the bug list, with a copy to the +person who maintains that port. The maintainers are named in the following list, as well as in the @file{README} file in the @command{gawk} distribution. Information in the @file{README} file should be considered authoritative if it conflicts with this @value{DOCUMENT}. @@ -37416,30 +37746,26 @@ The people maintaining the various @command{gawk} ports are: @cindex Robbins, Arnold @cindex Zaretskii, Eli @multitable {MS-Windows with MinGW} {123456789012345678901234567890123456789001234567890} -@item Unix and POSIX systems @tab Arnold Robbins, @EMAIL{arnold@@skeeve.com,arnold at skeeve dot com}. +@item Unix and POSIX systems @tab Arnold Robbins, @EMAIL{arnold@@skeeve.com,arnold at skeeve dot com} -@item MS-DOS with DJGPP @tab Scott Deifik, @EMAIL{scottd.mail@@sbcglobal.net,scottd dot mail at sbcglobal dot net}. +@item MS-DOS with DJGPP @tab Scott Deifik, @EMAIL{scottd.mail@@sbcglobal.net,scottd dot mail at sbcglobal dot net} -@item MS-Windows with MinGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org}. +@item MS-Windows with MinGW @tab Eli Zaretskii, @EMAIL{eliz@@gnu.org,eliz at gnu dot org} @c Leave this in the print version on purpose. @c OS/2 is not mentioned anywhere else in the print version though. -@item OS/2 @tab Andreas Buening, @EMAIL{andreas.buening@@nexgo.de,andreas dot buening at nexgo dot de}. +@item OS/2 @tab Andreas Buening, @EMAIL{andreas.buening@@nexgo.de,andreas dot buening at nexgo dot de} -@item VMS @tab John Malmberg, @EMAIL{wb8tyw@@qsl.net,wb8tyw at qsl.net}. +@item VMS @tab John Malmberg, @EMAIL{wb8tyw@@qsl.net,wb8tyw at qsl.net} -@item z/OS (OS/390) @tab Dave Pitts, @EMAIL{dpitts@@cozx.com,dpitts at cozx dot com}. +@item z/OS (OS/390) @tab Dave Pitts, @EMAIL{dpitts@@cozx.com,dpitts at cozx dot com} @end multitable -If your bug is also reproducible under Unix, please send a copy of your -report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email -list as well. -@c ENDOFRANGE dbugg -@c ENDOFRANGE tblgawb +If your bug is also reproducible under Unix, send a copy of your +report to the @EMAIL{bug-gawk@@gnu.org,bug-gawk at gnu dot org} email list as well. @node Other Versions @appendixsec Other Freely Available @command{awk} Implementations -@c STARTOFRANGE awkim @cindex @command{awk}, implementations @ignore From: emory!amc.com!brennan (Michael Brennan) @@ -37451,7 +37777,7 @@ Date: Wed, 4 Sep 1996 08:11:48 -0700 (PDT) @cindex Brennan, Michael @ifnotdocbook @quotation -@i{It's kind of fun to put comments like this in your awk code.}@* +@i{It's kind of fun to put comments like this in your awk code:}@* @ @ @ @ @ @ @code{// Do C++ comments work? answer: yes! of course} @author Michael Brennan @end quotation @@ -37464,8 +37790,6 @@ Date: Wed, 4 Sep 1996 08:11:48 -0700 (PDT) </blockquote> @end docbook - - There are a number of other freely available @command{awk} implementations. This @value{SECTION} briefly describes where to get them: @@ -37478,7 +37802,7 @@ This @value{SECTION} briefly describes where to get them: Brian Kernighan, one of the original designers of Unix @command{awk}, has made his implementation of @command{awk} freely available. -You can retrieve this version via the World Wide Web from +You can retrieve this version via @uref{http://www.cs.princeton.edu/~bwk, his home page}. It is available in several archive formats: @@ -37494,14 +37818,14 @@ It is available in several archive formats: @end table @cindex @command{git} utility -You can also retrieve it from Git Hub: +You can also retrieve it from GitHub: @example git clone git://github.com/onetrueawk/awk bwkawk @end example @noindent -The above command creates a copy of the @uref{http://www.git-scm.com, Git} +This command creates a copy of the @uref{http://git-scm.com, Git} repository in a directory named @file{bwkawk}. If you leave that argument off the @command{git} command line, the repository copy is created in a directory named @file{awk}. @@ -37509,9 +37833,13 @@ directory named @file{awk}. This version requires an ISO C (1990 standard) compiler; the C compiler from GCC (the GNU Compiler Collection) works quite nicely. -@xref{Common Extensions}, +@DBXREF{Common Extensions} for a list of extensions in this @command{awk} that are not in POSIX @command{awk}. +As a side note, Dan Bornstein has created a Git repository tracking +all the versions of BWK @command{awk} that he could find. It's +available at @uref{git://github.com/danfuzz/one-true-awk}. + @cindex Brennan, Michael @cindex @command{mawk} utility @cindex source code, @command{mawk} @@ -37541,7 +37869,7 @@ Once you have it, is similar to @command{gawk}'s (@pxref{Unix Installation}). -@xref{Common Extensions}, +@DBXREF{Common Extensions} for a list of extensions in @command{mawk} that are not in POSIX @command{awk}. @cindex Sumner, Andrew @@ -37550,7 +37878,7 @@ for a list of extensions in @command{mawk} that are not in POSIX @command{awk}. @item @command{awka} Written by Andrew Sumner, @command{awka} translates @command{awk} programs into C, compiles them, -and links them with a library of functions that provides the core +and links them with a library of functions that provide the core @command{awk} functionality. It also has a number of extensions. @@ -37562,7 +37890,7 @@ To get @command{awka}, go to @url{http://sourceforge.net/projects/awka}. @c andrewsumner@@yahoo.net The project seems to be frozen; no new code changes have been made -since approximately 2003. +since approximately 2001. @cindex Beebe, Nelson H.F.@: @cindex @command{pawk} (profiling version of Brian Kernighan's @command{awk}) @@ -37571,17 +37899,17 @@ since approximately 2003. Nelson H.F.@: Beebe at the University of Utah has modified BWK @command{awk} to provide timing and profiling information. It is different from @command{gawk} with the @option{--profile} option -(@pxref{Profiling}), +(@pxref{Profiling}) in that it uses CPU-based profiling, not line-count profiling. You may find it at either @uref{ftp://ftp.math.utah.edu/pub/pawk/pawk-20030606.tar.gz} or @uref{http://www.math.utah.edu/pub/pawk/pawk-20030606.tar.gz}. -@item Busybox Awk -@cindex Busybox Awk -@cindex source code, Busybox Awk -Busybox is a GPL-licensed program providing small versions of many +@item BusyBox @command{awk} +@cindex BusyBox Awk +@cindex source code, BusyBox Awk +BusyBox is a GPL-licensed program providing small versions of many applications within a single executable. It is aimed at embedded systems. It includes a full implementation of POSIX @command{awk}. When building it, be careful not to do @samp{make install} as it will overwrite @@ -37593,7 +37921,7 @@ information, see the @uref{http://busybox.net, project's home page}. @cindex source code, Solaris @command{awk} @item The OpenSolaris POSIX @command{awk} The versions of @command{awk} in @file{/usr/xpg4/bin} and -@file{/usr/xpg6/bin} on Solaris are more-or-less POSIX-compliant. +@file{/usr/xpg6/bin} on Solaris are more or less POSIX-compliant. They are based on the @command{awk} from Mortice Kern Systems for PCs. We were able to make this code compile and work under GNU/Linux with 1--2 hours of work. Making it more generally portable (using @@ -37603,8 +37931,8 @@ has not been done, at least to our knowledge. @cindex Illumos @cindex Illumos, POSIX-compliant @command{awk} @cindex source code, Illumos @command{awk} -The source code used to be available from the OpenSolaris web site. -However, that project was ended and the web site shut down. Fortunately, the +The source code used to be available from the OpenSolaris website. +However, that project was ended and the website shut down. Fortunately, the @uref{http://wiki.illumos.org/display/illumos/illumos+Home, Illumos project} makes this implementation available. You can view the files one at a time from @uref{https://github.com/joyent/illumos-joyent/blob/master/usr/src/cmd/awk_xpg4}. @@ -37623,7 +37951,7 @@ from POSIX @command{awk}. More information is available on the @cindex libmawk @cindex source code, libmawk This is an embeddable @command{awk} interpreter derived from -@command{mawk}. For more information see +@command{mawk}. For more information, see @uref{http://repo.hu/projects/libmawk/}. @item @code{pawk} @@ -37634,10 +37962,10 @@ features to Python. See @uref{https://github.com/alecthomas/pawk} for more information. (This is not related to Nelson Beebe's modified version of BWK @command{awk}, described earlier.) -@item @w{QSE Awk} -@cindex QSE Awk -@cindex source code, QSE Awk -This is an embeddable @command{awk} interpreter. For more information +@item @w{QSE @command{awk}} +@cindex QSE @command{awk} +@cindex source code, QSE @command{awk} +This is an embeddable @command{awk} interpreter. For more information, see @uref{http://code.google.com/p/qse/} and @uref{http://awk.info/?tools/qse}. @item @command{QTawk} @@ -37652,19 +37980,19 @@ including the manual and a download link. The project may also be frozen; no new code changes have been made since approximately 2008. -@item Other Versions -See also the @uref{http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations, -Wikipedia article}, for information on additional versions. +@item Other versions +See also the ``Versions and implementations'' section of the +@uref{http://en.wikipedia.org/wiki/Awk_language#Versions_and_implementations, +Wikipedia article} on @command{awk} for information on additional versions. @end table -@c ENDOFRANGE awkim @node Installation summary @appendixsec Summary @itemize @value{BULLET} @item -The @command{gawk} distribution is available from GNU project's main +The @command{gawk} distribution is available from the GNU Project's main distribution site, @code{ftp.gnu.org}. The canonical build recipe is: @example @@ -37676,34 +38004,30 @@ cd gawk-@value{VERSION}.@value{PATCHLEVEL} @item @command{gawk} may be built on non-POSIX systems as well. The currently -supported systems are MS-Windows using DJGPP, MSYS, MinGW and Cygwin, +supported systems are MS-Windows using DJGPP, MSYS, MinGW, and Cygwin, @ifclear FOR_PRINT OS/2 using EMX, @end ifclear and both Vax/VMS and OpenVMS. -Instructions for each system are included in this @value{CHAPTER}. +Instructions for each system are included in this @value{APPENDIX}. @item Bug reports should be sent via email to @email{bug-gawk@@gnu.org}. -Bug reports should be in English, and should include the version of @command{gawk}, -how it was compiled, and a short program and @value{DF} which demonstrate +Bug reports should be in English and should include the version of @command{gawk}, +how it was compiled, and a short program and @value{DF} that demonstrate the problem. @item There are a number of other freely available @command{awk} -implementations. Many are POSIX compliant; others are less so. +implementations. Many are POSIX-compliant; others are less so. @end itemize -@c ENDOFRANGE gligawk -@c ENDOFRANGE ingawk @ifclear FOR_PRINT @node Notes @appendix Implementation Notes -@c STARTOFRANGE gawii @cindex @command{gawk}, implementation issues -@c STARTOFRANGE impis @cindex implementation issues, @command{gawk} This appendix contains information mainly of interest to implementers and @@ -37779,7 +38103,7 @@ However, if you want to modify @command{gawk} and contribute back your changes, you will probably wish to work with the development version. To do so, you will need to access the @command{gawk} source code repository. The code is maintained using the -@uref{http://git-scm.com/, Git distributed version control system}. +@uref{http://git-scm.com, Git distributed version control system}. You will need to install it if your system doesn't have it. Once you have done so, use the command: @@ -37808,11 +38132,8 @@ that has a Git plug-in for working with Git repositories. @node Adding Code @appendixsubsec Adding New Features -@c STARTOFRANGE adfgaw @cindex adding, features to @command{gawk} -@c STARTOFRANGE fadgaw @cindex features, adding to @command{gawk} -@c STARTOFRANGE gawadf @cindex @command{gawk}, features, adding You are free to add any new features you like to @command{gawk}. However, if you want your changes to be incorporated into the @command{gawk} @@ -37847,7 +38168,7 @@ for information on getting the latest version of @command{gawk}.) @item @ifnotinfo -Follow the @uref{http://www.gnu.org/prep/standards/, @cite{GNU Coding Standards}}. +Follow the @cite{GNU Coding Standards}. @end ifnotinfo @ifinfo See @inforef{Top, , Version, standards, GNU Coding Standards}. @@ -37856,7 +38177,7 @@ This document describes how GNU software should be written. If you haven't read it, please do so, preferably @emph{before} starting to modify @command{gawk}. (The @cite{GNU Coding Standards} are available from the GNU Project's -@uref{http://www.gnu.org/prep/standards_toc.html, web site}. +@uref{http://www.gnu.org/prep/standards/, website}. Texinfo, Info, and DVI versions are also available.) @cindex @command{gawk}, coding style in @@ -37979,9 +38300,6 @@ Although this sounds like a lot of work, please remember that while you may write the new code, I have to maintain it and support it. If it isn't possible for me to do that with a minimum of extra work, then I probably will not. -@c ENDOFRANGE adfgaw -@c ENDOFRANGE gawadf -@c ENDOFRANGE fadgaw @node New Ports @appendixsubsec Porting @command{gawk} to a New Operating System @@ -38115,7 +38433,6 @@ coding style and brace layout that suits your taste. @node Derived Files @appendixsubsec Why Generated Files Are Kept In Git -@c STARTOFRANGE gawkgit @cindex Git, use of for @command{gawk} source code @c From emails written March 22, 2012, to the gawk developers list. @@ -38304,7 +38621,6 @@ wget http://git.savannah.gnu.org/cgit/gawk.git/snapshot/gawk-@var{branchname}.ta @noindent to retrieve a snapshot of the given branch. -@c ENDOFRANGE gawkgit @node Future Extensions @appendixsec Probable Future Extensions @@ -38685,13 +39001,10 @@ of @command{gawk}, but it @emph{will} be removed in the next major release. @end itemize -@c ENDOFRANGE impis -@c ENDOFRANGE gawii @node Basic Concepts @appendix Basic Programming Concepts @cindex programming, concepts -@c STARTOFRANGE procon @cindex programming, concepts This @value{APPENDIX} attempts to define some of the basic concepts @@ -38929,7 +39242,6 @@ standard for C. This standard became an ISO standard in 1990. In 1999, a revised ISO C standard was approved and released. Where it makes sense, POSIX @command{awk} is compatible with 1999 ISO C. -@c ENDOFRANGE procon @node Glossary @unnumbered Glossary @@ -38941,6 +39253,13 @@ pattern matches an input record, @command{awk} executes the rule's action. Actions are always enclosed in braces. (@xref{Action Overview}.) +@cindex Ada programming language +@cindex programming languages, Ada +@item Ada +A programming language originally defined by the U.S.@: Department of +Defense for embedded programming. It was designed to enforce good +Software Engineering practices. + @cindex Spencer, Henry @cindex @command{sed} utility @cindex amazing @command{awk} assembler (@command{aaa}) @@ -38952,13 +39271,6 @@ microcomputers. It is a good example of a program that would have been better written in another language. You can get it from @uref{http://awk.info/?awk100/aaa}. -@cindex Ada programming language -@cindex programming languages, Ada -@item Ada -A programming language originally defined by the U.S.@: Department of -Defense for embedded programming. It was designed to enforce good -Software Engineering practices. - @cindex amazingly workable formatter (@command{awf}) @cindex @command{awf} (amazingly workable formatter) program @item Amazingly Workable Formatter (@command{awf}) @@ -38980,6 +39292,21 @@ languages. These standards often become international standards as well. See also ``ISO.'' +@item Argument +An argument can be two different things. It can be an option or a +@value{FN} passed to a command while invoking it from the command line, or +it can be something passed to a @dfn{function} inside a program, e.g. +inside @command{awk}. + +In the latter case, an argument can be passed to a function in two ways. +Either it is given to the called function by value, i.e., a copy of the +value of the variable is made available to the called function, but the +original variable cannot be modified by the function itself; or it is +given by reference, i.e., a pointer to the interested variable is passed to +the function, which can then directly modify it. In @command{awk} +scalars are passed by value, and arrays are passed by reference. +See ``Pass By Value/Reference.'' + @item Array A grouping of multiple values under the same name. Most languages just provide sequential arrays. @@ -39021,6 +39348,25 @@ The GNU version of the standard shell @end ifinfo See also ``Bourne Shell.'' +@item Binary +Base-two notation, where the digits are @code{0}--@code{1}. Since +electronic circuitry works ``naturally'' in base 2 (just think of Off/On), +everything inside a computer is calculated using base 2. Each digit +represents the presence (or absence) of a power of 2 and is called a +@dfn{bit}. So, for example, the base-two number @code{10101} is +the same as decimal 21, ((1 x 16) + (1 x 4) + (1 x 1)). + +Since base-two numbers quickly become +very long to read and write, they are usually grouped by 3 (i.e., they are +read as octal numbers), or by 4 (i.e., they are read as hexadecimal +numbers). There is no direct way to insert base 2 numbers in a C program. +If need arises, such numbers are usually inserted as octal or hexadecimal +numbers. The number of base-two digits that fit into registers used for +representing integer numbers in computers is a rough indication of the +computing power of the computer itself. Most computers nowadays use 64 +bits for representing integer numbers in their registers, but 32-bit, +16-bit and 8-bit registers have been widely used in the past. +@xref{Nondecimal-numbers}. @item Bit Short for ``Binary Digit.'' All values in computer memory ultimately reduce to binary digits: values @@ -39047,6 +39393,24 @@ originally written by Steven R.@: Bourne at Bell Laboratories. Many shells (Bash, @command{ksh}, @command{pdksh}, @command{zsh}) are generally upwardly compatible with the Bourne shell. +@item Braces +The characters @samp{@{} and @samp{@}}. Braces are used in +@command{awk} for delimiting actions, compound statements, and function +bodies. + +@item Bracket Expression +Inside a @dfn{regular expression}, an expression included in square +brackets, meant to designate a single character as belonging to a +specified character class. A bracket expression can contain a list of one +or more characters, like @samp{[abc]}, a range of characters, like +@samp{[A-Z]}, or a name, delimited by @samp{:}, that designates a known set +of characters, like @samp{[:digit:]}. The form of bracket expression +enclosed between @samp{:} is independent of the underlying representation +of the character themselves, which could utilize the ASCII, ECBDIC, or +Unicode codesets, depending on the architecture of the computer system, and on +localization. +See also ``Regular Expression.'' + @item Built-in Function The @command{awk} language provides built-in functions that perform various numerical, I/O-related, and string computations. Examples are @@ -39092,11 +39456,6 @@ are the variables that have special meaning to @command{gawk}. Changing some of them affects @command{awk}'s running environment. (@xref{Built-in Variables}.) -@item Braces -The characters @samp{@{} and @samp{@}}. Braces are used in -@command{awk} for delimiting actions, compound statements, and function -bodies. - @item C The system programming language that most GNU software is written in. The @command{awk} programming language has C-like syntax, and this @value{DOCUMENT} @@ -39105,9 +39464,25 @@ points out similarities between @command{awk} and C when appropriate. In general, @command{gawk} attempts to be as similar to the 1990 version of ISO C as makes sense. +@item C Shell +The C Shell (@command{csh} or its improved version, @command{tcsh}) is a Unix shell that was +created by Bill Joy in the late 1970s. The C shell was differentiated from +other shells by its interactive features and overall style, which +looks more like C. The C Shell is not backward compatible with the Bourne +Shell, so special attention is required when converting scripts +written for other Unix shells to the C shell, especially with regard to the management of +shell variables. +See also ``Bourne Shell.'' + @item C++ A popular object-oriented programming language derived from C. +@item Character Class +See ``Bracket Expression.'' + +@item Character List +See ``Bracket Expression.'' + @cindex ASCII @cindex ISO 8859-1 @cindex ISO Latin-1 @@ -39131,7 +39506,59 @@ A preprocessor for @command{pic} that reads descriptions of molecules and produces @command{pic} input for drawing them. It was written in @command{awk} by Brian Kernighan and Jon Bentley, and is available from -@uref{http://netlib.sandia.gov/netlib/typesetting/chem.gz}. +@uref{http://netlib.org/typesetting/chem}. + +@item Comparison Expression +A relation that is either true or false, such as @samp{a < b}. +Comparison expressions are used in @code{if}, @code{while}, @code{do}, +and @code{for} +statements, and in patterns to select which input records to process. +(@xref{Typing and Comparison}.) + +@cindex compiled programs +@item Compiler +A program that translates human-readable source code into +machine-executable object code. The object code is then executed +directly by the computer. +See also ``Interpreter.'' + +@item Complemented Bracket Expression +The negation of a @dfn{bracket expression}. All that is @emph{not} +described by a given bracket expression. The symbol @samp{^} precedes +the negated bracket expression. E.g.: @samp{[[^:digit:]} +designates whatever character is not a digit. @samp{[^bad]} +designates whatever character is not one of the letters @samp{b}, @samp{a}, +or @samp{d}. +See ``Bracket Expression.'' + +@item Compound Statement +A series of @command{awk} statements, enclosed in curly braces. Compound +statements may be nested. +(@xref{Statements}.) + +@item Computed Regexps +See ``Dynamic Regular Expressions.'' + +@item Concatenation +Concatenating two strings means sticking them together, one after another, +producing a new string. For example, the string @samp{foo} concatenated with +the string @samp{bar} gives the string @samp{foobar}. +(@xref{Concatenation}.) + +@item Conditional Expression +An expression using the @samp{?:} ternary operator, such as +@samp{@var{expr1} ? @var{expr2} : @var{expr3}}. The expression +@var{expr1} is evaluated; if the result is true, the value of the whole +expression is the value of @var{expr2}; otherwise the value is +@var{expr3}. In either case, only one of @var{expr2} and @var{expr3} +is evaluated. (@xref{Conditional Exp}.) + +@item Control Statement +A control statement is an instruction to perform a given operation or a set +of operations inside an @command{awk} program, if a given condition is +true. Control statements are: @code{if}, @code{for}, @code{while}, and +@code{do} +(@pxref{Statements}). @cindex McIlroy, Doug @cindex cookie @@ -39181,39 +39608,6 @@ Doug @item Coprocess A subordinate program with which two-way communications is possible. -@cindex compiled programs -@item Compiler -A program that translates human-readable source code into -machine-executable object code. The object code is then executed -directly by the computer. -See also ``Interpreter.'' - -@item Compound Statement -A series of @command{awk} statements, enclosed in curly braces. Compound -statements may be nested. -(@xref{Statements}.) - -@item Concatenation -Concatenating two strings means sticking them together, one after another, -producing a new string. For example, the string @samp{foo} concatenated with -the string @samp{bar} gives the string @samp{foobar}. -(@xref{Concatenation}.) - -@item Conditional Expression -An expression using the @samp{?:} ternary operator, such as -@samp{@var{expr1} ? @var{expr2} : @var{expr3}}. The expression -@var{expr1} is evaluated; if the result is true, the value of the whole -expression is the value of @var{expr2}; otherwise the value is -@var{expr3}. In either case, only one of @var{expr2} and @var{expr3} -is evaluated. (@xref{Conditional Exp}.) - -@item Comparison Expression -A relation that is either true or false, such as @samp{a < b}. -Comparison expressions are used in @code{if}, @code{while}, @code{do}, -and @code{for} -statements, and in patterns to select which input records to process. -(@xref{Typing and Comparison}.) - @item Curly Braces See ``Braces.'' @@ -39259,15 +39653,15 @@ ordinary expression. It could be a string constant, such as @code{"foo"}, but it may also be an expression whose value can vary. (@xref{Computed Regexps}.) +@item Empty String +See ``Null String.'' + @item Environment A collection of strings, of the form @samp{@var{name}=@var{val}}, that each program has available to it. Users generally place values into the environment in order to provide information to various programs. Typical examples are the environment variables @env{HOME} and @env{PATH}. -@item Empty String -See ``Null String.'' - @cindex epoch, definition of @item Epoch The date used as the ``beginning of time'' for timestamps. @@ -39320,19 +39714,15 @@ Format strings control the appearance of output in the are controlled by the format strings contained in the predefined variables @code{CONVFMT} and @code{OFMT}. (@xref{Control Letters}.) +@item Fortran +Shorthand for FORmula TRANslator, one of the first programming languages +available for scientific calculations. It was created by John Backus, +and has been available since 1957. It is still in use today. + @item Free Documentation License This document describes the terms under which this @value{DOCUMENT} is published and may be copied. (@xref{GNU Free Documentation License}.) -@item Function -A specialized group of statements used to encapsulate general -or program-specific tasks. @command{awk} has a number of built-in -functions, and also allows you to define your own. -(@xref{Functions}.) - -@item FSF -See ``Free Software Foundation.'' - @cindex FSF (Free Software Foundation) @cindex Free Software Foundation (FSF) @cindex Stallman, Richard @@ -39342,6 +39732,26 @@ to the production and distribution of freely distributable software. It was founded by Richard M.@: Stallman, the author of the original Emacs editor. GNU Emacs is the most widely used version of Emacs today. +@item FSF +See ``Free Software Foundation.'' + +@item Function +A part of an @command{awk} program that can be invoked from every point of +the program, to perform a task. @command{awk} has several built-in +functions. +Users can define their own functions in every part of the program. +Function can be recursive, i.e., they may invoke themselves. +@xref{Functions}. +In @command{gawk} it is also possible to have functions shared +among different programs, and included where required using the +@code{@@include} directive +(@pxref{Include Files}). +In @command{gawk} the name of the function that should be invoked +can be generated at run time, i.e., dynamically. +The @command{gawk} extension API provides constructor functions +(@pxref{Constructor Functions}). + + @item @command{gawk} The GNU implementation of @command{awk}. @@ -39464,6 +39874,12 @@ meaning. Keywords are reserved and may not be used as variable names. and @code{while}. +@item Korn Shell +The Korn Shell (@command{ksh}) is a Unix shell which was developed by David Korn at Bell +Laboratories in the early 1980s. The Korn Shell is backward-compatible with the Bourne +shell and includes many features of the C shell. +See also ``Bourne Shell.'' + @cindex LGPL (Lesser General Public License) @cindex Lesser General Public License (LGPL) @cindex GNU Lesser General Public License @@ -39472,12 +39888,12 @@ This document describes the terms under which binary library archives or shared objects, and their source code may be distributed. -@item Linux -See ``GNU/Linux.'' - @item LGPL See ``Lesser General Public License.'' +@item Linux +See ``GNU/Linux.'' + @item Localization The process of providing the data necessary for an internationalized program to work in a particular language. @@ -39503,6 +39919,14 @@ Characters used within a regexp that do not stand for themselves. Instead, they denote regular expression operations, such as repetition, grouping, or alternation. +@item Nesting +Nesting is where information is organized in layers, or where objects +contain other similar objects. +In @command{gawk} the @code{@@include} +directive can be nested. The ``natural'' nesting of arithmetic and +logical operations can be changed using parentheses +(@pxref{Precedence}). + @item No-op An operation that does nothing. @@ -39523,6 +39947,11 @@ Octal numbers are written in C using a leading @samp{0}, to indicate their base. Thus, @code{013} is 11 ((1 x 8) + 3). @xref{Nondecimal-numbers}. +@item Output Record +A single chunk of data that is written out by @command{awk}. Usually, an +@command{awk} output record consists of one or more lines of text. +@xref{Records}. + @item Pattern Patterns tell @command{awk} which input records are interesting to which rules. @@ -39537,6 +39966,9 @@ An acronym describing what is possibly the most frequent source of computer usage problems. (Problem Exists Between Keyboard And Chair.) +@item Plug-in +See ``Extensions.'' + @item POSIX The name for a series of standards that specify a Portable Operating System interface. The ``IX'' denotes @@ -39561,6 +39993,9 @@ A sequence of consecutive lines from the input file(s). A pattern can specify ranges of input lines for @command{awk} to process or it can specify single lines. (@xref{Pattern Overview}.) +@item Record +See ``Input record'' and ``Output record.'' + @item Recursion When a function calls itself, either directly or indirectly. If this is clear, stop, and proceed to the next entry. @@ -39578,6 +40013,15 @@ operators. (@xref{Getline}, and @ref{Redirection}.) +@item Reference Counts +An internal mechanism in @command{gawk} to minimize the amount of memory +needed to store the value of string variables. If the value assumed by +a variable is used in more than one place, only one copy of the value +itself is kept, and the associated reference count is increased when the +same value is used by an additional variable, and decresed when the related +variable is no longer in use. When the reference count goes to zero, +the memory space used to store the value of the variable is freed. + @item Regexp See ``Regular Expression.'' @@ -39595,6 +40039,15 @@ slashes, such as @code{/foo/}. This regular expression is chosen when you write the @command{awk} program and cannot be changed during its execution. (@xref{Regexp Usage}.) +@item Regular Expression Operators +See ``Metacharacters.'' + +@item Rounding +Rounding the result of an arithmetic operation can be tricky. +More than one way of rounding exists, and in @command{gawk} +it is possible to choose which method should be used in a program. +@xref{Setting the rounding mode}. + @item Rule A segment of an @command{awk} program that specifies how to process single input records. A rule consists of a @dfn{pattern} and an @dfn{action}. @@ -39615,12 +40068,12 @@ Regular variables are scalars; arrays and functions are not. In @command{gawk}, a list of directories to search for @command{awk} program source files. In the shell, a list of directories to search for executable programs. -@item Seed -The initial value, or starting point, for a sequence of random numbers. - @item @command{sed} See ``Stream Editor.'' +@item Seed +The initial value, or starting point, for a sequence of random numbers. + @item Shell The command interpreter for Unix and POSIX-compliant systems. The shell works both interactively, and as a programming language @@ -39654,6 +40107,12 @@ A @value{FN} interpreted internally by @command{gawk}, instead of being handed directly to the underlying operating system---for example, @file{/dev/stderr}. (@xref{Special Files}.) +@item Statement +An expression inside an @command{awk} program in the action part +of a pattern--action rule, or inside an +@command{awk} function. A statement can be a variable assignment, +an array operation, a loop, etc. + @item Stream Editor A program that reads records from an input stream and processes them one or more at a time. This is in contrast with batch programs, which may @@ -39704,9 +40163,14 @@ This is standard time in Greenwich, England, which is used as a reference time for day and date calculations. See also ``Epoch'' and ``GMT.'' +@item Variable +A name for a value. In @command{awk}, variables may be either scalars +or arrays. + @item Whitespace A sequence of space, TAB, or newline characters occurring inside an input record or a string. + @end table @end ifclear @@ -39722,7 +40186,7 @@ record or a string. @end docbook @c This file is intended to be included within another document, -@c hence no sectioning command or @node. +@c hence no sectioning command or @node. @display Copyright @copyright{} 2007 Free Software Foundation, Inc. @url{http://fsf.org/} @@ -39944,7 +40408,7 @@ terms of section 4, provided that you also meet all of these conditions: @enumerate a -@item +@item The work must carry prominent notices stating that you modified it, and giving a relevant date. @@ -40394,7 +40858,7 @@ state the exclusion of warranty; and each file should have at least the ``copyright'' line and a pointer to where the full notice is found. @smallexample -@var{one line to give the program's name and a brief idea of what it does.} +@var{one line to give the program's name and a brief idea of what it does.} Copyright (C) @var{year} @var{name of author} This program is free software: you can redistribute it and/or modify @@ -40417,7 +40881,7 @@ If the program does terminal interaction, make it output a short notice like this when it starts in an interactive mode: @smallexample -@var{program} Copyright (C) @var{year} @var{name of author} +@var{program} Copyright (C) @var{year} @var{name of author} This program comes with ABSOLUTELY NO WARRANTY; for details type @samp{show w}. This is free software, and you are welcome to redistribute it under certain conditions; type @samp{show c} for details. @@ -41101,3 +41565,4 @@ But to use it you have to say which sorta sucks. TODO: +Check that all dark corners are indexed properly. diff --git a/doc/texinfo.tex b/doc/texinfo.tex index 7506dffb..8236d7d2 100644 --- a/doc/texinfo.tex +++ b/doc/texinfo.tex @@ -3,11 +3,12 @@ % Load plain if necessary, i.e., if running under initex. \expandafter\ifx\csname fmtname\endcsname\relax\input plain\fi % -\def\texinfoversion{2014-03-18.17} +\def\texinfoversion{2015-02-05.16} % % Copyright 1985, 1986, 1988, 1990, 1991, 1992, 1993, 1994, 1995, % 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, -% 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014 Free Software Foundation, Inc. +% 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015 +% Free Software Foundation, Inc. % % This texinfo.tex file is free software: you can redistribute it and/or % modify it under the terms of the GNU General Public License as @@ -96,7 +97,9 @@ \let\ptexraggedright=\raggedright \let\ptexrbrace=\} \let\ptexslash=\/ +\let\ptexsp=\sp \let\ptexstar=\* +\let\ptexsup=\sup \let\ptext=\t \let\ptextop=\top {\catcode`\'=\active \global\let\ptexquoteright'}% active in plain's math mode @@ -1010,24 +1013,15 @@ where each line of input produces a line of output.} % paragraph. % \gdef\dosuppressfirstparagraphindent{% - \gdef\indent{% - \restorefirstparagraphindent - \indent - }% - \gdef\noindent{% - \restorefirstparagraphindent - \noindent - }% - \global\everypar = {% - \kern -\parindent - \restorefirstparagraphindent - }% + \gdef\indent {\restorefirstparagraphindent \indent}% + \gdef\noindent{\restorefirstparagraphindent \noindent}% + \global\everypar = {\kern -\parindent \restorefirstparagraphindent}% } - +% \gdef\restorefirstparagraphindent{% - \global \let \indent = \ptexindent - \global \let \noindent = \ptexnoindent - \global \everypar = {}% + \global\let\indent = \ptexindent + \global\let\noindent = \ptexnoindent + \global\everypar = {}% } @@ -2090,12 +2084,9 @@ end \endgroup } - % In order for the font changes to affect most math symbols and letters, -% we have to define the \textfont of the standard families. Since -% texinfo doesn't allow for producing subscripts and superscripts except -% in the main text, we don't bother to reset \scriptfont and -% \scriptscriptfont (which would also require loading a lot more fonts). +% we have to define the \textfont of the standard families. We don't +% bother to reset \scriptfont and \scriptscriptfont; awaiting user need. % \def\resetmathfonts{% \textfont0=\tenrm \textfont1=\teni \textfont2=\tensy @@ -2109,8 +2100,8 @@ end % \tenSTYLE to set the current font. % % Each font-changing command also sets the names \lsize (one size lower) -% and \lllsize (three sizes lower). These relative commands are used in -% the LaTeX logo and acronyms. +% and \lllsize (three sizes lower). These relative commands are used +% in, e.g., the LaTeX logo and acronyms. % % This all needs generalizing, badly. % @@ -2146,7 +2137,7 @@ end \let\tenttsl=\secttsl \def\curfontsize{sec}% \def\lsize{subsec}\def\lllsize{reduced}% - \resetmathfonts \setleading{16pt}} + \resetmathfonts \setleading{17pt}} \def\subsecfonts{% \let\tenrm=\ssecrm \let\tenit=\ssecit \let\tensl=\ssecsl \let\tenbf=\ssecbf \let\tentt=\ssectt \let\smallcaps=\ssecsc @@ -2851,6 +2842,8 @@ end \let\v=\check \let\~=\tilde \let\dotaccent=\dot + % have to provide another name for sup operator + \let\mathopsup=\sup $\finishmath } \def\finishmath#1{#1$\endgroup} % Close the group opened by \tex. @@ -2874,6 +2867,18 @@ end } } +% for @sub and @sup, if in math mode, just do a normal sub/superscript. +% If in text, use math to place as sub/superscript, but switch +% into text mode, with smaller fonts. This is a different font than the +% one used for real math sub/superscripts (8pt vs. 7pt), but let's not +% fix it (significant additions to font machinery) until someone notices. +% +\def\sub{\ifmmode \expandafter\sb \else \expandafter\finishsub\fi} +\def\finishsub#1{$\sb{\hbox{\selectfonts\lllsize #1}}$}% +% +\def\sup{\ifmmode \expandafter\ptexsp \else \expandafter\finishsup\fi} +\def\finishsup#1{$\ptexsp{\hbox{\selectfonts\lllsize #1}}$}% + % ctrl is no longer a Texinfo command, but leave this definition for fun. \def\ctrl #1{{\tt \rawbackslash \hat}#1} @@ -4484,7 +4489,6 @@ end % Called from \indexdummies and \atdummies. % \def\commondummies{% - % % \definedummyword defines \#1 as \string\#1\space, thus effectively % preventing its expansion. This is used only for control words, % not control letters, because the \space would be incorrect for @@ -4561,6 +4565,7 @@ end \definedummyword\guilsinglright \definedummyword\lbracechar \definedummyword\leq + \definedummyword\mathopsup \definedummyword\minus \definedummyword\ogonek \definedummyword\pounds @@ -4574,6 +4579,8 @@ end \definedummyword\quotesinglbase \definedummyword\rbracechar \definedummyword\result + \definedummyword\sub + \definedummyword\sup \definedummyword\textdegree % % We want to disable all macros so that they are not expanded by \write. @@ -4648,6 +4655,7 @@ end \definedummyword\samp \definedummyword\strong \definedummyword\tie + \definedummyword\U \definedummyword\uref \definedummyword\url \definedummyword\var @@ -5739,13 +5747,16 @@ end % % #1 is the text, #2 is the section type (Ynumbered, Ynothing, % Yappendix, Yomitfromtoc), #3 the chapter number. +% Not used for @heading series. % % To test against our argument. \def\Ynothingkeyword{Ynothing} -\def\Yomitfromtockeyword{Yomitfromtoc} \def\Yappendixkeyword{Yappendix} +\def\Yomitfromtockeyword{Yomitfromtoc} % \def\chapmacro#1#2#3{% + \checkenv{}% chapters, etc., should not start inside an environment. + % % Insert the first mark before the heading break (see notes for \domark). \let\prevchapterdefs=\lastchapterdefs \let\prevsectiondefs=\lastsectiondefs @@ -5798,6 +5809,7 @@ end % {% \chapfonts \rmisbold + \let\footnote=\errfootnoteheading % give better error message % % Have to define \lastsection before calling \donoderef, because the % xref code eventually uses it. On the other hand, it has to be called @@ -5891,22 +5903,29 @@ end % Print any size, any type, section title. % -% #1 is the text, #2 is the section level (sec/subsec/subsubsec), #3 is -% the section type for xrefs (Ynumbered, Ynothing, Yappendix), #4 is the -% section number. +% #1 is the text of the title, +% #2 is the section level (sec/subsec/subsubsec), +% #3 is the section type (Ynumbered, Ynothing, Yappendix, Yomitfromtoc), +% #4 is the section number. % \def\seckeyword{sec} % \def\sectionheading#1#2#3#4{% {% - \checkenv{}% should not be in an environment. + \def\sectionlevel{#2}% + \def\temptype{#3}% + % + % It is ok for the @heading series commands to appear inside an + % environment (it's been historically allowed, though the logic is + % dubious), but not the others. + \ifx\temptype\Yomitfromtockeyword\else + \checkenv{}% non-@*heading should not be in an environment. + \fi + \let\footnote=\errfootnoteheading % % Switch to the right set of fonts. \csname #2fonts\endcsname \rmisbold % - \def\sectionlevel{#2}% - \def\temptype{#3}% - % % Insert first mark before the heading break (see notes for \domark). \let\prevsectiondefs=\lastsectiondefs \ifx\temptype\Ynothingkeyword @@ -6333,6 +6352,7 @@ end % other math active characters (just in case), to plain's definitions. \mathactive % + % Inverse of the list at the beginning of the file. \let\b=\ptexb \let\bullet=\ptexbullet \let\c=\ptexc @@ -6348,7 +6368,9 @@ end \let\+=\tabalign \let\}=\ptexrbrace \let\/=\ptexslash + \let\sp=\ptexsp \let\*=\ptexstar + %\let\sup=\ptexsup % do not redefine, we want @sup to work in math mode \let\t=\ptext \expandafter \let\csname top\endcsname=\ptextop % we've made it outer \let\frenchspacing=\plainfrenchspacing @@ -7414,7 +7436,6 @@ end % % \anythingelse will almost certainly be an error of some kind. - % \mbodybackslash is the definition of \ in @macro bodies. % It maps \foo\ => \csname macarg.foo\endcsname => #N % where N is the macro parameter number. @@ -7523,8 +7544,7 @@ end % the catcode regime underwhich the body was input). % % If you compile with TeX (not eTeX), and you have macros with 10 or more -% arguments, you need that no macro has more than 256 arguments, otherwise an -% error is produced. +% arguments, no macro can have more than 256 arguments (else error). \def\parsemargdef#1;{% \paramno=0\def\paramlist{}% \let\hash\relax @@ -8318,14 +8338,7 @@ end \catcode`\\=\other % % Make the characters 128-255 be printing characters. - {% - \count1=128 - \def\loop{% - \catcode\count1=\other - \advance\count1 by 1 - \ifnum \count1<256 \loop \fi - }% - }% + {\setnonasciicharscatcodenonglobal\other}% % % @ is our escape character in .aux files, and we need braces. \catcode`\{=1 @@ -8359,9 +8372,6 @@ end % % Auto-number footnotes. Otherwise like plain. \gdef\footnote{% - \let\indent=\ptexindent - \let\noindent=\ptexnoindent - % \global\advance\footnoteno by \@ne \edef\thisfootno{$^{\the\footnoteno}$}% % @@ -8388,7 +8398,7 @@ end % % Nested footnotes are not supported in TeX, that would take a lot % more work. (\startsavinginserts does not suffice.) - \let\footnote=\errfootnote + \let\footnote=\errfootnotenest % % We want to typeset this text as a normal paragraph, even if the % footnote reference occurs in (for example) a display environment. @@ -8427,12 +8437,17 @@ end } }%end \catcode `\@=11 -\def\errfootnote{% +\def\errfootnotenest{% \errhelp=\EMsimple \errmessage{Nested footnotes not supported in texinfo.tex, even though they work in makeinfo; sorry} } +\def\errfootnoteheading{% + \errhelp=\EMsimple + \errmessage{Footnotes in chapters, sections, etc., are not supported} +} + % In case a @footnote appears in a vbox, save the footnote text and create % the real \insert just after the vbox finished. Otherwise, the insertion % would be lost. @@ -8856,20 +8871,20 @@ end { \catcode`\_ = \active \globaldefs=1 -\parseargdef\documentlanguage{\begingroup - \let_=\normalunderscore % normal _ character for filenames +\parseargdef\documentlanguage{% \tex % read txi-??.tex file in plain TeX. % Read the file by the name they passed if it exists. + \let_ = \normalunderscore % normal _ character for filename test \openin 1 txi-#1.tex \ifeof 1 - \documentlanguagetrywithoutunderscore{#1_\finish}% + \documentlanguagetrywithoutunderscore #1_\finish \else \globaldefs = 1 % everything in the txi-LL files needs to persist \input txi-#1.tex \fi \closein 1 \endgroup % end raw TeX -\endgroup} +} % % If they passed de_DE, and txi-de_DE.tex doesn't exist, % try txi-de.tex. @@ -8934,6 +8949,7 @@ directory should work if nowhere else does.} \catcode\count255=#1\relax \advance\count255 by 1 \repeat + } % @documentencoding sets the definition of non-ASCII characters @@ -8968,10 +8984,12 @@ directory should work if nowhere else does.} % \else \ifx \declaredencoding \utfeight \setnonasciicharscatcode\active - \utfeightchardefs + % since we already invoked \utfeightchardefs at the top level + % (below), do not re-invoke it, then our check for duplicated + % definitions triggers. Making non-ascii chars active is enough. % \else - \message{Unknown document encoding #1, ignoring.}% + \message{Ignoring unknown document encoding: #1.}% % \fi % utfeight \fi % latnine @@ -8980,10 +8998,11 @@ directory should work if nowhere else does.} \fi % ascii } +% emacs-page % A message to be logged when using a character that isn't available % the default font encoding (OT1). % -\def\missingcharmsg#1{\message{Character missing in OT1 encoding: #1.}} +\def\missingcharmsg#1{\message{Character missing, sorry: #1.}} % Take account of \c (plain) vs. \, (Texinfo) difference. \def\cedilla#1{\ifx\c\ptexc\c{#1}\else\,{#1}\fi} @@ -9019,12 +9038,10 @@ directory should work if nowhere else does.} \gdef^^b4{\'{}} \gdef^^b5{$\mu$} \gdef^^b6{\P} - % - \gdef^^b7{$^.$} + \gdef^^b7{\ifmmode\cdot\else $\cdot$\fi} \gdef^^b8{\cedilla\ } \gdef^^b9{$^1$} \gdef^^ba{\ordm} - % \gdef^^bb{\guillemetright} \gdef^^bc{$1\over4$} \gdef^^bd{$1\over2$} @@ -9279,6 +9296,18 @@ directory should work if nowhere else does.} \UTFviiiLoop \endgroup +\def\globallet{\global\let} % save some \expandafter's below + +% @U{xxxx} to produce U+xxxx, if we support it. +\def\U#1{% + \expandafter\ifx\csname uni:#1\endcsname \relax + \errhelp = \EMsimple + \errmessage{Unicode character U+#1 not supported, sorry}% + \else + \csname uni:#1\endcsname + \fi +} + \begingroup \catcode`\"=12 \catcode`\<=12 @@ -9287,7 +9316,6 @@ directory should work if nowhere else does.} \catcode`\;=12 \catcode`\!=12 \catcode`\~=13 - \gdef\DeclareUnicodeCharacter#1#2{% \countUTFz = "#1\relax %\wlog{\space\space defining Unicode char U+#1 (decimal \the\countUTFz)}% @@ -9302,6 +9330,13 @@ directory should work if nowhere else does.} \expandafter\expandafter\expandafter\expandafter \expandafter\expandafter\expandafter \gdef\UTFviiiTmp{#2}% + % + \expandafter\ifx\csname uni:#1\endcsname \relax \else + \errmessage{Internal error, already defined: #1}% + \fi + % + % define an additional control sequence for this code point. + \expandafter\globallet\csname uni:#1\endcsname \UTFviiiTmp \endgroup} \gdef\parseXMLCharref{% @@ -9339,23 +9374,49 @@ directory should work if nowhere else does.} \uppercase{\gdef\UTFviiiTmp{#2#3#4}}} \endgroup +% https://en.wikipedia.org/wiki/Plane_(Unicode)#Basic_M +% U+0000..U+007F = https://en.wikipedia.org/wiki/Basic_Latin_(Unicode_block) +% U+0080..U+00FF = https://en.wikipedia.org/wiki/Latin-1_Supplement_(Unicode_block) +% U+0100..U+017F = https://en.wikipedia.org/wiki/Latin_Extended-A +% U+0180..U+024F = https://en.wikipedia.org/wiki/Latin_Extended-B +% +% Many of our renditions are less than wonderful, and all the missing +% characters are available somewhere. Loading the necessary fonts +% awaits user request. We can't truly support Unicode without +% reimplementing everything that's been done in LaTeX for many years, +% plus probably using luatex or xetex, and who knows what else. +% We won't be doing that here in this simple file. But we can try to at +% least make most of the characters not bomb out. +% \def\utfeightchardefs{% \DeclareUnicodeCharacter{00A0}{\tie} \DeclareUnicodeCharacter{00A1}{\exclamdown} \DeclareUnicodeCharacter{00A3}{\pounds} + \DeclareUnicodeCharacter{00A7}{\S} \DeclareUnicodeCharacter{00A8}{\"{ }} \DeclareUnicodeCharacter{00A9}{\copyright} \DeclareUnicodeCharacter{00AA}{\ordf} \DeclareUnicodeCharacter{00AB}{\guillemetleft} + \DeclareUnicodeCharacter{00AC}{\ifmmode\lnot\else $\lnot$\fi} \DeclareUnicodeCharacter{00AD}{\-} \DeclareUnicodeCharacter{00AE}{\registeredsymbol} \DeclareUnicodeCharacter{00AF}{\={ }} \DeclareUnicodeCharacter{00B0}{\ringaccent{ }} + \DeclareUnicodeCharacter{00B1}{\ifmmode\pm\else $\pm$\fi} + \DeclareUnicodeCharacter{00B2}{$^2$} + \DeclareUnicodeCharacter{00B3}{$^3$} \DeclareUnicodeCharacter{00B4}{\'{ }} + \DeclareUnicodeCharacter{00B5}{$\mu$} + \DeclareUnicodeCharacter{00B6}{\P} + \DeclareUnicodeCharacter{00B7}{\ifmmode\cdot\else $\cdot$\fi} \DeclareUnicodeCharacter{00B8}{\cedilla{ }} + \DeclareUnicodeCharacter{00B9}{$^1$} \DeclareUnicodeCharacter{00BA}{\ordm} \DeclareUnicodeCharacter{00BB}{\guillemetright} + \DeclareUnicodeCharacter{00BC}{$1\over4$} + \DeclareUnicodeCharacter{00BD}{$1\over2$} + \DeclareUnicodeCharacter{00BE}{$3\over4$} \DeclareUnicodeCharacter{00BF}{\questiondown} \DeclareUnicodeCharacter{00C0}{\`A} @@ -9382,6 +9443,7 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{00D4}{\^O} \DeclareUnicodeCharacter{00D5}{\~O} \DeclareUnicodeCharacter{00D6}{\"O} + \DeclareUnicodeCharacter{00D7}{\ifmmode\times\else $\times$\fi} \DeclareUnicodeCharacter{00D8}{\O} \DeclareUnicodeCharacter{00D9}{\`U} \DeclareUnicodeCharacter{00DA}{\'U} @@ -9415,6 +9477,7 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{00F4}{\^o} \DeclareUnicodeCharacter{00F5}{\~o} \DeclareUnicodeCharacter{00F6}{\"o} + \DeclareUnicodeCharacter{00F7}{\ifmmode\div\else $\div$\fi} \DeclareUnicodeCharacter{00F8}{\o} \DeclareUnicodeCharacter{00F9}{\`u} \DeclareUnicodeCharacter{00FA}{\'u} @@ -9434,20 +9497,23 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0107}{\'c} \DeclareUnicodeCharacter{0108}{\^C} \DeclareUnicodeCharacter{0109}{\^c} - \DeclareUnicodeCharacter{0118}{\ogonek{E}} - \DeclareUnicodeCharacter{0119}{\ogonek{e}} \DeclareUnicodeCharacter{010A}{\dotaccent{C}} \DeclareUnicodeCharacter{010B}{\dotaccent{c}} \DeclareUnicodeCharacter{010C}{\v{C}} \DeclareUnicodeCharacter{010D}{\v{c}} \DeclareUnicodeCharacter{010E}{\v{D}} + \DeclareUnicodeCharacter{010F}{d'} + \DeclareUnicodeCharacter{0110}{\DH} + \DeclareUnicodeCharacter{0111}{\dh} \DeclareUnicodeCharacter{0112}{\=E} \DeclareUnicodeCharacter{0113}{\=e} \DeclareUnicodeCharacter{0114}{\u{E}} \DeclareUnicodeCharacter{0115}{\u{e}} \DeclareUnicodeCharacter{0116}{\dotaccent{E}} \DeclareUnicodeCharacter{0117}{\dotaccent{e}} + \DeclareUnicodeCharacter{0118}{\ogonek{E}} + \DeclareUnicodeCharacter{0119}{\ogonek{e}} \DeclareUnicodeCharacter{011A}{\v{E}} \DeclareUnicodeCharacter{011B}{\v{e}} \DeclareUnicodeCharacter{011C}{\^G} @@ -9457,14 +9523,20 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0120}{\dotaccent{G}} \DeclareUnicodeCharacter{0121}{\dotaccent{g}} + \DeclareUnicodeCharacter{0122}{\cedilla{G}} + \DeclareUnicodeCharacter{0123}{\cedilla{g}} \DeclareUnicodeCharacter{0124}{\^H} \DeclareUnicodeCharacter{0125}{\^h} + \DeclareUnicodeCharacter{0126}{\missingcharmsg{H WITH STROKE}} + \DeclareUnicodeCharacter{0127}{\missingcharmsg{h WITH STROKE}} \DeclareUnicodeCharacter{0128}{\~I} \DeclareUnicodeCharacter{0129}{\~{\dotless{i}}} \DeclareUnicodeCharacter{012A}{\=I} \DeclareUnicodeCharacter{012B}{\={\dotless{i}}} \DeclareUnicodeCharacter{012C}{\u{I}} \DeclareUnicodeCharacter{012D}{\u{\dotless{i}}} + \DeclareUnicodeCharacter{012E}{\ogonek{I}} + \DeclareUnicodeCharacter{012F}{\ogonek{i}} \DeclareUnicodeCharacter{0130}{\dotaccent{I}} \DeclareUnicodeCharacter{0131}{\dotless{i}} @@ -9472,15 +9544,29 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0133}{ij} \DeclareUnicodeCharacter{0134}{\^J} \DeclareUnicodeCharacter{0135}{\^{\dotless{j}}} + \DeclareUnicodeCharacter{0136}{\cedilla{K}} + \DeclareUnicodeCharacter{0137}{\cedilla{k}} + \DeclareUnicodeCharacter{0138}{\ifmmode\kappa\else $\kappa$\fi} \DeclareUnicodeCharacter{0139}{\'L} \DeclareUnicodeCharacter{013A}{\'l} + \DeclareUnicodeCharacter{013B}{\cedilla{L}} + \DeclareUnicodeCharacter{013C}{\cedilla{l}} + \DeclareUnicodeCharacter{013D}{L'}% should kern + \DeclareUnicodeCharacter{013E}{l'}% should kern + \DeclareUnicodeCharacter{013F}{L\U{00B7}} + \DeclareUnicodeCharacter{0140}{l\U{00B7}} \DeclareUnicodeCharacter{0141}{\L} \DeclareUnicodeCharacter{0142}{\l} \DeclareUnicodeCharacter{0143}{\'N} \DeclareUnicodeCharacter{0144}{\'n} + \DeclareUnicodeCharacter{0145}{\cedilla{N}} + \DeclareUnicodeCharacter{0146}{\cedilla{n}} \DeclareUnicodeCharacter{0147}{\v{N}} \DeclareUnicodeCharacter{0148}{\v{n}} + \DeclareUnicodeCharacter{0149}{'n} + \DeclareUnicodeCharacter{014A}{\missingcharmsg{ENG}} + \DeclareUnicodeCharacter{014B}{\missingcharmsg{eng}} \DeclareUnicodeCharacter{014C}{\=O} \DeclareUnicodeCharacter{014D}{\=o} \DeclareUnicodeCharacter{014E}{\u{O}} @@ -9492,6 +9578,8 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0153}{\oe} \DeclareUnicodeCharacter{0154}{\'R} \DeclareUnicodeCharacter{0155}{\'r} + \DeclareUnicodeCharacter{0156}{\cedilla{R}} + \DeclareUnicodeCharacter{0157}{\cedilla{r}} \DeclareUnicodeCharacter{0158}{\v{R}} \DeclareUnicodeCharacter{0159}{\v{r}} \DeclareUnicodeCharacter{015A}{\'S} @@ -9503,10 +9591,12 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0160}{\v{S}} \DeclareUnicodeCharacter{0161}{\v{s}} - \DeclareUnicodeCharacter{0162}{\cedilla{t}} - \DeclareUnicodeCharacter{0163}{\cedilla{T}} + \DeclareUnicodeCharacter{0162}{\cedilla{T}} + \DeclareUnicodeCharacter{0163}{\cedilla{t}} \DeclareUnicodeCharacter{0164}{\v{T}} - + \DeclareUnicodeCharacter{0165}{\v{t}} + \DeclareUnicodeCharacter{0166}{\missingcharmsg{H WITH STROKE}} + \DeclareUnicodeCharacter{0167}{\missingcharmsg{h WITH STROKE}} \DeclareUnicodeCharacter{0168}{\~U} \DeclareUnicodeCharacter{0169}{\~u} \DeclareUnicodeCharacter{016A}{\=U} @@ -9518,6 +9608,8 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{0170}{\H{U}} \DeclareUnicodeCharacter{0171}{\H{u}} + \DeclareUnicodeCharacter{0172}{\ogonek{U}} + \DeclareUnicodeCharacter{0173}{\ogonek{u}} \DeclareUnicodeCharacter{0174}{\^W} \DeclareUnicodeCharacter{0175}{\^w} \DeclareUnicodeCharacter{0176}{\^Y} @@ -9529,6 +9621,7 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{017C}{\dotaccent{z}} \DeclareUnicodeCharacter{017D}{\v{Z}} \DeclareUnicodeCharacter{017E}{\v{z}} + \DeclareUnicodeCharacter{017F}{\missingcharmsg{LONG S}} \DeclareUnicodeCharacter{01C4}{D\v{Z}} \DeclareUnicodeCharacter{01C5}{D\v{z}} @@ -9734,12 +9827,51 @@ directory should work if nowhere else does.} \DeclareUnicodeCharacter{2261}{\equiv} }% end of \utfeightchardefs - % US-ASCII character definitions. \def\asciichardefs{% nothing need be done \relax } +% Latin1 (ISO-8859-1) character definitions. +\def\nonasciistringdefs{% + \setnonasciicharscatcode\active + \def\defstringchar##1{\def##1{\string##1}}% + \defstringchar^^a0\defstringchar^^a1\defstringchar^^a2\defstringchar^^a3% + \defstringchar^^a4\defstringchar^^a5\defstringchar^^a6\defstringchar^^a7% + \defstringchar^^a8\defstringchar^^a9\defstringchar^^aa\defstringchar^^ab% + \defstringchar^^ac\defstringchar^^ad\defstringchar^^ae\defstringchar^^af% + % + \defstringchar^^b0\defstringchar^^b1\defstringchar^^b2\defstringchar^^b3% + \defstringchar^^b4\defstringchar^^b5\defstringchar^^b6\defstringchar^^b7% + \defstringchar^^b8\defstringchar^^b9\defstringchar^^ba\defstringchar^^bb% + \defstringchar^^bc\defstringchar^^bd\defstringchar^^be\defstringchar^^bf% + % + \defstringchar^^c0\defstringchar^^c1\defstringchar^^c2\defstringchar^^c3% + \defstringchar^^c4\defstringchar^^c5\defstringchar^^c6\defstringchar^^c7% + \defstringchar^^c8\defstringchar^^c9\defstringchar^^ca\defstringchar^^cb% + \defstringchar^^cc\defstringchar^^cd\defstringchar^^ce\defstringchar^^cf% + % + \defstringchar^^d0\defstringchar^^d1\defstringchar^^d2\defstringchar^^d3% + \defstringchar^^d4\defstringchar^^d5\defstringchar^^d6\defstringchar^^d7% + \defstringchar^^d8\defstringchar^^d9\defstringchar^^da\defstringchar^^db% + \defstringchar^^dc\defstringchar^^dd\defstringchar^^de\defstringchar^^df% + % + \defstringchar^^e0\defstringchar^^e1\defstringchar^^e2\defstringchar^^e3% + \defstringchar^^e4\defstringchar^^e5\defstringchar^^e6\defstringchar^^e7% + \defstringchar^^e8\defstringchar^^e9\defstringchar^^ea\defstringchar^^eb% + \defstringchar^^ec\defstringchar^^ed\defstringchar^^ee\defstringchar^^ef% + % + \defstringchar^^f0\defstringchar^^f1\defstringchar^^f2\defstringchar^^f3% + \defstringchar^^f4\defstringchar^^f5\defstringchar^^f6\defstringchar^^f7% + \defstringchar^^f8\defstringchar^^f9\defstringchar^^fa\defstringchar^^fb% + \defstringchar^^fc\defstringchar^^fd\defstringchar^^fe\defstringchar^^ff% +} + + +% define all the unicode characters we know about, for the sake of @U. +\utfeightchardefs + + % Make non-ASCII characters printable again for compatibility with % existing Texinfo documents that may use them, even without declaring a % document encoding. @@ -10093,6 +10225,7 @@ directory should work if nowhere else does.} % {@catcode`- = @active @gdef@normalturnoffactive{% + @nonasciistringdefs @let-=@normaldash @let"=@normaldoublequote @let$=@normaldollar %$ font-lock fix @@ -10161,7 +10294,7 @@ directory should work if nowhere else does.} @c Local variables: @c eval: (add-hook 'write-file-hooks 'time-stamp) -@c page-delimiter: "^\\\\message" +@c page-delimiter: "^\\\\message\\|emacs-page" @c time-stamp-start: "def\\\\texinfoversion{" @c time-stamp-format: "%:y-%02m-%02d.%02H" @c time-stamp-end: "}" |