diff options
Diffstat (limited to 'gawk-info')
| -rw-r--r-- | gawk-info | 6151 |
1 files changed, 0 insertions, 6151 deletions
diff --git a/gawk-info b/gawk-info deleted file mode 100644 index 361bd0c5..00000000 --- a/gawk-info +++ /dev/null @@ -1,6151 +0,0 @@ -Info file gawk-info, produced by Makeinfo, -*- Text -*- from input -file gawk.texinfo. - -This file documents `awk', a program that you can use to select -particular records in a file and perform operations upon them. - -Copyright (C) 1989 Free Software Foundation, Inc. - -Permission is granted to make and distribute verbatim copies of this -manual provided the copyright notice and this permission notice are -preserved on all copies. - -Permission is granted to copy and distribute modified versions of -this manual under the conditions for verbatim copying, provided that -the entire resulting derived work is distributed under the terms of a -permission notice identical to this one. - -Permission is granted to copy and distribute translations of this -manual into another language, under the above conditions for modified -versions, except that this permission notice may be stated in a -translation approved by the Foundation. - - - -File: gawk-info, Node: Top, Next: Preface, Prev: (dir), Up: (dir) - -This file documents `awk', a program that you can use to select -particular records in a file and perform operations upon them; it -contains the following chapters: - -* Menu: - -* Preface:: What you can do with `awk'; brief history - and acknowledgements. - -* License:: Your right to copy and distribute `gawk'. - -* This Manual:: Using this manual. - - Includes sample input files that you can use. - -* Getting Started:: A basic introduction to using `awk'. - How to run an `awk' program. Command line syntax. - -* Reading Files:: How to read files and manipulate fields. - -* Printing:: How to print using `awk'. Describes the - `print' and `printf' statements. - Also describes redirection of output. - -* One-liners:: Short, sample `awk' programs. - -* Patterns:: The various types of patterns explained in detail. - -* Actions:: The various types of actions are introduced here. - Describes expressions and the various operators in - detail. Also describes comparison expressions. - -* Statements:: The various control statements are described in - detail. - -* Arrays:: The description and use of arrays. Also includes - array--oriented control statements. - -* User-defined:: User--defined functions are described in detail. - -* Built-in:: The built--in functions are summarized here. - -* Special:: The special variables are summarized here. - -* Sample Program:: A sample `awk' program with a complete explanation. - -* Notes:: Something about the implementation of `gawk'. - -* Glossary:: An explanation of some unfamiliar terms. - -* Index:: - - - -File: gawk-info, Node: Preface, Next: License, Prev: Top, Up: Top - -Preface -******* - -If you are like many computer users, you frequently would like to -make changes in various text files wherever certain patterns appear, -or extract data from parts of certain lines while discarding the -rest. To write a program to do this in a language such as C or -Pascal is a time--consuming inconvenience that may take many lines of -code. The job may be easier with `awk'. - -The `awk' utility interprets a special--purpose programming language -that makes it possible to handle simple data--reformatting jobs -easily with just a few lines of code. - -The GNU implementation of `awk' is called `gawk'; it is fully upward -compatible with the System V Release 3.1 and later version of `awk'. -All properly written `awk' programs should work with `gawk'. So we -usually don't distinguish between `gawk' and other `awk' -implementations in this manual. - -This manual teaches you what `awk' does and how you can use `awk' -effectively. You should already be familiar with basic, -general--purpose, operating system commands such as `ls'. Using -`awk' you can: - - * manage small, personal databases, - - * generate reports, - - * validate data, - - * produce indexes, and perform other document preparation tasks, - - * even experiment with algorithms that can be adapted later to - other computer languages! - -* Menu: - -* History:: The history of gawk and awk. Acknowledgements. - - - -File: gawk-info, Node: History, Up: Preface - -History of `awk' and `gawk' -=========================== - -The name `awk' comes from the initials of its designers: Alfred V. -Aho, Peter J. Weinberger, and Brian W. Kernighan. The original -version of `awk' was written in 1977. In 1985 a new version made the -programming language more powerful, introducing user--defined -functions, multiple input streams, and computed regular expressions. - -The GNU implementation, `gawk', was written in 1986 by Paul Rubin and -Jay Fenlason, with advice from Richard Stallman. John Woods -contributed parts of the code as well. In 1988, David Trueman, with -help from Arnold Robbins, reworked `gawk' for compatibility with the -newer `awk'. - -Many people need to be thanked for their assistance in producing this -manual. Jay Fenlason contributed many ideas and sample programs. -Richard Mlynarik and Robert Chassell gave helpful comments on drafts -of this manual. The paper ``A Supplemental Document for `awk''' by -John W. Pierce of the Chemistry Department at UC San Diego, -pinpointed several issues relevant both to `awk' implementation and -to this manual, that would otherwise have escaped us. - -Finally, we would like to thank Brian Kernighan of Bell Labs for -invaluable assistance during the testing and debugging of `gawk', and -for help in clarifying several points about the language. - - - -File: gawk-info, Node: License, Next: This Manual, Prev: Preface, Up: Top - -GNU GENERAL PUBLIC LICENSE -************************** - - Version 1, February 1989 - - Copyright (C) 1989 Free Software Foundation, Inc. - 675 Mass Ave, Cambridge, MA 02139, USA - - Everyone is permitted to copy and distribute verbatim copies - of this license document, but changing it is not allowed. - - Preamble -========= - - The license agreements of most software companies try to keep users -at the mercy of those companies. By contrast, our General Public -License is intended to guarantee your freedom to share and change -free software--to make sure the software is free for all its users. -The General Public License applies to the Free Software Foundation's -software and to any other program whose authors commit to using it. -You can use it for your programs, too. - - When we speak of free software, we are referring to freedom, not -price. Specifically, the General Public License is designed to make -sure that you have the freedom to give away or sell copies of free -software, that you receive source code or can get it if you want it, -that you can change the software or use pieces of it in new free -programs; and that you know you can do these things. - - To protect your rights, we need to make restrictions that forbid -anyone to deny you these rights or to ask you to surrender the rights. -These restrictions translate to certain responsibilities for you if -you distribute copies of the software, or if you modify it. - - For example, if you distribute copies of a such a program, whether -gratis or for a fee, you must give the recipients all the rights that -you have. You must make sure that they, too, receive or can get the -source code. And you must tell them their rights. - - We protect your rights with two steps: (1) copyright the software, -and (2) offer you this license which gives you legal permission to -copy, distribute and/or modify the software. - - Also, for each author's protection and ours, we want to make certain -that everyone understands that there is no warranty for this free -software. If the software is modified by someone else and passed on, -we want its recipients to know that what they have is not the -original, so that any problems introduced by others will not reflect -on the original authors' reputations. - - The precise terms and conditions for copying, distribution and -modification follow. - - TERMS AND CONDITIONS - - 1. This License Agreement applies to any program or other work - which contains a notice placed by the copyright holder saying it - may be distributed under the terms of this General Public - License. The ``Program'', below, refers to any such program or - work, and a ``work based on the Program'' means either the - Program or any work containing the Program or a portion of it, - either verbatim or with modifications. Each licensee is - addressed as ``you''. - - 2. You may copy and distribute verbatim copies of the Program's - source code as you receive it, in any medium, provided that you - conspicuously and appropriately publish on each copy an - appropriate copyright notice and disclaimer of warranty; keep - intact all the notices that refer to this General Public License - and to the absence of any warranty; and give any other - recipients of the Program a copy of this General Public License - along with the Program. You may charge a fee for the physical - act of transferring a copy. - - 3. You may modify your copy or copies of the Program or any portion - of it, and copy and distribute such modifications under the - terms of Paragraph 1 above, provided that you also do the - following: - - * cause the modified files to carry prominent notices stating - that you changed the files and the date of any change; and - - * cause the whole of any work that you distribute or publish, - that in whole or in part contains the Program or any part - thereof, either with or without modifications, to be - licensed at no charge to all third parties under the terms - of this General Public License (except that you may choose - to grant warranty protection to some or all third parties, - at your option). - - * If the modified program normally reads commands - interactively when run, you must cause it, when started - running for such interactive use in the simplest and most - usual way, to print or display an announcement including an - appropriate copyright notice and a notice that there is no - warranty (or else, saying that you provide a warranty) and - that users may redistribute the program under these - conditions, and telling the user how to view a copy of this - General Public License. - - * You may charge a fee for the physical act of transferring a - copy, and you may at your option offer warranty protection - in exchange for a fee. - - Mere aggregation of another independent work with the Program - (or its derivative) on a volume of a storage or distribution - medium does not bring the other work under the scope of these - terms. - - 4. You may copy and distribute the Program (or a portion or - derivative of it, under Paragraph 2) in object code or - executable form under the terms of Paragraphs 1 and 2 above - provided that you also do one of the following: - - * accompany it with the complete corresponding - machine-readable source code, which must be distributed - under the terms of Paragraphs 1 and 2 above; or, - - * accompany it with a written offer, valid for at least three - years, to give any third party free (except for a nominal - charge for the cost of distribution) a complete - machine-readable copy of the corresponding source code, to - be distributed under the terms of Paragraphs 1 and 2 above; - or, - - * accompany it with the information you received as to where - the corresponding source code may be obtained. (This - alternative is allowed only for noncommercial distribution - and only if you received the program in object code or - executable form alone.) - - Source code for a work means the preferred form of the work for - making modifications to it. For an executable file, complete - source code means all the source code for all modules it - contains; but, as a special exception, it need not include - source code for modules which are standard libraries that - accompany the operating system on which the executable file - runs, or for standard header files or definitions files that - accompany that operating system. - - 5. You may not copy, modify, sublicense, distribute or transfer the - Program except as expressly provided under this General Public - License. Any attempt otherwise to copy, modify, sublicense, - distribute or transfer the Program is void, and will - automatically terminate your rights to use the Program under - this License. However, parties who have received copies, or - rights to use copies, from you under this General Public License - will not have their licenses terminated so long as such parties - remain in full compliance. - - 6. By copying, distributing or modifying the Program (or any work - based on the Program) you indicate your acceptance of this - license to do so, and all its terms and conditions. - - 7. Each time you redistribute the Program (or any work based on the - Program), the recipient automatically receives a license from - the original licensor to copy, distribute or modify the Program - subject to these terms and conditions. You may not impose any - further restrictions on the recipients' exercise of the rights - granted herein. - - 8. The Free Software Foundation may publish revised and/or new - versions of the General Public License from time to time. Such - new versions will be similar in spirit to the present version, - but may differ in detail to address new problems or concerns. - - Each version is given a distinguishing version number. If the - Program specifies a version number of the license which applies - to it and ``any later version'', you have the option of - following the terms and conditions either of that version or of - any later version published by the Free Software Foundation. If - the Program does not specify a version number of the license, - you may choose any version ever published by the Free Software - Foundation. - - 9. If you wish to incorporate parts of the Program into other free - programs whose distribution conditions are different, write to - the author to ask for permission. For software which is - copyrighted by the Free Software Foundation, write to the Free - Software Foundation; we sometimes make exceptions for this. Our - decision will be guided by the two goals of preserving the free - status of all derivatives of our free software and of promoting - the sharing and reuse of software generally. - - NO WARRANTY - - 10. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO - WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE - LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT - HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM ``AS IS'' - WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, - INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF - MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE - ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS - WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE - COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION. - - 11. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN - WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY - MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE - LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, - INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR - INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS - OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY - YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH - ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN - ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. - - END OF TERMS AND CONDITIONS - -Appendix: How to Apply These Terms to Your New Programs -======================================================= - - If you develop a new program, and you want it to be of the greatest -possible use to humanity, the best way to achieve this is to make it -free software which everyone can redistribute and change under these -terms. - - To do so, attach the following notices to the program. It is safest -to attach them to the start of each source file to most effectively -convey the exclusion of warranty; and each file should have at least -the ``copyright'' line and a pointer to where the full notice is found. - - ONE LINE TO GIVE THE PROGRAM'S NAME AND A BRIEF IDEA OF WHAT IT DOES. - Copyright (C) 19YY NAME OF AUTHOR - - This program is free software; you can redistribute it and/or modify - it under the terms of the GNU General Public License as published by - the Free Software Foundation; either version 1, or (at your option) - any later version. - - This program is distributed in the hope that it will be useful, - but WITHOUT ANY WARRANTY; without even the implied warranty of - MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the - GNU General Public License for more details. - - You should have received a copy of the GNU General Public License - along with this program; if not, write to the Free Software - Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA. - - Also add information on how to contact you by electronic and paper -mail. - -If the program is interactive, make it output a short notice like -this when it starts in an interactive mode: - - Gnomovision version 69, Copyright (C) 19YY NAME OF AUTHOR - Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w'. - This is free software, and you are welcome to redistribute it - under certain conditions; type `show c' for details. - - The hypothetical commands `show w' and `show c' should show the -appropriate parts of the General Public License. Of course, the -commands you use may be called something other than `show w' and -`show c'; they could even be mouse-clicks or menu items--whatever -suits your program. - -You should also get your employer (if you work as a programmer) or -your school, if any, to sign a ``copyright disclaimer'' for the -program, if necessary. Here a sample; alter the names: - - Yoyodyne, Inc., hereby disclaims all copyright interest in the - program `Gnomovision' (a program to direct compilers to make passes - at assemblers) written by James Hacker. - - SIGNATURE OF TY COON, 1 April 1989 - Ty Coon, President of Vice - -That's all there is to it! - - - -File: gawk-info, Node: This Manual, Next: Getting Started, Prev: License, Up: Top - -Using This Manual -***************** - -The term `gawk' refers to a program (a version of `awk') developed by -the Free Software Foundation, and to the language you use to tell it -what to do. When we need to be careful, we call the program ``the -`awk' utility'' and the language ``the `awk' language''. The purpose -of this manual is to explain the `awk' language and how to run the -`awk' utility. - -The term "`awk' program" refers to a program written by you in the -`awk' programming language. - -*Note Getting Started::, for the bare essentials you need to know to -start using `awk'. - -Useful ``one--liners'' are included to give you a feel for the `awk' -language (*note One-liners::.). - -A sizable sample `awk' program has been provided for you (*note -Sample Program::.). - -If you find terms that you aren't familiar with, try looking them up -in the glossary (*note Glossary::.). - -Most of the time complete `awk' programs are used as examples, but in -some of the more advanced sections, only the part of the `awk' -program that illustrates the concept being described is shown. - -* Menu: - -This chapter contains the following sections: - -* The Files:: Sample data files for use in the `awk' programs - illustrated in this manual. - - - -File: gawk-info, Node: The Files, Up: This Manual - -Input Files for the Examples -============================ - -This manual contains many sample programs. The data for many of -those programs comes from two files. The first file, called -`BBS-list', represents a list of computer bulletin board systems and -information about those systems. - -Each line of this file is one "record". Each record contains the -name of a computer bulletin board, its phone number, the board's baud -rate, and a code for the number of hours it is operational. An `A' -in the last column means the board operates 24 hours all week. A `B' -in the last column means the board operates evening and weekend -hours, only. A `C' means the board operates only on weekends. - - aardvark 555-5553 1200/300 B - alpo-net 555-3412 2400/1200/300 A - barfly 555-7685 1200/300 A - bites 555-1675 2400/1200/300 A - camelot 555-0542 300 C - core 555-2912 1200/300 C - fooey 555-1234 2400/1200/300 B - foot 555-6699 1200/300 B - macfoo 555-6480 1200/300 A - sdace 555-3430 2400/1200/300 A - sabafoo 555-2127 1200/300 C - -The second data file, called `inventory-shipped', represents -information about shipments during the year. Each line of this file -is also one record. Each record contains the month of the year, the -number of green crates shipped, the number of red boxes shipped, the -number of orange bags shipped, and the number of blue packages -shipped, respectively. - - Jan 13 25 15 115 - Feb 15 32 24 226 - Mar 15 24 34 228 - Apr 31 52 63 420 - May 16 34 29 208 - Jun 31 42 75 492 - Jul 24 34 67 436 - Aug 15 34 47 316 - Sep 13 55 37 277 - Oct 29 54 68 525 - Nov 20 87 82 577 - Dec 17 35 61 401 - - Jan 21 36 64 620 - Feb 26 58 80 652 - Mar 24 75 70 495 - Apr 21 70 74 514 - -If you are reading this in GNU Emacs using Info, you can copy the -regions of text showing these sample files into your own test files. -This way you can try out the examples shown in the remainder of this -document. You do this by using the command `M-x write-region' to -copy text from the Info file into a file for use with `awk' (see your -``GNU Emacs Manual'' for more information). Using this information, -create your own `BBS-list' and `inventory-shipped' files, and -practice what you learn in this manual. - - - -File: gawk-info, Node: Getting Started, Next: Reading Files, Prev: This Manual, Up: Top - -Getting Started With `awk' -************************** - -The basic function of `awk' is to search files for lines (or other -units of text) that contain certain patterns. When a line matching -any of those patterns is found, `awk' performs specified actions on -that line. Then `awk' keeps processing input lines until the end of -the file is reached. - -An `awk' "program" or "script" consists of a series of "rules". -(They may also contain "function definitions", but that is an -advanced feature, so let's ignore it for now. *Note User-defined::.) - -A rule contains a "pattern", an "action", or both. Actions are -enclosed in curly braces to distinguish them from patterns. -Therefore, an `awk' program is a sequence of rules in the form: - - PATTERN { ACTION } - PATTERN { ACTION } - ... - - * Menu: - -* Very Simple:: A very simple example. -* Two Rules:: A less simple one--line example with two rules. -* More Complex:: A more complex example. -* Running gawk:: How to run gawk programs; includes command line syntax. -* Comments:: Adding documentation to gawk programs. -* Statements/Lines:: Subdividing or combining statements into lines. - -* When:: When to use gawk and when to use other things. - - - -File: gawk-info, Node: Very Simple, Next: Two Rules, Up: Getting Started - -A Very Simple Example -===================== - -The following command runs a simple `awk' program that searches the -input file `BBS-list' for the string of characters: `foo'. (A string -of characters is usually called, quite simply, a "string".) - - awk '/foo/ { print $0 }' BBS-list - -When lines containing `foo' are found, they are printed, because -`print $0' means print the current line. (Just `print' by itself -also means the same thing, so we could have written that instead.) - -You will notice that slashes, `/', surround the string `foo' in the -actual `awk' program. The slashes indicate that `foo' is a pattern -to search for. This type of pattern is called a "regular -expression", and is covered in more detail later (*note Regexp::.). -There are single quotes around the `awk' program so that the shell -won't interpret any of it as special shell characters. - -Here is what this program prints: - - fooey 555-1234 2400/1200/300 B - foot 555-6699 1200/300 B - macfoo 555-6480 1200/300 A - sabafoo 555-2127 1200/300 C - -In an `awk' rule, either the pattern or the action can be omitted, -but not both. - -If the pattern is omitted, then the action is performed for *every* -input line. - -If the action is omitted, the default action is to print all lines -that match the pattern. We could leave out the action (the print -statement and the curly braces) in the above example, and the result -would be the same: all lines matching the pattern `foo' would be -printed. (By comparison, omitting the print statement but retaining -the curly braces makes an empty action that does nothing; then no -lines would be printed.) - - - -File: gawk-info, Node: Two Rules, Next: More Complex, Prev: Very Simple, Up: Getting Started - -An Example with Two Rules -========================= - -The `awk' utility reads the input files one line at a time. For each -line, `awk' tries the patterns of all the rules. If several patterns -match then several actions are run, in the order in which they appear -in the `awk' program. If no patterns match, then no actions are run. - -After processing all the rules (perhaps none) that match the line, -`awk' reads the next line (however, *note Next::.). This continues -until the end of the file is reached. - -For example, the `awk' program: - - /12/ { print $0 } - /21/ { print $0 } - -contains two rules. The first rule has the string `12' as the -pattern and `print $0' as the action. The second rule has the string -`21' as the pattern and also has `print $0' as the action. Each -rule's action is enclosed in its own pair of braces. - -This `awk' program prints every line that contains the string `12' -*or* the string `21'. If a line contains both strings, it is printed -twice, once by each rule. - -If we run this program on our two sample data files, `BBS-list' and -`inventory-shipped', as shown here: - - awk '/12/ { print $0 } - /21/ { print $0 }' BBS-list inventory-shipped - -we get the following output: - - aardvark 555-5553 1200/300 B - alpo-net 555-3412 2400/1200/300 A - barfly 555-7685 1200/300 A - bites 555-1675 2400/1200/300 A - core 555-2912 1200/300 C - fooey 555-1234 2400/1200/300 B - foot 555-6699 1200/300 B - macfoo 555-6480 1200/300 A - sdace 555-3430 2400/1200/300 A - sabafoo 555-2127 1200/300 C - sabafoo 555-2127 1200/300 C - Jan 21 36 64 620 - Apr 21 70 74 514 - -Note how the line in `BBS-list' beginning with `sabafoo' was printed -twice, once for each rule. - - - -File: gawk-info, Node: More Complex, Next: Running gawk, Prev: Two Rules, Up: Getting Started - -A More Complex Example -====================== - -Here is an example to give you an idea of what typical `awk' programs -do. This example shows how `awk' can be used to summarize, select, -and rearrange the output of another utility. It uses features that -haven't been covered yet, so don't worry if you don't understand all -the details. - - ls -l | awk '$5 == "Nov" { sum += $4 } - END { print sum }' - -This command prints the total number of bytes in all the files in the -current directory that were last modified in November (of any year). -(In the C shell you would need to type a semicolon and then a -backslash at the end of the first line; in the Bourne shell you can -type the example as shown.) - -The `ls -l' part of this example is a command that gives you a full -listing of all the files in a directory, including file size and date. -Its output looks like this: - - -rw-r--r-- 1 close 1933 Nov 7 13:05 Makefile - -rw-r--r-- 1 close 10809 Nov 7 13:03 gawk.h - -rw-r--r-- 1 close 983 Apr 13 12:14 gawk.tab.h - -rw-r--r-- 1 close 31869 Jun 15 12:20 gawk.y - -rw-r--r-- 1 close 22414 Nov 7 13:03 gawk1.c - -rw-r--r-- 1 close 37455 Nov 7 13:03 gawk2.c - -rw-r--r-- 1 close 27511 Dec 9 13:07 gawk3.c - -rw-r--r-- 1 close 7989 Nov 7 13:03 gawk4.c - -The first field contains read--write permissions, the second field -contains the number of links to the file, and the third field -identifies the owner of the file. The fourth field contains the size -of the file in bytes. The fifth, sixth, and seventh fields contain -the month, day, and time, respectively, that the file was last -modified. Finally, the eighth field contains the name of the file. - -The `$5 == "Nov"' in our `awk' program is an expression that tests -whether the fifth field of the output from `ls -l' matches the string -`Nov'. Each time a line has the string `Nov' in its fifth field, the -action `{ sum += $4 }' is performed. This adds the fourth field (the -file size) to the variable `sum'. As a result, when `awk' has -finished reading all the input lines, `sum' will be the sum of the -sizes of files whose lines matched the pattern. - -After the last line of output from `ls' has been processed, the `END' -pattern is executed, and the value of `sum' is printed. In this -example, the value of `sum' would be 80600. - -These more advanced `awk' techniques are covered in later sections -(*note Actions::.). Before you can move on to more advanced `awk' -programming, you have to know how `awk' interprets your input and -displays your output. By manipulating "fields" and using special -"print" statements, you can produce some very useful and spectacular -looking reports. - - - -File: gawk-info, Node: Running gawk, Next: Comments, Prev: More Complex, Up: Getting Started - -How to Run `awk' Programs -========================= - -There are several ways to run an `awk' program. If the program is -short, it is easiest to include it in the command that runs `awk', -like this: - - awk 'PROGRAM' INPUT-FILE1 INPUT-FILE2 ... - - where PROGRAM consists of a series of PATTERNS and ACTIONS, as -described earlier. - -When the program is long, you would probably prefer to put it in a -file and run it with a command like this: - - awk -f PROGRAM-FILE INPUT-FILE1 INPUT-FILE2 ... - - * Menu: - -* One-shot:: Running a short throw--away `awk' program. -* Read Terminal:: Using no input files (input from terminal instead). -* Long:: Putting permanent `awk' programs in files. -* Executable Scripts:: Making self--contained `awk' programs. -* Command Line:: How the `awk' command line is laid out. - - - -File: gawk-info, Node: One-shot, Next: Read Terminal, Up: Running gawk - -One--shot Throw--away `awk' Programs ------------------------------------- - -Once you are familiar with `awk', you will often type simple programs -at the moment you want to use them. Then you can write the program -as the first argument of the `awk' command, like this: - - awk 'PROGRAM' INPUT-FILE1 INPUT-FILE2 ... - - where PROGRAM consists of a series of PATTERNS and ACTIONS, as -described earlier. - -This command format tells the shell to start `awk' and use the -PROGRAM to process records in the input file(s). There are single -quotes around the PROGRAM so that the shell doesn't interpret any -`awk' characters as special shell characters. They cause the shell -to treat all of PROGRAM as a single argument for `awk'. They also -allow PROGRAM to be more than one line long. - -This format is also useful for running short or medium--sized `awk' -programs from shell scripts, because it avoids the need for a -separate file for the `awk' program. A self--contained shell script -is more reliable since there are no other files to misplace. - - - -File: gawk-info, Node: Read Terminal, Next: Long, Prev: One-shot, Up: Running gawk - -Running `awk' without Input Files ---------------------------------- - -You can also use `awk' without any input files. If you type the -command line: - - awk 'PROGRAM' - -then `awk' applies the PROGRAM to the "standard input", which usually -means whatever you type on the terminal. This continues until you -indicate end--of--file by typing `Control-d'. - -For example, if you type: - - awk '/th/' - -whatever you type next will be taken as data for that `awk' program. -If you go on to type the following data, - - Kathy - Ben - Tom - Beth - Seth - Karen - Thomas - `Control-d' - -then `awk' will print - - Kathy - Beth - Seth - -as matching the pattern `th'. Notice that it did not recognize -`Thomas' as matching the pattern. The `awk' language is "case -sensitive", and matches patterns *exactly*. - - - -File: gawk-info, Node: Long, Next: Executable Scripts, Prev: Read Terminal, Up: Running gawk - -Running Long Programs ---------------------- - -Sometimes your `awk' programs can be very long. In this case it is -more convenient to put the program into a separate file. To tell -`awk' to use that file for its program, you type: - - awk -f SOURCE-FILE INPUT-FILE1 INPUT-FILE2 ... - - The `-f' tells the `awk' utility to get the `awk' program from the -file SOURCE-FILE. Any file name can be used for SOURCE-FILE. For -example, you could put the program: - - /th/ - -into the file `th-prog'. Then the command: - - awk -f th-prog - -does the same thing as this one: - - awk '/th/' - -which was explained earlier (*note Read Terminal::.). Note that you -don't usually need single quotes around the file name that you -specify with `-f', because most file names don't contain any of the -shell's special characters. - -If you want to identify your `awk' program files clearly as such, you -can add the extension `.awk' to the filename. This doesn't affect -the execution of the `awk' program, but it does make ``housekeeping'' -easier. - - - -File: gawk-info, Node: Executable Scripts, Next: Command Line, Prev: Long, Up: Running gawk - -Executable `awk' Programs -------------------------- - -(The following section assumes that you are already somewhat familiar -with `awk'.) - -Once you have learned `awk', you may want to write self--contained -`awk' scripts, using the `#!' script mechanism. You can do this on -BSD Unix systems and GNU. - -For example, you could create a text file named `hello', containing -the following (where `BEGIN' is a feature we have not yet discussed): - - #! /bin/awk -f - - # a sample awk program - - BEGIN { print "hello, world" } - -After making this file executable (with the `chmod' command), you can -simply type: - - hello - -at the shell, and the system will arrange to run `awk' as if you had -typed: - - awk -f hello - -Self--contained `awk' scripts are particularly useful for putting -`awk' programs into production on your system, without your users -having to know that they are actually using an `awk' program. - -If your system does not support the `#!' mechanism, you can get a -similar effect using a regular shell script. It would look something -like this: - - : a sample awk program - - awk 'PROGRAM' "$@" - -Using this technique, it is *vital* to enclose the PROGRAM in single -quotes to protect it from interpretation by the shell. If you omit -the quotes, only a shell wizard can predict the result. - -The `"$@"' causes the shell to forward all the command line arguments -to the `awk' program, without interpretation. - - - -File: gawk-info, Node: Command Line, Prev: Executable Scripts, Up: Running gawk - -Details of the `awk' Command Line ---------------------------------- - -(The following section assumes that you are already familiar with -`awk'.) - -There are two ways to run `awk'. Here are templates for both of -them; items enclosed in `[' and `]' in these templates are optional. - - awk [ -FFS ] [ -- ] 'PROGRAM' FILE ... - awk [ -FFS ] -f SOURCE-FILE [ -f SOURCE-FILE ... ] [ -- ] FILE ... - - Options begin with a minus sign, and consist of a single character. -The options and their meanings are as follows: - -`-FFS' - This sets the `FS' variable to FS (*note Special::.). As a - special case, if FS is `t', then `FS' will be set to the tab - character (`"\t"'). - -`-f SOURCE-FILE' - Indicates that the `awk' program is to be found in SOURCE-FILE - instead of in the first non--option argument. - -`--' - This signals the end of the command line options. If you wish - to specify an input file named `-f', you can precede it with the - `--' argument to prevent the `-f' from being interpreted as an - option. This handling of `--' follows the POSIX argument - parsing conventions. - -Any other options will be flagged as invalid with a warning message, -but are otherwise ignored. - -If the `-f' option is *not* used, then the first non--option command -line argument is expected to be the program text. - -The `-f' option may be used more than once on the command line. -`awk' will read its program source from all of the named files, as if -they had been concatenated together into one big file. This is -useful for creating libraries of `awk' functions. Useful functions -can be written once, and then retrieved from a standard place, -instead of having to be included into each individual program. You -can still type in a program at the terminal and use library -functions, by specifying `/dev/tty' as one of the arguments to a -`-f'. Type your program, and end it with the keyboard end--of--file -character `Control-d'. - -Any additional arguments on the command line are made available to -your `awk' program in the `ARGV' array (*note Special::.). These -arguments are normally treated as input files to be processed in the -order specified. However, an argument that has the form VAR`='VALUE, -means to assign the value VALUE to the variable VAR--it does not -specify a file at all. - -Command line options and the program text (if present) are omitted -from the `ARGV' array. All other arguments, including variable -assignments, are included (*note Special::.). - -The distinction between file name arguments and variable--assignment -arguments is made when `awk' is about to open the next input file. -At that point in execution, it checks the ``file name'' to see -whether it is really a variable assignment; if so, instead of trying -to read a file it will, *at that point in the execution*, assign the -variable. - -Therefore, the variables actually receive the specified values after -all previously specified files have been read. In particular, the -values of variables assigned in this fashion are *not* available -inside a `BEGIN' rule (*note BEGIN/END::.), since such rules are run -before `awk' begins scanning the argument list. - -The variable assignment feature is most useful for assigning to -variables such as `RS', `OFS', and `ORS', which control input and -output formats, before listing the data files. It is also useful for -controlling state if multiple passes are needed over a data file. -For example: - - awk 'pass == 1 { PASS 1 STUFF } - pass == 2 { PASS 2 STUFF }' pass=1 datafile pass=2 datafile - - - -File: gawk-info, Node: Comments, Next: Statements/Lines, Prev: Running gawk, Up: Getting Started - -Comments in `awk' Programs -========================== - -When you write a complicated `awk' program, you can put "comments" in -the program file to help you remember what the program does, and how -it works. - -A comment starts with the the sharp sign character, `#', and -continues to the end of the line. The `awk' language ignores the -rest of a line following a sharp sign. For example, we could have -put the following into `th-prog': - - # This program finds records containing the pattern `th'. This is how - # you continue comments on additional lines. - /th/ - -You can put comment lines into keyboard--composed throw--away `awk' -programs also, but this usually isn't very useful; the purpose of a -comment is to help yourself or another person understand the program -at another time. - - - -File: gawk-info, Node: Statements/Lines, Next: When, Prev: Comments, Up: Getting Started - -`awk' Statements versus Lines -============================= - -Most often, each line in an `awk' program is a separate statement or -separate rule, like this: - - awk '/12/ { print $0 } - /21/ { print $0 }' BBS-list inventory-shipped - -But sometimes statements can be more than one line, and lines can -contain several statements. - -You can split a statement into multiple lines by inserting a newline -after any of the following: - - , { ? : || && - -Lines ending in `do' or `else' automatically have their statements -continued on the following line(s). A newline at any other point -ends the statement. - -If you would like to split a single statement into two lines at a -point where a newline would terminate it, you can "continue" it by -ending the first line with a backslash character, `\'. This is -allowed absolutely anywhere in the statement, even in the middle of a -string or regular expression. For example: - - awk '/This program is too long, so continue it\ - on the next line/ { print $1 }' - -We have generally not used backslash continuation in the sample -programs in this manual. Since there is no limit on the length of a -line, it is never strictly necessary; it just makes programs -prettier. We have preferred to make them even more pretty by keeping -the statements short. Backslash continuation is most useful when -your `awk' program is in a separate source file, instead of typed in -on the command line. - -*Warning: this does not work if you are using the C shell.* -Continuation with backslash works for `awk' programs in files, and -also for one--shot programs *provided* you are using the Bourne -shell, the Korn shell, or the Bourne--again shell. But the C shell -used on Berkeley Unix behaves differently! There, you must use two -backslashes in a row, followed by a newline. - -When `awk' statements within one rule are short, you might want to -put more than one of them on a line. You do this by separating the -statements with semicolons, `;'. This also applies to the rules -themselves. Thus, the above example program could have been written: - - /12/ { print $0 } ; /21/ { print $0 } - -*Note:* It is a new requirement that rules on the same line require -semicolons as a separator in the `awk' language; it was done for -consistency with the statements in the action part of rules. - - - -File: gawk-info, Node: When, Prev: Statements/Lines, Up: Getting Started - -When to Use `awk' -================= - -What use is all of this to me, you might ask? Using additional -operating system utilities, more advanced patterns, field separators, -arithmetic statements, and other selection criteria, you can produce -much more complex output. The `awk' language is very useful for -producing reports from large amounts of raw data, like summarizing -information from the output of standard operating system programs -such as `ls'. (*Note A More Complex Example: More Complex.) - -Programs written with `awk' are usually much smaller than they would -be in other languages. This makes `awk' programs easy to compose and -use. Often `awk' programs can be quickly composed at your terminal, -used once, and thrown away. Since `awk' programs are interpreted, -you can avoid the usually lengthy edit--compile--test--debug cycle of -software development. - -Complex programs have been written in `awk', including a complete -retargetable assembler for 8--bit microprocessors (*note Glossary::. -for more information) and a microcode assembler for a special purpose -Prolog computer. However, `awk''s capabilities are strained by tasks -of such complexity. - -If you find yourself writing `awk' scripts of more than, say, a few -hundred lines, you might consider using a different programming -language. Emacs Lisp is a good choice if you need sophisticated -string or pattern matching capabilities. The shell is also good at -string and pattern matching; in addition it allows powerful use of -the standard utilities. More conventional languages like C, C++, or -Lisp offer better facilities for system programming and for managing -the complexity of large programs. Programs in these languages may -require more lines of source code than the equivalent `awk' programs, -but they will be easier to maintain and usually run more efficiently. - - - -File: gawk-info, Node: Reading Files, Next: Printing, Prev: Getting Started, Up: Top - -Reading Files (Input) -********************* - -In the typical `awk' program, all input is read either from the -standard input (usually the keyboard) or from files whose names you -specify on the `awk' command line. If you specify input files, `awk' -reads data from the first one until it reaches the end; then it reads -the second file until it reaches the end, and so on. The name of the -current input file can be found in the special variable `FILENAME' -(*note Special::.). - -The input is split automatically into "records", and processed by the -rules one record at a time. (Records are the units of text mentioned -in the introduction; by default, a record is a line of text.) Each -record read is split automatically into "fields", to make it more -convenient for a rule to work on parts of the record under -consideration. - -On rare occasions you will need to use the `getline' command, which -can do explicit input from any number of files. - -* Menu: - -* Records:: Controlling how data is split into records. -* Fields:: An introduction to fields. -* Field Separators:: The field separator and how to change it. -* Multiple:: Reading multi--line records. - -* Assignment Options:: Setting variables on the command line and a summary - of command line syntax. This is an advanced method - of input. - -* Getline:: Reading files under explicit program control - using the `getline' function. -* Close Input:: Closing an input file (so you can read from - the beginning once more). - - - -File: gawk-info, Node: Records, Next: Fields, Up: Reading Files - -How Input is Split into Records -=============================== - -The `awk' language divides its input into records and fields. -Records are separated from each other by the "record separator". By -default, the record separator is the "newline" character. Therefore, -normally, a record is a line of text. - -Sometimes you may want to use a different character to separate your -records. You can use different characters by changing the special -variable `RS'. - -The value of `RS' is a string that says how to separate records; the -default value is `"\n"', the string of just a newline character. -This is why lines of text are the default record. Although `RS' can -have any string as its value, only the first character of the string -will be used as the record separator. The other characters are -ignored. `RS' is exceptional in this regard; `awk' uses the full -value of all its other special variables. - -The value of `RS' is changed by "assigning" it a new value (*note -Assignment Ops::.). One way to do this is at the beginning of your -`awk' program, before any input has been processed, using the special -`BEGIN' pattern (*note BEGIN/END::.). This way, `RS' is changed to -its new value before any input is read. The new value of `RS' is -enclosed in quotation marks. For example: - - awk 'BEGIN { RS = "/" } ; { print $0 }' BBS-list - -changes the value of `RS' to `/', the slash character, before reading -any input. Records are now separated by a slash. The second rule in -the `awk' program (the action with no pattern) will proceed to print -each record. Since each `print' statement adds a newline at the end -of its output, the effect of this `awk' program is to copy the input -with each slash changed to a newline. - -Another way to change the record separator is on the command line, -using the variable--assignment feature (*note Command Line::.). - - awk '...' RS="/" SOURCE-FILE - -`RS' will be set to `/' before processing SOURCE-FILE. - -The empty string (a string of no characters) has a special meaning as -the value of `RS': it means that records are separated only by blank -lines. *Note Multiple::, for more details. - -The `awk' utility keeps track of the number of records that have been -read so far from the current input file. This value is stored in a -special variable called `FNR'. It is reset to zero when a new file -is started. Another variable, `NR', is the total number of input -records read so far from all files. It starts at zero but is never -automatically reset to zero. - -If you change the value of `RS' in the middle of an `awk' run, the -new value is used to delimit subsequent records, but the record -currently being processed (and records already finished) are not -affected. - - - -File: gawk-info, Node: Fields, Next: Non-Constant Fields, Prev: Records, Up: Reading Files - -Examining Fields -================ - -When `awk' reads an input record, the record is automatically -separated or "parsed" by the interpreter into pieces called "fields". -By default, fields are separated by whitespace, like words in a line. -Whitespace in `awk' means any string of one or more spaces and/or -tabs; other characters such as newline, formfeed, and so on, that are -considered whitespace by other languages are *not* considered -whitespace by `awk'. - -The purpose of fields is to make it more convenient for you to refer -to these pieces of the record. You don't have to use them--you can -operate on the whole record if you wish--but fields are what make -simple `awk' programs so powerful. - -To refer to a field in an `awk' program, you use a dollar--sign, `$', -followed by the number of the field you want. Thus, `$1' refers to -the first field, `$2' to the second, and so on. For example, suppose -the following is a line of input: - - This seems like a pretty nice example. - - Here the first field, or `$1', is `This'; the second field, or `$2', -is `seems'; and so on. Note that the last field, `$7', is -`example.'. Because there is no space between the `e' and the `.', -the period is considered part of the seventh field. - -No matter how many fields there are, the last field in a record can -be represented by `$NF'. So, in the example above, `$NF' would be -the same as `$7', which is `example.'. Why this works is explained -below (*note Non-Constant Fields::.). If you try to refer to a field -beyond the last one, such as `$8' when the record has only 7 fields, -you get the empty string. - -Plain `NF', with no `$', is a special variable whose value is the -number of fields in the current record. - -`$0', which looks like an attempt to refer to the zeroth field, is a -special case: it represents the whole input record. This is what you -would use when you aren't interested in fields. - -Here are some more examples: - - awk '$1 ~ /foo/ { print $0 }' BBS-list - -This example contains the "matching" operator `~' (*note Comparison -Ops::.). Using this operator, all records in the file `BBS-list' -whose first field contains the string `foo' are printed. - -By contrast, the following example: - - awk '/foo/ { print $1, $NF }' BBS-list - -looks for the string `foo' in *the entire record* and prints the -first field and the last field for each input record containing the -pattern. - -The following program will search the system password file, and print -the entries for users who have no password. - - awk -F: '$2 == ""' /etc/passwd - -This program uses the `-F' option on the command line to set the file -separator. (Fields in `/etc/passwd' are separated by colons. The -second field represents a user's encrypted password, but if the field -is empty, that user has no password.) - - - -File: gawk-info, Node: Non-Constant Fields, Next: Changing Fields, Prev: Fields, Up: Reading Files - -Non-constant Field Numbers -========================== - -The number of a field does not need to be a constant. Any expression -in the `awk' language can be used after a `$' to refer to a field. -The `awk' utility evaluates the expression and uses the "numeric -value" as a field number. Consider this example: - - awk '{ print $NR }' - -Recall that `NR' is the number of records read so far: 1 in the first -record, 2 in the second, etc. So this example will print the first -field of the first record, the second field of the second record, and -so on. For the twentieth record, field number 20 will be printed; -most likely this will make a blank line, because the record will not -have 20 fields. - -Here is another example of using expressions as field numbers: - - awk '{ print $(2*2) }' BBS-list - -The `awk' language must evaluate the expression `(2*2)' and use its -value as the field number to print. The `*' sign represents -multiplication, so the expression `2*2' evaluates to 4. This -example, then, prints the hours of operation (the fourth field) for -every line of the file `BBS-list'. - -When you use non--constant field numbers, you may ask for a field -with a negative number. This always results in an empty string, just -like a field whose number is too large for the input record. For -example, `$(1-4)' would try to examine field number -3; it would -result in an empty string. - -If the field number you compute is zero, you get the entire record. - -The number of fields in the current record is stored in the special -variable `NF' (*note Special::.). The expression `$NF' is not a -special feature: it is the direct consequence of evaluating `NF' and -using its value as a field number. - - - -File: gawk-info, Node: Changing Fields, Next: Field Separators, Prev: Non-Constant Fields, Up: Reading Files - -Changing the Contents of a Field -================================ - -You can change the contents of a field as seen by `awk' within an -`awk' program; this changes what `awk' perceives as the current input -record. (The actual input is untouched: `awk' never modifies the -input file.) - -Look at this example: - - awk '{ $3 = $2 - 10; print $2, $3 }' inventory-shipped - -The `-' sign represents subtraction, so this program reassigns field -three, `$3', to be the value of field two minus ten, ``$2' - 10'. -(*Note Arithmetic Ops::.) Then field two, and the new value for -field three, are printed. - -In order for this to work, the text in field `$2' must make sense as -a number; the string of characters must be converted to a number in -order for the computer to do arithmetic on it. The number resulting -from the subtraction is converted back to a string of characters -which then becomes field 3. *Note Conversion::. - -When you change the value of a field (as perceived by `awk'), the -text of the input record is recalculated to contain the new field -where the old one was. `$0' will from that time on reflect the -altered field. Thus, - - awk '{ $2 = $2 - 10; print $0 }' inventory-shipped - -will print a copy of the input file, with 10 subtracted from the -second field of each line. - -You can also assign contents to fields that are out of range. For -example: - - awk '{ $6 = ($5 + $4 + $3 + $2)/4) ; print $6 }' inventory-shipped - -We've just created `$6', whose value is the average of fields `$2', -`$3', `$4', and `$5'. The `+' sign represents addition, and the `/' -sign represents division. For the file `inventory-shipped' `$6' -represents the average number of parcels shipped for a particular -month. - -Creating a new field changes what `awk' interprets as the current -input record. The value of `$0' will be recomputed. This -recomputation affects and is affected by features not yet discussed, -in particular, the "Output Field Separator", `OFS', which is used to -separate the fields (*note Output Separators::.), and `NF' (the -number of fields; *note Fields::.). For example, the value of `NF' -will be set to the number of the highest out--of--range field you -create. - -Note, however, that merely *referencing* an out--of--range field will -*not* change the value of either `$0' or `NF'. Referencing an -out--of--range field merely produces a null string. For example: - - if ($(NF+1) != "") - print "can't happen" - else - print "everything is normal" - -should print `everything is normal'. (*Note If::, for more -information about `awk''s `if-else' statements.) - - - -File: gawk-info, Node: Field Separators, Next: Multiple, Prev: Changing Fields, Up: Reading Files - -Specifying How Fields Are Separated -=================================== - -You can change the way `awk' splits a record into fields by changing -the value of the "field separator". The field separator is -represented by the special variable `FS' in an `awk' program, and can -be set by `-F' on the command line. The `awk' language scans each -input line for the field separator character to determine the -positions of fields within that line. Shell programmers take note! -`awk' uses the variable `FS', not `IFS'. - -The default value of the field separator is a string containing a -single space. This value is actually a special case; as you know, by -default, fields are separated by whitespace sequences, not by single -spaces: two spaces in a row do not delimit an empty field. -``Whitespace'' is defined as sequences of one or more spaces or tab -characters. - -You change the value of `FS' by "assigning" it a new value. You can -do this using the special `BEGIN' pattern (*note BEGIN/END::.). This -pattern allows you to change the value of `FS' before any input is -read. The new value of `FS' is enclosed in quotations. For example, -set the value of `FS' to the string `","': - - awk 'BEGIN { FS = "," } ; { print $2 }' - -and use the input line: - - John Q. Smith, 29 Oak St., Walamazoo, MI 42139 - -This `awk' program will extract the string `29 Oak St.'. - -Sometimes your input data will contain separator characters that -don't separate fields the way you thought they would. For instance, -the person's name in the example we've been using might have a title -or suffix attached, such as `John Q. Smith, LXIX'. If you assigned -`FS' to be `,' then: - - awk 'BEGIN { FS = "," } ; { print $2 } - -would extract `LXIX', instead of `29 Oak St.'. If you were expecting -the program to print the address, you would be surprised. So, choose -your data layout and separator characters carefully to prevent -problems like this from happening. - -You can assign `FS' to be a series of characters. For example, the -assignment: - - FS = ", \t" - -makes every area of an input line that consists of a comma followed -by a space and a tab, into a field separator. (`\t' stands for a tab.) - -If `FS' is any single character other than a blank, then that -character is used as the field separator, and two successive -occurrences of that character do delimit an empty field. - -If you assign `FS' to a string longer than one character, that string -is evaluated as a "regular expression" (*note Regexp::.). The value -of the regular expression is used as a field separator. - -`FS' can be set on the command line. You use the `-F' argument to do -so. For example: - - awk -F, 'PROGRAM' INPUT-FILES - -sets `FS' to be the `,' character. Notice that the argument uses a -capital `F'. Contrast this with `-f', which specifies a file -containing an `awk' program. Case is significant in command options: -the `-F' and `-f' options have nothing to do with each other. You -can use both options at the same time to set the `FS' argument *and* -get an `awk' program from a file. - -As a special case, if the argument to `-F' is `t', then `FS' is set -to the tab character. (This is because if you type `-F\t', without -the quotes, at the shell, the `\' gets deleted, so `awk' figures that -you really want your fields to be separated with tabs, and not `t's. -Use `FS="t"' if you really do want to separate your fields with `t's.) - -For example, let's use an `awk' program file called `baud.awk' that -contains the pattern `/300/', and the action `print $1'. We'll use -the operating system utility `cat' to ``look'' at our program: - - % cat baud.awk - /300/ { print $1 } - -Let's also set `FS' to be the `-' character. We will apply all this -information to the file `BBS-list'. This `awk' program will now -print a list of the names of the bulletin boards that operate at 300 -baud and the first three digits of their phone numbers. - - awk -F- -f baud.awk BBS-list - -produces this output: - - aardvark 555 - alpo - barfly 555 - bites 555 - camelot 555 - core 555 - fooey 555 - foot 555 - macfoo 555 - sdace 555 - sabafoo 555 - -Note the second line of output. If you check the original file, you -will see that the second line looked like this: - - alpo-net 555-3412 2400/1200/300 A - -The `-' as part of the system's name was used as the field separator, -instead of the `-' in the phone number that was originally intended. -This demonstrates why you have to be careful in choosing your field -and record separators. - - - -File: gawk-info, Node: Multiple, Next: Assignment Options, Prev: Field Separators, Up: Reading Files - -Multiple--Line Records -====================== - -In some data bases, a single line cannot conveniently hold all the -information in one entry. Then you will want to use multi--line -records. - -The first step in doing this is to choose your data format: when -records are not defined as single lines, how will you want to define -them? What should separate records? - -One technique is to use an unusual character or string to separate -records. For example, you could use the formfeed character (written -`\f' in `awk', as in C) to separate them, making each record a page -of the file. To do this, just set the variable `RS' to `"\f"' (a -string containing the formfeed character), or whatever string you -prefer to use. - -Another technique is to have blank lines separate records. By a -special dispensation, a null string as the value of `RS' indicates -that records are separated by one or more blank lines. If you set -`RS' to the null string, a record will always end at the first blank -line encountered. And the next record won't start until the first -nonblank line that follows--no matter how many blank lines appear in -a row, they will be considered one record--separator. - -The second step is to separate the fields in the record. One way to -do this is to put each field on a separate line: to do this, just set -the variable `FS' to the string `"\n"'. (This simple regular -expression matches a single newline.) Another idea is to divide each -of the lines into fields in the normal manner; the regular expression -`"[ \t\n]+"' will do this nicely by treating the newlines inside the -record just like spaces. - -When `RS' is set to the null string, the newline character *always* -acts as a field separator. This is in addition to whatever value -`FS' has. The probable reason for this rule is so that you get -rational behavior in the default case (i.e. `FS == " "'). This can -be a problem if you really don't want the newline character to -separate fields, since there is no way to do that. However, you can -work around this by using the `split' function to manually break up -your data (*note String Functions::.). - -Here is how to use records separated by blank lines and break each -line into fields normally: - - awk 'BEGIN { RS = ""; FS = "[ \t\n]+" } ; { print $0 }' BBS-list - - - -File: gawk-info, Node: Assignment Options, Next: Getline, Prev: Multiple, Up: Reading Files - -Assigning Variables on the Command Line -======================================= - -You can include variable "assignments" among the file names on the -command line used to invoke `awk' (*note Command Line::.). Such -assignments have the form: - - VARIABLE=TEXT - -and allow you to change variables either at the beginning of the -`awk' run or in between input files. The variable assignment is -performed at a time determined by its position among the input file -arguments: after the processing of the preceding input file argument. -For example: - - awk '{ print $n }' n=4 inventory-shipped n=2 BBS-list - -prints the value of field number `n' for all input records. Before -the first file is read, the command line sets the variable `n' equal -to 4. This causes the fourth field of the file `inventory-shipped' -to be printed. After the first file has finished, but before the -second file is started, `n' is set to 2, so that the second field of -the file `BBS-list' will be printed. - -Command line arguments are made available for explicit examination by -the `awk' program in an array named `ARGV' (*note Special::.). - - - -File: gawk-info, Node: Getline, Prev: Assignment Options, Up: Reading Files - -Explicit Input with `getline' -============================= - -So far we have been getting our input files from `awk''s main input -stream--either the standard input (usually your terminal) or the -files specified on the command line. The `awk' language has a -special built--in function called `getline' that can be used to read -input under your explicit control. - -This command is quite complex and should *not* be used by beginners. -The command (and its variations) is covered here because this is the -section about input. The examples that follow the explanation of the -`getline' command include material that has not been covered yet. -Therefore, come back and attempt the `getline' command *after* you -have reviewed the rest of this manual and have a good knowledge of -how `awk' works. - -When retrieving input, `getline' returns a 1 if it found a record, -and a 0 if the end of the file was encountered. If there was some -error in getting a record, such as a file that could not be opened, -then `getline' returns a -1. - -In the following examples, COMMAND stands for a string value that -represents a shell command. - -`getline' - The `getline' function can be used by itself, in an `awk' - program, to read input from the current input. All it does in - this case is read the next input record and split it up into - fields. This is useful if you've finished processing the - current record, but you want to do some special processing - *right now* on the next record. Here's an example: - - awk '{ - if (t = index($0, "/*")) { - if(t > 1) - tmp = substr($0, 1, t - 1) - else - tmp = "" - u = index(substr($0, t + 2), "*/") - while (! u) { - getline - t = -1 - u = index($0, "*/") - } - if(u <= length($0) - 2) - $0 = tmp substr($0, t + u + 3) - else - $0 = tmp - } - print $0 - }' - - This `awk' program deletes all comments, `/* ... */', from the - input. By replacing the `print $0' with other statements, you - could perform more complicated processing on the de--commented - input, such as search it for matches for a regular expression. - - This form of the `getline' command sets `NF' (the number of - fields; *note Fields::.), `NR' (the number of records read so - far), the `FNR' variable (*note Records::.), and the value of - `$0'. - - *Note:* The new value of `$0' will be used in testing the - patterns of any subsequent rules. The original value of `$0' - that triggered the rule which executed `getline' is lost. By - contrast, the `next' statement reads a new record but - immediately begins processing it normally, starting with the - first rule in the program. *Note Next::. - -`getline VAR' - This form of `getline' reads a record into the variable VAR. - This is useful when you want your program to read the next - record from the input file, but you don't want to subject the - record to the normal input processing. - - For example, suppose the next line is a comment, or a special - string, and you want to read it, but you must make certain that - it won't accidentally trigger any rules. This version of - `getline' will allow you to read that line and store it in a - variable so that the main read--a--line--and--check--each--rule - loop of `awk' never sees it. - - The following example swaps every two lines of input. For - example, given: - - wan - tew - free - phore - - it outputs: - - tew - wan - phore - free - - Here's the program: - - awk '{ - if ((getline tmp) > 0) { - print tmp - print $0 - } else - print $0 - }' - - The `getline' function used in this way sets only `NR' and `FNR' - (and of course, VAR). The record is not split into fields, so - the values of the fields (including `$0') and the value of `NF' - do not change. - -`getline < FILE' - This form of the `getline' function takes its input from the - file FILE. Here FILE is a string--valued expression that - specifies the file name. - - This form is useful if you want to read your input from a - particular file, instead of from the main input stream. For - example, the following program reads its input record from the - file `foo.input' when it encounters a first field with a value - equal to 10 in the current input file. - - awk '{ - if ($1 == 10) { - getline < "foo.input" - print - } else - print - }' - - Since the main input stream is not used, the values of `NR' and - `FNR' are not changed. But the record read is split into fields - in the normal manner, so the values of `$0' and other fields are - changed. So is the value of `NF'. - - This does not cause the record to be tested against all the - patterns in the `awk' program, in the way that would happen if - the record were read normally by the main processing loop of - `awk'. However the new record is tested against any subsequent - rules, just as when `getline' is used without a redirection. - -`getline VAR < FILE' - This form of the `getline' function takes its input from the - file FILE and puts it in the variable VAR. As above, FILE is a - string--valued expression that specifies the file to read from. - - In this version of `getline', none of the built--in variables - are changed, and the record is not split into fields. The only - variable changed is VAR. - - For example, the following program copies all the input files to - the output, except for records that say `@include FILENAME'. - Such a record is replaced by the contents of the file FILENAME. - - awk '{ - if (NF == 2 && $1 == "@include") { - while ((getline line < $2) > 0) - print line - close($2) - } else - print - }' - - Note here how the name of the extra input file is not built into - the program; it is taken from the data, from the second field on - the `@include' line. - - The `close' command is used to ensure that if two identical - `@include' lines appear in the input, the entire specified file - is included twice. *Note Close Input::. - - One deficiency of this program is that it does not process - nested `@include' statements the way a true macro preprocessor - would. - -`COMMAND | getline' - You can "pipe" the output of a command into `getline'. A pipe - is simply a way to link the output of one program to the input - of another. In this case, the string COMMAND is run as a shell - command and its output is piped into `awk' to be used as input. - This form of `getline' reads one record from the pipe. - - For example, the following program copies input to output, - except for lines that begin with `@execute', which are replaced - by the output produced by running the rest of the line as a - shell command: - - awk '{ - if ($1 == "@execute") { - tmp = substr($0, 10) - while ((tmp | getline) > 0) - print - close(tmp) - } else - print - }' - - The `close' command is used to ensure that if two identical - `@execute' lines appear in the input, the command is run again - for each one. *Note Close Input::. - - Given the input: - - foo - bar - baz - @execute who - bletch - - the program might produce: - - foo - bar - baz - hack ttyv0 Jul 13 14:22 - hack ttyp0 Jul 13 14:23 (gnu:0) - hack ttyp1 Jul 13 14:23 (gnu:0) - hack ttyp2 Jul 13 14:23 (gnu:0) - hack ttyp3 Jul 13 14:23 (gnu:0) - bletch - - Notice that this program ran the command `who' and printed the - result. (If you try this program yourself, you will get - different results, showing you logged in.) - - This variation of `getline' splits the record into fields, sets - the value of `NF' and recomputes the value of `$0'. The values - of `NR' and `FNR' are not changed. - -`COMMAND | getline VAR' - The output of the command COMMAND is sent through a pipe to - `getline' and into the variable VAR. For example, the following - program reads the current date and time into the variable - `current_time', using the utility called `date', and then prints - it. - - awk 'BEGIN { - "date" | getline current_time - close("date") - print "Report printed on " current_time - }' - - In this version of `getline', none of the built--in variables - are changed, and the record is not split into fields. - - - -File: gawk-info, Node: Close Input, Up: Getline - -Closing Input Files -------------------- - -If the same file name or the same shell command is used with -`getline' more than once during the execution of the `awk' program, -the file is opened (or the command is executed) only the first time. -At that time, the first record of input is read from that file or -command. The next time the same file or command is used in -`getline', another record is read from it, and so on. - -What this implies is that if you want to start reading the same file -again from the beginning, or if you want to rerun a shell command -(rather that reading more output from the command), you must take -special steps. What you can do is use the `close' statement: - - close (FILENAME) - -This statement closes a file or pipe, represented here by FILENAME. -The string value of FILENAME must be the same value as the string -used to open the file or pipe to begin with. - -Once this statement is executed, the next `getline' from that file or -command will reopen the file or rerun the command. - - - -File: gawk-info, Node: Printing, Next: One-liners, Prev: Reading Files, Up: Top - -Printing Output -*************** - -One of the most common things that actions do is to output or "print" -some or all of the input. For simple output, use the `print' -statement. For fancier formatting use the `printf' statement. Both -are described in this chapter. - -* Menu: - -* Print:: The `print' statement. -* Print Examples:: Simple examples of `print' statements. -* Output Separators:: The output separators and how to change them. - -* Redirection:: How to redirect output to multiple files and pipes. -* Close Output:: How to close output files and pipes. - -* Printf:: The `printf' statement. - - - -File: gawk-info, Node: Print, Next: Print Examples, Up: Printing - -The `print' Statement -===================== - -The `print' statement does output with simple, standardized -formatting. You specify only the strings or numbers to be printed, -in a list separated by commas. They are output, separated by single -spaces, followed by a newline. The statement looks like this: - - print ITEM1, ITEM2, ... - - The entire list of items may optionally be enclosed in parentheses. -The parentheses are necessary if any of the item expressions uses a -relational operator; otherwise it could be confused with a -redirection (*note Redirection::.). The relational operators are -`==', `!=', `<', `>', `>=', `<=', `~' and `!~' (*note Comparison -Ops::.). - -The items printed can be constant strings or numbers, fields of the -current record (such as `$1'), variables, or any `awk' expressions. -The `print' statement is completely general for computing *what* -values to print. With one exception (*note Output Separators::.), -what you can't do is specify *how* to print them--how many columns to -use, whether to use exponential notation or not, and so on. For -that, you need the `printf' statement (*note Printf::.). - -To print a fixed piece of text, write a string constant as one item, -such as `"Hello there"'. If you forget to use the double--quote -characters, your text will be taken as an `awk' expression, and you -will probably get an error. Keep in mind that a space will be -printed between any two items. - -The simple statement `print' with no items is equivalent to `print -$0': it prints the entire current record. To print a blank line, use -`print ""', where `""' is the null, or empty, string. - -Most often, each `print' statement makes one line of output. But it -isn't limited to one line. If an item value is a string that -contains a newline, the newline is output along with the rest of the -string. A single `print' can make any number of lines this way. - - - -File: gawk-info, Node: Print Examples, Next: Output Separators, Prev: Print, Up: Printing - -Examples of `print' Statements -============================== - -Here is an example that prints the first two fields of each input -record, with a space between them: - - awk '{ print $1, $2 }' inventory-shipped - -Its output looks like this: - - Jan 13 - Feb 15 - Mar 15 - ... - - A common mistake in using the `print' statement is to omit the comma -between two items. This often has the effect of making the items run -together in the output, with no space. The reason for this is that -juxtaposing two string expressions in `awk' means to concatenate -them. For example, without the comma: - - awk '{ print $1 $2 }' inventory-shipped - -prints: - - Jan13 - Feb15 - Mar15 - ... - - Neither example's output makes much sense to someone unfamiliar with -the file `inventory-shipped'. A heading line at the beginning would -make it clearer. Let's add some headings to our table of months -(`$1') and green crates shipped (`$2'). We do this using the BEGIN -pattern (*note BEGIN/END::.) to cause the headings to be printed only -once: - - awk 'BEGIN { print "Month Crates" - print "---- -----" } - { print $1, $2 }' inventory-shipped - -Did you already guess what will happen? This program prints the -following: - - Month Crates - ---- ----- - Jan 13 - Feb 15 - Mar 15 - ... - - The headings and the table data don't line up! We can fix this by -printing some spaces between the two fields: - - awk 'BEGIN { print "Month Crates" - print "---- -----" } - { print $1, " ", $2 }' inventory-shipped - -You can imagine that this way of lining up columns can get pretty -complicated when you have many columns to fix. Counting spaces for -two or three columns can be simple, but more than this and you can -get ``lost'' quite easily. This is why the `printf' statement was -created (*note Printf::.); one of its specialties is lining up -columns of data. - - - -File: gawk-info, Node: Output Separators, Next: Redirection, Prev: Print Examples, Up: Printing - -Output Separators -================= - -As mentioned previously, a `print' statement contains a list of -items, separated by commas. In the output, the items are normally -separated by single spaces. But they do not have to be spaces; a -single space is only the default. You can specify any string of -characters to use as the "output field separator", by setting the -special variable `OFS'. The initial value of this variable is the -string `" "'. - -The output from an entire `print' statement is called an "output -record". Each `print' statement outputs one output record and then -outputs a string called the "output record separator". The special -variable `ORS' specifies this string. The initial value of the -variable is the string `"\n"' containing a newline character; thus, -normally each `print' statement makes a separate line. - -You can change how output fields and records are separated by -assigning new values to the variables `OFS' and/or `ORS'. The usual -place to do this is in the `BEGIN' rule (*note BEGIN/END::.), so that -it happens before any input is processed. You may also do this with -assignments on the command line, before the names of your input files. - -The following example prints the first and second fields of each -input record separated by a semicolon, with a blank line added after -each line: - - awk 'BEGIN { OFS = ";"; ORS = "\n\n" } - { print $1, $2 }' BBS-list - -If the value of `ORS' does not contain a newline, all your output -will be run together on a single line, unless you output newlines -some other way. - - - -File: gawk-info, Node: Redirection, Next: Printf, Prev: Output Separators, Up: Printing - -Redirecting Output of `print' and `printf' -========================================== - -So far we have been dealing only with output that prints to the -standard output, usually your terminal. Both `print' and `printf' -can be told to send their output to other places. This is called -"redirection". - -A redirection appears after the `print' or `printf' statement. -Redirections in `awk' are written just like redirections in shell -commands, except that they are written inside the `awk' program. - -Here are the three forms of output redirection. They are all shown -for the `print' statement, but they work for `printf' also. - -`print ITEMS > OUTPUT-FILE' - This type of redirection prints the items onto the output file - OUTPUT-FILE. The file name OUTPUT-FILE can be any expression. - Its value is changed to a string and then used as a filename - (*note Expressions::.). - - When this type of redirection is used, the OUTPUT-FILE is erased - before the first output is written to it. Subsequent writes do - not erase OUTPUT-FILE, but append to it. If OUTPUT-FILE does - not exist, then it is created. - - For example, here is how one `awk' program can write a list of - BBS names to a file `name-list' and a list of phone numbers to a - file `phone-list'. Each output file contains one name or number - per line. - - awk '{ print $2 > "phone-list" - print $1 > "name-list" }' BBS-list - -`print ITEMS >> OUTPUT-FILE' - This type of redirection prints the items onto the output file - OUTPUT-FILE. The difference between this and the single--`>' - redirection is that the old contents (if any) of OUTPUT-FILE are - not erased. Instead, the `awk' output is appended to the file. - -`print ITEMS | COMMAND' - It is also possible to send output through a "pipe" instead of - into a file. This type of redirection opens a pipe to COMMAND - and writes the values of ITEMS through this pipe, to another - process created to execute COMMAND. - - The redirection argument COMMAND is actually an `awk' - expression. Its value is converted to a string, whose contents - give the shell command to be run. - - For example, this produces two files, one unsorted list of BBS - names and one list sorted in reverse alphabetical order: - - awk '{ print $1 > "names.unsorted" - print $1 | "sort -r > names.sorted" }' BBS-list - - Here the unsorted list is written with an ordinary redirection - while the sorted list is written by piping through the `sort' - utility. - - Here is an example that uses redirection to mail a message to a - mailing list `bug-system'. This might be useful when trouble is - encountered in an `awk' script run periodically for system - maintenance. - - print "Awk script failed:", $0 | "mail bug-system" - print "processing record number", FNR, "of", FILENAME | "mail bug-system" - close ("mail bug-system") - - We use a `close' statement here because it's a good idea to - close the pipe as soon as all the intended output has been sent - to it. *Note Close Output::, for more information on this. - -Redirecting output using `>', `>>', or `|' asks the system to open a -file or pipe only if the particular FILE or COMMAND you've specified -has not already been written to by your program. - - - -File: gawk-info, Node: Close Output, Up: Redirection - -Closing Output Files and Pipes ------------------------------- - -When a file or pipe is opened, the filename or command associated -with it is remembered by `awk' and subsequent writes to the same file -or command are appended to the previous writes. The file or pipe -stays open until `awk' exits. This is usually convenient. - -Sometimes there is a reason to close an output file or pipe earlier -than that. To do this, use the `close' command, as follows: - - close (FILENAME) - -or - - close (COMMAND) - -The argument FILENAME or COMMAND can be any expression. Its value -must exactly equal the string used to open the file or pipe to begin -with--for example, if you open a pipe with this: - - print $1 | "sort -r > names.sorted" - -then you must close it with this: - - close ("sort -r > names.sorted") - -Here are some reasons why you might need to close an output file: - - * To write a file and read it back later on in the same `awk' - program. Close the file when you are finished writing it; then - you can start reading it with `getline' (*note Getline::.). - - * To write numerous files, successively, in the same `awk' - program. If you don't close the files, eventually you will - exceed the system limit on the number of open files in one - process. So close each one when you are finished writing it. - - * To make a command finish. When you redirect output through a - pipe, the command reading the pipe normally continues to try to - read input as long as the pipe is open. Often this means the - command cannot really do its work until the pipe is closed. For - example, if you redirect output to the `mail' program, the - message will not actually be sent until the pipe is closed. - - * To run the same subprogram a second time, with the same arguments. - This is not the same thing as giving more input to the first run! - - For example, suppose you pipe output to the `mail' program. If - you output several lines redirected to this pipe without closing - it, they make a single message of several lines. By contrast, - if you close the pipe after each line of output, then each line - makes a separate message. - - - -File: gawk-info, Node: Printf, Prev: Redirection, Up: Printing - -Using `printf' Statements For Fancier Printing -============================================== - -If you want more precise control over the output format than `print' -gives you, use `printf'. With `printf' you can specify the width to -use for each item, and you can specify various stylistic choices for -numbers (such as what radix to use, whether to print an exponent, -whether to print a sign, and how many digits to print after the -decimal point). You do this by specifying a "format string". - -* Menu: - -* Basic Printf:: Syntax of the `printf' statement. -* Format-Control:: Format-control letters. -* Modifiers:: Format--specification modifiers. -* Printf Examples:: Several examples. - - - -File: gawk-info, Node: Basic Printf, Next: Format-Control, Up: Printf - -Introduction to the `printf' Statement --------------------------------------- - -The `printf' statement looks like this: - - printf FORMAT, ITEM1, ITEM2, ... - - The entire list of items may optionally be enclosed in parentheses. -The parentheses are necessary if any of the item expressions uses a -relational operator; otherwise it could be confused with a -redirection (*note Redirection::.). The relational operators are -`==', `!=', `<', `>', `>=', `<=', `~' and `!~' (*note Comparison -Ops::.). - -The difference between `printf' and `print' is the argument FORMAT. -This is an expression whose value is taken as a string; its job is to -say how to output each of the other arguments. It is called the -"format string". - -The format string is essentially the same as in the C library -function `printf'. Most of FORMAT is text to be output verbatim. -Scattered among this text are "format specifiers", one per item. -Each format specifier says to output the next item at that place in -the format. - -The `printf' statement does not automatically append a newline to its -output. It outputs nothing but what the format specifies. So if you -want a newline, you must include one in the format. The output -separator variables `OFS' and `ORS' have no effect on `printf' -statements. - - - -File: gawk-info, Node: Format-Control, Next: Modifiers, Prev: Basic Printf, Up: Printf - -Format--Control Characters --------------------------- - -A format specifier starts with the character `%' and ends with a -"format--control letter"; it tells the `printf' statement how to -output one item. (If you actually want to output a `%', write `%%'.) -The format--control letter specifies what kind of value to print. -The rest of the format specifier is made up of optional "modifiers" -which are parameters such as the field width to use. - -Here is a list of them: - -`c' - This prints a number as an ASCII character. Thus, `printf "%c", - 65' outputs the letter `A'. The output for a string value is - the first character of the string. - -`d' - This prints a decimal integer. - -`e' - This prints a number in scientific (exponential) notation. For - example, - - printf "%4.3e", 1950 - - prints `1.950e+03', with a total of 4 significant figures of - which 3 follow the decimal point. The `4.3' are "modifiers", - discussed below. - -`f' - This prints a number in floating point notation. - -`g' - This prints either scientific notation or floating point - notation, whichever is shorter. - -`o' - This prints an unsigned octal integer. - -`s' - This prints a string. - -`x' - This prints an unsigned hexadecimal integer. - -`%' - This isn't really a format--control letter, but it does have a - meaning when used after a `%': the sequence `%%' outputs one - `%'. It does not consume an argument. - - - -File: gawk-info, Node: Modifiers, Next: Printf Examples, Prev: Format-Control, Up: Printf - -Modifiers for `printf' Formats ------------------------------- - -A format specification can also include "modifiers" that can control -how much of the item's value is printed and how much space it gets. -The modifiers come between the `%' and the format--control letter. -Here are the possible modifiers, in the order in which they may appear: - -`-' - The minus sign, used before the width modifier, says to - left--justify the argument within its specified width. Normally - the argument is printed right--justified in the specified width. - -`WIDTH' - This is a number representing the desired width of a field. - Inserting any number between the `%' sign and the format control - character forces the field to be expanded to this width. The - default way to do this is to pad with spaces on the left. - -`.PREC' - This is a number that specifies the precision to use when - printing. This specifies the number of digits you want printed - to the right of the decimal place. - -The C library `printf''s dynamic WIDTH and PREC capability (for -example, `"%*.*s"') is not supported. However, it can be easily -simulated using concatenation to dynamically build the format string. - - - -File: gawk-info, Node: Printf Examples, Prev: Modifiers, Up: Printf - -Examples of Using `printf' --------------------------- - -Here is how to use `printf' to make an aligned table: - - awk '{ printf "%-10s %s\n", $1, $2 }' BBS-list - -prints the names of bulletin boards (`$1') of the file `BBS-list' as -a string of 10 characters, left justified. It also prints the phone -numbers (`$2') afterward on the line. This will produce an aligned -two--column table of names and phone numbers, like so: - - aardvark 555-5553 - alpo-net 555-3412 - barfly 555-7685 - bites 555-1675 - camelot 555-0542 - core 555-2912 - fooey 555-1234 - foot 555-6699 - macfoo 555-6480 - sdace 555-3430 - sabafoo 555-2127 - -Did you notice that we did not specify that the phone numbers be -printed as numbers? They had to be printed as strings because the -numbers are separated by a dash. This dash would be interpreted as a -"minus" sign if we had tried to print the phone numbers as numbers. -This would have led to some pretty confusing results. - -We did not specify a width for the phone numbers because they are the -last things on their lines. We don't need to put spaces after them. - -We could make our table look even nicer by adding headings to the -tops of the columns. To do this, use the BEGIN pattern (*note -BEGIN/END::.) to cause the header to be printed only once, at the -beginning of the `awk' program: - - awk 'BEGIN { print "Name Number" - print "--- -----" } - { printf "%-10s %s\n", $1, $2 }' BBS-list - -Did you notice that we mixed `print' and `printf' statements in the -above example? We could have used just `printf' statements to get -the same results: - - awk 'BEGIN { printf "%-10s %s\n", "Name", "Number" - printf "%-10s %s\n", "---", "-----" } - { printf "%-10s %s\n", $1, $2 }' BBS-list - -By outputting each column heading with the same format specification -used for the elements of the column, we have made sure that the -headings will be aligned just like the columns. - -The fact that the same format specification is used can be emphasized -by storing it in a variable, like so: - - awk 'BEGIN { format = "%-10s %s\n" - printf format, "Name", "Number" - printf format, "---", "-----" } - { printf format, $1, $2 }' BBS-list - -See if you can use the `printf' statement to line up the headings and -table data for our `inventory-shipped' example covered earlier in the -section on the `print' statement (*note Print::.). - - - -File: gawk-info, Node: One-liners, Next: Patterns, Prev: Printing, Up: Top - -Useful ``One-liners'' -********************* - -Useful `awk' programs are often short, just a line or two. Here is a -collection of useful, short programs to get you started. Some of -these programs contain constructs that haven't been covered yet. The -description of the program will give you a good idea of what is going -on, but please read the rest of the manual to become an `awk' expert! - -`awk '{ num_fields = num_fields + NF }' -`` END { print num_fields }''' - This program prints the total number of fields in all input lines. - -`awk 'length($0) > 80'' - This program prints every line longer than 80 characters. The - sole rule has a relational expression as its pattern, and has no - action (so the default action, printing the record, is used). - -`awk 'NF > 0'' - This program prints every line that has at least one field. - This is an easy way to delete blank lines from a file (or - rather, to create a new file similar to the old file but from - which the blank lines have been deleted). - -`awk '{ if (NF > 0) print }'' - This program also prints every line that has at least one field. - Here we allow the rule to match every line, then decide in the - action whether to print. - -`awk 'BEGIN { for (i = 1; i <= 7; i++)' -`` print int(101 * rand()) }''' - This program prints 7 random numbers from 0 to 100, inclusive. - -`ls -l FILES | awk '{ x += $4 } ; END { print "total bytes: " x }'' - This program prints the total number of bytes used by FILES. - -`expand FILE | awk '{ if (x < length()) x = length() }' -`` END { print "maximum line length is " x }''' - This program prints the maximum line length of FILE. The input - is piped through the `expand' program to change tabs into - spaces, so the widths compared are actually the right--margin - columns. - - - -File: gawk-info, Node: Patterns, Next: Actions, Prev: One-liners, Up: Top - -Patterns -******** - -Patterns control the execution of rules: a rule is executed when its -pattern matches the input record. The `awk' language provides -several special patterns that are described in the sections that -follow. Patterns include: - -NULL - The empty pattern, which matches every input record. (*Note The - Empty Pattern: Empty.) - -/REGULAR EXPRESSION/ - A regular expression as a pattern. It matches when the text of - the input record fits the regular expression. (*Note Regular - Expressions as Patterns: Regexp.) - -CONDEXP - A single comparison expression. It matches when it is true. - (*Note Comparison Expressions as Patterns: Comparison Patterns.) - -`BEGIN' -`END' - Special patterns to supply start--up or clean--up information to - `awk'. (*Note Specifying Record Ranges With Patterns: BEGIN/END.) - -PAT1, PAT2 - A pair of patterns separated by a comma, specifying a range of - records. (*Note Specifying Record Ranges With Patterns: Ranges.) - -CONDEXP1 BOOLEAN CONDEXP2 - A "compound" pattern, which combines expressions with the - operators `and', `&&', and `or', `||'. (*Note Boolean - Operators and Patterns: Boolean.) - -! CONDEXP - The pattern CONDEXP is evaluated. Then the `!' performs a - boolean ``not'' or logical negation operation; if the input line - matches the pattern in CONDEXP then the associated action is - *not* executed. If the input line did not match that pattern, - then the action *is* executed. (*Note Boolean Operators and - Patterns: Boolean.) - -(EXPR) - Parentheses may be used to control how operators nest. - -PAT1 ? PAT2 : PAT3 - The first pattern is evaluated. If it is true, the input line - is tested against the second pattern, otherwise it is tested - against the third. (*Note Conditional Patterns: Conditional - Patterns.) - -* Menu: - -The following subsections describe these forms in detail: - -* Empty:: The empty pattern, which matches every record. - -* Regexp:: Regular expressions such as `/foo/'. - -* Comparison Patterns:: Comparison expressions such as `$1 > 10'. - -* Boolean:: Combining comparison expressions. - -* Ranges:: Using pairs of patterns to specify record ranges. - -* BEGIN/END:: Specifying initialization and cleanup rules. - -* Conditional Patterns:: Patterns such as `pat1 ? pat2 : pat3'. - - - -File: gawk-info, Node: Empty, Next: Regexp, Up: Patterns - -The Empty Pattern -================= - -An empty pattern is considered to match *every* input record. For -example, the program: - - awk '{ print $1 }' BBS-list - -prints just the first field of every record. - - - -File: gawk-info, Node: Regexp, Next: Comparison Patterns, Prev: Empty, Up: Patterns - -Regular Expressions as Patterns -=============================== - -A "regular expression", or "regexp", is a way of describing classes -of strings. When enclosed in slashes (`/'), it makes an `awk' -pattern that matches every input record that contains a match for the -regexp. - -The simplest regular expression is a sequence of letters, numbers, or -both. Such a regexp matches any string that contains that sequence. -Thus, the regexp `foo' matches any string containing `foo'. (More -complicated regexps let you specify classes of similar strings.) - -* Menu: - -* Usage: Regexp Usage. How regexps are used in patterns. -* Operators: Regexp Operators. How to write a regexp. - - - -File: gawk-info, Node: Regexp Usage, Next: Regexp Operators, Up: Regexp - -How to use Regular Expressions ------------------------------- - -When you enclose `foo' in slashes, you get a pattern that matches a -record that contains `foo'. For example, this prints the second -field of each record that contains `foo' anywhere: - - awk '/foo/ { print $2 }' BBS-list - -Regular expressions can also be used in comparison expressions. Then -you can specify the string to match against; it need not be the -entire current input record. These comparison expressions can be -used as patterns or in `if' and `while' statements. - -`EXP ~ /REGEXP/' - This is true if the expression EXP (taken as a character string) - is matched by REGEXP. The following example matches, or - selects, all input records with the letter `J' in the first field: - - awk '$1 ~ /J/' inventory-shipped - - So does this: - - awk '{ if ($1 ~ /J/) print }' inventory-shipped - -`EXP !~ /REGEXP/' - This is true if the expression EXP (taken as a character string) - is *not* matched by REGEXP. The following example matches, or - selects, all input records whose first field *does not* contain - the letter `J': - - awk '$1 !~ /J/' inventory-shipped - -The right hand side of a `~' or `!~' operator need not be a constant -regexp (i.e. a string of characters between `/'s). It can also be -"computed", or "dynamic". For example: - - identifier = "[A-Za-z_][A-Za-z_0-9]+" - $0 ~ identifier - -sets `identifier' to a regexp that describes `awk' variable names, -and tests if the input record matches this regexp. - -A dynamic regexp may actually be any expression. The expression is -evaluated, and the result is treated as a string that describes a -regular expression. - - - -File: gawk-info, Node: Regexp Operators, Prev: Regexp Usage, Up: Regexp - -Regular Expression Operators ----------------------------- - -You can combine regular expressions with the following characters, -called "regular expression operators", or "metacharacters", to -increase the power and versatility of regular expressions. This is a -table of metacharacters: - -`\' - This is used to suppress the special meaning of a character when - matching. For example: - - \$ - - matches the character `$'. - -`^' - This matches the beginning of the string or the beginning of a - line within the string. For example: - - ^@chapter - - matches the `@chapter' at the beginning of a string, and can be - used to identify chapter beginnings in Texinfo source files. - -`$' - This is similar to `^', but it matches only at the end of a - string or the end of a line within the string. For example: - - /p$/ - - as a pattern matches a record that ends with a `p'. - -`.' - This matches any single character except a newline. For example: - - .P - - matches any single character followed by a `P' in a string. - Using concatenation we can make regular expressions like `U.A', - which matches any three--character string that begins with `U' - and ends with `A'. - -`[...]' - This is called a "character set". It matches any one of a group - of characters that are enclosed in the square brackets. For - example: - - [MVX] - - matches any of the characters `M', `V', or `X' in a string. - - Ranges of characters are indicated by using a hyphen between the - beginning and ending characters, and enclosing the whole thing - in brackets. For example: - - [0-9] - - matches any string that contains a digit. - - Note that special patterns have to be followed to match the - characters, `]', `-', and `^' when they are enclosed in the - square brackets. To match a `]', make it the first character in - the set. For example: - - []d] - - matches either `]', or `d'. - - To match `-', write it as `--', which is a range containing only - `-'. You may also make the `-' be the first or last character - in the set. To match `^', make it any character except the - first one of a set. - -`[^ ...]' - This is the "complemented character set". The first character - after the `[' *must* be a `^'. This matches any characters - *except* those in the square brackets. For example: - - [^0-9] - - matches any characters that are not digits. - -`|' - This is the "alternation operator" and it is used to specify - alternatives. For example: - - ^P|[0-9] - - matches any string that matches either `^P' or `[0-9]'. This - means it matches any string that contains a digit or starts with - `P'. - -`(...)' - Parentheses are used for grouping in regular expressions as in - arithmetic. They can be used to concatenate regular expressions - containing the alternation operator, `|'. - -`*' - This symbol means that the preceding regular expression is to be - repeated as many times as possible to find a match. For example: - - ph* - - applies the `*' symbol to the preceding `h' and looks for - matches to one `p' followed by any number of `h''s. This will - also match just `p' if no `h''s are present. - - The `*' means repeat the *smallest* possible preceding - expression in order to find a match. The `awk' language - processes a `*' by matching as many repetitions as can be found. - For example: - - awk '/\(c[ad][ad]*r x\)/ { print }' sample - - matches every record in the input containing a string of the - form `(car x)', `(cdr x)', `(cadr x)', and so on. - -`+' - This symbol is similar to `*', but the preceding expression must - be matched at least once. This means that: - - wh+y - - would match `why' and `whhy' but not `wy', whereas `wh*y' would - match all three of these strings. And this is a simpler way of - writing the last `*' example: - - awk '/\(c[ad]+r x\)/ { print }' sample - -`?' - This symbol is similar to `*', but the preceding expression can - be matched once or not at all. For example: - - fe?d - - will match `fed' or `fd', but nothing else. - -In regular expressions, the `*', `+', and `?' operators have the -highest precedence, followed by concatenation, and finally by `|'. -As in arithmetic, parentheses can change how operators are grouped. - -Any other character stands for itself. However, it is important to -note that case in regular expressions *is* significant, both when -matching ordinary (i.e. non--metacharacter) characters, and inside -character sets. Thus a `w' in a regular expression matches only a -lower case `w' and not either an uppercase or lowercase `w'. When -you want to do a case--independent match, you have to use a character -set: `[Ww]'. - - - -File: gawk-info, Node: Comparison Patterns, Next: Ranges, Prev: Regexp, Up: Patterns - -Comparison Expressions as Patterns -================================== - -"Comparison patterns" use "relational operators" to compare strings -or numbers. The relational operators are the same as in C. Here is -a table of them: - -`X < Y' - True if X is less than Y. - -`X <= Y' - True if X is less than or equal to Y. - -`X > Y' - True if X is greater than Y. - -`X >= Y' - True if X is greater than or equal to Y. - -`X == Y' - True if X is equal to Y. - -`X != Y' - True if X is not equal to Y. - -Comparison expressions can be used as patterns to control whether a -rule is executed. The expression is evaluated for each input record -read, and the pattern is considered matched if the condition is "true". - -The operands of a relational operator are compared as numbers if they -are both numbers. Otherwise they are converted to, and compared as, -strings (*note Conversion::.). Strings are compared by comparing the -first character of each, then the second character of each, and so on. -Thus, `"10"' is less than `"9"'. - -The following example prints the second field of each input record -whose first field is precisely `foo'. - - awk '$1 == "foo" { print $2 }' BBS-list - -Contrast this with the following regular expression match, which -would accept any record with a first field that contains `foo': - - awk '$1 ~ "foo" { print $2 }' BBS-list - - - -File: gawk-info, Node: Ranges, Next: BEGIN/END, Prev: Comparison Patterns, Up: Patterns - -Specifying Record Ranges With Patterns -====================================== - -A "range pattern" is made of two patterns separated by a comma: -`BEGPAT, ENDPAT'. It matches ranges of consecutive input records. -The first pattern BEGPAT controls where the range begins, and the -second one ENDPAT controls where it ends. - -They work as follows: BEGPAT is matched against every input record; -when a record matches BEGPAT, the range pattern becomes "turned on". -The range pattern matches this record. As long as it stays turned -on, it automatically matches every input record read. But meanwhile, -ENDPAT is matched against every input record, and when it matches, -the range pattern is turned off again for the following record. Now -we go back to checking BEGPAT against each record. For example: - - awk '$1 == "on", $1 == "off"' - -prints every record between on/off pairs, inclusive. - -The record that turns on the range pattern and the one that turns it -off both match the range pattern. If you don't want to operate on -these records, you can write `if' statements in the rule's action to -distinguish them. - -It is possible for a pattern to be turned both on and off by the same -record, if both conditions are satisfied by that record. Then the -action is executed for just that record. - - - -File: gawk-info, Node: BEGIN/END, Next: Boolean, Prev: Ranges, Up: Patterns - -`BEGIN' and `END' Special Patterns -================================== - -`BEGIN' and `END' are special patterns. They are not used to match -input records. Rather, they are used for supplying start--up or -clean--up information to your `awk' script. A `BEGIN' rule is -executed, once, before the first input record has been read. An -`END' rule is executed, once, after all the input has been read. For -example: - - awk 'BEGIN { print "Analysis of ``foo'' program" } - /foo/ { ++foobar } - END { print "``foo'' appears " foobar " times." }' BBS-list - -This program finds out how many times the string `foo' appears in the -input file `BBS-list'. The `BEGIN' pattern prints out a title for -the report. There is no need to use the `BEGIN' pattern to -initialize the counter `foobar' to zero, as `awk' does this for us -automatically (*note Variables::.). The second rule increments the -variable `foobar' every time a record containing the pattern `foo' is -read. The last rule prints out the value of `foobar' at the end of -the run. - -The special patterns `BEGIN' and `END' do not combine with other -kinds of patterns. - -An `awk' program may have multiple `BEGIN' and/or `END' rules. The -contents of multiple `BEGIN' or `END' rules are treated as if they -had been enclosed in a single rule, in the order that the rules are -encountered in the `awk' program. (This feature was introduced with -the new version of `awk'.) - -Multiple `BEGIN' and `END' sections are also useful for writing -library functions that need to do initialization and/or cleanup of -their own. Note that the order in which library functions are named -on the command line will affect the order in which their `BEGIN' and -`END' rules will be executed. Therefore you have to be careful how -you write your library functions. (*Note Command Line::, for more -information on using library functions.) - -If an `awk' program only has a `BEGIN' rule, and no other rules, then -the program will exit after the `BEGIN' rule has been run. Older -versions of `awk' used to read their input until end of file was -seen. However, if an `END' rule exists as well, then the input will -be read, even if there are no other rules in the program. - -`BEGIN' and `END' rules must have actions; there is no default action -for these rules since there is no current record when they run. - - - -File: gawk-info, Node: Boolean, Next: Conditional Patterns, Prev: BEGIN/END, Up: Patterns - -Boolean Operators and Patterns -============================== - -A boolean pattern is a combination of other patterns using the -boolean operators ``or'' (`||'), ``and'' (`&&'), and ``not'' (`!'), -along with parentheses to control nesting. Whether the boolean -pattern matches an input record is computed from whether its -subpatterns match. - -The subpatterns of a boolean pattern can be regular expressions, -matching expressions, comparisons, or other boolean combinations of -such. Range patterns cannot appear inside boolean operators, since -they don't make sense for classifying a single record, and neither -can the special patterns `BEGIN' and `END', which never match any -input record. - -Here are descriptions of the three boolean operators. - -`PAT1 && PAT2' - Matches if both PAT1 and PAT2 match by themselves. For example, - the following command prints all records in the input file - `BBS-list' that contain both `2400' and `foo'. - - awk '/2400/ && /foo/' BBS-list - - Whether PAT2 matches is tested only if PAT1 succeeds. This can - make a difference when PAT2 contains expressions that have side - effects: in the case of `/foo/ && ($2 == bar++)', the variable - `bar' is not incremented if there is no `foo' in the record. - -`PAT1 || PAT2' - Matches if at least one of PAT1 and PAT2 matches the current - input record. For example, the following command prints all - records in the input file `BBS-list' that contain *either* - `2400' or `foo', or both. - - awk '/2400/ || /foo/' BBS-list - - Whether PAT2 matches is tested only if PAT1 fails to match. - This can make a difference when PAT2 contains expressions that - have side effects. - -`!PAT' - Matches if PAT does not match. For example, the following - command prints all records in the input file `BBS-list' that do - *not* contain the string `foo'. - - awk '! /foo/' BBS-list - -Note that boolean patterns are built from other patterns just as -boolean expressions are built from other expressions (*note Boolean -Ops::.). Any boolean expression is also a valid boolean pattern. -But the converse is not true: simple regular expression patterns such -as `/foo/' are not allowed in boolean expressions. Regular -expressions can appear in boolean expressions only in conjunction -with the matching operators, `~' and `!~'. - - - -File: gawk-info, Node: Conditional Patterns, Prev: Boolean, Up: Patterns - -Conditional Patterns -==================== - -Patterns may use a "conditional expression" much like the conditional -expression of the C language. This takes the form: - - PAT1 ? PAT2 : PAT3 - -The first pattern is evaluated. If it evaluates to TRUE, then the -input record is tested against PAT2. Otherwise it is tested against -PAT3. The conditional pattern matches if PAT2 or PAT3 (whichever one -is selected) matches. - - - -File: gawk-info, Node: Actions, Next: Expressions, Prev: Patterns, Up: Top - -Actions: The Basics -******************* - -The "action" part of an `awk' rule tells `awk' what to do once a -match for the pattern is found. An action consists of one or more -`awk' "statements", enclosed in curly braces (`{' and `}'). The -curly braces must be used even if the action contains only one -statement, or even if it contains no statements at all. Action -statements are separated by newlines or semicolons. - -Besides the print statements already covered (*note Printing::.), -there are four kinds of action statements: expressions, control -statements, compound statements, and function definitions. - - * "Expressions" include assignments, arithmetic, function calls, - and more (*note Expressions::.). - - * "Control statements" specify the control flow of `awk' programs. - The `awk' language gives you C--like constructs (`if', `for', - `while', and so on) as well as a few special ones (*note - Statements::.). - - * A "compound statement" is just one or more `awk' statements - enclosed in curly braces. This way you can group several - statements to form the body of an `if' or similar statement. - - * You can define "user--defined functions" for use elsewhere in - the `awk' program (*note User-defined::.). - - - -File: gawk-info, Node: Expressions, Next: Statements, Prev: Actions, Up: Top - -Actions: Expressions -******************** - -Expressions are the basic building block of `awk' actions. An -expression evaluates to a value, which you can print, test, store in -a variable or pass to a function. - -But, beyond that, an expression can assign a new value to a variable -or a field, with an assignment operator. - -An expression can serve as a statement on its own. Most other action -statements are made up of various combinations of expressions. As in -other languages, expressions in `awk' include variables, array -references, constants, and function calls, as well as combinations of -these with various operators. - -* Menu: - -* Constants:: String and numeric constants. -* Variables:: Variables give names to values for future use. -* Fields:: Field references such as `$1' are also expressions. -* Arrays:: Array element references are expressions. - -* Arithmetic Ops:: Arithmetic operations (`+', `-', etc.) -* Concatenation:: Concatenating strings. -* Comparison Ops:: Comparison of numbers and strings with `<', etc. -* Boolean Ops:: Combining comparison expressions using boolean operators - `||' (``or''), `&&' (``and'') and `!' (``not''). - -* Assignment Ops:: Changing the value of a variable or a field. -* Increment Ops:: Incrementing the numeric value of a variable. - -* Conversion:: The conversion of strings to numbers and vice versa. -* Conditional Exp:: Conditional expressions select between two subexpressions - under control of a third subexpression. -* Function Calls:: A function call is an expression. - - - -File: gawk-info, Node: Constants, Next: Variables, Up: Expressions - -Constant Expressions -==================== - -There are two types of constants: numeric constants and string -constants. - -The "numeric constant" is a number. This number can be an integer, a -decimal fraction, or a number in scientific (exponential) notation. -Note that all numeric values are represented within `awk' in -double--precision floating point. Here are some examples of numeric -constants, which all have the same value: - - 105 - 1.05e+2 - 1050e-1 - -A string constant consists of a sequence of characters enclosed in -double--quote marks. For example: - - "parrot" - -represents the string constant `parrot'. Strings in `gawk' can be of -any length and they can contain all the possible 8--bit ASCII -characters including ASCII NUL. Other `awk' implementations may have -difficulty with some character codes. - -Some characters cannot be included literally in a string. You -represent them instead with "escape sequences", which are character -sequences beginning with a backslash (`\'). - -One use of the backslash is to include double--quote characters in a -string. Since a plain double--quote would end the string, you must -use `\"'. Backslash itself is another character that can't be -included normally; you write `\\' to put one backslash in the string. - -Another use of backslash is to represent unprintable characters such -as newline. While there is nothing to stop you from writing these -characters directly in an `awk' program, they may look ugly. - -`\b' - Represents a backspaced, H'. - -`\f' - Represents a formfeed, L'. - -`\n' - Represents a newline, J'. - -`\r' - Represents a carriage return, M'. - -`\t' - Represents a horizontal tab, I'. - -`\v' - Represents a vertical tab, K'. - -`\NNN' - Represents the octal value NNN, where NNN is one to three digits - between 0 and 7. For example, the code for the ASCII ESC - (escape) character is `\033'. - - - -File: gawk-info, Node: Variables, Next: Arithmetic Ops, Prev: Constants, Up: Expressions - -Variables -========= - -Variables let you give names to values and refer to them later. You -have already seen variables in many of the examples. The name of a -variable must be a sequence of letters, digits and underscores, but -it may not begin with a digit. Case is significant in variable -names; `a' and `A' are distinct variables. - -A variable name is a valid expression by itself; it represents the -variable's current value. Variables are given new values with -"assignment operators" and "increment operators". *Note Assignment -Ops::. - -A few variables have special built--in meanings, such as `FS', the -field separator, and `NF', the number of fields in the current input -record. *Note Special::, for a list of them. Special variables can -be used and assigned just like all other variables, but their values -are also used or changed automatically by `awk'. Each special -variable's name is made entirely of upper case letters. - -Variables in `awk' can be assigned either numeric values or string -values. By default, variables are initialized to the null string, -which has the numeric value zero. So there is no need to -``initialize'' each variable explicitly in `awk', the way you would -need to do in C or most other traditional programming languages. - - - -File: gawk-info, Node: Arithmetic Ops, Next: Concatenation, Prev: Variables, Up: Expressions - -Arithmetic Operators -==================== - -The `awk' language uses the common arithmetic operators when -evaluating expressions. All of these arithmetic operators follow -normal precedence rules, and work as you would expect them to. This -example divides field 3 by field 4, adds field 2, stores the result -into field 1, and prints the results: - - awk '{ $1 = $2 + $3 / $4; print }' inventory-shipped - -The arithmetic operators in `awk' are: - -`X + Y' - Addition. - -`X - Y' - Subtraction. - -`- X' - Negation. - -`X / Y' - Division. Since all numbers in `awk' are double--precision - floating point, the result is not rounded to an integer: `3 / 4' - has the value 0.75. - -`X * Y' - Multiplication. - -`X % Y' - Remainder. The quotient is rounded toward zero to an integer, - multiplied by Y and this result is subtracted from X. This - operation is sometimes known as ``trunc--mod''. The following - relation always holds: - - `b * int(a / b) + (a % b) == a' - - One undesirable effect of this definition of remainder is that X - % Y is negative if X is negative. Thus, - - -17 % 8 = -1 - -`X ^ Y' -`X ** Y' - Exponentiation: X raised to the Y power. `2 ^ 3' has the value - 8. The character sequence `**' is equivalent to `^'. - - - -File: gawk-info, Node: Concatenation, Next: Comparison Ops, Prev: Arithmetic Ops, Up: Expressions - -String Concatenation -==================== - -There is only one string operation: concatenation. It does not have -a specific operator to represent it. Instead, concatenation is -performed by writing expressions next to one another, with no -operator. For example: - - awk '{ print "Field number one: " $1 }' BBS-list - -produces, for the first record in `BBS-list': - - Field number one: aardvark - -If you hadn't put the space after the `:', the line would have run -together. For example: - - awk '{ print "Field number one:" $1 }' BBS-list - -produces, for the first record in `BBS-list': - - Field number one:aardvark - - - -File: gawk-info, Node: Comparison Ops, Next: Boolean Ops, Prev: Concatenation, Up: Expressions - -Comparison Expressions -====================== - -"Comparison expressions" use "relational operators" to compare -strings or numbers. The relational operators are the same as in C. -Here is a table of them: - -`X < Y' - True if X is less than Y. - -`X <= Y' - True if X is less than or equal to Y. - -`X > Y' - True if X is greater than Y. - -`X >= Y' - True if X is greater than or equal to Y. - -`X == Y' - True if X is equal to Y. - -`X != Y' - True if X is not equal to Y. - -`X ~ REGEXP' - True if regexp REGEXP matches the string X. - -`X !~ REGEXP' - True if regexp REGEXP does not match the string X. - -`SUBSCRIPT in ARRAY' - True if array ARRAY has an element with the subscript SUBSCRIPT. - -Comparison expressions have the value 1 if true and 0 if false. - -The operands of a relational operator are compared as numbers if they -are both numbers. Otherwise they are converted to, and compared as, -strings (*note Conversion::.). Strings are compared by comparing the -first character of each, then the second character of each, and so on. -Thus, `"10"' is less than `"9"'. - -For example, - - $1 == "foo" - -has the value of 1, or is true, if the first field of the current -input record is precisely `foo'. By contrast, - - $1 ~ /foo/ - -has the value 1 if the first field contains `foo'. - - - -File: gawk-info, Node: Boolean Ops, Next: Assignment Ops, Prev: Comparison Ops, Up: Expressions - -Boolean Operators -================= - -A boolean expression is combination of comparison expressions or -matching expressions, using the boolean operators ``or'' (`||'), -``and'' (`&&'), and ``not'' (`!'), along with parentheses to control -nesting. The truth of the boolean expression is computed by -combining the truth values of the component expressions. - -Boolean expressions can be used wherever comparison and matching -expressions can be used. They can be used in `if' and `while' -statements. They have numeric values (1 if true, 0 if false). - -In addition, every boolean expression is also a valid boolean -pattern, so you can use it as a pattern to control the execution of -rules. - -Here are descriptions of the three boolean operators, with an example -of each. It may be instructive to compare these examples with the -analogous examples of boolean patterns (*note Boolean::.), which use -the same boolean operators in patterns instead of expressions. - -`BOOLEAN1 && BOOLEAN2' - True if both BOOLEAN1 and BOOLEAN2 are true. For example, the - following statement prints the current input record if it - contains both `2400' and `foo'. - - if ($0 ~ /2400/ && $0 ~ /foo/) print - - The subexpression BOOLEAN2 is evaluated only if BOOLEAN1 is - true. This can make a difference when BOOLEAN2 contains - expressions that have side effects: in the case of `$0 ~ /foo/ - && ($2 == bar++)', the variable `bar' is not incremented if - there is no `foo' in the record. - -`BOOLEAN1 || BOOLEAN2' - True if at least one of BOOLEAN1 and BOOLEAN2 is true. For - example, the following command prints all records in the input - file `BBS-list' that contain *either* `2400' or `foo', or both. - - awk '{ if ($0 ~ /2400/ || $0 ~ /foo/) print }' BBS-list - - The subexpression BOOLEAN2 is evaluated only if BOOLEAN1 is - true. This can make a difference when BOOLEAN2 contains - expressions that have side effects. - -`!BOOLEAN' - True if BOOLEAN is false. For example, the following program - prints all records in the input file `BBS-list' that do *not* - contain the string `foo'. - - awk '{ if (! ($0 ~ /foo/)) print }' BBS-list - - - -File: gawk-info, Node: Assignment Ops, Next: Increment Ops, Prev: Boolean Ops, Up: Expressions - -Assignment Operators -==================== - -An "assignment" is an expression that stores a new value into a -variable. For example, let's assign the value 1 to the variable `z': - - z = 1 - -After this expression is executed, the variable `z' has the value 1. -Whatever old value `z' had before the assignment is forgotten. - -The `=' sign is called an "assignment operator". It is the simplest -assignment operator because the value of the right--hand operand is -stored unchanged. - -The left--hand operand of an assignment can be a variable (*note -Variables::.), a field (*note Changing Fields::.) or an array element -(*note Arrays::.). These are all called "lvalues", which means they -can appear on the left side of an assignment operator. The -right--hand operand may be any expression; it produces the new value -which the assignment stores in the specified variable, field or array -element. - -Assignments can store string values also. For example, this would -store the value `"this food is good"' in the variable `message': - - thing = "food" - predicate = "good" - message = "this " thing " is " predicate - -(This also illustrates concatenation of strings.) - -It is important to note that variables do *not* have permanent types. -The type of a variable is simply the type of whatever value it -happens to hold at the moment. In the following program fragment, -the variable `foo' has a numeric value at first, and a string value -later on: - - foo = 1 - print foo - foo = "bar" - print foo - -When the second assignment gives `foo' a string value, the fact that -it previously had a numeric value is forgotten. - -An assignment is an expression, so it has a value: the same value -that is assigned. Thus, `z = 1' as an expression has the value 1. -One consequence of this is that you can write multiple assignments -together: - - x = y = z = 0 - -stores the value 0 in all three variables. It does this because the -value of `z = 0', which is 0, is stored into `y', and then the value -of `y = z = 0', which is 0, is stored into `x'. - -You can use an assignment anywhere an expression is called for. For -example, it is valid to write `x != (y = 1)' to set `y' to 1 and then -test whether `x' equals 1. But this style tends to make programs -hard to read; except in a one--shot program, you should rewrite it to -get rid of such nesting of assignments. This is never very hard. - -Aside from `=', there are several other assignment operators that do -arithmetic with the old value of the variable. For example, the -operator `+=' computes a new value by adding the right--hand value to -the old value of the variable. Thus, the following assignment adds 5 -to the value of `foo': - - foo += 5 - -This is precisely equivalent to the following: - - foo = foo + 5 - -Use whichever one makes the meaning of your program clearer. - -Here is a table of the arithmetic assignment operators. In each -case, the right--hand operand is an expression whose value is -converted to a number. - -`LVALUE += INCREMENT' - Adds INCREMENT to the value of LVALUE to make the new value of - LVALUE. - -`LVALUE -= DECREMENT' - Subtracts DECREMENT from the value of LVALUE. - -`LVALUE *= COEFFICIENT' - Multiplies the value of LVALUE by COEFFICIENT. - -`LVALUE /= QUOTIENT' - Divides the value of LVALUE by QUOTIENT. - -`LVALUE %= MODULUS' - Sets LVALUE to its remainder by MODULUS. - -`LVALUE ^= POWER' -`LVALUE **= POWER' - Raises LVALUE to the power POWER. - - - -File: gawk-info, Node: Increment Ops, Next: Conversion, Prev: Assignment Ops, Up: Expressions - -Increment Operators -=================== - -"Increment operators" increase or decrease the value of a variable by -1. You could do the same thing with an assignment operator, so the -increment operators add no power to the `awk' language; but they are -convenient abbreviations for something very common. - -The operator to add 1 is written `++'. There are two ways to use -this operator: pre--incrementation and post--incrementation. - -To pre--increment a variable V, write `++V'. This adds 1 to the -value of V and that new value is also the value of this expression. -The assignment expression `V += 1' is completely equivalent. - -Writing the `++' after the variable specifies post--increment. This -increments the variable value just the same; the difference is that -the value of the increment expression itself is the variable's *old* -value. Thus, if `foo' has value 4, then the expression `foo++' has -the value 4, but it changes the value of `foo' to 5. - -The post--increment `foo++' is nearly equivalent to writing `(foo += -1) - 1'. It is not perfectly equivalent because all numbers in `awk' -are floating point: in floating point, `foo + 1 - 1' does not -necessarily equal `foo'. But the difference will be minute as long -as you stick to numbers that are fairly small (less than a trillion). - -Any lvalue can be incremented. Fields and array elements are -incremented just like variables. - -The decrement operator `--' works just like `++' except that it -subtracts 1 instead of adding. Like `++', it can be used before the -lvalue to pre--decrement or after it to post--decrement. - -Here is a summary of increment and decrement expressions. - -`++LVALUE' - This expression increments LVALUE and the new value becomes the - value of this expression. - -`LVALUE++' - This expression causes the contents of LVALUE to be incremented. - The value of the expression is the *old* value of LVALUE. - -`--LVALUE' - Like `++LVALUE', but instead of adding, it subtracts. It - decrements LVALUE and delivers the value that results. - -`LVALUE--' - Like `LVALUE++', but instead of adding, it subtracts. It - decrements LVALUE. The value of the expression is the *old* - value of LVALUE. - - - -File: gawk-info, Node: Conversion, Next: Conditional Exp, Prev: Increment Ops, Up: Expressions - -Conversion of Strings and Numbers -================================= - -Strings are converted to numbers, and numbers to strings, if the -context of your `awk' statement demands it. For example, if the -values of `foo' or `bar' in the expression `foo + bar' happen to be -strings, they are converted to numbers before the addition is -performed. If numeric values appear in string concatenation, they -are converted to strings. Consider this: - - two = 2; three = 3 - print (two three) + 4 - -This eventually prints the (numeric) value `27'. The numeric -variables `two' and `three' are converted to strings and concatenated -together, and the resulting string is converted back to a number -before adding `4'. The resulting numeric value `27' is printed. - -If, for some reason, you need to force a number to be converted to a -string, concatenate the null string with that number. To force a -string to be converted to a number, add zero to that string. Strings -that can't be interpreted as valid numbers are given the numeric -value zero. - -The exact manner in which numbers are converted into strings is -controlled by the `awk' special variable `OFMT' (*note Special::.). -Numbers are converted using a special version of the `sprintf' -function (*note Built-in::.) with `OFMT' as the format specifier. - -`OFMT''s default value is `"%.6g"', which prints a value with at -least six significant digits. You might want to change it to specify -more precision, if your version of `awk' uses double precision -arithmetic. Double precision on most modern machines gives you 16 or -17 decimal digits of precision. - -Strange results can happen if you set `OFMT' to a string that doesn't -tell `sprintf' how to format floating point numbers in a useful way. -For example, if you forget the `%' in the format, all numbers will be -converted to the same constant string. - - - -File: gawk-info, Node: Conditional Exp, Next: Function Calls, Prev: Conversion, Up: Expressions - -Conditional Expressions -======================= - -A "conditional expression" is a special kind of expression with three -operands. It allows you to use one expression's value to select one -of two other expressions. - -The conditional expression looks the same as in the C language: - - SELECTOR ? IF-TRUE-EXP : IF-FALSE-EXP - -There are three subexpressions. The first, SELECTOR, is always -computed first. If it is ``true'' (not zero) then IF-TRUE-EXP is -computed next and its value becomes the value of the whole expression. -Otherwise, IF-FALSE-EXP is computed next and its value becomes the -value of the whole expression. - -For example, this expression produces the absolute value of `x': - - x > 0 ? x : -x - -Each time the conditional expression is computed, exactly one of -IF-TRUE-EXP and IF-FALSE-EXP is computed; the other is ignored. This -is important when the expressions contain side effects. For example, -this conditional expression examines element `i' of either array `a' -or array `b', and increments `i'. - - x == y ? a[i++] : b[i++] - -This is guaranteed to increment `i' exactly once, because each time -one or the other of the two increment expressions will be executed -and the other will not be. - - - -File: gawk-info, Node: Function Calls, Prev: Conditional Exp, Up: Expressions - -Function Calls -============== - -A "function" is a name for a particular calculation. Because it has -a name, you can ask for it by name at any point in the program. For -example, the function `sqrt' computes the square root of a number. - -A fixed set of functions are "built in", which means they are -available in every `awk' program. The `sqrt' function is one of -these. *Note Built-in::, for a list of built--in functions and their -descriptions. In addition, you can define your own functions in the -program for use elsewhere in the same program. *Note User-defined::, -for how to do this. - -The way to use a function is with a "function call" expression, which -consists of the function name followed by a list of "arguments" in -parentheses. The arguments are expressions which give the raw -materials for the calculation that the function will do. When there -is more than one argument, they are separated by commas. If there -are no arguments, write just `()' after the function name. - -*Do not put any space between the function name and the -open--parenthesis!* A user--defined function name looks just like -the name of a variable, and space would make the expression look like -concatenation of a variable with an expression inside parentheses. -Space before the parenthesis is harmless with built--in functions, -but it is best not to get into the habit of using space, lest you do -likewise for a user--defined function one day by mistake. - -Each function needs a particular number of arguments. For example, -the `sqrt' function must be called with a single argument, like this: - - sqrt(ARGUMENT) - -The argument is the number to take the square root of. - -Some of the built--in functions allow you to omit the final argument. -If you do so, they will use a reasonable default. *Note Built-in::, -for full details. If arguments are omitted in calls to user--defined -functions, then those arguments are treated as local variables, -initialized to the null string (*note User-defined::.). - -Like every other expression, the function call has a value, which is -computed by the function based on the arguments you give it. In this -example, the value of `sqrt(ARGUMENT)' is the square root of the -argument. A function can also have side effects, such as assigning -the values of certain variables or doing I/O. - -Here is a command to read numbers, one number per line, and print the -square root of each one: - - awk '{ print "The square root of", $1, "is", sqrt($1) }' - - - -File: gawk-info, Node: Statements, Next: Arrays, Prev: Expressions, Up: Top - -Actions: Statements -******************* - -"Control statements" such as `if', `while', and so on control the -flow of execution in `awk' programs. Most of the control statements -in `awk' are patterned on similar statements in C. - -The simplest kind of statement is an expression. The other kinds of -statements start with special keywords such as `if' and `while', to -distinguish them from simple expressions. - -In all the examples in this chapter, BODY can be either a single -statement or a group of statements. Groups of statements are -enclosed in braces, and separated by newlines or semicolons. - -* Menu: - -* Expressions:: One kind of statement simply computes an expression. - -* If:: Conditionally execute some `awk' statements. - -* While:: Loop until some condition is satisfied. - -* Do:: Do specified action while looping until some - condition is satisfied. - -* For:: Another looping statement, that provides - initialization and increment clauses. - -* Break:: Immediately exit the innermost enclosing loop. - -* Continue:: Skip to the end of the innermost enclosing loop. - -* Next:: Stop processing the current input record. - -* Exit:: Stop execution of `awk'. - - - -File: gawk-info, Node: If, Next: While, Up: Statements - -The `if' Statement -================== - -The `if'-`else' statement is `awk''s decision--making statement. The -`else' part of the statement is optional. - - `if (CONDITION) BODY1 else BODY2' - -Here CONDITION is an expression that controls what the rest of the -statement will do. If CONDITION is true, BODY1 is executed; -otherwise, BODY2 is executed (assuming that the `else' clause is -present). The condition is considered true if it is nonzero or -nonnull. - -Here is an example: - - awk '{ if (x % 2 == 0) - print "x is even" - else - print "x is odd" }' - -In this example, if the statement containing `x' is found to be true -(that is, x is divisible by 2), then the first `print' statement is -executed, otherwise the second `print' statement is performed. - -If the `else' appears on the same line as BODY1, and BODY1 is a -single statement, then a semicolon must separate BODY1 from `else'. -To illustrate this, let's rewrite the previous example: - - awk '{ if (x % 2 == 0) print "x is even"; else - print "x is odd" }' - -If you forget the `;', `awk' won't be able to parse it, and you will -get a syntax error. - -We would not actually write this example this way, because a human -reader might fail to see the `else' if it were not the first thing on -its line. - - - -File: gawk-info, Node: While, Next: Do, Prev: If, Up: Statements - -The `while' Statement -===================== - -In programming, a loop means a part of a program that is (or at least -can be) executed two or more times in succession. - -The `while' statement is the simplest looping statement in `awk'. It -repeatedly executes a statement as long as a condition is true. It -looks like this: - - while (CONDITION) - BODY - -Here BODY is a statement that we call the "body" of the loop, and -CONDITION is an expression that controls how long the loop keeps -running. - -The first thing the `while' statement does is test CONDITION. If -CONDITION is true, it executes the statement BODY. After BODY has -been executed, CONDITION is tested again and this process is repeated -until CONDITION is no longer true. If CONDITION is initially false, -the body of the loop is never executed. - - awk '{ i = 1 - while (i <= 3) { - print $i - i++ - } - }' - -This example prints the first three input fields, one per line. - -The loop works like this: first, the value of `i' is set to 1. Then, -the `while' tests whether `i' is less than or equal to three. This -is the case when `i' equals one, so the `i'-th field is printed. -Then the `i++' increments the value of `i' and the loop repeats. - -When `i' reaches 4, the loop exits. Here BODY is a compound -statement enclosed in braces. As you can see, a newline is not -required between the condition and the body; but using one makes the -program clearer unless the body is a compound statement or is very -simple. - - - -File: gawk-info, Node: Do, Next: For, Prev: While, Up: Statements - -The `do'--`while' Statement -=========================== - -The `do' loop is a variation of the `while' looping statement. The -`do' loop executes the BODY once, then repeats BODY as long as -CONDITION is true. It looks like this: - - do - BODY - while (CONDITION) - -Even if CONDITION is false at the start, BODY is executed at least -once (and only once, unless executing BODY makes CONDITION true). -Contrast this with the corresponding `while' statement: - - while (CONDITION) - BODY - -This statement will not execute BODY even once if CONDITION is false -to begin with. - -Here is an example of a `do' statement: - - awk '{ i = 1 - do { - print $0 - i++ - } while (i <= 10) - }' - -prints each input record ten times. It isn't a very realistic -example, since in this case an ordinary `while' would do just as -well. But this is normal; there is only occasionally a real use for -a `do' statement. - - - -File: gawk-info, Node: For, Next: Break, Prev: Do, Up: Statements - -The `for' Statement -=================== - -The `for' statement makes it more convenient to count iterations of a -loop. The general form of the `for' statement looks like this: - - for (INITIALIZATION; CONDITION; INCREMENT) - BODY - -This statement starts by executing INITIALIZATION. Then, as long as -CONDITION is true, it repeatedly executes BODY and then INCREMENT. -Typically INITIALIZATION sets a variable to either zero or one, -INCREMENT adds 1 to it, and CONDITION compares it against the desired -number of iterations. - -Here is an example of a `for' statement: - - awk '{ for (i = 1; i <= 3; i++) - print $i - }' - -This prints the first three fields of each input record, one field -per line. - -In the `for' statement, BODY stands for any statement, but -INITIALIZATION, CONDITION and INCREMENT are just expressions. You -cannot set more than one variable in the INITIALIZATION part unless -you use a multiple assignment statement such as `x = y = 0', which is -possible only if all the initial values are equal. (But you can -initialize additional variables by writing their assignments as -separate statements preceding the `for' loop.) - -The same is true of the INCREMENT part; to increment additional -variables, you must write separate statements at the end of the loop. -The C compound expression, using C's comma operator, would be useful -in this context, but it is not supported in `awk'. - -Most often, INCREMENT is an increment expression, as in the example -above. But this is not required; it can be any expression whatever. -For example, this statement prints odd numbers from 1 to 100: - - # print odd numbers from 1 to 100 - for (i = 1; i <= 100; i += 2) - print i - -Any of the three expressions following `for' may be omitted if you -don't want it to do anything. Thus, `for (;x > 0;)' is equivalent to -`while (x > 0)'. If the CONDITION part is empty, it is treated as -TRUE, effectively yielding an infinite loop. - -In most cases, a `for' loop is an abbreviation for a `while' loop, as -shown here: - - INITIALIZATION - while (CONDITION) { - BODY - INCREMENT - } - -(The only exception is when the `continue' statement (*note -Continue::.) is used inside the loop; changing a `for' statement to a -`while' statement in this way can change the effect of the `continue' -statement inside the loop.) - -The `awk' language has a `for' statement in addition to a `while' -statement because often a `for' loop is both less work to type and -more natural to think of. Counting the number of iterations is very -common in loops. It can be easier to think of this counting as part -of looping rather than as something to do inside the loop. - -The next section has more complicated examples of `for' loops. - -There is an alternate version of the `for' loop, for iterating over -all the indices of an array: - - for (i in array) - PROCESS array[i] - -*Note Arrays::, for more information on this version of the `for' loop. - - - -File: gawk-info, Node: Break, Next: Continue, Prev: For, Up: Statements - -The `break' Statement -===================== - -The `break' statement jumps out of the innermost `for', `while', or -`do'--`while' loop that encloses it. The following example finds the -smallest divisor of any number, and also identifies prime numbers: - - awk '# find smallest divisor of num - { num = $1 - for (div = 2; div*div <= num; div++) - if (num % div == 0) - break - if (num % div == 0) - printf "Smallest divisor of %d is %d\n", num, div - else - printf "%d is prime\n", num }' - -When the remainder is zero in the first `if' statement, `awk' -immediately "breaks" out of the containing `for' loop. This means -that `awk' proceeds immediately to the statement following the loop -and continues processing. (This is very different from the `exit' -statement (*note Exit::.) which stops the entire `awk' program.) - -Here is another program equivalent to the previous one. It -illustrates how the CONDITION of a `for' or `while' could just as -well be replaced with a `break' inside an `if': - - awk '# find smallest divisor of num - { num = $1 - for (div = 2; ; div++) { - if (num % div == 0) { - printf "Smallest divisor of %d is %d\n", num, div - break - } - if (div*div > num) { - printf "%d is prime\n", num - break - } - } - }' - - - -File: gawk-info, Node: Continue, Next: Next, Prev: Break, Up: Statements - -The `continue' Statement -======================== - -The `continue' statement, like `break', is used only inside `for', -`while', and `do'--`while' loops. It skips over the rest of the loop -body, causing the next cycle around the loop to begin immediately. -Contrast this with `break', which jumps out of the loop altogether. -Here is an example: - - # print names that don't contain the string "ignore" - - # first, save the text of each line - { names[NR] = $0 } - - # print what we're interested in - END { - for (x in names) { - if (names[x] ~ /ignore/) - continue - print names[x] - } - } - -If any of the input records contain the string `ignore', this example -skips the print statement and continues back to the first statement -in the loop. - -This isn't a practical example of `continue', since it would be just -as easy to write the loop like this: - - for (x in names) - if (x !~ /ignore/) - print x - -The `continue' statement causes `awk' to skip the rest of what is -inside a `for' loop, but it resumes execution with the increment part -of the `for' loop. The following program illustrates this fact: - - awk 'BEGIN { - for (x = 0; x <= 20; x++) { - if (x == 5) - continue - printf ("%d ", x) - } - print "" - }' - -This program prints all the numbers from 0 to 20, except for 5, for -which the `printf' is skipped. Since the increment `x++' is not -skipped, `x' does not remain stuck at 5. - - - -File: gawk-info, Node: Next, Next: Exit, Prev: Continue, Up: Statements - -The `next' Statement -==================== - -The `next' statement forces `awk' to immediately stop processing the -current record and go on to the next record. This means that no -further rules are executed for the current record. The rest of the -current rule's action is not executed either. - -Contrast this with the effect of the `getline' function (*note -Getline::.). That too causes `awk' to read the next record -immediately, but it does not alter the flow of control in any way. -So the rest of the current action executes with a new input record. - -At the grossest level, `awk' program execution is a loop that reads -an input record and then tests each rule pattern against it. If you -think of this loop as a `for' statement whose body contains the -rules, then the `next' statement is analogous to a `continue' -statement: it skips to the end of the body of the loop, and executes -the increment (which reads another record). - -For example, if your `awk' program works only on records with four -fields, and you don't want it to fail when given bad input, you might -use the following rule near the beginning of the program: - - NF != 4 { - printf ("line %d skipped: doesn't have 4 fields", FNR) > "/dev/tty" - next - } - -so that the following rules will not see the bad record. The error -message is redirected to `/dev/tty' (the terminal), so that it won't -get lost amid the rest of the program's regular output. - - - -File: gawk-info, Node: Exit, Prev: Next, Up: Statements - -The `exit' Statement -==================== - -The `exit' statement causes `awk' to immediately stop executing the -current rule and to stop processing input; any remaining input is -ignored. - -If an `exit' statement is executed from a `BEGIN' rule the program -stops processing everything immediately. No input records will be -read. However, if an `END' rule is present, it will be executed -(*note BEGIN/END::.). - -If `exit' is used as part of an `END' rule, it causes the program to -stop immediately. - -An `exit' statement that is part an ordinary rule (that is, not part -of a `BEGIN' or `END' rule) stops the execution of any further -automatic rules, but the `END' rule is executed if there is one. If -you don't want the `END' rule to do its job in this case, you can set -a variable to nonzero before the `exit' statement, and check that -variable in the `END' rule. - -If an argument is supplied to `exit', its value is used as the exit -status code for the `awk' process. If no argument is supplied, -`exit' returns status zero (success). - -For example, let's say you've discovered an error condition you -really don't know how to handle. Conventionally, programs report -this by exiting with a nonzero status. Your `awk' program can do -this using an `exit' statement with a nonzero argument. Here's an -example of this: - - BEGIN { - if (("date" | getline date_now) < 0) { - print "Can't get system date" - exit 4 - } - } - - - -File: gawk-info, Node: Arrays, Next: Built-in, Prev: Statements, Up: Top - -Actions: Using Arrays in `awk' -****************************** - -An "array" is a table of various values, called "elements". The -elements of an array are distinguished by their "indices". Names of -arrays in `awk' are strings of alphanumeric characters and -underscores, just like regular variables. - -You cannot use the same identifier as both a variable and as an array -name in one `awk' program. - -* Menu: - -* Intro: Array Intro. Basic facts abou arrays in `awk'. -* Reference to Elements:: How to examine one element of an array. -* Assigning Elements:: How to change an element of an array. -* Example: Array Example. Sample program explained. - -* Scanning an Array:: A variation of the `for' statement. It loops - through the indices of an array's existing elements. - -* Delete:: The `delete' statement removes an element from an array. - -* Multi-dimensional:: Emulating multi--dimensional arrays in `awk'. -* Multi-scanning:: Scanning multi--dimensional arrays. - - - -File: gawk-info, Node: Array Intro, Next: Reference to Elements, Up: Arrays - -Introduction to Arrays -====================== - -The `awk' language has one--dimensional "arrays" for storing groups -of related strings or numbers. Each array must have a name; valid -array names are the same as valid variable names, and they do -conflict with variable names: you can't have both an array and a -variable with the same name at any point in an `awk' program. - -Arrays in `awk' superficially resemble arrays in other programming -languages; but there are fundamental differences. In `awk', you -don't need to declare the size of an array before you start to use it. -What's more, in `awk' any number or even a string may be used as an -array index. - -In most other languages, you have to "declare" an array and specify -how many elements or components it has. In such languages, the -declaration causes a contiguous block of memory to be allocated for -that many elements. An index in the array must be a positive -integer; for example, the index 0 specifies the first element in the -array, which is actually stored at the beginning of the block of -memory. Index 1 specifies the second element, which is stored in -memory right after the first element, and so on. It is impossible to -add more elements to the array, because it has room for only as many -elements as you declared. (Some languages have arrays whose first -index is 1, others require that you specify both the first and last -index when you declare the array. In such a language, an array could -be indexed, for example, from -3 to 17.) A contiguous array of four -elements might look like this, conceptually, if the element values -are 8, `"foo"', `""' and 30: - - +--------+--------+-------+--------+ - | 8 | "foo" | "" | 30 | value - +--------+--------+-------+--------+ - 0 1 2 3 index - -Only the values are stored; the indices are implicit from the order -of the values. 8 is the value at index 0, because 8 appears in the -position with 0 elements before it. - -Arrays in `awk' are different: they are "associative". This means -that each array is a collection of pairs: an index, and its -corresponding array element value: - - Element 4 Value 30 - Element 2 Value "foo" - Element 1 Value 8 - Element 3 Value "" - -We have shown the pairs in jumbled order because their order doesn't -mean anything. - -One advantage of an associative array is that new pairs can be added -at any time. For example, suppose we add to that array a tenth -element whose value is `"number ten"'. The result is this: - - Element 10 Value "number ten" - Element 4 Value 30 - Element 2 Value "foo" - Element 1 Value 8 - Element 3 Value "" - -Now the array is "sparse" (i.e. some indices are missing): it has -elements number 4 and 10, but doesn't have an element 5, 6, 7, 8, or 9. - -Another consequence of associative arrays is that the indices don't -have to be positive integers. Any number, or even a string, can be -an index. For example, here is an array which translates words from -English into French: - - Element "dog" Value "chien" - Element "cat" Value "chat" - Element "one" Value "un" - Element 1 Value "un" - -Here we decided to translate the number 1 in both spelled--out and -numeral form--thus illustrating that a single array can have both -numbers and strings as indices. - -When `awk' creates an array for you, e.g. with the `split' built--in -function (*note String Functions::.), that array's indices start at -the number one. - - - -File: gawk-info, Node: Reference to Elements, Next: Assigning Elements, Prev: Array Intro, Up: Arrays - -Referring to an Array Element -============================= - -The principal way of using an array is to refer to one of its elements. -An array reference is an expression which looks like this: - - ARRAY[INDEX] - -Here ARRAY is the name of an array. The expression INDEX is the -index of the element of the array that you want. The value of the -array reference is the current value of that array element. - -For example, `foo[4.3]' is an expression for the element of array -`foo' at index 4.3. - -If you refer to an array element that has no recorded value, the -value of the reference is `""', the null string. This includes -elements to which you have not assigned any value, and elements that -have been deleted (*note Delete::.). Such a reference automatically -creates that array element, with the null string as its value. (In -some cases, this is unfortunate, because it might waste memory inside -`awk'). - -You can find out if an element exists in an array at a certain index -with the expression: - - INDEX in ARRAY - -This expression tests whether or not the particular index exists, -without the side effect of creating that element if it is not present. -The expression has the value 1 (true) if `ARRAY[SUBSCRIPT]' exists, -and 0 (false) if it does not exist. - -For example, to find out whether the array `frequencies' contains the -subscript `"2"', you would ask: - - if ("2" in frequencies) print "Subscript \"2\" is present." - -Note that this is *not* a test of whether or not the array -`frequencies' contains an element whose *value* is `"2"'. (There is -no way to that except to scan all the elements.) Also, this *does -not* create `frequencies["2"]', while the following (incorrect) -alternative would: - - if (frequencies["2"] != "") print "Subscript \"2\" is present." - - - -File: gawk-info, Node: Assigning Elements, Next: Array Example, Prev: Reference to Elements, Up: Arrays - -Assigning Array Elements -======================== - -Array elements are lvalues: they can be assigned values just like -`awk' variables: - - ARRAY[SUBSCRIPT] = VALUE - -Here ARRAY is the name of your array. The expression SUBSCRIPT is -the index of the element of the array that you want to assign a -value. The expression VALUE is the value you are assigning to that -element of the array. - - - -File: gawk-info, Node: Array Example, Next: Scanning an Array, Prev: Assigning Elements, Up: Arrays - -Basic Example of an Array -========================= - -The following program takes a list of lines, each beginning with a -line number, and prints them out in order of line number. The line -numbers are not in order, however, when they are first read: they -are scrambled. This program sorts the lines by making an array using -the line numbers as subscripts. It then prints out the lines in -sorted order of their numbers. It is a very simple program, and will -get confused if it encounters repeated numbers, gaps, or lines that -don't begin with a number. - - BEGIN { - max=0 - } - - { - if ($1 > max) - max = $1 - arr[$1] = $0 - } - - END { - for (x = 1; x <= max; x++) - print arr[x] - } - -The first rule just initializes the variable `max'. (This is not -strictly necessary, since an uninitialized variable has the null -string as its value, and the null string is effectively zero when -used in a context where a number is required.) - -The second rule keeps track of the largest line number seen so far; -it also stores each line into the array `arr', at an index that is -the line's number. - -The third rule runs after all the input has been read, to print out -all the lines. - -When this program is run with the following input: - - 5 I am the Five man - 2 Who are you? The new number two! - 4 . . . And four on the floor - 1 Who is number one? - 3 I three you. - - its output is this: - - 1 Who is number one? - 2 Who are you? The new number two! - 3 I three you. - 4 . . . And four on the floor - 5 I am the Five man - - - -File: gawk-info, Node: Scanning an Array, Next: Delete, Prev: Array Example, Up: Arrays - -Scanning All Elements of an Array -================================= - -In programs that use arrays, often you need a loop that will execute -once for each element of an array. In other languages, where arrays -are contiguous and indices are limited to positive integers, this is -easy: the largest index is one less than the length of the array, and -you can find all the valid indices by counting from zero up to that -value. This technique won't do the job in `awk', since any number or -string may be an array index. So `awk' has a special kind of `for' -statement for scanning an array: - - for (VAR in ARRAY) - BODY - -This loop executes BODY once for each different value that your -program has previously used as an index in ARRAY, with the variable -VAR set to that index. - -Here is a program that uses this form of the `for' statement. The -first rule scans the input records and notes which words appear (at -least once) in the input, by storing a 1 into the array `used' with -the word as index. The second rule scans the elements of `used' to -find all the distinct words that appear in the input. It prints each -word that is more than 10 characters long, and also prints the number -of such words. *Note Built-in::, for more information on the -built--in function `length'. - - # Record a 1 for each word that is used at least once. - { - for (i = 0; i < NF; i++) - used[$i] = 1 - } - - # Find number of distinct words more than 10 characters long. - END { - num_long_words = 0 - for (x in used) - if (length(x) > 10) { - ++num_long_words - print x - } - print num_long_words, "words longer than 10 characters" - } - -*Note Sample Program::, for a more detailed example of this type. - -The order in which elements of the array are accessed by this -statement is determined by the internal arrangement of the array -elements within `awk' and cannot be controlled or changed. This can -lead to problems if new elements are added to ARRAY by statements in -BODY; you cannot predict whether or not the `for' loop will reach -them. Similarly, changing VAR inside the loop can produce strange -results. It is best to avoid such things. - - - -File: gawk-info, Node: Delete, Next: Multi-dimensional, Prev: Scanning an Array, Up: Arrays - -The `delete' Statement -====================== - -You can remove an individual element of an array using the `delete' -statement: - - delete ARRAY[INDEX] - -When an array element is deleted, it is as if you had never referred -to it and had never given it any value. Any value the element -formerly had can no longer be obtained. - -Here is an example of deleting elements in an array: - - awk '{ for (i in frequencies) - delete frequencies[i] - }' - -This example removes all the elements from the array `frequencies'. - -If you delete an element, the `for' statement to scan the array will -not report that element, and the `in' operator to check for the -presence of that element will return 0: - - delete foo[4] - if (4 in foo) - print "This will never be printed" - - - -File: gawk-info, Node: Multi-dimensional, Next: Multi-scanning, Prev: Delete, Up: Arrays - -Multi--dimensional arrays -========================= - -A multi--dimensional array is an array in which an element is -identified by a sequence of indices, not a single index. For -example, a two--dimensional array requires two indices. The usual -way (in most languages, including `awk') to refer to an element of a -two--dimensional array named `grid' is with `grid[x,y]'. - -Multi--dimensional arrays are supported in `awk' through -concatenation of indices into one string. What happens is that `awk' -converts the indices into strings (*note Conversion::.) and -concatenates them together, with a separator between them. This -creates a single string that describes the values of the separate -indices. The combined string is used as a single index into an -ordinary, one--dimensional array. The separator used is the value of -the special variable `SUBSEP'. - -For example, suppose the value of `SUBSEP' is `","' and the -expression `foo[5,12]="value"' is executed. The numbers 5 and 12 -will be concatenated with a comma between them, yielding `"5,12"'; -thus, the array element `foo["5,12"]' will be set to `"value"'. - -Once the element's value is stored, `awk' has no record of whether it -was stored with a single index or a sequence of indices. The two -expressions `foo[5,12]' and `foo[5 SUBSEP 12]' always have the same -value. - -The default value of `SUBSEP' is not a comma; it is the string -`"\034"', which contains a nonprinting character that is unlikely to -appear in an `awk' program or in the input data. - -The usefulness of choosing an unlikely character comes from the fact -that index values that contain a string matching `SUBSEP' lead to -combined strings that are ambiguous. Suppose that `SUBSEP' is a -comma; then `foo["a,b", "c"]' and `foo["a", "b,c"]' will be -indistinguishable because both are actually stored as `foo["a,b,c"]'. -Because `SUBSEP' is `"\034"', such confusion can actually happen only -when an index contains the character `"\034"', which is a rare event. - -You can test whether a particular index--sequence exists in a -``multi--dimensional'' array with the same operator `in' used for -single dimensional arrays. Instead of a single index as the -left--hand operand, write the whole sequence of indices, separated by -commas, in parentheses: - - (SUBSCRIPT1, SUBSCRIPT2, ...) in ARRAY - -The following example treats its input as a two--dimensional array of -fields; it rotates this array 90 degrees clockwise and prints the -result. It assumes that all lines have the same number of elements. - - awk 'BEGIN { - max_nf = max_nr = 0 - } - - { - if (max_nf < NF) - max_nf = NF - max_nr = NR - for (x = 1; x <= NF; x++) - vector[x, NR] = $x - } - - END { - for (x = 1; x <= max_nf; x++) { - for (y = max_nr; y >= 1; --y) - printf("%s ", vector[x, y]) - printf("\n") - } - }' - -When given the input: - - 1 2 3 4 5 6 - 2 3 4 5 6 1 - 3 4 5 6 1 2 - 4 5 6 1 2 3 - -it produces: - - 4 3 2 1 - 5 4 3 2 - 6 5 4 3 - 1 6 5 4 - 2 1 6 5 - 3 2 1 6 - - - -File: gawk-info, Node: Multi-scanning, Prev: Multi-dimensional, Up: Arrays - -Scanning Multi--dimensional Arrays -================================== - -There is no special `for' statement for scanning a -``multi--dimensional'' array; there cannot be one, because in truth -there are no multi--dimensional arrays or elements; there is only a -multi--dimensional *way of accessing* an array. - -However, if your program has an array that is always accessed as -multi--dimensional, you can get the effect of scanning it by -combining the scanning `for' statement (*note Scanning an Array::.) -with the `split' built--in function (*note String Functions::.). It -works like this: - - for (combined in ARRAY) { - split (combined, separate, SUBSEP) - ... - } - -This finds each concatenated, combined index in the array, and splits -it into the individual indices by breaking it apart where the value -of `SUBSEP' appears. The split--out indices become the elements of -the array `separate'. - -Thus, suppose you have previously stored in `ARRAY[1, "foo"]'; then -an element with index `"1\034foo"' exists in ARRAY. (Recall that the -default value of `SUBSEP' contains the character with code 034.) -Sooner or later the `for' statement will find that index and do an -iteration with `combined' set to `"1\034foo"'. Then the `split' -function will be called as follows: - - split ("1\034foo", separate, "\034") - -The result of this is to set `separate[1]' to 1 and `separate[2]' to -`"foo"'. Presto, the original sequence of separate indices has been -recovered. - - - -File: gawk-info, Node: Built-in, Next: User-defined, Prev: Arrays, Up: Top - -Built--in functions -******************* - -"Built--in" functions are functions always available for your `awk' -program to call. This chapter defines all the built--in functions -that exist; some of them are mentioned in other sections, but they -are summarized here for your convenience. (You can also define new -functions yourself. *Note User-defined::.) - -In most cases, any extra arguments given to built--in functions are -ignored. The defaults for omitted arguments vary from function to -function and are described under the individual functions. - -The name of a built--in function need not be followed immediately by -the opening left parenthesis of the arguments; whitespace is allowed. -However, it is wise to write no space there, since user--defined -functions do not allow space. - -When a function is called, expressions that create the function's -actual parameters are evaluated completely before the function call -is performed. For example, in the code fragment: - - i = 4 - j = myfunc(i++) - -the variable `i' will be set to 5 before `myfunc' is called with a -value of 4 for its actual parameter. - -* Menu: - -* Numeric Functions:: Functions that work with numbers, - including `int', `sin' and `rand'. - -* String Functions:: Functions for string manipulation, - such as `split', `match', and `sprintf'. - -* I/O Functions:: Functions for files and shell commands - - - -File: gawk-info, Node: Numeric Functions, Next: String Functions, Up: Built-in - -Numeric Built--in Functions -=========================== - -The general syntax of the numeric built--in functions is the same for -each. Here is an example of that syntax: - - awk '# Read input records containing a pair of points: x0, y0, x1, y1. - # Print the points and the distance between them. - { printf "%f %f %f %f %f\n", $1, $2, $3, $4, - sqrt(($2-$1) * ($2-$1) + ($4-$3) * ($4-$3)) }' - -This calculates the square root of a calculation that uses the values -of the fields. It then prints the first four fields of the input -record and the result of the square root calculation. - -Here is the full list of numeric built--in functions: - -`int(X)' - This gives you the integer part of X, truncated toward 0. This - produces the nearest integer to X, located between X and 0. - - For example, `int(3)' is 3, `int(3.9)' is 3, `int(-3.9)' is -3, - and `int(-3)' is -3 as well. - -`sqrt(X)' - This gives you the positive square root of X. It reports an - error if X is negative. - -`exp(X)' - This gives you the exponential of X, or reports an error if X is - out of range. The range of values X can have depends on your - machine's floating point representation. - -`log(X)' - This gives you the natural logarithm of X, if X is positive; - otherwise, it reports an error. - -`sin(X)' - This gives you the sine of X, with X in radians. - -`cos(X)' - This gives you the cosine of X, with X in radians. - -`atan2(Y, X)' - This gives you the arctangent of Y/X, with both in radians. - -`rand()' - This gives you a random number. The values of `rand()' are - uniformly--distributed between 0 and 1. The value is never 0 - and never 1. - - Often you want random integers instead. Here is a user--defined - function you can use to obtain a random nonnegative integer less - than N: - - function randint(n) { - return int(n * rand()) - } - - The multiplication produces a random real number at least 0, and - less than N. We then make it an integer (using `int') between 0 - and `N-1'. - - Here is an example where a similar function is used to produce - random integers between 1 and N: - - awk ' - # Function to roll a simulated die. - function roll(n) { return 1 + int(rand() * n) } - - # Roll 3 six--sided dice and print total number of points. - { - printf("%d points\n", roll(6)+roll(6)+roll(6)) - }' - - *Note* that `rand()' starts generating numbers from the same - point, or "seed", each time you run `awk'. This means that the - same program will produce the same results each time you run it. - The numbers are random within one `awk' run, but predictable - from run to run. This is convenient for debugging, but if you - want a program to do different things each time it is used, you - must change the seed to a value that will be different in each - run. To do this, use `srand'. - -`srand(X)' - The function `srand(X)' sets the starting point, or "seed", for - generating random numbers to the value X. - - Each seed value leads to a particular sequence of ``random'' - numbers. Thus, if you set the seed to the same value a second - time, you will get the same sequence of ``random'' numbers again. - - If you omit the argument X, as in `srand()', then the current - date and time of day are used for a seed. This is the way to - get random numbers that are truly unpredictable. - - The return value of `srand()' is the previous seed. This makes - it easy to keep track of the seeds for use in consistently - reproducing sequences of random numbers. - - - -File: gawk-info, Node: String Functions, Next: I/O Functions, Prev: Numeric Functions, Up: Built-in - -Built--in Functions for String Manipulation -=========================================== - -`index(IN, FIND)' - This searches the string IN for the first occurrence of the - string FIND, and returns the position where that occurrence - begins in the string IN. For example: - - awk 'BEGIN { print index("peanut", "an") }' - - prints `3'. If FIND is not found, `index' returns 0. - -`length(STRING)' - This gives you the number of characters in STRING. If STRING is - a number, the length of the digit string representing that - number is returned. For example, `length("abcde")' is 5. - Whereas, `length(15 * 35)' works out to 3. How? Well, 15 * 35 - = 525, and 525 is then converted to the string `"525"', which - has three characters. - -`match(STRING, REGEXP)' - The `match' function searches the string, STRING, for the - longest, leftmost substring matched by the regular expression, - REGEXP. It returns the character position, or "index", of where - that substring begins (1, if it starts at the beginning of - STRING). If no match if found, it returns 0. - - The `match' function sets the special variable `RSTART' to the - index. It also sets the special variable `RLENGTH' to the - length of the matched substring. If no match is found, `RSTART' - is set to 0, and `RLENGTH' to -1. - - For example: - - awk '{ - if ($1 == "FIND") - regex = $2 - else { - where = match($0, regex) - if (where) - print "Match of", regex, "found at", where, "in", $0 - } - }' - - This program looks for lines that match the regular expression - stored in the variable `regex'. This regular expression can be - changed. If the first word on a line is `FIND', `regex' is - changed to be the second word on that line. Therefore, given: - - FIND fo*bar - My program was a foobar - But none of it would doobar - FIND Melvin - JF+KM - This line is property of The Reality Engineering Co. - This file was created by Melvin. - - `awk' prints: - - Match of fo*bar found at 18 in My program was a foobar - Match of Melvin found at 26 in This file was created by Melvin. - -`split(STRING, ARRAY, FIELD_SEPARATOR)' - This divides STRING up into pieces separated by FIELD_SEPARATOR, - and stores the pieces in ARRAY. The first piece is stored in - `ARRAY[1]', the second piece in `ARRAY[2]', and so forth. The - string value of the third argument, FIELD_SEPARATOR, is used as - a regexp to search for to find the places to split STRING. If - the FIELD_SEPARATOR is omitted, the value of `FS' is used. - `split' returns the number of elements created. - - The `split' function, then, splits strings into pieces in a - manner similar to the way input lines are split into fields. - For example: - - split("auto-da-fe", a, "-") - - splits the string `auto-da-fe' into three fields using `-' as - the separator. It sets the contents of the array `a' as follows: - - a[1] = "auto" - a[2] = "da" - a[3] = "fe" - - The value returned by this call to `split' is 3. - -`sprintf(FORMAT, EXPRESSION1,...)' - This returns (without printing) the string that `printf' would - have printed out with the same arguments (*note Printf::.). For - example: - - sprintf("pi = %.2f (approx.)", 22/7) - - returns the string `"pi = 3.14 (approx.)"'. - -`sub(REGEXP, REPLACEMENT_STRING, TARGET_VARIABLE)' - The `sub' function alters the value of TARGET_VARIABLE. It - searches this value, which should be a string, for the leftmost - substring matched by the regular expression, REGEXP, extending - this match as far as possible. Then the entire string is - changed by replacing the matched text with REPLACEMENT_STRING. - The modified string becomes the new value of TARGET_VARIABLE. - - This function is peculiar because TARGET_VARIABLE is not simply - used to compute a value, and not just any expression will do: it - must be a variable, field or array reference, so that `sub' can - store a modified value there. If this argument is omitted, then - the default is to use and alter `$0'. - - For example: - - str = "water, water, everywhere" - sub(/at/, "ith", str) - - sets `str' to `"wither, water, everywhere"', by replacing the - leftmost, longest occurrence of `at' with `ith'. - - The `sub' function returns the number of substitutions made - (either one or zero). - - The special character, `&', in the replacement string, - REPLACEMENT_STRING, stands for the precise substring that was - matched by REGEXP. (If the regexp can match more than one - string, then this precise substring may vary.) For example: - - awk '{ sub(/candidate/, "& and his wife"); print }' - - will change the first occurrence of ``candidate'' to ``candidate - and his wife'' on each input line. - - The effect of this special character can be turned off by - preceding it with a backslash (`\&'). To include a backslash in - the replacement string, it too must be preceded with a (second) - backslash. - - Note: if you use `sub' with a third argument that is not a - variable, field or array element reference, then it will still - search for the pattern and return 0 or 1, but the modified - string is thrown away because there is no place to put it. For - example: - - sub(/USA/, "United States", "the USA and Canada") - - will indeed produce a string `"the United States and Canada"', - but there will be no way to use that string! - -`gsub(REGEXP, REPLACEMENT_STRING, TARGET_VARIABLE)' - This is similar to the `sub' function, except `gsub' replaces - *all* of the longest, leftmost, *non--overlapping* matching - substrings it can find. The ``g'' in `gsub' stands for - "global", which means replace *everywhere*. For example: - - awk '{ gsub(/Britain/, "United Kingdom"); print }' - - replaces all occurrences of the string `Britain' with `United - Kingdom' for all input records. - - The `gsub' function returns the number of substitutions made. - If the variable to be searched and altered, TARGET_VARIABLE, is - omitted, then the entire input record, `$0', is used. - - The characters `&' and `\' are special in `gsub' as they are in - `sub' (see immediately above). - -`substr(STRING, START, LENGTH)' - This returns a LENGTH--character--long substring of STRING, - starting at character number START. The first character of a - string is character number one. For example, - `substr("washington", 5, 3)' returns `"ing"'. - - If LENGTH is not present, this function returns the whole suffix - of STRING that begins at character number START. For example, - `substr("washington", 5)' returns `"ington"'. - - - -File: gawk-info, Node: I/O Functions, Prev: String Functions, Up: Built-in - -Built--in Functions for I/O to Files and Commands -================================================= - -`close(FILENAME)' - Close the file FILENAME. The argument may alternatively be a - shell command that was used for redirecting to or from a pipe; - then the pipe is closed. - - *Note Close Input::, regarding closing input files and pipes. - *Note Close Output::, regarding closing output files and pipes. - -`system(COMMAND)' - The system function allows the user to execute operating system - commands and then return to the `awk' program. The `system' - function executes the command given by the string value of - COMMAND. It returns, as its value, the status returned by the - command that was executed. This is known as returning the "exit - status". - - For example, if the following fragment of code is put in your - `awk' program: - - END { - system("mail -s 'awk run done' operator < /dev/null") - } - - the system operator will be sent mail when the `awk' program - finishes processing input and begins its end--of--input - processing. - - Note that much the same result can be obtained by redirecting - `print' or `printf' into a pipe. However, if your `awk' program - is interactive, this function is useful for cranking up large - self--contained programs, such as a shell or an editor. - - - -File: gawk-info, Node: User-defined, Next: Special, Prev: Built-in, Up: Top - -User--defined Functions -*********************** - -Complicated `awk' programs can often be simplified by defining your -own functions. User--defined functions can be called just like -built--in ones (*note Function Calls::.), but it is up to you to -define them--to tell `awk' what they should do. - -* Menu: - -* Definition Syntax:: How to write definitions and what they mean. -* Function Example:: An example function definition and what it does. -* Function Caveats:: Things to watch out for. -* Return Statement:: Specifying the value a function returns. - - - -File: gawk-info, Node: Definition Syntax, Next: Function Example, Up: User-defined - -Syntax of Function Definitions -============================== - -The definition of a function named NAME looks like this: - - function NAME (PARAMETER-LIST) { - BODY-OF-FUNCTION - } - -A valid function name is like a valid variable name: a sequence of -letters, digits and underscores, not starting with a digit. - -Such function definitions can appear anywhere between the rules of -the `awk' program. The general format of an `awk' program, then, is -now modified to include sequences of rules *and* user--defined -function definitions. - -The function definition need not precede all the uses of the function. -This is because `awk' reads the entire program before starting to -execute any of it. - -The PARAMETER-LIST is a list of the function's "local" variable -names, separated by commas. Within the body of the function, local -variables refer to arguments with which the function is called. If -the function is called with fewer arguments than it has local -variables, this is not an error; the extra local variables are simply -set as the null string. - -The local variable values hide or "shadow" any variables of the same -names used in the rest of the program. The shadowed variables are -not accessible in the function definition, because there is no way to -name them while their names have been taken away for the local -variables. All other variables used in the `awk' program can be -referenced or set normally in the function definition. - -The local variables last only as long as the function is executing. -Once the function finishes, the shadowed variables come back. - -The BODY-OF-FUNCTION part of the definition is the most important -part, because this is what says what the function should actually *do*. -The local variables exist to give the body a way to talk about the -arguments. - -Functions may be "recursive", i.e., they can call themselves, either -directly, or indirectly (via calling a second function that calls the -first again). - -The keyword `function' may also be written `func'. - - - -File: gawk-info, Node: Function Example, Next: Function Caveats, Prev: Definition Syntax, Up: User-defined - -Function Definition Example -=========================== - -Here is an example of a user--defined function, called `myprint', -that takes a number and prints it in a specific format. - - function myprint(num) - { - printf "%6.3g\n", num - } - -To illustrate, let's use the following `awk' rule to use, or "call", -our `myprint' function: - - $3 > 0 { myprint($3) }' - -This program prints, in our special format, all the third fields that -contain a positive number in our input. Therefore, when given: - - 1.2 3.4 5.6 7.8 - 9.10 11.12 13.14 15.16 - 17.18 19.20 21.22 23.24 - -this program, using our function to format the results, will print: - - 5.6 - 13.1 - 21.2 - -Here is a rather contrived example of a recursive function. It -prints a string backwards: - - function rev (str, len) { - if (len == 0) { - printf "\n" - return - } - printf "%c", substr(str, len, 1) - rev(str, len - 1) - } - - - -File: gawk-info, Node: Function Caveats, Next: Return Statement, Prev: Function Example, Up: User-defined - -Caveats of Function Calling -=========================== - -*Note* that there cannot be any blanks between the function name and -the left parenthesis of the argument list, when calling a function. -This is so `awk' can tell you are not trying to concatenate the value -of a variable with the value of an expression inside the parentheses. - -When a function is called, it is given a *copy* of the values of its -arguments. This is called "passing by value". The caller may use a -variable as the expression for the argument, but the called function -does not know this: all it knows is what value the argument had. For -example, if you write this code: - - foo = "bar" - z = myfunc(foo) - -then you should not think of the argument to `myfunc' as being ``the -variable `foo'''. Instead, think of the argument as the string -value, `"bar"'. - -If the function `myfunc' alters the values of its local variables, -this has no effect on any other variables. In particular, if -`myfunc' does this: - - function myfunc (win) { - print win - win = "zzz" - print win - } - -to change its first argument variable `win', this *does not* change -the value of `foo' in the caller. The role of `foo' in calling -`myfunc' ended when its value, `"bar"', was computed. If `win' also -exists outside of `myfunc', this definition will not change it--that -value is shadowed during the execution of `myfunc' and cannot be seen -or changed from there. - -However, when arrays are the parameters to functions, they are *not* -copied. Instead, the array itself is made available for direct -manipulation by the function. This is usually called "passing by -reference". Changes made to an array parameter inside the body of a -function *are* visible outside that function. *This can be very -dangerous if you don't watch what you are doing.* For example: - - function changeit (array, ind, nvalue) { - array[ind] = nvalue - } - - BEGIN { - a[1] = 1 ; a[2] = 2 ; a[3] = 3 - changeit(a, 2, "two") - printf "a[1] = %s, a[2] = %s, a[3] = %s\n", a[1], a[2], a[3] - } - -will print `a[1] = 1, a[2] = two, a[3] = 3', because the call to -`changeit' stores `"two"' in the second element of `a'. - - - -File: gawk-info, Node: Return Statement, Prev: Function Caveats, Up: User-defined - -The `return' statement -====================== - -The body of a user--defined function can contain a `return' statement. -This statement returns control to the rest of the `awk' program. It -can also be used to return a value for use in the rest of the `awk' -program. It looks like: - - `return EXPRESSION' - -The EXPRESSION part is optional. If it is omitted, then the returned -value is undefined and, therefore, unpredictable. - -A `return' statement with no value expression is assumed at the end -of every function definition. So if control reaches the end of the -function definition, then the function returns an unpredictable value. - -Here is an example of a user--defined function that returns a value -for the largest number among the elements of an array: - - function maxelt (vec, i, ret) { - for (i in vec) { - if (ret == "" || vec[i] > ret) - ret = vec[i] - } - return ret - } - -You call `maxelt' with one argument, an array name. The local -variables `i' and `ret' are not intended to be arguments; while there -is nothing to stop you from passing two or three arguments to -`maxelt', the results would be strange. - -When writing a function definition, it is conventional to separate -the parameters from the local variables with extra spaces, as shown -above in the definition of `maxelt'. - -Here is a program that uses, or calls, our `maxelt' function. This -program loads an array, calls `maxelt', and then reports the maximum -number in that array: - - awk ' - function maxelt (vec, i, ret) { - for (i in vec) { - if (ret == "" || vec[i] > ret) - ret = vec[i] - } - return ret - } - - # Load all fields of each record into nums. - { - for(i = 1; i <= NF; i++) - nums[NR, i] = $i - } - - END { - print maxelt(nums) - }' - -Given the following input: - - 1 5 23 8 16 - 44 3 5 2 8 26 - 256 291 1396 2962 100 - -6 467 998 1101 - 99385 11 0 225 - -our program tells us (predictably) that: - - 99385 - -is the largest number in our array. - - - -File: gawk-info, Node: Special, Next: Sample Program, Prev: User-defined, Up: Top - -Special Variables -***************** - -Most `awk' variables are available for you to use for your own -purposes; they will never change except when your program assigns -them, and will never affect anything except when your program -examines them. - -A few variables have special meanings. Some of them `awk' examines -automatically, so that they enable you to tell `awk' how to do -certain things. Others are set automatically by `awk', so that they -carry information from the internal workings of `awk' to your program. - -Most of these variables are also documented in the chapters where -their areas of activity are described. - -* Menu: - -* User-modified:: Special variables that you change to control `awk'. - -* Auto-set:: Special variables where `awk' gives you information. - - - -File: gawk-info, Node: User-modified, Next: Auto-set, Up: Special - -Special Variables That Control `awk' -==================================== - -This is a list of the variables which you can change to control how -`awk' does certain things. - -`FS' - `FS' is the input field separator (*note Field Separators::.). - The value is a regular expression that matches the separations - between fields in an input record. - - The default value is `" "', a string consisting of a single - space. As a special exception, this value actually means that - any sequence of spaces and tabs is a single separator. It also - causes spaces and tabs at the beginning or end of a line to be - ignored. - - You can set the value of `FS' on the command line using the `-F' - option: - - awk -F, 'PROGRAM' INPUT-FILES - -`OFMT' - This string is used by `awk' to control conversion of numbers to - strings (*note Conversion::.). It works by being passed, in - effect, as the first argument to the `sprintf' function. Its - default value is `"%.6g"'. - -`OFS' - This is the output field separator (*note Output Separators::.). - It is output between the fields output by a `print' statement. - Its default value is `" "', a string consisting of a single space. - -`ORS' - This is the output record separator (*note Output - Separators::.). It is output at the end of every `print' - statement. Its default value is the newline character, often - represented in `awk' programs as `\n'. - -`RS' - This is `awk''s record separator (*note Records::.). Its - default value is a string containing a single newline character, - which means that an input record consists of a single line of - text. - -`SUBSEP' - `SUBSEP' is a subscript separator (*note Multi-dimensional::.). - It has the default value of `"\034"', and is used to separate - the parts of the name of a multi--dimensional array. Thus, if - you access `foo[12,3]', it really accesses `foo["12\0343"]'. - - - -File: gawk-info, Node: Auto-set, Prev: User-modified, Up: Special - -Special Variables That Convey Information to You -================================================ - -This is a list of the variables that are set automatically by `awk' -on certain occasions so as to provide information for your program. - -`ARGC' -`ARGV' - The command--line arguments available to `awk' are stored in an - array called `ARGV'. `ARGC' is the number of command--line - arguments present. `ARGV' is indexed from zero to `ARGC' - 1. - For example: - - awk '{ print ARGV[$1] }' inventory-shipped BBS-list - - In this example, `ARGV[0]' contains `"awk"', `ARGV[1]' contains - `"inventory-shipped"', and `ARGV[2]' contains `"BBS-list"'. - `ARGC' is 3, one more than the index of the last element in - `ARGV' since the elements are numbered from zero. - - Notice that the `awk' program is not treated as an argument. - The `-f' `FILENAME' option, and the `-F' option, are also not - treated as arguments for this purpose. - - Variable assignments on the command line *are* treated as - arguments, and do show up in the `ARGV' array. - - Your program can alter `ARGC' the elements of `ARGV'. Each time - `awk' reaches the end of an input file, it uses the next element - of `ARGV' as the name of the next input file. By storing a - different string there, your program can change which files are - read. You can use `-' to represent the standard input. By - storing additional elements and incrementing `ARGC' you can - cause additional files to be read. - - If you decrease the value of `ARGC', that eliminates input files - from the end of the list. By recording the old value of `ARGC' - elsewhere, your program can treat the eliminated arguments as - something other than file names. - - To eliminate a file from the middle of the list, store the null - string (`""') into `ARGV' in place of the file's name. As a - special feature, `awk' ignores file names that have been - replaced with the null string. - -`ENVIRON' - This is an array that contains the values of the environment. - The array indices are the environment variable names; the values - are the values of the particular environment variables. For - example, `ENVIRON["HOME"]' might be `/u/close'. Changing this - array does not affect the environment passed on to any programs - that `awk' may spawn via redirection or the `system' function. - (This may not work under operating systems other than MS-DOS, - Unix, or GNU.) - -`FILENAME' - This is the name of the file that `awk' is currently reading. - If `awk' is reading from the standard input (in other words, - there are no files listed on the command line), `FILENAME' is - set to `"-"'. `FILENAME' is changed each time a new file is - read (*note Reading Files::.). - -`FNR' - `FNR' is the current record number in the current file. `FNR' - is incremented each time a new record is read (*note Getline::.). - It is reinitialized to 0 each time a new input file is started. - -`NF' - `NF' is the number of fields in the current input record. `NF' - is set each time a new record is read, when a new field is - created, or when $0 changes (*note Fields::.). - -`NR' - This is the number of input records `awk' has processed since - the beginning of the program's execution. (*note Records::.). - `NR' is set each time a new record is read. - -`RLENGTH' - `RLENGTH' is the length of the string matched by the `match' - function (*note String Functions::.). `RLENGTH' is set by - invoking the `match' function. Its value is the length of the - matched string, or -1 if no match was found. - -`RSTART' - `RSTART' is the start of the string matched by the `match' - function (*note String Functions::.). `RSTART' is set by - invoking the `match' function. Its value is the position of the - string where the matched string starts, or 0 if no match was - found. - - - -File: gawk-info, Node: Sample Program, Next: Notes, Prev: Special, Up: Top - -Sample Program -************** - -The following example is a complete `awk' program, which prints the -number of occurrences of each word in its input. It illustrates the -associative nature of `awk' arrays by using strings as subscripts. -It also demonstrates the `for X in ARRAY' construction. Finally, it -shows how `awk' can be used in conjunction with other utility -programs to do a useful task of some complexity with a minimum of -effort. Some explanations follow the program listing. - - awk ' - # Print list of word frequencies - { - for (i = 1; i <= NF; i++) - freq[$i]++ - } - - END { - for (word in freq) - printf "%s\t%d\n", word, freq[word] - }' - -The first thing to notice about this program is that it has two -rules. The first rule, because it has an empty pattern, is executed -on every line of the input. It uses `awk''s field--accessing -mechanism (*note Fields::.) to pick out the individual words from the -line, and the special variable `NF' (*note Special::.) to know how -many fields are available. - -For each input word, an element of the array `freq' is incremented to -reflect that the word has been seen an additional time. - -The second rule, because it has the pattern `END', is not executed -until the input has been exhausted. It prints out the contents of -the `freq' table that has been built up inside the first action. - -Note that this program has several problems that would prevent it -from being useful by itself on real text files: - - * Words are detected using the `awk' convention that fields are - separated by whitespace and that other characters in the input - (except newlines) don't have any special meaning to `awk'. This - means that punctuation characters count as part of words. - - * The `awk' language considers upper and lower case characters to - be distinct. Therefore, `foo' and `Foo' will not be treated by - this program as the same word. This is undesirable since in - normal text, words are capitalized if they begin sentences, and - a frequency analyzer should not be sensitive to that. - - * The output does not come out in any useful order. You're more - likely to be interested in which words occur most frequently, or - having an alphabetized table of how frequently each word occurs. - -The way to solve these problems is to use other operating system -utilities to process the input and output of the `awk' script. -Suppose the script shown above is saved in the file `frequency.awk'. -Then the shell command: - - tr A-Z a-z < file1 | tr -cd 'a-z\012' \ - | awk -f frequency.awk \ - | sort +1 -nr - -produces a table of the words appearing in `file1' in order of -decreasing frequency. - -The first `tr' command in this pipeline translates all the upper case -characters in `file1' to lower case. The second `tr' command deletes -all the characters in the input except lower case characters and -newlines. The second argument to the second `tr' is quoted to -protect the backslash in it from being interpreted by the shell. The -`awk' program reads this suitably massaged data and produces a word -frequency table, which is not ordered. - -The `awk' script's output is now sorted by the `sort' command and -printed on the terminal. The options given to `sort' in this example -specify to sort by the second field of each input line (skipping one -field), that the sort keys should be treated as numeric quantities -(otherwise `15' would come before `5'), and that the sorting should -be done in descending (reverse) order. - -See the general operating system documentation for more information -on how to use the `tr' and `sort' commands. - - - -File: gawk-info, Node: Notes, Next: Glossary, Prev: Sample Program, Up: Top - -Implementation Notes -******************** - -This appendix contains information mainly of interest to implementors -and maintainers of `gawk'. Everything in it applies specifically to -`gawk', and not to other implementations. - -* Menu: - -* Extensions:: Things`gawk' does that Unix `awk' does not. - -* Future Extensions:: Things likely to appear in a future release. - -* Improvements:: Suggestions for future improvements. - -* Manual Improvements:: Suggestions for improvements to this manual. - - - -File: gawk-info, Node: Extensions, Next: Future Extensions, Up: Notes - -GNU Extensions to the AWK Language -================================== - -Several new features are in a state of flux. They are described here -merely to document them somewhat, but they will probably change. We -hope they will be incorporated into other versions of `awk', too. - -All of these features can be turned off either by compiling `gawk' -with `-DSTRICT', or by invoking `gawk' as `awk'. - -The `AWKPATH' environment variable - When opening a file supplied via the `-f' option, if the - filename does not contain a `/', `gawk' will perform a "path - search" for the file, similar to that performed by the shell. - `gawk' gets its search path from the `AWKPATH' environment - variable. If that variable does not exist, it uses the default - path `".:/usr/lib/awk:/usr/local/lib/awk"'. - -Case Independent Matching - Two new operators have been introduced, `~~', and `!~~'. These - perform regular expression match and no-match operations that - are case independent. In other words, `A' and `a' would both - match `/a/'. - -The `-i' option - This option causes the `~' and `!~' operators to behave like the - `~~' and `!~~' operators described above. - -The `-v' option - This option prints version information for this particular copy - of `gawk'. This is so you can determine if your copy of `gawk' - is up to date with respect to whatever the Free Software - Foundation is currently distributing. It may disappear in a - future version of `gawk'. - - - -File: gawk-info, Node: Future Extensions, Next: Improvements, Prev: Extensions, Up: Notes - -Extensions Likely To Appear In A Future Release -=============================================== - -Here are some more extensions that indicate the directions we are -currently considering for `gawk'. Like the previous section, this -section is also subject to change. None of these are implemented yet. - -The `IGNORECASE' special variable - If `IGNORECASE' is non--zero, then *all* regular expression - matching will be done in a case--independent fashion. The `-i' - option and the `~~' and `!~~' operators will go away, as this - mechanism generalizes those facilities. - -More Escape Sequences - The ANSI C `\a', and `\x' escape sequences will be recognized. - Unix `awk' does not recognize `\v', although `gawk' does. - -`RS' as a regexp - The meaning of `RS' will be generalized along the lines of `FS'. - -Transliteration Functions - We are planning on adding `toupper' and `tolower' functions - which will take string arguments, and return strings where the - case of each letter has been transformed to upper-- or - lower--case respectively. - -Access To System File Descriptors - `gawk' will recognize the special file names `/dev/stdin', - `/dev/stdout', `/dev/stderr', and `/dev/fd/N' internally. These - will allow access to inherited file descriptors from within an - `awk' program. - - - -File: gawk-info, Node: Improvements, Next: Manual Improvements, Prev: Future Extensions, Up: Notes - -Suggestions for Future Improvements -=================================== - -Here are some projects that would--be `gawk' hackers might like to -take on. They vary in size from a few days to a few weeks of -programming, depending on which one you choose and how fast a -programmer you are. Please send any improvements you write to the -maintainers at the GNU project. - - 1. State machine regexp matcher: At present, `gawk' uses the - backtracking regular expression matcher from the GNU subroutine - library. If a regexp is really going to be used a lot of times, - it is faster to convert it once to a description of a finite - state machine, then run a routine simulating that machine every - time you want to match the regexp. You could use the matching - routines used by GNU `egrep'. - - 2. Compilation of `awk' programs: `gawk' uses a `Bison' - (YACC--like) parser to convert the script given it into a syntax - tree; the syntax tree is then executed by a simple recursive - evaluator. Both of these steps incur a lot of overhead, since - parsing can be slow (especially if you also do the previous - project and convert regular expressions to finite state machines - at compile time) and the recursive evaluator performs many - procedure calls to do even the simplest things. - - It should be possible for `gawk' to convert the script's parse - tree into a C program which the user would then compile, using - the normal C compiler and a special `gawk' library to provide - all the needed functions (regexps, fields, associative arrays, - type coercion, and so on). - - An easier possibility might be for an intermediate phase of - `awk' to convert the parse tree into a linear byte code form - like the one used in GNU Emacs Lisp. The recursive evaluator - would then be replaced by a straight line byte code interpreter - that would be intermediate in speed between running a compiled - program and doing what `gawk' does now. - - - -File: gawk-info, Node: Manual Improvements, Prev: Improvements, Up: Notes - -Suggestions For Future Improvements of This Manual -================================================== - - 1. An error message section has not been included in this version - of the manual. Perhaps some nice beta testers will document - some of the messages for the future. - - 2. A summary page has not been included, as the ``man'', or help, - page that comes with the `gawk' code should suffice. - - GNU only supports Info, so this manual itself should contain - whatever forms of information it would be useful to have on an - Info summary page. - - 3. A function and variable index has not been included as we are - not sure what to put in it. - - 4. A section summarizing the differences between V7 `awk' and - System V Release 4 `awk' would be useful for long--time `awk' - hackers. - - - -File: gawk-info, Node: Glossary, Next: Index, Prev: Notes, Up: Top - -Glossary -******** - -Action - A series of `awk' statements attached to a rule. If the rule's - pattern matches an input record, the `awk' language executes the - rule's action. Actions are always enclosed in curly braces. - -Amazing `awk' assembler - Henry Spencer at the University of Toronto wrote a retargetable - assembler completely as `awk' scripts. It is thousands of lines - long, including machine descriptions for several 8--bit - microcomputers. It is distributed with `gawk' and is a good - example of a program that would have been better written in - another language. - -Assignment - An `awk' expression that changes the value of some `awk' - variable or data object. An object that you can assign to is - called an "lvalue". - -Built-in function - The `awk' language provides built--in functions that perform - various numerical and string computations. Examples are `sqrt' - (for the square root of a number) and `substr' (for a substring - of a string). - -C - The system programming language that most of GNU is written in. - The `awk' programming language has C--like syntax, and this - manual points out similarities between `awk' and C when - appropriate. - -Compound statement - A series of `awk' statements, enclosed in curly braces. - Compound statements may be nested. - -Concatenation - Concatenating two strings means sticking them together, one - after another, giving a new string. For example, the string - `foo' concatenated with the string `bar' gives the string - `foobar'. - -Conditional expression - A relation that is either true or false, such as `(a < b)'. - Conditional expressions are used in `if' and `while' statements, - and in patterns to select which input records to process. - -Curly braces - The characters `{' and `}'. Curly braces are used in `awk' for - delimiting actions, compound statements, and function bodies. - -Data objects - These are numbers and strings of characters. Numbers are - converted into strings and vice versa, as needed. - -Escape Sequences - A special sequence of characters used for describing - non--printable characters, such as `\n' for newline, or `\033' - for the ASCII ESC (escape) character. - -Field - When `awk' reads an input record, it splits the record into - pieces separated by whitespace (or by a separator regexp which - you can change by setting the special variable `FS'). Such - pieces are called fields. - -Format - Format strings are used to control the appearance of output in - the `printf' statement. Also, data conversions from numbers to - strings are controlled by the format string contained in the - special variable `OFMT'. - -Function - A specialized group of statements often used to encapsulate - general or program--specific tasks. `awk' has a number of - built--in functions, and also allows you to define your own. - -`gawk' - The GNU implementation of `awk'. - -`awk' language - The language in which `awk' programs are written. - -`awk' program - An `awk' program consists of a series of "patterns" and - "actions", collectively known as "rules". For each input record - given to the program, the program's rules are all processed in - turn. `awk' programs may also contain function definitions. - -`awk' script - Another name for an `awk' program. - -Input record - A single chunk of data read in by `awk'. Usually, an `awk' - input record consists of one line of text. - -Keyword - In the `awk' language, a keyword is a word that has special - meaning. Keywords are reserved and may not be used as variable - names. - - The keywords are: `if', `else', `while', `do...while', `for', - `for...in', `break', `continue', `delete', `next', `function', - `func', and `exit'. - -Lvalue - An expression that can appear on the left side of an assignment - operator. In most languages, lvalues can be variables or array - elements. In `awk', a field designator can also be used as an - lvalue. - -Number - A numeric valued data object. The `gawk' implementation uses - double precision floating point to represent numbers. - -Pattern - Patterns tell `awk' which input records are interesting to which - rules. - - A pattern is an arbitrary conditional expression against which - input is tested. If the condition is satisfied, the pattern is - said to "match" the input record. A typical pattern might - compare the input record against a regular expression. - -Range (of input lines) - A sequence of consecutive lines from the input file. A pattern - can specify ranges of input lines for `awk' to process, or it - can specify single lines. - -Recursion - When a function calls itself, either directly or indirectly. If - this isn't clear, refer to the entry for ``recursion''. - -Redirection - Redirection means performing input from other than the standard - input stream, or output to other than the standard output stream. - - You can redirect the output of the `print' and `printf' - statements to a file or a system command, using the `>', `>>', - and `|' operators. You can redirect input to the `getline' - statement using the `<' and `|' operators. - -Regular Expression - See ``regexp''. - -Regexp - Short for "regular expression". A regexp is a pattern that - denotes a set of strings, possibly an infinite set. For - example, the regexp `R.*xp' matches any string starting with the - letter `R' and ending with the letters `xp'. In `awk', regexps - are used in patterns and in conditional expressions. - -Rule - A segment of an `awk' program, that specifies how to process - single input records. A rule consists of a "pattern" and an - "action". `awk' reads an input record; then, for each rule, if - the input record satisfies the rule's pattern, `awk' executes - the rule's action. Otherwise, the rule does nothing for that - input record. - -Special Variable - The variables `ARGC', `ARGV', `ENVIRON', `FILENAME', `FNR', - `FS', `NF', `NR', `OFMT', `OFS', `ORS', `RLENGTH', `RSTART', - `RS', `SUBSEP', have special meaning to `awk'. Changing some of - them affects `awk''s running environment. - -Stream Editor - A program that reads records from an input stream and processes - them one or more at a time. This is in contrast with batch - programs, which may expect to read their input files in entirety - before starting to do anything, and with interactive programs, - which require input from the user. - -String - A datum consisting of a sequence of characters, such as `I am a - string'. Constant strings are written with double--quotes in - the `awk' language, and may contain "escape sequences". - -Whitespace - A sequence of blank or tab characters occurring inside an input - record or a string. - - - -File: gawk-info, Node: Index, Prev: Glossary, Up: Top - -Index -***** - -* Menu: - -* #!: Executable Scripts. -* -f option: Long. -* `$NF', last field in record: Fields. -* `$' (field operator): Fields. -* `>>': Redirection. -* `>': Redirection. -* `BEGIN', special pattern: BEGIN/END. -* `END', special pattern: BEGIN/END. -* `awk' language: This Manual. -* `awk' program: This Manual. -* `break' statement: Break. -* `close' statement for input: Close Input. -* `close' statement for output: Close Output. -* `continue' statement: Continue. -* `delete' statement: Delete. -* `exit' statement: Exit. -* `for (x in ...)': Scanning an Array. -* `for' statement: For. -* `if' statement: If. -* `next' statement: Next. -* `print $0': Very Simple. -* `printf' statement, format of: Basic Printf. -* `printf', format-control characters: Format-Control. -* `printf', modifiers: Modifiers. -* `print' statement: Print. -* `return' statement: Return Statement. -* `while' statement: While. -* `|': Redirection. -* `BBS-list' file: The Files. -* `inventory-shipped' file: The Files. -* Accessing fields: Fields. -* Acronym: History. -* Action, curly braces: Actions. -* Action, curly braces: Getting Started. -* Action, default: Very Simple. -* Action, definition of: Getting Started. -* Action, general: Actions. -* Action, separating statements: Actions. -* Applications of `awk': When. -* Arguments in function call: Function Calls. -* Arguments, Command Line: Command Line. -* Arithmetic operators: Arithmetic Ops. -* Array assignment: Assigning Elements. -* Array reference: Reference to Elements. -* Arrays: Array Intro. -* Arrays, definition of: Array Intro. -* Arrays, deleting an element: Delete. -* Arrays, determining presence of elements: Reference to Elements. -* Arrays, multi-dimensional subscripts: Multi-dimensional. -* Arrays, special `for' statement: Scanning an Array. -* Assignment operators: Assignment Ops. -* Associative arrays: Array Intro. -* Backslash Continuation: Statements/Lines. -* Basic function of `gawk': Getting Started. -* Body of a loop: While. -* Boolean expressions: Boolean Ops. -* Boolean operators: Boolean Ops. -* Boolean patterns: Boolean. -* Built-in functions, list of: Built-in. -* Built-in variables: Variables. -* Calling a function: Function Calls. -* Case sensitivity and gawk: Read Terminal. -* Changing contents of a field: Changing Fields. -* Changing the record separator: Records. -* Closing files and pipes: Close Output. -* Command Line: Command Line. -* Command line formats: Running gawk. -* Command line, setting `FS' on: Field Separators. -* Comments: Comments. -* Comparison expressions: Comparison Ops. -* Comparison expressions as patterns: Comparison Patterns. -* Compound statements: Actions. -* Computed Regular Expressions: Regexp Usage. -* Concatenation: Concatenation. -* Conditional Patterns: Conditional Patterns. -* Conditional expression: Conditional Exp. -* Constants, types of: Constants. -* Continuing statements on the next line: Statements/Lines. -* Conversion of strings and numbers: Conversion. -* Curly braces: Actions. -* Curly braces: Getting Started. -* Default action: Very Simple. -* Default pattern: Very Simple. -* Deleting elements of arrays: Delete. -* Differences between `gawk' and `awk': Arithmetic Ops. -* Differences between `gawk' and `awk': Constants. -* Documenting `awk' programs: Comments. -* Dynamic Regular Expressions: Regexp Usage. -* Element assignment: Assigning Elements. -* Element of array: Reference to Elements. -* Emacs Lisp: When. -* Empty pattern: Empty. -* Escape sequence notation: Constants. -* Examining fields: Fields. -* Executable Scripts: Executable Scripts. -* Expression, conditional: Conditional Exp. -* Expressions: Actions. -* Expressions, boolean: Boolean Ops. -* Expressions, comparison: Comparison Ops. -* Field separator, `FS': Field Separators. -* Field separator, choice of: Field Separators. -* Field separator, setting on command line: Field Separators. -* Field, changing contents of: Changing Fields. -* Fields: Fields. -* Fields, negative-numbered: Non-Constant Fields. -* Fields, semantics of: Field Separators. -* Fields, separating: Field Separators. -* Format specifier: Format-Control. -* Format string: Basic Printf. -* Formatted output: Printf. -* Function call: Function Calls. -* Function definitions: Actions. -* Functions, user-defined: User-defined. -* General input: Reading Files. -* History of `awk': History. -* How gawk works: Two Rules. -* Increment operators: Increment Ops. -* Input file, sample: The Files. -* Input, `getline' function: Getline. -* Input, general: Reading Files. -* Input, multiple line records: Multiple. -* Input, standard: Read Terminal. -* Input, standard: Reading Files. -* Interaction of `awk' with other programs: I/O Functions. -* Invocation of `gawk': Command Line. -* Language, `awk': This Manual. -* Loop: While. -* Loops, breaking out of: Break. -* Lvalue: Assignment Ops. -* Manual, using this: This Manual. -* Metacharacters: Regexp Operators. -* Mod function, semantics of: Arithmetic Ops. -* Modifiers (in format specifiers): Modifiers. -* Multiple line records: Multiple. -* Multiple passes over data: Command Line. -* Multiple statements on one line: Statements/Lines. -* Negative-numbered fields: Non-Constant Fields. -* Number of fields, `NF': Fields. -* Number of records, `FNR': Records. -* Number of records, `NR': Records. -* Numerical constant: Constants. -* Numerical value: Constants. -* One-liners: One-liners. -* Operator, Ternary: Conditional Patterns. -* Operators, `$': Fields. -* Operators, arithmetic: Arithmetic Ops. -* Operators, assignment: Assignment Ops. -* Operators, boolean: Boolean Ops. -* Operators, increment: Increment Ops. -* Operators, regular expression matching: Regexp Usage. -* Operators, relational: Comparison Ops. -* Operators, relational: Comparison Patterns. -* Operators, string: Concatenation. -* Operators, string-matching: Regexp Usage. -* Options, Command Line: Command Line. -* Output: Printing. -* Output field separator, `OFS': Output Separators. -* Output record separator, `ORS': Output Separators. -* Output redirection: Redirection. -* Output, formatted: Printf. -* Output, piping: Redirection. -* Passes, Multiple: Command Line. -* Pattern, case sensitive: Read Terminal. -* Pattern, comparison expressions: Comparison Patterns. -* Pattern, default: Very Simple. -* Pattern, definition of: Getting Started. -* Pattern, empty: Empty. -* Pattern, regular expressions: Regexp. -* Patterns, `BEGIN': BEGIN/END. -* Patterns, `END': BEGIN/END. -* Patterns, Conditional: Conditional Patterns. -* Patterns, boolean: Boolean. -* Patterns, definition of: Patterns. -* Patterns, types of: Patterns. -* Pipes for output: Redirection. -* Printing, general: Printing. -* Program, `awk': This Manual. -* Program, Self contained: Executable Scripts. -* Program, definition of: Getting Started. -* Programs, documenting: Comments. -* Range pattern: Ranges. -* Reading files, `getline' function: Getline. -* Reading files, general: Reading Files. -* Reading files, multiple line records: Multiple. -* Record separator, `RS': Records. -* Records, multiple line: Multiple. -* Redirection of output: Redirection. -* Reference to array: Reference to Elements. -* Regexp: Regexp. -* Regular Expressions, Computed: Regexp Usage. -* Regular Expressions, Dynamic: Regexp Usage. -* Regular expression matching operators: Regexp Usage. -* Regular expression, metacharacters: Regexp Operators. -* Regular expressions as patterns: Regexp. -* Regular expressions, field separators and: Field Separators. -* Relational operators: Comparison Patterns. -* Relational operators: Comparison Ops. -* Removing elements of arrays: Delete. -* Rule, definition of: Getting Started. -* Running gawk programs: Running gawk. -* Sample input file: The Files. -* Scanning an array: Scanning an Array. -* Script, definition of: Getting Started. -* Scripts, Executable: Executable Scripts. -* Scripts, Shell: Executable Scripts. -* Self contained Programs: Executable Scripts. -* Separator character, choice of: Field Separators. -* Shell Scripts: Executable Scripts. -* Single quotes, why they are needed: One-shot. -* Special variables, user modifiable: User-modified. -* Standard input: Read Terminal. -* Standard input: Reading Files. -* Statements: Statements. -* Statements: Actions. -* String constants: Constants. -* String operators: Concatenation. -* String value: Constants. -* String-matching operators: Regexp Usage. -* Subscripts, multi-dimensional in arrays: Multi-dimensional. -* Ternary Operator: Conditional Patterns. -* Use of comments: Comments. -* User-defined functions: User-defined. -* User-defined variables: Variables. -* Uses of `awk': Preface. -* Using this manual: This Manual. -* Variables, built-in: Variables. -* Variables, user-defined: Variables. -* What is `awk': Preface. -* When to use `awk': When. -* file, `awk' program: Long. -* patterns, range: Ranges. -* program file: Long. -* regexp search operators: Regexp Usage. -* running long programs: Long. - - - -Tag Table: -Node: Top918 -Node: Preface2804 -Node: History4267 -Node: License5644 -Node: This Manual18989 -Node: The Files20330 -Node: Getting Started22914 -Node: Very Simple24249 -Node: Two Rules26030 -Node: More Complex28066 -Node: Running gawk30908 -Node: One-shot31827 -Node: Read Terminal32945 -Node: Long33862 -Node: Executable Scripts34991 -Node: Command Line36534 -Node: Comments40168 -Node: Statements/Lines41067 -Node: When43498 -Node: Reading Files45420 -Node: Records47119 -Node: Fields49902 -Node: Non-Constant Fields52789 -Node: Changing Fields54591 -Node: Field Separators57302 -Node: Multiple62004 -Node: Assignment Options64393 -Node: Getline65608 -Node: Close Input74958 -Node: Printing76023 -Node: Print76748 -Node: Print Examples78712 -Node: Output Separators80751 -Node: Redirection82417 -Node: Close Output85886 -Node: Printf88132 -Node: Basic Printf88908 -Node: Format-Control90261 -Node: Modifiers91806 -Node: Printf Examples93108 -Node: One-liners95707 -Node: Patterns97642 -Node: Empty100130 -Node: Regexp100402 -Node: Regexp Usage101173 -Node: Regexp Operators102947 -Node: Comparison Patterns107890 -Node: Ranges109336 -Node: BEGIN/END110722 -Node: Boolean113151 -Node: Conditional Patterns115605 -Node: Actions116105 -Node: Expressions117435 -Node: Constants119124 -Node: Variables121097 -Node: Arithmetic Ops122454 -Node: Concatenation123840 -Node: Comparison Ops124569 -Node: Boolean Ops125973 -Node: Assignment Ops128266 -Node: Increment Ops131817 -Node: Conversion134112 -Node: Conditional Exp136066 -Node: Function Calls137384 -Node: Statements139939 -Node: If141253 -Node: While142627 -Node: Do144232 -Node: For145265 -Node: Break148306 -Node: Continue149848 -Node: Next151476 -Node: Exit152985 -Node: Arrays154514 -Node: Array Intro155624 -Node: Reference to Elements159227 -Node: Assigning Elements161115 -Node: Array Example161615 -Node: Scanning an Array163336 -Node: Delete165642 -Node: Multi-dimensional166529 -Node: Multi-scanning169746 -Node: Built-in171303 -Node: Numeric Functions172806 -Node: String Functions176601 -Node: I/O Functions183717 -Node: User-defined185189 -Node: Definition Syntax185834 -Node: Function Example187928 -Node: Function Caveats189034 -Node: Return Statement191386 -Node: Special193612 -Node: User-modified194478 -Node: Auto-set196511 -Node: Sample Program200558 -Node: Notes204316 -Node: Extensions204909 -Node: Future Extensions206490 -Node: Improvements207922 -Node: Manual Improvements210034 -Node: Glossary210928 -Node: Index217934 - -End Tag Table |