diff options
Diffstat (limited to 'gawk.info-2')
| -rw-r--r-- | gawk.info-2 | 1236 |
1 files changed, 1236 insertions, 0 deletions
diff --git a/gawk.info-2 b/gawk.info-2 new file mode 100644 index 00000000..954b4309 --- /dev/null +++ b/gawk.info-2 @@ -0,0 +1,1236 @@ +This is Info file gawk.info, produced by Makeinfo-1.54 from the input +file gawk.texi. + + This file documents `awk', a program that you can use to select +particular records in a file and perform operations upon them. + + This is Edition 0.15 of `The GAWK Manual', +for the 2.15 version of the GNU implementation +of AWK. + + Copyright (C) 1989, 1991, 1992, 1993 Free Software Foundation, Inc. + + Permission is granted to make and distribute verbatim copies of this +manual provided the copyright notice and this permission notice are +preserved on all copies. + + Permission is granted to copy and distribute modified versions of +this manual under the conditions for verbatim copying, provided that +the entire resulting derived work is distributed under the terms of a +permission notice identical to this one. + + Permission is granted to copy and distribute translations of this +manual into another language, under the above conditions for modified +versions, except that this permission notice may be stated in a +translation approved by the Foundation. + + +File: gawk.info, Node: Statements/Lines, Next: When, Prev: Comments, Up: Getting Started + +`awk' Statements versus Lines +============================= + + Most often, each line in an `awk' program is a separate statement or +separate rule, like this: + + awk '/12/ { print $0 } + /21/ { print $0 }' BBS-list inventory-shipped + + But sometimes statements can be more than one line, and lines can +contain several statements. You can split a statement into multiple +lines by inserting a newline after any of the following: + + , { ? : || && do else + +A newline at any other point is considered the end of the statement. +(Splitting lines after `?' and `:' is a minor `gawk' extension. The +`?' and `:' referred to here is the three operand conditional +expression described in *Note Conditional Expressions: Conditional Exp.) + + If you would like to split a single statement into two lines at a +point where a newline would terminate it, you can "continue" it by +ending the first line with a backslash character, `\'. This is allowed +absolutely anywhere in the statement, even in the middle of a string or +regular expression. For example: + + awk '/This program is too long, so continue it\ + on the next line/ { print $1 }' + +We have generally not used backslash continuation in the sample +programs in this manual. Since in `gawk' there is no limit on the +length of a line, it is never strictly necessary; it just makes +programs prettier. We have preferred to make them even more pretty by +keeping the statements short. Backslash continuation is most useful +when your `awk' program is in a separate source file, instead of typed +in on the command line. You should also note that many `awk' +implementations are more picky about where you may use backslash +continuation. For maximal portability of your `awk' programs, it is +best not to split your lines in the middle of a regular expression or a +string. + + *Warning: backslash continuation does not work as described above +with the C shell.* Continuation with backslash works for `awk' +programs in files, and also for one-shot programs *provided* you are +using a POSIX-compliant shell, such as the Bourne shell or the +Bourne-again shell. But the C shell used on Berkeley Unix behaves +differently! There, you must use two backslashes in a row, followed by +a newline. + + When `awk' statements within one rule are short, you might want to +put more than one of them on a line. You do this by separating the +statements with a semicolon, `;'. This also applies to the rules +themselves. Thus, the previous program could have been written: + + /12/ { print $0 } ; /21/ { print $0 } + +*Note:* the requirement that rules on the same line must be separated +with a semicolon is a recent change in the `awk' language; it was done +for consistency with the treatment of statements within an action. + + +File: gawk.info, Node: When, Prev: Statements/Lines, Up: Getting Started + +When to Use `awk' +================= + + You might wonder how `awk' might be useful for you. Using additional +utility programs, more advanced patterns, field separators, arithmetic +statements, and other selection criteria, you can produce much more +complex output. The `awk' language is very useful for producing +reports from large amounts of raw data, such as summarizing information +from the output of other utility programs like `ls'. (*Note A More +Complex Example: More Complex.) + + Programs written with `awk' are usually much smaller than they would +be in other languages. This makes `awk' programs easy to compose and +use. Often `awk' programs can be quickly composed at your terminal, +used once, and thrown away. Since `awk' programs are interpreted, you +can avoid the usually lengthy edit-compile-test-debug cycle of software +development. + + Complex programs have been written in `awk', including a complete +retargetable assembler for 8-bit microprocessors (*note Glossary::., for +more information) and a microcode assembler for a special purpose Prolog +computer. However, `awk''s capabilities are strained by tasks of such +complexity. + + If you find yourself writing `awk' scripts of more than, say, a few +hundred lines, you might consider using a different programming +language. Emacs Lisp is a good choice if you need sophisticated string +or pattern matching capabilities. The shell is also good at string and +pattern matching; in addition, it allows powerful use of the system +utilities. More conventional languages, such as C, C++, and Lisp, offer +better facilities for system programming and for managing the complexity +of large programs. Programs in these languages may require more lines +of source code than the equivalent `awk' programs, but they are easier +to maintain and usually run more efficiently. + + +File: gawk.info, Node: Reading Files, Next: Printing, Prev: Getting Started, Up: Top + +Reading Input Files +******************* + + In the typical `awk' program, all input is read either from the +standard input (by default the keyboard, but often a pipe from another +command) or from files whose names you specify on the `awk' command +line. If you specify input files, `awk' reads them in order, reading +all the data from one before going on to the next. The name of the +current input file can be found in the built-in variable `FILENAME' +(*note Built-in Variables::.). + + The input is read in units called records, and processed by the +rules one record at a time. By default, each record is one line. Each +record is split automatically into fields, to make it more convenient +for a rule to work on its parts. + + On rare occasions you will need to use the `getline' command, which +can do explicit input from any number of files (*note Explicit Input +with `getline': Getline.). + +* Menu: + +* Records:: Controlling how data is split into records. +* Fields:: An introduction to fields. +* Non-Constant Fields:: Non-constant Field Numbers. +* Changing Fields:: Changing the Contents of a Field. +* Field Separators:: The field separator and how to change it. +* Constant Size:: Reading constant width data. +* Multiple Line:: Reading multi-line records. +* Getline:: Reading files under explicit program control + using the `getline' function. +* Close Input:: Closing an input file (so you can read from + the beginning once more). + + +File: gawk.info, Node: Records, Next: Fields, Prev: Reading Files, Up: Reading Files + +How Input is Split into Records +=============================== + + The `awk' language divides its input into records and fields. +Records are separated by a character called the "record separator". By +default, the record separator is the newline character, defining a +record to be a single line of text. + + Sometimes you may want to use a different character to separate your +records. You can use a different character by changing the built-in +variable `RS'. The value of `RS' is a string that says how to separate +records; the default value is `"\n"', the string containing just a +newline character. This is why records are, by default, single lines. + + `RS' can have any string as its value, but only the first character +of the string is used as the record separator. The other characters are +ignored. `RS' is exceptional in this regard; `awk' uses the full value +of all its other built-in variables. + + You can change the value of `RS' in the `awk' program with the +assignment operator, `=' (*note Assignment Expressions: Assignment +Ops.). The new record-separator character should be enclosed in +quotation marks to make a string constant. Often the right time to do +this is at the beginning of execution, before any input has been +processed, so that the very first record will be read with the proper +separator. To do this, use the special `BEGIN' pattern (*note `BEGIN' +and `END' Special Patterns: BEGIN/END.). For example: + + awk 'BEGIN { RS = "/" } ; { print $0 }' BBS-list + +changes the value of `RS' to `"/"', before reading any input. This is +a string whose first character is a slash; as a result, records are +separated by slashes. Then the input file is read, and the second rule +in the `awk' program (the action with no pattern) prints each record. +Since each `print' statement adds a newline at the end of its output, +the effect of this `awk' program is to copy the input with each slash +changed to a newline. + + Another way to change the record separator is on the command line, +using the variable-assignment feature (*note Invoking `awk': Command +Line.). + + awk '{ print $0 }' RS="/" BBS-list + +This sets `RS' to `/' before processing `BBS-list'. + + Reaching the end of an input file terminates the current input +record, even if the last character in the file is not the character in +`RS'. + + The empty string, `""' (a string of no characters), has a special +meaning as the value of `RS': it means that records are separated only +by blank lines. *Note Multiple-Line Records: Multiple Line, for more +details. + + The `awk' utility keeps track of the number of records that have +been read so far from the current input file. This value is stored in a +built-in variable called `FNR'. It is reset to zero when a new file is +started. Another built-in variable, `NR', is the total number of input +records read so far from all files. It starts at zero but is never +automatically reset to zero. + + If you change the value of `RS' in the middle of an `awk' run, the +new value is used to delimit subsequent records, but the record +currently being processed (and records already processed) are not +affected. + + +File: gawk.info, Node: Fields, Next: Non-Constant Fields, Prev: Records, Up: Reading Files + +Examining Fields +================ + + When `awk' reads an input record, the record is automatically +separated or "parsed" by the interpreter into chunks called "fields". +By default, fields are separated by whitespace, like words in a line. +Whitespace in `awk' means any string of one or more spaces and/or tabs; +other characters such as newline, formfeed, and so on, that are +considered whitespace by other languages are *not* considered +whitespace by `awk'. + + The purpose of fields is to make it more convenient for you to refer +to these pieces of the record. You don't have to use them--you can +operate on the whole record if you wish--but fields are what make +simple `awk' programs so powerful. + + To refer to a field in an `awk' program, you use a dollar-sign, `$', +followed by the number of the field you want. Thus, `$1' refers to the +first field, `$2' to the second, and so on. For example, suppose the +following is a line of input: + + This seems like a pretty nice example. + +Here the first field, or `$1', is `This'; the second field, or `$2', is +`seems'; and so on. Note that the last field, `$7', is `example.'. +Because there is no space between the `e' and the `.', the period is +considered part of the seventh field. + + No matter how many fields there are, the last field in a record can +be represented by `$NF'. So, in the example above, `$NF' would be the +same as `$7', which is `example.'. Why this works is explained below +(*note Non-constant Field Numbers: Non-Constant Fields.). If you try +to refer to a field beyond the last one, such as `$8' when the record +has only 7 fields, you get the empty string. + + Plain `NF', with no `$', is a built-in variable whose value is the +number of fields in the current record. + + `$0', which looks like an attempt to refer to the zeroth field, is a +special case: it represents the whole input record. This is what you +would use if you weren't interested in fields. + + Here are some more examples: + + awk '$1 ~ /foo/ { print $0 }' BBS-list + +This example prints each record in the file `BBS-list' whose first +field contains the string `foo'. The operator `~' is called a +"matching operator" (*note Comparison Expressions: Comparison Ops.); it +tests whether a string (here, the field `$1') matches a given regular +expression. + + By contrast, the following example: + + awk '/foo/ { print $1, $NF }' BBS-list + +looks for `foo' in *the entire record* and prints the first field and +the last field for each input record containing a match. + + +File: gawk.info, Node: Non-Constant Fields, Next: Changing Fields, Prev: Fields, Up: Reading Files + +Non-constant Field Numbers +========================== + + The number of a field does not need to be a constant. Any +expression in the `awk' language can be used after a `$' to refer to a +field. The value of the expression specifies the field number. If the +value is a string, rather than a number, it is converted to a number. +Consider this example: + + awk '{ print $NR }' + +Recall that `NR' is the number of records read so far: 1 in the first +record, 2 in the second, etc. So this example prints the first field +of the first record, the second field of the second record, and so on. +For the twentieth record, field number 20 is printed; most likely, the +record has fewer than 20 fields, so this prints a blank line. + + Here is another example of using expressions as field numbers: + + awk '{ print $(2*2) }' BBS-list + + The `awk' language must evaluate the expression `(2*2)' and use its +value as the number of the field to print. The `*' sign represents +multiplication, so the expression `2*2' evaluates to 4. The +parentheses are used so that the multiplication is done before the `$' +operation; they are necessary whenever there is a binary operator in +the field-number expression. This example, then, prints the hours of +operation (the fourth field) for every line of the file `BBS-list'. + + If the field number you compute is zero, you get the entire record. +Thus, `$(2-2)' has the same value as `$0'. Negative field numbers are +not allowed. + + The number of fields in the current record is stored in the built-in +variable `NF' (*note Built-in Variables::.). The expression `$NF' is +not a special feature: it is the direct consequence of evaluating `NF' +and using its value as a field number. + + +File: gawk.info, Node: Changing Fields, Next: Field Separators, Prev: Non-Constant Fields, Up: Reading Files + +Changing the Contents of a Field +================================ + + You can change the contents of a field as seen by `awk' within an +`awk' program; this changes what `awk' perceives as the current input +record. (The actual input is untouched: `awk' never modifies the input +file.) + + Consider this example: + + awk '{ $3 = $2 - 10; print $2, $3 }' inventory-shipped + +The `-' sign represents subtraction, so this program reassigns field +three, `$3', to be the value of field two minus ten, `$2 - 10'. (*Note +Arithmetic Operators: Arithmetic Ops.) Then field two, and the new +value for field three, are printed. + + In order for this to work, the text in field `$2' must make sense as +a number; the string of characters must be converted to a number in +order for the computer to do arithmetic on it. The number resulting +from the subtraction is converted back to a string of characters which +then becomes field three. *Note Conversion of Strings and Numbers: +Conversion. + + When you change the value of a field (as perceived by `awk'), the +text of the input record is recalculated to contain the new field where +the old one was. Therefore, `$0' changes to reflect the altered field. +Thus, + + awk '{ $2 = $2 - 10; print $0 }' inventory-shipped + +prints a copy of the input file, with 10 subtracted from the second +field of each line. + + You can also assign contents to fields that are out of range. For +example: + + awk '{ $6 = ($5 + $4 + $3 + $2) ; print $6 }' inventory-shipped + +We've just created `$6', whose value is the sum of fields `$2', `$3', +`$4', and `$5'. The `+' sign represents addition. For the file +`inventory-shipped', `$6' represents the total number of parcels +shipped for a particular month. + + Creating a new field changes the internal `awk' copy of the current +input record--the value of `$0'. Thus, if you do `print $0' after +adding a field, the record printed includes the new field, with the +appropriate number of field separators between it and the previously +existing fields. + + This recomputation affects and is affected by several features not +yet discussed, in particular, the "output field separator", `OFS', +which is used to separate the fields (*note Output Separators::.), and +`NF' (the number of fields; *note Examining Fields: Fields.). For +example, the value of `NF' is set to the number of the highest field +you create. + + Note, however, that merely *referencing* an out-of-range field does +*not* change the value of either `$0' or `NF'. Referencing an +out-of-range field merely produces a null string. For example: + + if ($(NF+1) != "") + print "can't happen" + else + print "everything is normal" + +should print `everything is normal', because `NF+1' is certain to be +out of range. (*Note The `if' Statement: If Statement, for more +information about `awk''s `if-else' statements.) + + It is important to note that assigning to a field will change the +value of `$0', but will not change the value of `NF', even when you +assign the null string to a field. For example: + + echo a b c d | awk '{ OFS = ":"; $2 = "" ; print ; print NF }' + +prints + + a::c:d + 4 + +The field is still there, it just has an empty value. You can tell +because there are two colons in a row. + + +File: gawk.info, Node: Field Separators, Next: Constant Size, Prev: Changing Fields, Up: Reading Files + +Specifying how Fields are Separated +=================================== + + (This section is rather long; it describes one of the most +fundamental operations in `awk'. If you are a novice with `awk', we +recommend that you re-read this section after you have studied the +section on regular expressions, *Note Regular Expressions as Patterns: +Regexp.) + + The way `awk' splits an input record into fields is controlled by +the "field separator", which is a single character or a regular +expression. `awk' scans the input record for matches for the +separator; the fields themselves are the text between the matches. For +example, if the field separator is `oo', then the following line: + + moo goo gai pan + +would be split into three fields: `m', ` g' and ` gai pan'. + + The field separator is represented by the built-in variable `FS'. +Shell programmers take note! `awk' does not use the name `IFS' which +is used by the shell. + + You can change the value of `FS' in the `awk' program with the +assignment operator, `=' (*note Assignment Expressions: Assignment +Ops.). Often the right time to do this is at the beginning of +execution, before any input has been processed, so that the very first +record will be read with the proper separator. To do this, use the +special `BEGIN' pattern (*note `BEGIN' and `END' Special Patterns: +BEGIN/END.). For example, here we set the value of `FS' to the string +`","': + + awk 'BEGIN { FS = "," } ; { print $2 }' + +Given the input line, + + John Q. Smith, 29 Oak St., Walamazoo, MI 42139 + +this `awk' program extracts the string ` 29 Oak St.'. + + Sometimes your input data will contain separator characters that +don't separate fields the way you thought they would. For instance, the +person's name in the example we've been using might have a title or +suffix attached, such as `John Q. Smith, LXIX'. From input containing +such a name: + + John Q. Smith, LXIX, 29 Oak St., Walamazoo, MI 42139 + +the previous sample program would extract ` LXIX', instead of ` 29 Oak +St.'. If you were expecting the program to print the address, you +would be surprised. So choose your data layout and separator +characters carefully to prevent such problems. + + As you know, by default, fields are separated by whitespace sequences +(spaces and tabs), not by single spaces: two spaces in a row do not +delimit an empty field. The default value of the field separator is a +string `" "' containing a single space. If this value were interpreted +in the usual way, each space character would separate fields, so two +spaces in a row would make an empty field between them. The reason +this does not happen is that a single space as the value of `FS' is a +special case: it is taken to specify the default manner of delimiting +fields. + + If `FS' is any other single character, such as `","', then each +occurrence of that character separates two fields. Two consecutive +occurrences delimit an empty field. If the character occurs at the +beginning or the end of the line, that too delimits an empty field. The +space character is the only single character which does not follow these +rules. + + More generally, the value of `FS' may be a string containing any +regular expression. Then each match in the record for the regular +expression separates fields. For example, the assignment: + + FS = ", \t" + +makes every area of an input line that consists of a comma followed by a +space and a tab, into a field separator. (`\t' stands for a tab.) + + For a less trivial example of a regular expression, suppose you want +single spaces to separate fields the way single commas were used above. +You can set `FS' to `"[ ]"'. This regular expression matches a single +space and nothing else. + + `FS' can be set on the command line. You use the `-F' argument to +do so. For example: + + awk -F, 'PROGRAM' INPUT-FILES + +sets `FS' to be the `,' character. Notice that the argument uses a +capital `F'. Contrast this with `-f', which specifies a file +containing an `awk' program. Case is significant in command options: +the `-F' and `-f' options have nothing to do with each other. You can +use both options at the same time to set the `FS' argument *and* get an +`awk' program from a file. + + The value used for the argument to `-F' is processed in exactly the +same way as assignments to the built-in variable `FS'. This means that +if the field separator contains special characters, they must be escaped +appropriately. For example, to use a `\' as the field separator, you +would have to type: + + # same as FS = "\\" + awk -F\\\\ '...' files ... + +Since `\' is used for quoting in the shell, `awk' will see `-F\\'. +Then `awk' processes the `\\' for escape characters (*note Constant +Expressions: Constants.), finally yielding a single `\' to be used for +the field separator. + + As a special case, in compatibility mode (*note Invoking `awk': +Command Line.), if the argument to `-F' is `t', then `FS' is set to the +tab character. (This is because if you type `-F\t', without the quotes, +at the shell, the `\' gets deleted, so `awk' figures that you really +want your fields to be separated with tabs, and not `t's. Use `-v +FS="t"' on the command line if you really do want to separate your +fields with `t's.) + + For example, let's use an `awk' program file called `baud.awk' that +contains the pattern `/300/', and the action `print $1'. Here is the +program: + + /300/ { print $1 } + + Let's also set `FS' to be the `-' character, and run the program on +the file `BBS-list'. The following command prints a list of the names +of the bulletin boards that operate at 300 baud and the first three +digits of their phone numbers: + + awk -F- -f baud.awk BBS-list + +It produces this output: + + aardvark 555 + alpo + barfly 555 + bites 555 + camelot 555 + core 555 + fooey 555 + foot 555 + macfoo 555 + sdace 555 + sabafoo 555 + +Note the second line of output. If you check the original file, you +will see that the second line looked like this: + + alpo-net 555-3412 2400/1200/300 A + + The `-' as part of the system's name was used as the field +separator, instead of the `-' in the phone number that was originally +intended. This demonstrates why you have to be careful in choosing +your field and record separators. + + The following program searches the system password file, and prints +the entries for users who have no password: + + awk -F: '$2 == ""' /etc/passwd + +Here we use the `-F' option on the command line to set the field +separator. Note that fields in `/etc/passwd' are separated by colons. +The second field represents a user's encrypted password, but if the +field is empty, that user has no password. + + According to the POSIX standard, `awk' is supposed to behave as if +each record is split into fields at the time that it is read. In +particular, this means that you can change the value of `FS' after a +record is read, but before any of the fields are referenced. The value +of the fields (i.e. how they were split) should reflect the old value +of `FS', not the new one. + + However, many implementations of `awk' do not do this. Instead, +they defer splitting the fields until a field reference actually +happens, using the *current* value of `FS'! This behavior can be +difficult to diagnose. The following example illustrates the results of +the two methods. (The `sed' command prints just the first line of +`/etc/passwd'.) + + sed 1q /etc/passwd | awk '{ FS = ":" ; print $1 }' + +will usually print + + root + +on an incorrect implementation of `awk', while `gawk' will print +something like + + root:nSijPlPhZZwgE:0:0:Root:/: + + There is an important difference between the two cases of `FS = " "' +(a single blank) and `FS = "[ \t]+"' (which is a regular expression +matching one or more blanks or tabs). For both values of `FS', fields +are separated by runs of blanks and/or tabs. However, when the value of +`FS' is `" "', `awk' will strip leading and trailing whitespace from +the record, and then decide where the fields are. + + For example, the following expression prints `b': + + echo ' a b c d ' | awk '{ print $2 }' + +However, the following prints `a': + + echo ' a b c d ' | awk 'BEGIN { FS = "[ \t]+" } ; { print $2 }' + +In this case, the first field is null. + + The stripping of leading and trailing whitespace also comes into +play whenever `$0' is recomputed. For instance, this pipeline + + echo ' a b c d' | awk '{ print; $2 = $2; print }' + +produces this output: + + a b c d + a b c d + +The first `print' statement prints the record as it was read, with +leading whitespace intact. The assignment to `$2' rebuilds `$0' by +concatenating `$1' through `$NF' together, separated by the value of +`OFS'. Since the leading whitespace was ignored when finding `$1', it +is not part of the new `$0'. Finally, the last `print' statement +prints the new `$0'. + + The following table summarizes how fields are split, based on the +value of `FS'. + +`FS == " "' + Fields are separated by runs of whitespace. Leading and trailing + whitespace are ignored. This is the default. + +`FS == ANY SINGLE CHARACTER' + Fields are separated by each occurrence of the character. Multiple + successive occurrences delimit empty fields, as do leading and + trailing occurrences. + +`FS == REGEXP' + Fields are separated by occurrences of characters that match + REGEXP. Leading and trailing matches of REGEXP delimit empty + fields. + + +File: gawk.info, Node: Constant Size, Next: Multiple Line, Prev: Field Separators, Up: Reading Files + +Reading Fixed-width Data +======================== + + (This section discusses an advanced, experimental feature. If you +are a novice `awk' user, you may wish to skip it on the first reading.) + + `gawk' 2.13 introduced a new facility for dealing with fixed-width +fields with no distinctive field separator. Data of this nature arises +typically in one of at least two ways: the input for old FORTRAN +programs where numbers are run together, and the output of programs +that did not anticipate the use of their output as input for other +programs. + + An example of the latter is a table where all the columns are lined +up by the use of a variable number of spaces and *empty fields are just +spaces*. Clearly, `awk''s normal field splitting based on `FS' will +not work well in this case. (Although a portable `awk' program can use +a series of `substr' calls on `$0', this is awkward and inefficient for +a large number of fields.) + + The splitting of an input record into fixed-width fields is +specified by assigning a string containing space-separated numbers to +the built-in variable `FIELDWIDTHS'. Each number specifies the width +of the field *including* columns between fields. If you want to ignore +the columns between fields, you can specify the width as a separate +field that is subsequently ignored. + + The following data is the output of the `w' utility. It is useful +to illustrate the use of `FIELDWIDTHS'. + + 10:06pm up 21 days, 14:04, 23 users + User tty login idle JCPU PCPU what + hzuo ttyV0 8:58pm 9 5 vi p24.tex + hzang ttyV3 6:37pm 50 -csh + eklye ttyV5 9:53pm 7 1 em thes.tex + dportein ttyV6 8:17pm 1:47 -csh + gierd ttyD3 10:00pm 1 elm + dave ttyD4 9:47pm 4 4 w + brent ttyp0 26Jun91 4:46 26:46 4:41 bash + dave ttyq4 26Jun9115days 46 46 wnewmail + + The following program takes the above input, converts the idle time +to number of seconds and prints out the first two fields and the +calculated idle time. (This program uses a number of `awk' features +that haven't been introduced yet.) + + BEGIN { FIELDWIDTHS = "9 6 10 6 7 7 35" } + NR > 2 { + idle = $4 + sub(/^ */, "", idle) # strip leading spaces + if (idle == "") idle = 0 + if (idle ~ /:/) { split(idle, t, ":"); idle = t[1] * 60 + t[2] } + if (idle ~ /days/) { idle *= 24 * 60 * 60 } + + print $1, $2, idle + } + + Here is the result of running the program on the data: + + hzuo ttyV0 0 + hzang ttyV3 50 + eklye ttyV5 0 + dportein ttyV6 107 + gierd ttyD3 1 + dave ttyD4 0 + brent ttyp0 286 + dave ttyq4 1296000 + + Another (possibly more practical) example of fixed-width input data +would be the input from a deck of balloting cards. In some parts of +the United States, voters make their choices by punching holes in +computer cards. These cards are then processed to count the votes for +any particular candidate or on any particular issue. Since a voter may +choose not to vote on some issue, any column on the card may be empty. +An `awk' program for processing such data could use the `FIELDWIDTHS' +feature to simplify reading the data. + + This feature is still experimental, and will likely evolve over time. + + +File: gawk.info, Node: Multiple Line, Next: Getline, Prev: Constant Size, Up: Reading Files + +Multiple-Line Records +===================== + + In some data bases, a single line cannot conveniently hold all the +information in one entry. In such cases, you can use multi-line +records. + + The first step in doing this is to choose your data format: when +records are not defined as single lines, how do you want to define them? +What should separate records? + + One technique is to use an unusual character or string to separate +records. For example, you could use the formfeed character (written +`\f' in `awk', as in C) to separate them, making each record a page of +the file. To do this, just set the variable `RS' to `"\f"' (a string +containing the formfeed character). Any other character could equally +well be used, as long as it won't be part of the data in a record. + + Another technique is to have blank lines separate records. By a +special dispensation, a null string as the value of `RS' indicates that +records are separated by one or more blank lines. If you set `RS' to +the null string, a record always ends at the first blank line +encountered. And the next record doesn't start until the first nonblank +line that follows--no matter how many blank lines appear in a row, they +are considered one record-separator. (End of file is also considered a +record separator.) + + The second step is to separate the fields in the record. One way to +do this is to put each field on a separate line: to do this, just set +the variable `FS' to the string `"\n"'. (This simple regular +expression matches a single newline.) + + Another way to separate fields is to divide each of the lines into +fields in the normal manner. This happens by default as a result of a +special feature: when `RS' is set to the null string, the newline +character *always* acts as a field separator. This is in addition to +whatever field separations result from `FS'. + + The original motivation for this special exception was probably so +that you get useful behavior in the default case (i.e., `FS == " "'). +This feature can be a problem if you really don't want the newline +character to separate fields, since there is no way to prevent it. +However, you can work around this by using the `split' function to +break up the record manually (*note Built-in Functions for String +Manipulation: String Functions.). + + +File: gawk.info, Node: Getline, Next: Close Input, Prev: Multiple Line, Up: Reading Files + +Explicit Input with `getline' +============================= + + So far we have been getting our input files from `awk''s main input +stream--either the standard input (usually your terminal) or the files +specified on the command line. The `awk' language has a special +built-in command called `getline' that can be used to read input under +your explicit control. + + This command is quite complex and should *not* be used by beginners. +It is covered here because this is the chapter on input. The examples +that follow the explanation of the `getline' command include material +that has not been covered yet. Therefore, come back and study the +`getline' command *after* you have reviewed the rest of this manual and +have a good knowledge of how `awk' works. + + `getline' returns 1 if it finds a record, and 0 if the end of the +file is encountered. If there is some error in getting a record, such +as a file that cannot be opened, then `getline' returns -1. In this +case, `gawk' sets the variable `ERRNO' to a string describing the error +that occurred. + + In the following examples, COMMAND stands for a string value that +represents a shell command. + +`getline' + The `getline' command can be used without arguments to read input + from the current input file. All it does in this case is read the + next input record and split it up into fields. This is useful if + you've finished processing the current record, but you want to do + some special processing *right now* on the next record. Here's an + example: + + awk '{ + if (t = index($0, "/*")) { + if (t > 1) + tmp = substr($0, 1, t - 1) + else + tmp = "" + u = index(substr($0, t + 2), "*/") + while (u == 0) { + getline + t = -1 + u = index($0, "*/") + } + if (u <= length($0) - 2) + $0 = tmp substr($0, t + u + 3) + else + $0 = tmp + } + print $0 + }' + + This `awk' program deletes all C-style comments, `/* ... */', + from the input. By replacing the `print $0' with other + statements, you could perform more complicated processing on the + decommented input, like searching for matches of a regular + expression. (This program has a subtle problem--can you spot it?) + + This form of the `getline' command sets `NF' (the number of + fields; *note Examining Fields: Fields.), `NR' (the number of + records read so far; *note How Input is Split into Records: + Records.), `FNR' (the number of records read from this input + file), and the value of `$0'. + + *Note:* the new value of `$0' is used in testing the patterns of + any subsequent rules. The original value of `$0' that triggered + the rule which executed `getline' is lost. By contrast, the + `next' statement reads a new record but immediately begins + processing it normally, starting with the first rule in the + program. *Note The `next' Statement: Next Statement. + +`getline VAR' + This form of `getline' reads a record into the variable VAR. This + is useful when you want your program to read the next record from + the current input file, but you don't want to subject the record + to the normal input processing. + + For example, suppose the next line is a comment, or a special + string, and you want to read it, but you must make certain that it + won't trigger any rules. This version of `getline' allows you to + read that line and store it in a variable so that the main + read-a-line-and-check-each-rule loop of `awk' never sees it. + + The following example swaps every two lines of input. For + example, given: + + wan + tew + free + phore + + it outputs: + + tew + wan + phore + free + + Here's the program: + + awk '{ + if ((getline tmp) > 0) { + print tmp + print $0 + } else + print $0 + }' + + The `getline' function used in this way sets only the variables + `NR' and `FNR' (and of course, VAR). The record is not split into + fields, so the values of the fields (including `$0') and the value + of `NF' do not change. + +`getline < FILE' + This form of the `getline' function takes its input from the file + FILE. Here FILE is a string-valued expression that specifies the + file name. `< FILE' is called a "redirection" since it directs + input to come from a different place. + + This form is useful if you want to read your input from a + particular file, instead of from the main input stream. For + example, the following program reads its input record from the + file `foo.input' when it encounters a first field with a value + equal to 10 in the current input file. + + awk '{ + if ($1 == 10) { + getline < "foo.input" + print + } else + print + }' + + Since the main input stream is not used, the values of `NR' and + `FNR' are not changed. But the record read is split into fields in + the normal manner, so the values of `$0' and other fields are + changed. So is the value of `NF'. + + This does not cause the record to be tested against all the + patterns in the `awk' program, in the way that would happen if the + record were read normally by the main processing loop of `awk'. + However the new record is tested against any subsequent rules, + just as when `getline' is used without a redirection. + +`getline VAR < FILE' + This form of the `getline' function takes its input from the file + FILE and puts it in the variable VAR. As above, FILE is a + string-valued expression that specifies the file from which to + read. + + In this version of `getline', none of the built-in variables are + changed, and the record is not split into fields. The only + variable changed is VAR. + + For example, the following program copies all the input files to + the output, except for records that say `@include FILENAME'. Such + a record is replaced by the contents of the file FILENAME. + + awk '{ + if (NF == 2 && $1 == "@include") { + while ((getline line < $2) > 0) + print line + close($2) + } else + print + }' + + Note here how the name of the extra input file is not built into + the program; it is taken from the data, from the second field on + the `@include' line. + + The `close' function is called to ensure that if two identical + `@include' lines appear in the input, the entire specified file is + included twice. *Note Closing Input Files and Pipes: Close Input. + + One deficiency of this program is that it does not process nested + `@include' statements the way a true macro preprocessor would. + +`COMMAND | getline' + You can "pipe" the output of a command into `getline'. A pipe is + simply a way to link the output of one program to the input of + another. In this case, the string COMMAND is run as a shell + command and its output is piped into `awk' to be used as input. + This form of `getline' reads one record from the pipe. + + For example, the following program copies input to output, except + for lines that begin with `@execute', which are replaced by the + output produced by running the rest of the line as a shell command: + + awk '{ + if ($1 == "@execute") { + tmp = substr($0, 10) + while ((tmp | getline) > 0) + print + close(tmp) + } else + print + }' + + The `close' function is called to ensure that if two identical + `@execute' lines appear in the input, the command is run for each + one. *Note Closing Input Files and Pipes: Close Input. + + Given the input: + + foo + bar + baz + @execute who + bletch + + the program might produce: + + foo + bar + baz + hack ttyv0 Jul 13 14:22 + hack ttyp0 Jul 13 14:23 (gnu:0) + hack ttyp1 Jul 13 14:23 (gnu:0) + hack ttyp2 Jul 13 14:23 (gnu:0) + hack ttyp3 Jul 13 14:23 (gnu:0) + bletch + + Notice that this program ran the command `who' and printed the + result. (If you try this program yourself, you will get different + results, showing you who is logged in on your system.) + + This variation of `getline' splits the record into fields, sets the + value of `NF' and recomputes the value of `$0'. The values of + `NR' and `FNR' are not changed. + +`COMMAND | getline VAR' + The output of the command COMMAND is sent through a pipe to + `getline' and into the variable VAR. For example, the following + program reads the current date and time into the variable + `current_time', using the `date' utility, and then prints it. + + awk 'BEGIN { + "date" | getline current_time + close("date") + print "Report printed on " current_time + }' + + In this version of `getline', none of the built-in variables are + changed, and the record is not split into fields. + + +File: gawk.info, Node: Close Input, Prev: Getline, Up: Reading Files + +Closing Input Files and Pipes +============================= + + If the same file name or the same shell command is used with +`getline' more than once during the execution of an `awk' program, the +file is opened (or the command is executed) only the first time. At +that time, the first record of input is read from that file or command. +The next time the same file or command is used in `getline', another +record is read from it, and so on. + + This implies that if you want to start reading the same file again +from the beginning, or if you want to rerun a shell command (rather than +reading more output from the command), you must take special steps. +What you must do is use the `close' function, as follows: + + close(FILENAME) + +or + + close(COMMAND) + + The argument FILENAME or COMMAND can be any expression. Its value +must exactly equal the string that was used to open the file or start +the command--for example, if you open a pipe with this: + + "sort -r names" | getline foo + +then you must close it with this: + + close("sort -r names") + + Once this function call is executed, the next `getline' from that +file or command will reopen the file or rerun the command. + + `close' returns a value of zero if the close succeeded. Otherwise, +the value will be non-zero. In this case, `gawk' sets the variable +`ERRNO' to a string describing the error that occurred. + + +File: gawk.info, Node: Printing, Next: One-liners, Prev: Reading Files, Up: Top + +Printing Output +*************** + + One of the most common things that actions do is to output or "print" +some or all of the input. For simple output, use the `print' +statement. For fancier formatting use the `printf' statement. Both +are described in this chapter. + +* Menu: + +* Print:: The `print' statement. +* Print Examples:: Simple examples of `print' statements. +* Output Separators:: The output separators and how to change them. +* OFMT:: Controlling Numeric Output With `print'. +* Printf:: The `printf' statement. +* Redirection:: How to redirect output to multiple + files and pipes. +* Special Files:: File name interpretation in `gawk'. + `gawk' allows access to + inherited file descriptors. + + +File: gawk.info, Node: Print, Next: Print Examples, Prev: Printing, Up: Printing + +The `print' Statement +===================== + + The `print' statement does output with simple, standardized +formatting. You specify only the strings or numbers to be printed, in a +list separated by commas. They are output, separated by single spaces, +followed by a newline. The statement looks like this: + + print ITEM1, ITEM2, ... + +The entire list of items may optionally be enclosed in parentheses. The +parentheses are necessary if any of the item expressions uses a +relational operator; otherwise it could be confused with a redirection +(*note Redirecting Output of `print' and `printf': Redirection.). The +relational operators are `==', `!=', `<', `>', `>=', `<=', `~' and `!~' +(*note Comparison Expressions: Comparison Ops.). + + The items printed can be constant strings or numbers, fields of the +current record (such as `$1'), variables, or any `awk' expressions. +The `print' statement is completely general for computing *what* values +to print. With two exceptions, you cannot specify *how* to print +them--how many columns, whether to use exponential notation or not, and +so on. (*Note Output Separators::, and *Note Controlling Numeric +Output with `print': OFMT.) For that, you need the `printf' statement +(*note Using `printf' Statements for Fancier Printing: Printf.). + + The simple statement `print' with no items is equivalent to `print +$0': it prints the entire current record. To print a blank line, use +`print ""', where `""' is the null, or empty, string. + + To print a fixed piece of text, use a string constant such as +`"Hello there"' as one item. If you forget to use the double-quote +characters, your text will be taken as an `awk' expression, and you +will probably get an error. Keep in mind that a space is printed +between any two items. + + Most often, each `print' statement makes one line of output. But it +isn't limited to one line. If an item value is a string that contains a +newline, the newline is output along with the rest of the string. A +single `print' can make any number of lines this way. + + +File: gawk.info, Node: Print Examples, Next: Output Separators, Prev: Print, Up: Printing + +Examples of `print' Statements +============================== + + Here is an example of printing a string that contains embedded +newlines: + + awk 'BEGIN { print "line one\nline two\nline three" }' + +produces output like this: + + line one + line two + line three + + Here is an example that prints the first two fields of each input +record, with a space between them: + + awk '{ print $1, $2 }' inventory-shipped + +Its output looks like this: + + Jan 13 + Feb 15 + Mar 15 + ... + + A common mistake in using the `print' statement is to omit the comma +between two items. This often has the effect of making the items run +together in the output, with no space. The reason for this is that +juxtaposing two string expressions in `awk' means to concatenate them. +For example, without the comma: + + awk '{ print $1 $2 }' inventory-shipped + +prints: + + Jan13 + Feb15 + Mar15 + ... + + Neither example's output makes much sense to someone unfamiliar with +the file `inventory-shipped'. A heading line at the beginning would +make it clearer. Let's add some headings to our table of months (`$1') +and green crates shipped (`$2'). We do this using the `BEGIN' pattern +(*note `BEGIN' and `END' Special Patterns: BEGIN/END.) to force the +headings to be printed only once: + + awk 'BEGIN { print "Month Crates" + print "----- ------" } + { print $1, $2 }' inventory-shipped + +Did you already guess what happens? This program prints the following: + + Month Crates + ----- ------ + Jan 13 + Feb 15 + Mar 15 + ... + +The headings and the table data don't line up! We can fix this by +printing some spaces between the two fields: + + awk 'BEGIN { print "Month Crates" + print "----- ------" } + { print $1, " ", $2 }' inventory-shipped + + You can imagine that this way of lining up columns can get pretty +complicated when you have many columns to fix. Counting spaces for two +or three columns can be simple, but more than this and you can get +"lost" quite easily. This is why the `printf' statement was created +(*note Using `printf' Statements for Fancier Printing: Printf.); one of +its specialties is lining up columns of data. + |