unknown_document

cat is a command that reads one or more files, con cat enates them, and writes them to standard output ( stdout ), which by default prints the contents to the screen. This command uses the call signature: cat { options } { paths } If no path is specified, then this command receives input from stdin . If one path is specified, then this file is read and passed to stdin . If multiple paths are specified, then they are each read in sequence and passed to stdin . Basic Usage As a quick review, lets start with a file fruits.txt with the following contents: apple banana watermelon grape strawberry We can print the contents to the screen using the following command: ninja$: cat fruits.txt apple banana watermelon grape strawberry ninja$: Now, suppose we have a second file animals.txt with the following content: bear chicken duck We can concatenate the files and print them to the screen with the following command: ninja$: cat animals.txt fruits.txt bear chicken duck apple banana watermelon grape strawberry ninja$: Options Although most invocations of the cat command use the simple call signature, there are a few useful options available. Adding line numbers cat --number (or cat -n for short) returns each line prefixed with a line number, which can be handy in some cases: ninja$: cat --number fruits.txt 1 apple 2 banana 3 watermelon 4 grape 5 strawberry ninja$: Squeezing empty lines The cat command also provides an option for "squeezing" multiple blank lines down to a single blank line. For example, given a file that contains various blank lines: one two three this option can be used to: ninja$: cat --squeeze-blank blanks.txt one two three ninja$:

The echo command is used to print a line of text to stdout . In many cases this is used to initiate a command or pipeline with the contents of a text string. Various options are supported, though most applications use the simple call signature: echo { string } ninja$: echo Hello World! Hello World! ninja$:

The head command filters a stream so that only the beginning of the stream is passed to stdout . This is often used to get a quick summary of the contents of a file, or to target specific lines by index in a pipeline . The call signature is: head { options } { path } If path is specified, that file will be read and streamed to stdin . If path is omitted or set to - , then this command will operate on the stream that is piped or redirected to stdin . The most common option is -n (or the --lines long option), which defines the index of the line after which content is filtered. By default the index is 10. To demonstrate, let's work with our numbers.txt file, which contains a simple list of numbers from one through twelve: ninja$: head numbers.txt one two three four five six seven eight nine ten ninja$: Although this file contains 12 lines of text, only lines up to the 10th line are passed to stdout , Now lets change the index to 3: ninja$: head -n 3 numbers.txt one two three ninja$: The head command also allows the index to be negative, so that lines can be filtered relative to the end of the file. For example, to pass all lines except for the last 5 lines the index can be set to -5 : ninja$: head -n -5 numbers.txt one two three four five six seven ninja$:

The tail command is the complement of the head command, and filters a stream so that only the end of the stream is passed to stdout . This is often used to get a quick summary of the contents of a file, or to target specific lines by index in a pipeline . The call signature is: tail { options } { path } If path is specified, that file will be read and streamed to stdin . If path is omitted or set to - , then this command will operate on the stream that is piped or redirected to stdin . The most common option is -n (or the --lines long option), which defines the index of the line from which content is passed to stdout . By default the index is 10. To demonstrate, let's work with our numbers.txt file, which contains a simple list of numbers from one through twelve.: ninja$: tail numbers.txt three four five six seven eight nine ten eleven twelve ninja$: Although this file contains 12 lines of text, only the last 10 lines are passed to stdout , Now lets change the index to 3: ninja$: tail -n 3 numbers.txt ten eleven twelve ninja$: The tail command also allows the index to be negative, so that lines can be filtered relative to the beginning of the file. For example, to filter out only the first 5 lines the index can be set to -5 : ninja$: tail -n -5 numbers.txt eight nine ten eleven twelve ninja$:

Some of the most common command line operations involve reading, writing, or transforming file contents. echo Pass a string to stdout cat Concatenate one or more files head Output the first part of files tail Output the last part of files

"coreutils" refers to the GNU core utilities , which are the basic set of file, shell, and text manipulation utilities that are expected to be installed on every UNIX-like operating system. The coreutils include a fairly wide range of commands that apply to many situations. In this section we will focus on just a few of the more common commands that are used for working with text files: Basics Basic commands that are used for viewing and summarizing file content File Modification Commands that are used for modifying files, such as sorting them.

Much of the terminology used when discussing modern terminals comes from history, which can make understanding terminals seem more confusing than it should be. A small amount history can help provide some background information to help some of the concepts be more relatable. Early computers were comprised of multiple refrigerator-sized cabinets, each of which performed one of the basic tasks of the system. For example, one cabinet would contain the " central processing unit " (CPU), while other cabinets held tape drives , disk drives , punched-card readers , a line printer , and any other peripherals that were available to the computer. The user-interface to these computers consisted of a keyboard and some means of displaying the output from the computer, such as a sequence of light bulbs or later a Cathode-ray Tube (CRT) display , connected to the computer via cables. The user-interface was typically arranged into a " console ", where the operator could sit and interact with the computer. Since the user-interface represented the point at which electrical signals "enter" and "exit" the network, the user-interfaces themselves became known as " terminals ". Standardization As often happens in technology development, there were initially a wide range of competing technologies, but over time market forces and economies of scale led these competing technologies to converge towards standardization around a handful of those technologies, the names of which have become much of the jargon that surrounds the terminal today. Early terminals were electro-mechanical "Teletype writers", generally shortened to " TTY ", which is a term still used today when interacting with the terminal. As with a modern keyboard, each key on the TTY represented a human-readable character such as a letter, number, punctuation, etc. As the user would press a key to input data into the computer, each key would be " encoded " into a sequence of 8 1s and 0s called bytes , which could be understood by the computer. These bytes were sent over the cable to the CPU, which performed the requested operations, then the resulting byte(s) were transmitted back to the terminal where they were "decoded" back into characters for human-consumption. The rules used for encoding and decoding each character eventually became the ASCII standard. Despite the benefits of standardization, the ASCII standard was written to support US English, which led to several extensions in order to support other languages, which eventually led to the UTF-8 standard which unified the various extensions into a single encoding that is broadly in use today. Over time display technologies evolved which led to the introduction of "video terminals" (VT) which allowed text and other graphical information to be displayed on a screen. Some of the most popular video terminals were the VT100 and later VT200 series, which introduced support for the ANSI escape codes that have become standards that are still in use today. Emulation As microprocessor technology advanced and the cost of memory and other peripherals dropped, terminals began to handle increasingly-advanced operations, which led to the introduction of " intelligent terminals ", which differentiated them from the "dumb terminals" they replaced. In order to make the newer terminals " backward compatible " with existing software, these terminals included hardware to " emulate " the older devices. Over time the emulation functions were implemented in software, which eventually led to the introduction of fully-software " terminal emulators " that could offer features that would have been difficult to impossible to implement in hardware, such as command-line completion and syntax highlighting . With the introduction of terminal emulators, "the terminal" became a software window that is opened in order to gain access to the operating system . The software that provides this access is called "the shell", which is the subject of the next section.

Some of the most common command line operations involve reading, writing, or transforming file contents. cut Extracts information from each line of input. paste Combines lines from multiple files into a single stream. tr Applies translation operations to characters from the input sort Sorts each line of input. shuf Randomly orders lines of input. uniq Filters out adjacent, duplicate lines.

The tr provides a means of translating (or transliterating) characters in text for simple search and replace operations. The call signature is: tr { option } { string1 } { string2 } where string1 defines the array of characters to search for, and string2 defines the array of characters to replace those characters with. The following example details the basic behavior. Starting with our colors.csv file: ninja$: cat colors.csv red,#f00 green,#0f0 blue,#00f yellow,#ff0 cyan,#0ff magenta,#f0f ninja$: suppose we want to adjust the colors slightly. The following command changes each occurrence of "f" with "c", and each occurrence of "0" with "4": ninja$: cat colors.csv | tr f0 c4 red,#c44 green,#4c4 blue,#44c yellow,#cc4 cyan,#4cc magenta,#c4c ninja$: When a character from string1 is encountered, it is replaced with the character at the same index of string2 . As such, string1 and string2 should generally be the same length. Cases where they are not are treated as follows: When string1 is longer than string2 and the -t option is not specified, the last character of string2 is repeated to achieve the same length as string1 . When string1 is longer than string2 and the -t option is specified, then string1 is truncated to achieve the same length as string2 . When string2 is longer than string1 , string2 is truncated to achieve the same length as string1 . string1 and string2 both support character ranges. For example, the following converts characters from lower-case to upper-case: ninja$: cat colors.csv | tr f0 c4 | tr a-z A-Z RED,#C44 GREEN,#4C4 BLUE,#44C YELLOW,#CC4 CYAN,#4CC MAGENTA,#C4C ninja$: Deleting Characters The tr command can also be used to delete unwanted characters from the input stream. This is achieved by specifying the -d option and omitting string2 . For example, to remove "#" from the output once can execute: ninja$: cat colors.csv | tr f0 c4 | tr a-z A-Z | tr -d # RED,C44 GREEN,4C4 BLUE,44C YELLOW,CC4 CYAN,4CC MAGENTA,C4C ninja$: Squeezing Characters Finally, the tr command can also "squeeze" adjacent runs of "E", "L", or "C" down to a single character, using the -s option, as follows: ninja$: cat colors.csv | tr f0 c4 | tr a-z A-Z | tr -d # | tr -s ELC RED,C44 GREN,4C4 BLUE,44C YELOW,C4 CYAN,4C MAGENTA,C4C ninja$:

The cut command is used to extract information from each line of content passed to stdin . The call signature is: cut { options } { path } When path is specified, the contents of that file are read and passed to stdin . cut is often used to either extract fields from delimited files, or to extract positional data by index. Let's look at each of those: Extract By Delimited Fields Data can be extracted from delimited fields in two steps. First, each line is split based on the delimiter. By default the cut command assumes tab-delimited data, though this can be changed using the -d option, followed by the character to use as a delimiter. Second, information is selected and returned by numerical index, starting from 1. Field index specifications can take several forms: Spec Meaning n include field n n,m include fields n and m n-m include fields from n to m n- include fields from n to the end -m include fields from 1 to m where n and m are integers. To demonstrate, suppose you have the following file: ninja$: cat colors.csv red,#f00 green,#0f0 blue,#00f yellow,#ff0 cyan,#0ff magenta,#f0f ninja$: and want to extract the color name from each line. This can be achieved by first setting the delimiter to , , then selecting the first field from each line: ninja$: cut -d , -f 1 colors.csv red green blue yellow cyan magenta ninja$: Alternately, if we wanted to extract the hex value for each color, simply select the second field: ninja$: cut -d , -f 2 colors.csv #f00 #0f0 #00f #ff0 #0ff #f0f ninja$: Multiple fields can be specified using comma-separated values. One quirk of the cut command is that fields are returned in the order they are read , not in the order they are specified . For example, if we wanted to reverse fields 1 and 2 and return them we might try the following: ninja$: cut -d , -f 2,1 colors.csv red,#f00 green,#0f0 blue,#00f yellow,#ff0 cyan,#0ff magenta,#f0f ninja$: Note that both fields are returned in the original order, despite the ordering of the spec. Extract By Index Data can be extracted from each line by column/position by specifying which column(s) to return. Columns are specified in a manner similar to above: Spec Meaning n include column n n,m include columns n and m n-m include columns from n to m n- include columns from n to the end -m include columns from 1 to m where n and m are integers. For example, to return only the first 6 characters from each line: ninja$: cut -c -6 colors.csv red,#f green, blue,# yellow cyan,# magent ninja$: or to return 6 columns, starting from column 3: ninja$: cut -c 3-9 colors.csv d,#f00 een,#0f ue,#00f llow,#f an,#0ff genta,# ninja$: Finally, more complex cases can combine multiple column-specifications as follows: ninja$: cut -c 1-3,9 colors.csv red gref bluf yelf cyaf mag# ninja$: