bash的重定向
-
1 背景知识
在bash中操作重定向,实际是在操作文件描述符:
在经典Unix实现中,内核为每个进程维护一份
file descriptor table
(进程级),其中的文件描述符指向另一张file table
(系统级),包含系统中所有打开的文件。file table
中的每一项再指向实际记录文件信息的inode table
。
进程要读写文件时,先通过系统调用(例如ssize_t read(int fd, void *buf, size_t count);
)将文件描述符传给内核,然后由内核代表进程访问文件。进程无法直接访问file table
或inode table
。Bash启动时会打开3个标准文件描述符:
stdin
(0),stdout
(1),stderr
(2),它们都指向当前terminal,例如/dev/tty0
Bash执行一条命令时,会:
1.fork
(man 2 fork
)一个子进程,继承父进程的所有文件描述符
2. 这之后,如果这条命令里有重定向,那么重定向将会被执行
3. 最后命令以exec
(man 2 exec
)的方式被执行
-
1.1 Bash Redirection Cheet Sheet
这是Peteris Krumins在2012年写的一份bash重定向用法简表,github仓库:pkrumins/bash-redirections-cheat-sheet
图片版
图片用的github图床,可能需要科学上网
txt版
.---------------------------------------------------------------------------. | | | Bash Redirections Cheat Sheet | | | +---------------------------------------------------------------------------+ | | | Created by Peteris Krumins (peter@catonmat.net) | | www.catonmat.net -- good coders code, great coders reuse | | | +-----------------------------.---------------------------------------------+ | Redirection | Description | '-----------------------------'---------------------------------------------' | cmd > file | Redirect the standard output (stdout) of | | | `cmd` to a file. | +-----------------------------'---------------------------------------------' | cmd 1> file | Same as `cmd > file`. 1 is the default file | | | descriptor for stdout. | +-----------------------------'---------------------------------------------' | cmd 2> file | Redirect the standard error (stderr) of | | | `cmd` to a file. 2 is the default file | | | descriptor for stderr. | +-----------------------------'---------------------------------------------' | cmd >> file | Append stdout of `cmd` to a file. | +-----------------------------'---------------------------------------------' | cmd 2>> file | Append stderr of `cmd` to a file. | +-----------------------------'---------------------------------------------' | cmd &> file | Redirect stdout and stderr to a file. | +-----------------------------'---------------------------------------------' | cmd > file 2>&1 | Another way to redirect both stdout and | | | stderr of `cmd` to a file. This *is not* | | | same as `cmd 2>&1 > file`. | | | Redirection order matters! | +-----------------------------'---------------------------------------------' | cmd > /dev/null | Discard stdout of `cmd`. | +-----------------------------'---------------------------------------------' | cmd 2> /dev/null | Discard stderr of `cmd`. | +-----------------------------'---------------------------------------------' | cmd &> /dev/null | Discard stdout and stderr. | +-----------------------------'---------------------------------------------' | cmd < file | Redirect the contents of the file to the | | | stdin of `cmd`. | +-----------------------------'---------------------------------------------' | cmd << EOL | | | foo | Redirect a bunch of lines to the stdin. | | bar | If 'EOL' is quoted, text is treated | | baz | literally. This is called a here-document. | | EOL | | +-----------------------------'---------------------------------------------' | cmd <<- EOL | | | <tab>foo | Redirect a bunch of lines to the stdin. | | <tab><tab>bar | The <tab>'s are ignored but not the | | EOL | whitespace. Helpful for formatting. | +-----------------------------'---------------------------------------------' | cmd <<< "string" | Redirect a single line of text to stdin. | | | This is called a here-string. | +-----------------------------'---------------------------------------------' | exec 2> file | Redirect stderr of all commands to a file | | | forever. | +-----------------------------'---------------------------------------------' | exec 3< file | Open a file for reading using a custom fd. | +-----------------------------'---------------------------------------------' | exec 3> file | Open a file for writing using a custom fd. | +-----------------------------'---------------------------------------------' | exec 3<> file | Open a file for reading and writing using | | | a custom file descriptor. | +-----------------------------'---------------------------------------------' | exec 3>&- | Close a file descriptor. | +-----------------------------'---------------------------------------------' | exec 4>&3 | Make file descriptor 4 to be a copy of file | | | descriptor 3. (Copy fd 3 to 4.) | +-----------------------------'---------------------------------------------' | exec 4>&3- | Copy file descriptor 3 to 4 and close fd 3 | +-----------------------------'---------------------------------------------' | echo "foo" >&3 | Write to a custom file descriptor. | +-----------------------------'---------------------------------------------' | cat <&3 | Read from a custom file descriptor. | +-----------------------------'---------------------------------------------' | (cmd1; cmd2) > file | Redirect stdout from multiple commands to a | | | file (using a sub-shell). | +-----------------------------'---------------------------------------------' | { cmd1; cmd2; } > file | Redirect stdout from multiple commands to a | | | file (faster; not using a sub-shell). | +-----------------------------'---------------------------------------------' | exec 3<> /dev/tcp/host/port | Open a TCP connection to host:port. | +-----------------------------'---------------------------------------------' | exec 3<> /dev/udp/host/port | Open a UDP connection to host:port. | +-----------------------------'---------------------------------------------' | cmd <(cmd1) | Redirect stdout of `cmd1` to an anonymous | | | fifo, then pass the fifo to `cmd` as an | | | argument. Useful when `cmd` doesn't read | | | from stdin directly. | +-----------------------------'---------------------------------------------' | cmd < <(cmd1) | Redirect stdout of `cmd1` to an anonymous | | | fifo, then redirect the fifo to stdin of | | ____' `cmd`. Best example: | | | diff <(find /path1 | sort) <(find /path2 | sort) | +------------------------'----.---------------------------------------------' | cmd <(cmd1) <(cmd2) | Redirect stdout of `cmd1` `cmd2` to two | | | anonymous fifos, then pass both fifos as | | | arguments to \verb|cmd|. | +-----------------------------.---------------------------------------------' | cmd1 >(cmd2) | Run `cmd2` with its stdin connected to an | | | anonymous fifo, and pass the filename of | | | the pipe as an argument to `cmd1`. | +-----------------------------.---------------------------------------------' | cmd1 | cmd2 | Redirect stdout of cmd1 to stdin of `cmd2`. | | | Pro-tip: This is the same as | | | `cmd1 > >(cmd2)`, same as `cmd2 < <(cmd1)`, | | | same as `> >(cmd2) cmd1`, same as | | | `< <(cmd1) cmd2`. | +-----------------------------'---------------------------------------------' | cmd1 |& cmd2 | Redirect stdout and stderr of `cmd1` to | | | stdin of `cmd2` (bash 4.0+ only). | | | Use `cmd1 2>&1 | cmd2` for older bashes. | +-----------------------------'---------------------------------------------' | cmd | tee file | Redirect stdout of `cmd` to a file and | | | print it to screen. | +-----------------------------'---------------------------------------------' | exec {filew}> file | Open a file for writing using a named file | | | descriptor called `{filew}` (bash 4.1+) | +-----------------------------'---------------------------------------------' | cmd 3>&1 1>&2 2>&3 | Swap stdout and stderr of `cmd`. | +-----------------------------'---------------------------------------------' | cmd > >(cmd1) 2> >(cmd2) | Send stdout of `cmd` to `cmd1` and stderr | | | `cmd` to `cmd2`. | +-----------------------------'---------------------------------------------' | cmd1 | cmd2 | cmd3 | cmd4 | Find out the exit codes of all piped cmds. | | echo ${PIPESTATUS[@]} | | +-----------------------------'---------------------------------------------' | | | I explained each one of these redirections in my article All About Bash | | Redirections: | | | | http://www.catonmat.net/blog/bash-one-liners-explained-part-three | | | +---------------------------------------------------------------------------+
-
1.2 Bash Manual, Redirection Section
这是Bash手册中的重定向小节,可以使用
man bash
+/^REDIRECTION
快速定位查看,大概在1700行左右的位置。REDIRECTION Before a command is executed, its input and output may be redirected using a special notation interpreted by the shell. Redirection allows commands' file handles to be duplicated, opened, closed, made to refer to different files, and can change the files the command reads from and writes to. Redirection may also be used to modify file handles in the current shell execution environment. The following redirection operators may precede or appear anywhere within a sim‐ ple command or may follow a command. Redirections are processed in the order they appear, from left to right. Each redirection that may be preceded by a file descriptor number may instead be preceded by a word of the form {varname}. In this case, for each redirection operator except >&- and <&-, the shell will allocate a file descriptor greater than or equal to 10 and assign it to var‐ name. If >&- or <&- is preceded by {varname}, the value of varname defines the file descrip‐ tor to close. If {varname} is supplied, the redirection persists beyond the scope of the com‐ mand, allowing the shell programmer to manage the file descriptor himself. In the following descriptions, if the file descriptor number is omitted, and the first charac‐ ter of the redirection operator is <, the redirection refers to the standard input (file de‐ scriptor 0). If the first character of the redirection operator is >, the redirection refers to the standard output (file descriptor 1). The word following the redirection operator in the following descriptions, unless otherwise noted, is subjected to brace expansion, tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, quote removal, pathname expansion, and word split‐ ting. If it expands to more than one word, bash reports an error. Note that the order of redirections is significant. For example, the command ls > dirlist 2>&1 directs both standard output and standard error to the file dirlist, while the command ls 2>&1 > dirlist directs only the standard output to file dirlist, because the standard error was duplicated from the standard output before the standard output was redirected to dirlist. Bash handles several filenames specially when they are used in redirections, as described in the following table. If the operating system on which bash is running provides these special files, bash will use them; otherwise it will emulate them internally with the behavior de‐ scribed below. /dev/fd/fd If fd is a valid integer, file descriptor fd is duplicated. /dev/stdin File descriptor 0 is duplicated. /dev/stdout File descriptor 1 is duplicated. /dev/stderr File descriptor 2 is duplicated. /dev/tcp/host/port If host is a valid hostname or Internet address, and port is an integer port number or service name, bash attempts to open the corresponding TCP socket. /dev/udp/host/port If host is a valid hostname or Internet address, and port is an integer port number or service name, bash attempts to open the corresponding UDP socket. A failure to open or create a file causes the redirection to fail. Redirections using file descriptors greater than 9 should be used with care, as they may con‐ flict with file descriptors the shell uses internally. Note that the exec builtin command can make redirections take effect in the current shell. Redirecting Input Redirection of input causes the file whose name results from the expansion of word to be opened for reading on file descriptor n, or the standard input (file descriptor 0) if n is not specified. The general format for redirecting input is: [n]<word Redirecting Output Redirection of output causes the file whose name results from the expansion of word to be opened for writing on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created; if it does exist it is truncated to zero size. The general format for redirecting output is: [n]>word If the redirection operator is >, and the noclobber option to the set builtin has been en‐ abled, the redirection will fail if the file whose name results from the expansion of word ex‐ ists and is a regular file. If the redirection operator is >|, or the redirection operator is > and the noclobber option to the set builtin command is not enabled, the redirection is at‐ tempted even if the file named by word exists. Appending Redirected Output Redirection of output in this fashion causes the file whose name results from the expansion of word to be opened for appending on file descriptor n, or the standard output (file descriptor 1) if n is not specified. If the file does not exist it is created. The general format for appending output is: [n]>>word Redirecting Standard Output and Standard Error This construct allows both the standard output (file descriptor 1) and the standard error out‐ put (file descriptor 2) to be redirected to the file whose name is the expansion of word. There are two formats for redirecting standard output and standard error: &>word and >&word Of the two forms, the first is preferred. This is semantically equivalent to >word 2>&1 When using the second form, word may not expand to a number or -. If it does, other redirect‐ ion operators apply (see Duplicating File Descriptors below) for compatibility reasons. Appending Standard Output and Standard Error This construct allows both the standard output (file descriptor 1) and the standard error out‐ put (file descriptor 2) to be appended to the file whose name is the expansion of word. The format for appending standard output and standard error is: &>>word This is semantically equivalent to >>word 2>&1 (see Duplicating File Descriptors below). Here Documents This type of redirection instructs the shell to read input from the current source until a line containing only delimiter (with no trailing blanks) is seen. All of the lines read up to that point are then used as the standard input (or file descriptor n if n is specified) for a command. The format of here-documents is: [n]<<[-]word here-document delimiter No parameter and variable expansion, command substitution, arithmetic expansion, or pathname expansion is performed on word. If any part of word is quoted, the delimiter is the result of quote removal on word, and the lines in the here-document are not expanded. If word is un‐ quoted, all lines of the here-document are subjected to parameter expansion, command substitu‐ tion, and arithmetic expansion, the character sequence \<newline> is ignored, and \ must be used to quote the characters \, $, and `. If the redirection operator is <<-, then all leading tab characters are stripped from input lines and the line containing delimiter. This allows here-documents within shell scripts to be indented in a natural fashion. Here Strings A variant of here documents, the format is: [n]<<<word The word undergoes tilde expansion, parameter and variable expansion, command substitution, arithmetic expansion, and quote removal. Pathname expansion and word splitting are not per‐ formed. The result is supplied as a single string, with a newline appended, to the command on its standard input (or file descriptor n if n is specified). Duplicating File Descriptors The redirection operator [n]<&word is used to duplicate input file descriptors. If word expands to one or more digits, the file descriptor denoted by n is made to be a copy of that file descriptor. If the digits in word do not specify a file descriptor open for input, a redirection error occurs. If word evalu‐ ates to -, file descriptor n is closed. If n is not specified, the standard input (file de‐ scriptor 0) is used. The operator [n]>&word is used similarly to duplicate output file descriptors. If n is not specified, the standard output (file descriptor 1) is used. If the digits in word do not specify a file descriptor open for output, a redirection error occurs. If word evaluates to -, file descriptor n is closed. As a special case, if n is omitted, and word does not expand to one or more digits or -, the standard output and standard error are redirected as described previously. Moving File Descriptors The redirection operator [n]<&digit- moves the file descriptor digit to file descriptor n, or the standard input (file descriptor 0) if n is not specified. digit is closed after being duplicated to n. Similarly, the redirection operator [n]>&digit- moves the file descriptor digit to file descriptor n, or the standard output (file descriptor 1) if n is not specified. Opening File Descriptors for Reading and Writing The redirection operator [n]<>word causes the file whose name is the expansion of word to be opened for both reading and writing on file descriptor n, or on file descriptor 0 if n is not specified. If the file does not ex‐ ist, it is created.
-
2 重定向命令
2.1 读写重定向
读重定向没有指定文件描述符时,默认为
0
(stdin);写重定向没有指定文件描述符时,默认为1
(stdout)。- 读
- 格式:
[n]<word
- 效果:
word
被Bash展开的结果作为文件名,以读的方式打开,然后将文件描述符n
指向那个文件
- 写
- 格式:
[n]>word
- 效果:以写的方式打开
word
文件,文件描述符n
指向该文件。该文件不存在则会被创建,文件已存在则会被清空。 - 注意:
- 文件已存在时,如果Bash启用了
noclobber
选项(set -o noclobber
),则重定向操作会失败(文件不会被清空) - 如果运算符是
>|
([n]>|word
),则始终覆盖文件
- 文件已存在时,如果Bash启用了
- 追加写
- 格式:
[n]>>word
- 效果:以写的方式打开
word
文件,文件描述符n
指向该文件。该文件不存在则会被创建;文件已存在,则写入内容时向文件末尾追加。
-
2.2 duplicate文件描述符(work in progress)
- dup写
- 格式:
[n]<&word
- dup读
- 格式:
[n]>&word
-
2.3 写重定向实例:
stdout
与stderr
(work in progress)- 重定向写入
- 格式:
&>word
,>&word
,>word 2>&1
- 追加重定向写入
- 格式:
&>>word
,>>word 2>&1
-
2.4 文件描述符 打开,移动,关闭(work in progress)
- 打开
- 格式:
[n]<>word
- 移动
- 格式:
[n]<&digit-
,[n]>&digit-
- 关闭
- 格式:
[n]<&-
,[n]>&-
-
2.5 Here Documents/Strings (work in progress)
- Here Documents
- 格式:
[n]<<[-]word here-document delimiter
- 效果
- 注意:
- 如果重定向运算符是
<<-
,行首的<tab>
和delimiter
那一行会被去掉(行首空格不会被去掉!)。这是为了使shell脚本的格式更自然
- 如果重定向运算符是
- Here Strings
- 格式:
[n]<<<word
-
3 杂项
- 每个重定向运算符前,指定文件描述符的
n
都可以换成{varname}
,这时一个大于等于10的文件描述符会被赋值给变量varname
。例如exec {my_fd}<filename
- 注意命令写成脚本之后,用zsh执行会报错
exec: {varname}: not found
,待排查
- 注意命令写成脚本之后,用zsh执行会报错
- 每个重定向运算符前,指定文件描述符的
-
此回复已被删除!
-
是会常用又常忘的东西,mark了!
-
mark了