I/O Redirection in Bash

Since Bash is a superset of Bourne shell, the IO redirection is the same across the Bourne shell family. To find out exactly which shell you are using in Unix or Linux, typing in command

echo $SHELL

shall display an executable command with its path, for example

Path	Shell
/bin/sh	Bourne shell
/bin/bash	Bourne Again SHell
/bin/csh	C SHell
/bin/ksh	Korn SHell

Bash takes input from standard input (stdin), writes output to standard output (stdout), and writes error output to standard error (stderr). Standard input is connected to the terminal keyboard, while standard output and error are connected to the terminal screen, by default. Redirection of I/O is accomplished by using a redirection metacharacter followed by the desired destination (stdout & stderr) or source (stdin). Bash uses file descriptor numbers to refer to the standard I/O, where 0 is standard input, 1 is standard output, and 2 is standard error. Below are the common redirection for the Bourne shell family:

Meta Character	Action
>	Redirect standard output
>>	Append to standard output
2>	Redirect standard error
2>&1	Redirect standard error to standard ouput
2>&1|	Redirect standard error to standard ouput, and pipe them to another command
<	Redirect standard input
|	Pipe standard output to another command

Please note that because < and > are referring to standard input and output respectively, the numbers 0 and 1 are not required in this case.

Useful Sed command examples in UNIX

1. Substitute a regular expression into a new value
A simple example is to change the first occurrence of 'old' on each line from file1 to 'new' and save as file2

sed 's|old|new|' < file1 > file2

where 's' is the substitute command, '|' is the delimiter, 'old' is the search pattern in regular expression, and 'new' is the replacement string. We can also test string substitution using 'echo' like this

echo "old, and some other old stuffs" | sed 's|old|new|'

which will output 'new, and some other old stuffs'. Note that in this case it only replace the first occurrence of 'old' with 'new'. If we want to replace all 'old' with 'new', we could append a global replacement symbol 'g' after the last delimiter '|'

echo "old, and some other old stuffs" | sed 's|old|new|g'

which will output 'new, and some other new stuffs'
2. Pick any delimiter you like
We were using '|' as the delimiter in the above examples. In fact, we can pick any character as the delimiter, as long as it is not in the string we are looking for. For example,

echo "old, and some other old stuffs" | sed 's/old/new/g'
echo "old, and some other old stuffs" | sed 's:old:new:g'

have the same effect.
3. Use & as the matched string
We could search for a pattern and add some characters on top of the original string like this

echo "old, and some other old stuffs" | sed 's|[a-z]*|(&)|'

which will output '(old), and some other old stuffs'. The special character '&' represents the matching pattern. We could have any number of '&' in the replacement string

echo "old, 123 and 456 some other old stuffs" | sed -r 's|[0-9]+|& &|'

which will output 'old, 123 123 and 456 some other old stuffs'. Note that we use '-r' here to indicate we are using extended regular expression such that sed will support the use of '+'.

Useful Find command examples in UNIX

Find command is one of the most useful commands in UNIX, as there are many options out there to accomplish various search tasks.

• Find files which have been modified for more than or less than a certain time
We can use 'find * -mtime n' to search files under current directory based on modification time. The numeric argument n can be specified as +n, meaning greater than n days; -n, meaning less than n days; or n, meaning exactly n days. For example,
to find files which were last modified for more than 30 days ago

find . -mtime +30

to find files which were last modified for less than 15 days ago

find . -mtime -15

to find files which were last modified for exactly 10 days ago

find . -mtime 10

There are other options like

-mmin n, which refers to file's data was last modified n minutes ago
-amin n, which refers to file's data was last accessed n minutes ago
-atime n, which refers to file's data was last accessed n days ago
-type f, which searches for regular file
-type d, which searches for directory,
-exec command, which executes command by passing in the find results,
-perm /mode, which searches files with given permission 

• Remove files which are older than n days
Since we understand how to search for files which were last modified for more than n days, we can then use the rm command to remove them

find /some-dir/* -mtimes +30 -type f -exec rm {} \;

'-exec' is used to pass in commands like 'rm'; '{}' is replaced by the file name that 'find' returns; and the '-exec' command has to end with '\;'.
Another way will be

find /some-dir/* -mtimes +30 -type f -print0 | xargs -0 rm

where 'xargs' reads 'find' results and execute 'rm' on each of the 'find' results
• Find files which contain particular strings
Below is an example to search all txt files starting from current directory for string "some-string"

find . -name "*.txt" -print | xargs grep "some-string"

Using Python SimpleHTTPServer to share files

There are cases when I want to share some files within the intranet to other users.

Python comes with a simple built-in module called SimpleHTTPServer that can turn any directory in the system into the web server directory. I literally just need a single command line to accomplish this task, given Python is shipped with openSUSE!

Assuming I am running linux, and I want to share my home directory ~

cd ~

Start up the http server on port 10001

youyang@monkey-dev-01:~> python -m SimpleHTTPServer 10001

Now the http server is running on port 10001. Open a browser and type the following address, where monkey-dev-01 happens to be my hostname

http://monkey-dev-01:10001

Since there is no index.html under my home directory, the files in the home directory will be listed. Now my purpose is served, but there is one slight problem -- the program is running on the foreground, meaning I can't do anything on that open terminal as it continues to display the http requests when I browse the directories. To change the running process from foreground to background, I need to press CTRL+Z to pause the process, as shown below

^Z
[1]+  Stopped                 python -m SimpleHTTPServer 10001

and type 'bg' to move the process to run in the background

youyang@monkey-dev-01:~> bg
[1]+ python -m SimpleHTTPServer 10001 &

We can also use 'jobs -l' to display the jobs in current session

youyang@monkey-dev-01:~> jobs -l
[1]+  4127 Running                 python -m SimpleHTTPServer 10001 &

We can also start up the http server running in the background directly, by appending a '&' at the end of the command

youyang@monkey-dev-01:~> python -m SimpleHTTPServer 10001&

Unix Cheat Sheet

List a directory		Alternatively, type 'ls --help' or 'man ls' for more information
	ls [path]	list given path without any options. Without the path, it is listing current directory
	ls -l [path]	list given path in long listing format
	ls -h [path]	list given path in human readable format
	ls -a [path]	list given path, including hidden files (which has prefix dot)
	ls -lrt [path]	list given path, combining different options. long listing format, sort by modification time in reverse order
	ls [path]	list given path, redirect listing results into a file
	ls [path] | more	list given path, showing listing results on one screen at a time
Change to a directory
	cd <directory>	change to the given directory
	cd ~	change to user's home directory
	cd /	change to root directory
	cd ..	go to parent directory
	cd ../<directory>	go to a sibling directory
Print current working directory
	pwd	display the current working directory
Create directory(ies)
	mkdir <directory>	create the directory under current working directory
	mkdir -p <dir1>/<dir2>	create directory2 under directory1; create parent directory2 as needed
Remove files or directory(ies)
	rm <file>	remove the file
	rm -rf <directory>	remove the contents of given directory recursively without prompt
Move a file or a directory
	mv <source> <dest>	rename source as dest, and move source to dest
Copy a file or a directory
	cp <source> <dest>	make a copy from source to dest
	cp -r <source> <dest>	copy recursively from source directory to dest directory
Download a file from http website
	wget <url>	download the resource to current directory
	curl <url>	download the resource to current directory
View a text file
	less <file>	view file page by page, with a lot of features
	more <file>	view file with screen length
	cat <file>	view file, but it will scroll to the end of file
	cat <file> | more	view file, and piped the file content to more command
Create a text file
	touch <file>	create an empty file
	vi <file>	use vi text editor to create a file (need to save the file though)
	echo "foo" >> <file>	create a file with content 'foo'
	cat > <file>	enter multi-line texts and press control-d to save the content to a file
Change a file's modification or access time
	stat <file1>	display the stat of given file
	touch -m -d '1 Mar 2013 02:53' file.txt	change the modification time of file.txt to be '1 Mar 2013 02:53', and create the file if it doesn't exist
	touch -a -d '1 Mar 2013 02:53' file.txt	change the access time of file.txt to be '1 Mar 2013 02:53', and create the file if it doesn't exist
Compare two files
	diff <file1> <file2>	show the differences between file1 and file2
	sdiff <file1> <file2>	show file1 and file2 side by side
Pipes and redirections
	<command> > <file>	redirect command output to a file, e.g. ll > file.txt writes current directory listings to file.txt
	<command> >> <file>	appends command output to the end of a file
	<command> < <file>	redirect the standard input from the file
	<command> < <file1> > <file2>	redirect the standard input from the file1, and then redirect the standard output to file2
	<command1> | <command2>	pipe the output of command1 as the input of command2
File permissions		read=4, write=2, execute=1
	chmod 700 <file(s)>	file(s) owner can read, write and execute
	chmod 600 <file(s)>	file(s) owner can read and write, but not execute
	chmod 400 <file(s)>	file(s) owner can read only
	chmod 755 <file(s)>	file(s) owner can read, write and execute; both group owner and others can read and write
	chmod 644 <file(s)>	file(s) owner can read and write; both group owner and others can read only
	chmod u+x <file(s)>	grant file(s) owner execute permission
	chmod u-x <file(s)>	remove execute permission from file(s) owner
	chmod a+rw <file(s)>	grant everyone read and write permission to the file(s)
Other useful comands
	df -h	report file system disk space usage in human readable format
	du -h	estimate file space usage in human readable format
	date	display or set the current system date and time
	top	display linux tasks, as well as CPU and memory stats
	ps -aef	display a snapshot of current processes
	ln -s <target> <link_name>	create a symbolic link to the target with link_name
	alias	display current defined aliases or create an alias
	!<command>	invoke previous run of the given command