Linux/Unix

For this purposes of this page, Linux is just the non-proprietary, open-source “clone” of the proprietary Unix operating system.  They are otherwise almost completely interchangeable as regards functionality, command syntax, file and directory organization, and the like. There are three common small computer operating systems in use in the present decade:  Microsoft Windows, Mac OS X, and Linux.  But Mac OS X is also built on Unix, so command line operations under Mac OS X (via the ‘Terminal’ utility) are almost indistinguishable from Linux.  Only MS Windows has no obvious roots in Unix, and it shows if you ever try to do anything moderately complicated at the Windows/DOS command line. Several general features make the Unix/Linux environment by far the most powerful and flexible modern operating system for creatively and efficiently working with programs, text files, and data files:
  • The existence of a huge variety of useful commands for manipulating, display, parsing, and modifying files.  This page lists just a few of the most common examples.
  • The existence of a huge body of free software packages to do more complex things, ranging from image manipulation and conversion to editors to compilers to numerical models and analysis tools.
  • The ability to redirect program input and output using the ‘<‘ and ‘>’ operators and to ‘pipe’ output from one command or program directly to the input of another program using the ‘|’ operator.   It is possible to accomplish surprisingly complex tasks via a single command line by stringing together commands in clever ways.
If you’re content to limit what you do with text and data files (for example) to what is possible and convenient in someone else’s software package, then by all means, skip learning Unix/Linux.  Otherwise, if you are serious about hands-dirty, under-the-hood scientific computing,  it’s hard to imagine making do with any other currently available operating system   (Note:  there is only one widely used “currently available operating system” that is NOT Linux or Unix.) The following web site has a very useful introduction to Linux/Unix for those who are new to it or who would like to refresh their knowledge of key commands:

How to ACCESS the Unix/Linux command line

In order to use Unix or Linux, you need to know how to access the operating system at the command line level.  This generally means opening a text terminal window on your local machine and either using Unix/Linux locally or connecting to a remote Unix/Linux machine and logging in there.

On a Macintosh

Go to the Applications folder, scroll to Utilities, open that folder, and then double-click on the Terminal application.   It will open with the command prompt, and you’re ready to begin entering Unix commands. If you wish to connect remotely to another Linux/Unix computer from your Mac, open a Terminal window as described above.  You can type a command similar to the following:
ssh remotemachine.aos.wisc.edu
and log in there.  A common way to end a remote ssh session is to simply type the character control-D.

On a Linux computer

Depending on the flavor of Linux, you can probably find various terminal programs that can be opened from a menu of applications or utilities.  One common one is ‘xterm’, but there are others.  All have a similar function,  and most have the string ‘term’ somewhere in the program name. If you need to connect remotely to another machine, use ‘ssh’ as described above for Mac OS X.

On a Windows computer

Windows machines don’t have anything like the Unix operating system, so the only way you can use Unix/Linux from a Windows machine is by remotely connecting to another machine that does have a Unix variant on it. To do this, you need the SecureCRT package or similar, which uses the SSH protocol to connect other machines.   My recollection is that you need to provide the remote IP address and login info via a form built in to SecureCRT, but this may not be up to date.

Especially Useful Unix/Linux Commands

Display the manual page for any standard Unix/Linux command (useful for finding out what the command does and what options exist):
man command_name
Identify the present working directory:
pwd
List the contents of the current directory:
ls
Same as above, but list the details of each file (size, permissions, mod time), with the most recently modified file first:
ls -lt
Same as above, but displays a screen full at a time (use space bar to scroll down a page, return to scroll just one line):
ls -lt | more
Same as above, but only display the 10 most recently modified files in the current directory:
ls -lt | head -10
Create a text file that lists all files in the present directory that have the suffix “.dat”:
ls *.dat > datfilelist.txt
Display the contents of a text file page by page:
more file.txt
Count the number of lines, words, and characters (bytes) in a file:
wc file.txt
Count the number of lines containing ‘string’:
grep string file.txt | wc
Count the number of files in the present working directory whose names contain ‘foo’:
ls *foo* | wc
Display only the first 10 lines of a text file:
head -10 file.txt
Display only the last 20 lines of a text file:
tail -20 file txt
Display lines 1000 through 1500 of a text file,  one screen at a time:
head -1500 file.txt | tail -501 | more
Run a compiled program a.out with standard input read from the keyboard and standard output displayed to the screen:
a.out
Same as above, but with output displayed a page at a time:
a.out | more
Same as above, but with output saved to a file:
a.out > outfile.txt
Same as above, but reading standard input from a file:
a.out < infile.txt > outfile.txt
Compare two text (ASCII) files and display any differences:
diff file1.txt file2.txt
Display all lines in a text file that contain ‘string’:
grep string file.txt
Display all lines in a text file that DON’T contain ‘string’:
grep -v string file.txt
Find all lines that contain BOTH ‘string1’ and ‘string2’ and save only those lines to a file:
grep string1 file.txt | grep string2 > outfile.txt
Combine (concatenate) two files end-to-end (works for binary files too), saving the result to a new file:
cat infile1.txt infile2.txt > outfile.txt
Paste the columns from two text files side-by-side (usually both files should have same number of lines):
paste file1.txt file2.txt > filemerge.txt
Select only certain character columns 3 through 50 from a text file, discarding the rest, and display to the screen via ‘more’:
cut -c3-50 infile.txt | more
Display the contents of a binary file, with all byte values displayed as hexadecimal:
od -h file.dat | more
Same as above, but with all byte values displayed as characters:
od -c file.dat | more
Same as above, but with all 2-byte sequences displayed as integers:
od -d file.dat | more
Same as above, but with all 4-byte sequences displayed as floats (see ‘man od’ for more options):
od -f file.dat | more
Find and display all files with suffix “.dat” in directory tree foo, descending into subdirectories as well:
find foo -name "*.dat"
Sort the lines in a text file and write the result to another file:
sort file.txt > sorted.txt
Same as above, but suppress duplicate lines:
sort -u file.txt > sorted_unique.txt