Introduction to Shell
In order to effectively work on the SPS website, you will need to have access to a shell. This guide will help provide a minimal introduction to using shells. Since our access to the OCF computers (where we setup the development environment) is over SSH (Secure SHell), this will be necessary in order to effectively debug the website.
The most common shell (at the time of writing) is
bash, the Bourne Again Shell.
Bash is the default shell provided on the OCF and the only shell available through git bash (for Windows). Most Linux distributions use
bash as their default shell, but a few distributions have switched to
zsh, the Z shell. The Terminal program in Mac OSX uses
zsh by default. These instructions will be written with
bash in mind, but they should work very similarly for
A common convention for shell commands is to specify a replaceable string by square braces, e.g.
[username] would represent your username (
carterturn for me).
Another common rule of shells is that pressing
Ctrl+c will cancel the current operation. Sometimes you will have to press it multiple times.
It will be useful to have access to a shell during this introduction. You should try to run these commands on the OCF computers while you read through this document. To open a shell on the OCF comptuers, connect via SSH. To do this, you will first need to open a shell on your computer. On Mac OSX or Linux, this can be done by running the Terminal program. On Windows, this can be done by opening git bash. Once you have done so, run the following command:
ssh [your OCF username]@ssh.ocf.berkeley.edu
The Shell as a File Browser
The most common use of the shell is to browse files, copy and move files, or delete files: the same functions that a typical GUI file browser performs. This section will go over some of the commands used to do this.
The first command is
cd, which stands for "change directory." Most shell commands take one or more arguments that tell the command what to do.
cd takes a single command, the path to the directory to change to. A path can be specified in a few different ways. The most common way is relative: if a path does not start with
/, the path will be relative to the current directory. If a path starts with
/, the path will be relative to the root directory of the system. Finally, if a path starts with
~, the path will be relative to the home directory (on most systems, this is
/home/[username], on the OCF, home is
/home/[first letter of username]/[first two letters of username]/[username]. Try running
cd public_html. This will change your current directory to the
public_html folder. You will notice your prompt (the part before your cursor) change to include
public_html. By default, the prompt shows your current directory (
~ for home). If you run
cd with no arguments, you will change back to your home directory. Run
cd with no arguments now.
You have probably noticed that
cd can only be used to change directories if you already know what directories are there. To view files and folders in your current directory, use the command
ls (short for "list"). If you run this command in your home directory, you should see the
ls can also list the contents of other directories. If you specify a path, similar to
ls will show files in that folder (or just the file at that path if a path to a file is specified). In addition to the path argument,
ls has a variety of flags: arguments that can be used to modify its behavior. To get a list of the flags, run the command
ls --help. You should see a lengthy explanation of the available flags. Many flags have a short and long form. In general, the short form is a single dash followed by a single letter, while the long form is two dashes followed by a word (most commands use this convention, but there are some exceptions). An important flag for
ls is the
-l flag. This changes
ls to a list mode, outputting additional information about the files and folders at the path.
-a is also useful. This shows hidden files and folders (files and folders with names beginning with a
.). Flags can be combined. Try running the command
ls -la in your home directory (note that the short flags can be combined with only a single
- preceding them. You should see a folder called
.. This is the current directory. The folder called
.. is the next directory up (the folder with the first two letters of your username). Note that you can use
.. as part of paths (e.g. for
Move, Copy, Remove:
ls you can move around and see files. Next, you will likely want to move, copy, and remove files. The commands for this are
rm respectively. The
cp commands are quite similar (except that
cp copies instead of moving). To see how these work, we will first create an empty file. Run the command
touch test.file to create an empty file named
ls to check this. Let us then copy that file.
cp copies all of its non-flag arguments to its last argument. If the last argument is a folder,
cp copies the file or folder into the folder. Run
cp test.file test_2.file. This creates another empty file (or to be precise, a copy of the empty file
test_2.file. Next, let us copy a file into a directory. Run the command
cp test.file public_html/ to create a copy of
test.file in the
public_html folder. Sometimes you want to copy a folder.
cp requires the flag
-r (for recurse into folders) to do this. You can see this by running
cp -r public_html/ new_folder. This will create a new folder in your home directory named
new_folder, which should contain a file named
mv is quite similar to
cp, so we will not go into detail on its use here. Let us now use
rm to cleanup all the files we just created. First, run
rm test.file test_2.file. This will delete the files we made in the current directory (verify this with
ls). Note that
rm will allow you to remove multiple files at once. Next, run
rm public_html/test.file. Finally, we remove
new_folder. If you try to do this with
rm new_folder, you will get an error message:
rm does not allow you to just remove folders. To delete folders, run
rm -r new_folder (again,
-r means recurse into folders). On the OCF computers, you will be prompted for each file that
rm attempts to remove. This is a safeguard against accidentally deleting large sets of files. Type
Y and press enter to confirm deletion (or
N to not delete if something seems wrong).
touch is used to create an empty file if none exists, and updates the timestamp on a file if it does exist (many system utilities use timestamps to determine if a file has changed, so this can be useful for telling a system utility to re-run. See
update_website_debug.sh for an example of this behavior). The command
mkdir [folder name] is used to "make" a "directory". If you are trying to delete an empty directory, the command
rmdir can be used.
rm -r can also be used to delete empty directories.
In this section, we will review the commands used for
git commands start with
git. Nearly all
git commands have a
--help flag that can be used to get more information about a command.
git command you will usually run is
clone. The full syntax is
git clone [remote url]. This will clone a remote url into a local folder, with the folder name based on the remote url. If you want to clone to a local folder with a different name, you can use the
-o [folder name] flag to specify a different local folder name.
When you are in a git repository, the command
git status provides useful information on the current state of your local copy of the repository. The first line of the output shows the current branch. After that, there are three lists (not all of them will be visible). The first list is a list of changes that are staged. Staged changes will be included in a commit when you run
git commit. The second list is a list of changes that are staged to be included in the next commit. The third and final list is a list of files that are not part of the git repository. These files will not be included in the next commit unless they are staged.
As you have probably heard, git uses a system of branches. A branch is effectively a set of commits in a chronological order with each commit being dependent on the commits before it. Two different branches have a different set of commits. To create a new branch, you can use the command
git checkout -b [new branch name]. The
-b flag is needed the first time a branch is checked out. After a branch is created, you can use the command
git checkout [existing branch name] to switch to it. When you checkout an existing branch, you should pull the latest version from the remote repository (Github). A pull is actually a combination of several steps, but they rarely need to be done separately, so we will defer a discussion of them. You can pull a branch by running
git add is used to stage changes to be committed. The simplest form is with the
--all flag. This will stage all the changes you have made (including new files). If you only want to prepare some changes, you can specify a path or set of paths, for example
git add file1.html folder1/file2.html. If you only want to add part of a file, you can use the
-p flag. This will open an interactive editor that allows you to select parts of a file.
Once all the changes you want staged are staged, run
git commit to make the commit. By default, the commit message is entered in interactive mode, so git will open the default text editor. For more information on common editors, see the Editors section of this page.
git commit has an option to provide the commit message on the command line using the
-m (for message) flag. For example,
git commit -m "Initial commit" will make a commit with the message
Initial commit without opening an editor.
Once you have added a commit to a branch, you will want to update the remote repository. This is done by pushing the branch using the command
git push. The first time you push a branch, you also need to specify the remote repository and remote branch. This is done with the
--set-upstream [remote repository] [remote branch] flag. The remote repository is identified by a short name, nearly always
origin. The remote branch should usually be the same as the local branch. The full command is
git push --set-upstream [remote repository] [remote branch]. After the first push, you can just use
git push; git will remember the remote repository and remote branch.
You will eventually want to combine your branch with
dev-stable. We will make changes to
dev-stable using Pull Requests in the Github web interface, but in order to make adding code to
dev-stable clean, you want to minimize the changes you have relative to
dev-stable. You do this by first merging
dev-stable into your branch. To do this, we want to latest copy of
dev-stable. To get this, run
git checkout dev-stable, then
git pull. Then, checkout your branch again. Finally, run
git merge dev-stable. This will bring the code from the latest version of
dev-stable into your branch. If all goes well, a text editor will open will a commit message for the merge already loaded. You can just save and close this to complete the merge. If you have edited a section of a file that has also changed in
dev-stable, you may get an error indicating that it was not possible to complete the merge automatically. In this case, contact Carter. After completing the merge, run
git push to sync the merged code to the remote repository.
Sometimes you want to edit files from the shell. To do this, you can use a text editor. All editors take a file as an argument, for example
[editor name] [file] will open a file in an editor. The three most common editors, all available on the OCF computers, are
nano is the easiest to use; all of
nano's shortcuts are written on the button of the screen when
nano is running.
vim are much more powerful (it has been joked that
emacs is practically an entire operating system disguised as a text editor, which may be fair, considering that it has the game snake built in), but rely on a variety of keyboard shortcuts that are only apparent after reading their manuals. Reading the shortcuts can be accomplished in a few hours, but memorizing them tends to take a few weeks. A few important shortcuts for these editors are listed below.
To start entering text in
emacs, just start typing. You can move around a file using the arrow keys (though this is not the most efficient method, it is the easiest to remember). To save a file, use the keyboard shortcut
Ctrl+x Ctrl+s (that is, hold down
Ctrl, then press
x, then release
x, then press
s, then release
s, then release
Ctrl). To close
emacs, use the keyboard shortcut
emacs will prompt you if the file is unsaved.
emacs is the best text editor, but it does take some time to learn.
vim (Vi iMproved) are very similar text editors. In many cases, the command
vi simply opens
vim. I will use
vim to refer to both here.
vim is a modal text editor, which means that some commands only work at some times. Pressing
Esc several times will let you exit to the "root" mode. To start entering text, press
i (for "insert" mode). Depending on your keyboard configuration, the arrow keys may not allow you to move around a file while in insert mode. To leave insert mode, press
Esc. To save a file, leave insert mode, then enter
:w (to write a file). To quit, enter
:q. To quit without saving, enter
Using a shell may seem tedious due to all of the typing required to make your instructions explicit. Fortunately, there are a variety of keyboard shortcuts that make shell usage much faster (faster than GUI usage in most cases). The keyboard shortcuts here apply to the
bash shell; similar shortcuts exist for
zsh, but I am less familiar with them.
The most important keyboard shortcut is the tab key. If you press tab,
bash will attempt to automatically complete your current command or path (sometimes,
bash is even able to complete flags.
git also has excellent tab compatibility, try pressing tab in the middle of typing a
git command to see what happens). This does not work too well with two letter commands, but for anything longer than three letters, it is worth trying to press tab to see if
bash can automatically figure it out. If you have two files or folders that begin with the same letters,
bash will fill in as many letters as are in common. Once you have cloned our repos, trying typing
cd s, then pressing tab. It should automatically fill in
cd sps_web_. If you then type
2 and press tab again, the entire command
cd sps_web_2020 will be filled in. You only had to press seven keys!
If you use mice frequently while typing, you probably find yourself clicking on the middle of phrases to fix a typo. Your mouse does not work in the shell, but there are keyboard shortcuts for rapidly moving around a line.
Ctrl+a takes you to the beginning of a line,
Ctrl+e takes you to the end of a line.
Alt+f jumps forward a "word" (defined as a set of letters and numbers),
Alt+b jumps backwards a word. On some systems,
Alt+Backspace backspaces an entire word (this depends fairly strongly on your keyboard layout).
The shell also has its own built-in cut and paste functionality (usually referred to as "kill" for cut and "yank" for paste).
Ctrl+u cuts everything to the left of the cursor.
Ctrl+k cuts everything to the right of the cursor.
Ctrl+y then pastes that content.
bash maintains a history of commands you have run. If you press the up arrow (or
Ctrl+p), the previous command will be loaded, ready to run again. You can keep moving up through your history. Use the down arrow (or
Ctrl+n) to move forwards in history. You can even search backwards in the history by pressing
Ctrl+r, then (with
r released) entering a phrase. History is retained between sessions. If you log out and log in again, your history will still be there. If you visit the same directories often, using history may be faster than typing a
Hopefully this guide has provided you with enough information on using shells to provide "a landing you can walk away from". This is nowhere near a complete description of what shells are capable of. All shells have complete programming languages built in, with
for loops, and variables. Nearly anything (except viewing pictures and watching videos) can be done entirely from the shell, then automated. If you would like to know more, the Advanced Bash-Scripting Guide is a good resource. Some other very powerful tools are
awk; they are installed with most shells.