Interactive Session 1A

Don’t fear the Terminal

Source materials

These materials are modified from the following source:

S. Jeanette Clark, Matthew B. Jones, Samantha Csik, Carmen Galaz García, Bryce Mecum, Natasha Haycock-ChavezDaphne Virlar-Knight. 2022. Scalable and Computationally Reproducible Approaches to Arctic Research.

https://learning.nceas.ucsb.edu/2022-09-arctic/

1. Commonly used (and very helpful) bash commands

Below are just a few bash commands that you’re likely to use. Some may be extended with options (more on that in the next section) or even piped together (i.e. where the output of one command gets sent to the next command, using the | operator). You can also find some nice bash cheat sheets online, like this one. Alternatively, the Bash Reference Manual has all the content you need, albeit a bit dense.

bash command what it does
pwd print your current working directory
cd change directory
ls list contents of a directory
tree display the contents of a directory in the form of a tree structure (not installed by default)
echo print text that is passed in as an argument
mv move or rename a file
cp copy a file(s) or directory(ies)
touch create a new empty file
mkdir create a new directory
rm/rmdir remove a file/ empty directory (be careful – there is no “trash” folder!)
grep searches a given file(s) for lines containing a match to a given pattern list
awk a text processing language that can be used in shell scripts or at a shell prompt for actions like pattern matching, printing specified fields, etc.
sed stands for Stream Editor; a versatile command for editing files
cut extract a specific portion of text in a file
join join two files based on a key field present in both
top, htop view running processes in a Linux system (press Q to quit)

2. General command syntax

Bash commands are typically are written as: command [options] [arguments] where the command must be an executable on your PATH and where options (settings that change the shell and/or script behavior) take one of two forms: short form (e.g. command -option-abbrev) or long form (e.g. command --option-name or command -o option-name). An example:

# the `ls` command lists the files in a directory
ls file/path/to/directory

# adding on the `-a` or `--all` option lists all files (including hidden files) in a directory
ls -a file/path/to/directory # short form
ls --all file/path/to/directory # long form
ls -o all file/path/to/directory # long form

3. Some useful keyboard shortcuts

It can sometimes feel messy working on the command line. These keyboard shortcuts can make it a little easier:

  • Ctrl + L: clear your terminal window
  • Ctrl + U: delete the current line
  • Ctrl + C: abort a command
  • up & down arrow keys: recall previously executed commands in chronological order
  • TAB key: autocompletion

4. Practice using some common bash commands

  1. Use the pwd command to print your current location, or working directory. You should be in your home directory on the server (e.g. /home/yourusername).

  2. Use the ls command to list the contents (any files or subdirectories) of your home directory

  3. Use the mkdir command to create a new directory named bash_practice:

mkdir bash_practice
  1. Use the cd command to move into your new bash_practice directory:
# move from /home/yourusername to home/yourusername/bash_practice
cd bash_practice
  • To move up a directory level, use two dots, .. :
# move from /home/yourusername/bash_practice back to /home/yourusername
$ cd ..
Note

To quickly navigate back to your home directory from wherever you may be on your computer, use a tilde, ~ :

# e.g. to move from from some subdirectory, /home/yourusername/Projects/project1/data, back to your home directory, home/yourusername
$ cd ~

# or use .. to back out three subdirectories
$ cd ../../..
  1. Add some .txt files (file1.txt, file2.txt, file3.txt) to your bash_practice subdirectory using the touch command (Note: be sure to cd into bash_practice if you’re not already there):
# add one file at a time
touch file1.txt
touch file2.txt
touch file3.txt

# or add all files simultanously like this:
touch file{1..3}.txt

# or like this:
touch file1.txt file2.txt file3.txt
  1. You can also add other file types (e.g. .py, .csv, etc.)
touch mypython.py mycsv.csv
  1. Print out all the .txt files in bash_practice using a wildcard, *:
ls *.txt
  1. Count the number of .txt files in bash_practice by combining the ls and wc (word count) funtions using the pipe, |, operator:
# `wc` returns a word count (lines, words, chrs)
# the `-l` option only returns the number of lines
# use a pipe, `|`, to send the output from `ls *.txt` to `wc -l`
ls *.txt | wc -l
  1. Delete mypython.py using the rm command:
rm mypython.py
  1. Create a new directory inside bash_practice called data and move mycsv.csv into it.
mkdir data
mv mycsv.csv ~/bash_practice/data

# add the --interactive option (-i for short) to prevent a file from being overwritten by accident (e.g. in case there's a file with the same name in the destination location)
mv -i mycsv.csv ~/bash_practice/data
  1. Use mv to rename mycsv.csv to mydata.csv
mv mycsv.csv mydata.csv
  1. Add column headers col1, col2, col3 to mydata.csv using echo + the > operator
echo "col1, col2, col3" > mydata.csv
Tip

You can check to see that mydata.csv was updated using GNU nano, a text editor for the command line that comes preinstalled on Linux machines (you can edit your file in nano as well). To do so, use the nano command followed by the file you want to open/edit:

nano mydata.csv

To save and quit out of nano, use the control + X keyboard shortcut.

You can also create and open a file in nano in just one line of code. For example, running nano hello_world.sh is the same as creating the file first using touch hello_world.sh, then opening it with nano using nano hello_world.sh.

  1. Append a row of data to mydata.csv using echo + the >> operator
# using `>` will overwrite the contents of an existing file; `>>` appends new information to an existing file
echo "1, 2, 3" >> mydata.csv

5. Git via a shell

Git, a popular version control system and command line tool can be accessed via a shell. While there are lots of graphical user interfaces (GUIs) that faciliatate version control with Git, they often only implement a small subset of Git’s most-used functionality. By interacting with Git via the command line, you have access to all Git commands. While all-things Git is outside the scope of this workshop, we will use some basic Git commands in the shell to clone GitHub (remote) repositories to the server and save/store our changes to files. A few important Git commands:

Git command what it does
git clone create a copy (clone) of repository in a new directory in a different location
git add add a change in the working directory to the staging area
git commit record a snapshot of a repository; the -m option adds a commit message
git push send commits from a local repository to a remote repository
git fetch downloads contents (e.g. files, commits, refs) from a remote repo to a local repo
git pull fetches contents of a remote repo and merges changes into the local repo

End Interactive Session 1A