Day-1: Introduction to UNIX and Linux

Essentials for Scientific Computing: Introduction to UNIX and Linux Day 1 Ershaad Ahamed TUE-CMS, JNCASR

May 2012

1

Operating Systems

Computers consist of one or more CPUs, memory and peripheral devices. The CPU fetches instructions from memory and executes them. When a computer boots up, the first instructions that it fetches and executes is from the BIOS. These instructions direct the CPU in various stages to eventually load the operating system and execute it. From that point onwards the operating system (or specifically the kernel) runs at all times and has the following functions. 1. Hardware abstraction 2. Resource management and multitasking 3. Standardised interfaces

1.1

Hardware Abstraction

The manner in which hardware is instructed to perform their specific functions is highly dependent on the hardware itself. This in turn depends on the manufacturer, design and even the specific revision of the hardware in question. In case we didn’t have an OS, every programmer would have to know the exact details of how to interact with the specific hardware on which their program will run whenever they write a program.

1.2

Resource Management and Multitasking

On a computer which is running an operating system that supports multitasking (which includes most modern OSes), several programs appear to be running simultaneously although there may be only one or a few processors present. This is called multitasking and is achieved by running each program for a short amount of time on the CPU and quickly switching between them, giving the user the illusion that all programs are executing concurrently. Multiple programs running also means that they are sharing common resources such as memory and even devices such as the video display. It is the operating system that manages this sharing of resources and prevents rogue programs from disrupting 1

the functioning of other programs especially by protecting each program’s share of memory from other running programs.

1.3

Standardised Interfaces

Programs or applications interact with the operating system by invoking functions provided by the operating system called system calls. For example when an application needs to open a file, it invokes a specific system call with appropriate parameters. The operating system then performs hardware specific tasks that will eventually make the data of the file available to the application program. The specific names, parameters that need to be passed and return values of these system calls constitutes the Application Programming Interface (API) presented by the operating system to an application programmer. By standardising these APIs, an application program’s source code can be compiled unmodified on all operating systems that present the same API regardless of the specific vendor or version of the operating system and independent of the hardware it is running on.

2

UNIX, Linux and the GNU Project

In the early 1970’s, Ken Thompson, Dennis Ritchie and others began to design and develop the UNIX operating system at AT&T Bell Labs. Work on UNIX development continued into the 80’s and by the 1990’s several variants of the UNIX operating system were developed by various entities, a notable one being the Berkeley Software Distribution (BSD). The large number of variants eventually led to the creation of the IEEE POSIX standard which initially defined the API, and eventually all aspects of the behaviour of a UNIX-like system including commands, utilities and standard libraries. POSIX then evolved into what is now the Single Unix Specification (SUS). In 1983, Richard Stallman initiated the GNU project aiming to create a UNIX compatible software system consisting of free software. Free software can be modified, copied and redistributed without restriction. Several licenses qualify for designation as free software. The GPL (GNU General Public License) is the license adopted by the GNU project. In essence it dictates that the source code of any software released under the GPL should be available without restriction and that any modified or derivative software will also be governed by the GPL. Linux began as a hobby kernel by Linus Torvalds in 1991. It aimed to be POSIX compliant and was licensed under the GPL. The combination of Linux kernel and GNU project software is what is now popularly known as Linux, although it is more accurate to refer to it as GNU/Linux. Fedora, Ubuntu, SUSE etc. are all distributions of the GNU/Linux operating system. Software from the GNU project is extensive and includes everything from compilers for a large number of languages to editors to complete desktop graphical user interfaces (GUIs).

2

3

Users and Groups

UNIX was designed around the concept of multiple users. Each user on a system owns a set of files. The owner can define whether other users may view, modify or execute (if they are scripts or executables) their files. Users may be divided into sets called groups. A user may be a member of several groups. Groups allow users to allow specific permissions to members of a group to which they belong. In UNIX, users and groups are identified by names which are mapped to unique user and group ids respectively. Any file on disk has a user and group id associated with it, and this defines its ownership. As special user exists on UNIX systems called the root user. The root user has access to all file and functions on the system. A person logs in as the root user in order to perform system administration tasks such as configuring a network or changing operating system policy and settings. The root user can edit files critical to the functioning of the system and can delete all files on the system, while normal users can only edit files that are owned by them or for which they are given write permissions.

4

Processes

Once the programmer has written the source code in a programming language (like C or FORTRAN), it is compiled into machine code that is ready to be executed. This machine code executable when stored on disk is called a program. When a program is executed, the operating system reads it from disk and loads all or parts of the program into main memory. From there it initiates the execution of these instructions by the CPU and the program becomes a process. Each process in a UNIX system is identified by a process ID. A process may have a limited lifetime or may run for the entire duration the system is up. processes may be in a “running”, “stopped” or “sleeping” state.

5

Files (from the point of view of processes)

A process may open a number of files for input or output. In the UNIX design files need not necessarily need to be data stored on disk. They may be a network device or even a sound device. This is one of the abstractions built into the UNIX design. The operating system can present any device (real or virtual) to an application program as a regular file. Three files are always opened whenever a process is started. They are the standard input, standard output and standard error (stdin, stdout and stderr). these files do not exist on disk.

6

Directory Layout, Pathnames and Symbolic Links

In UNIX all files are stored within directories (folders). UNIX uses a hierarchical directory structure similar to other operating systems. In UNIX all directories are subdirectories of the uppermost level root directory represented by ‘/’. 3

The ‘/’ is also the symbol for separating elements in a pathname. An absolute pathname specifies how to reach a particular file or directory from the root directory. For example,. ‘/home/user/thesis.txt’

6.1

Relative and Absolute Pathnames

If a pathname is specified without a leading ‘/’, then it is a relative path. Relative pathnames are interpreted as relative to the current working directory. Picture it as an absolute pathname that is constructed by joining the current working directory and the relative pathname. Every directory has two special subdirectories named ‘.’ and ‘..’. ‘.’ is equivalent to the current working directory and ‘..’ is equivalent to the parent of the current working directory.

6.2

Symbolic Links

Symbolic links are files on a disk that point to other files (shortcuts to Windows users). Deleting a symbolic link does not delete the file that it points to. Symbolic links take up very little actual space on the disk. One use of symbolic links is the make a direct link to a file that has a very long pathname. Symbolic links are created with the ln command.

7 7.1

Basic Commands The Bash Shell

In operating system, the shell is the interactive interface presented to the user. The user interacts with the shell in order to achieve specific tasks, and the shell in turn communicates with the operating system kernel. The shell is thus a layer of abstraction that hides details of the underlying operating system and presents a uniform and convenient way for the user to interact with the OS. The most common text mode shell for Linux based operating systems is the Bash shell. The Bash shell presents the user with a programmable interface along with several built-in commands. If you need to get a shell on a Linux system running a graphical front end, you usually need to open the ‘terminal’ menu item. The Bash shell presents a prompt to the user when it is ready to accept commands. A user prompt may look like the following. ershaad@strobe:~$ or [ershaad@n138 ~]$ And a root prompt will be similar to root@strobe:~# or [root@n138 ~]# 4

Notice the # at the end of the prompt for the root user. As we mentioned above, the root user can affect the working of the system and therefore, this account should be used with caution.

7.2

ls Command and the Current Working Directory

The ls command lists the contents of the current working directory. The current working directory for a session of the shell can be found by entering the pwd command. Files that begin with a period ‘.’ are hidden files and do not appear in the output of the ls command. In order to view all files including hidden files pass the -a command line option to ls ls -a. Command line options may have a long and a short form. The corresponding long option for -a is --all, so that you would type ls --all to see the full listing of files. Short options can be strung together or supplied separately. Like ls -l -t or ls -lt. The same cannot be done with long options. Both long and short options can be used in the same command ls -t --all. Most commands usually print a usage summary when passed either of the -h or --help options. In order to get detailed usage of a command, the man page can be referred. It is an online documentation that can be viewed from the shell. To view the man page of the cp command simply type man cp at the shell prompt. The first few lines of the cp man page looks like. CP(1)

User Commands

NAME cp - copy files and directories SYNOPSIS cp [OPTION]... [-T] SOURCE DEST cp [OPTION]... SOURCE... DIRECTORY cp [OPTION]... -t DIRECTORY SOURCE... DESCRIPTION Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY. The words in CAPS or in the form or in italics means that they need to be substituted with an appropriate word which may be a filename or keyword. In the above example SOURCE needs to be replaced with the path of the source file. Words or options within square brackets ‘[]’ are optional. If a ‘|’ symbol separates one or more words or options like -r|-s, it means only one of them may be specified at a time. An ellipsis ‘. . . ’ after a word means that more can be specified. In the above man page SOURCE... means one or more source files can be specified separated by spaces.

7.3

Essential Commands

• pwd Print the present working directory • cd Change the current working directory 5

cd [dir] cd /home/ershaad/workdir cd workdir • rm Delete file(s) rm draftfile.txt rm -r tempdir • cp Copy files cp thesis.tex thesiscopy.tex cp -r datadir datadircopy • mv Move or rename files mv oldfilename.txt newfilename.txt mv /home/ershaad/olddirname /home/backup/newdirname mv myfile.txt .. • mkdir Create a directory mkdir newdir mkdir -p newdir/newsubdir/subsubdir • cat Write the concatenation of contents to stdout cat file1.txt cat file1.txt file2.txt • less View the contents of files less readme.txt • file Information on the type of a file file image01.jpg • find Search for files based on criteria find sourcedir -name "*.f90" • locate Search for filename within index of files locate workreport.doc 6

• chmod Manage permissions for a file chmod u+x file.sh chmod go-rwx file.txt chmod o-w file.txt • gzip Compress a file gzip bigfile.dat • gunzip Uncompress a file gunzip bigfile.dat.gz • tar Create/Extract an archive tar -cvf homebackup.tar /home/ershaad tar -czvf homebackup.tar.gz /home/ershaad tar -xvf homebackup.tar tar -xzvf homebackup.tar.gz • df Summary of space on filesystems df -h • exit Exit the shell • head, tail Print the first/last few lines of a file head bankstatement.txt head -n 15 bankstatement.txt tail bankstatement.txt tail -n 12 bankstatement.txt • du Find how much disk space a directory is using du -sh Downloads • date Print or set the system date/time • grep Print lines in file that match some pattern 7

grep orange fruits.txt

8