Design of the Unix Kernel Goals of this Lecture

Design of the Unix Kernel

2 – Kernel

© Hannes Lubich, 2003–2005

Page 1

Goals of this Lecture • Understand the design and functionality of a typical Unix kernel • Understand how processes are structured and managed by the kernel • Understand the kernel support for the Unix file system • Understand the kernel support for the Input / Output system • Understand how program execution and standard program I/O are supported by the kernel to facilitate interprocess communication 2 – Kernel

© Hannes Lubich, 2003–2005

Page 2

1

Tasks of an Operating System • Provide a controlled interface between a user and its applications, and the computer hardware • Control re-entrant process execution by providing operations for the creation, suspension and termination of processes • Provide mechanisms for user-to-process, kernel-to-process and interprocess communication • Provide resources (CPU time, memory, disk space, I/O channels etc.) and avoid deadlocks, starvation etc. by: – – – – – –

Process scheduling (e.g. time sharing, batch, …) Memory management/protection and allocation for all processes Transfer of processes to secondary memory (swapping / paging) Management of secondary memory through a file system Organisation of I/O via a defined driver interface Proactive management of system resources

2 – Kernel

© Hannes Lubich, 2003–2005

Page 3

Block Diagram of the Unix Kernel M.J. Bach, The Design of The UNIX Operating System, Figure 2.1

User Programs Libraries

User Level

System Level

• Must be re-entrant • Can be monolithic or modular

System Level Hardware Level

2 – Kernel

System Call Interface Inter-Process Communication

File Subsystem Process Control Subsystem

Buffer Cache

Scheduler Memory Management

Character Block Device Drivers Hardware Control Hardware © Hannes Lubich, 2003–2005

Page 4

2

Process Hierarchies Initial process (Process 0)

/etc/init (Process 1)

Further system processes

Generated by the Kernel

Overlap process New process

getty (Process a)

login (Process a)

Login-Shell (Process a)

Inetd (Process b)

telnetd (Process c)

Login-Shell (Process c)

httpd (Process d)

Send Web Page

Adapted From: R. Adamov, VL: UNIX-Betriebssystem und Werkzeuge, Univ. Zürich

2 – Kernel

© Hannes Lubich, 2003–2005

Page 5

Design of a Unix Process Per Proc. Region Tables (Virtual Addresses)

Process A

Text

8K

Data

16 K

Stack

32 K

Regions b

c Kernel Text, Data, ...

(Kernel) a

Process B

Text

4K

Data

8K

Stack

32 K

e

d Adapted From: M.J. Bach, The Design of the UNIX Operating System, Figure 6.2

2 – Kernel

(Kernel) © Hannes Lubich, 2003–2005

Page 6

3

Process Modes and Memory Structures Execution Mode: User/Kernel User Mode (Regions)

Memory Structures Read Only MachineCode (Text)

Read/Write Static Process Data (Data)

Read/Write Dynamic Data (Stack)

Read Only Kernel Text

Read/Write Process Information in the Process Table

Read/Write Process private data (can only be manipulated by the kernel) within the u_area

Kernel Read/Write Kernel Stack Dynamic Data

… Symbol Table

Execution Mode: • Selects the right set of registers, pointing either to kernel or user regions • Kernel mode: execution of privileged instructions, access to kernel memory regions • User mode: execution of user-supplied instructions, access to process memory only • Switch of execution mode trough interrupt, or system call 2 – Kernel

© Hannes Lubich, 2003–2005

Page 7

Kernel Data Structures to Manage Processes Per Process Region Table Region Table u_area

Main Memory Process Table M.J. Bach, The Design of the UNIX Operating System, Figure 2.5

2 – Kernel

© Hannes Lubich, 2003–2005

Page 8

4

State Diagram of a Unix Process user running System Call or Interrupt

return

Interrupt, Interrupt Return

kernel running schedule process sleep asleep

ready to run

wakeup

Context Switch Possible 2 – Kernel

© Hannes Lubich, 2003–2005

Page 9

Activating/Suspending Processes Time

Proc A

Proc B

Buffer locked, Sleeps : Buffer locked, Sleeps : :

Proc C

Buffer locked, Sleeps

Buffer is unlocked, Wake up all sleeping processes Ready to run : : : : : Runs, Buffer locked, Sleeps : : : : Ready to run : : Runs 2 – Kernel

Ready to run

Ready to run : Runs, Buffer unlocked : Lock buffer : : : Sleeps for some reason : : : : : : Runs, Buffer : locked, Sleeps Wakes up, Unlocks : Buffer, Wake up all : sleeping processes Ready to run : : Context switch : : :

© Hannes Lubich, 2003–2005

Page 10

5

The Unix File System: Basic Concepts • • • • • • • •

Each physical file system is tree-structured and has a root Directories are the building blocks for the hierarchy Plain files are simple, byte streams Special files represent devices, communication end-points, processes, connections between file systems etc. Links allow references through the file system Navigation can be relative or absolute „.“ denotes the actual directory „..“ denotes the parent directory

2 – Kernel

© Hannes Lubich, 2003–2005

Page 11

The Unix File System Plain File Byte

Virtual File System with Root and and Mounted File Systems

1 2 3

...

n

/

Directories Directory

Plain File

/

bin

date

dev

etc

lib

tmp

usr

src

lib

sd1c

sd0c

cat console tty00

dev

Special File

Disk 0

lubich

cad

local

Disk 1

“mount “mount /dev/sd1c /dev/sd1c /lib” /lib”

sd0a From: R. Adamov, VL: UNIX-Betriebssystem und Werkzeuge, Univ. Zürich

2 – Kernel

© Hannes Lubich, 2003–2005

Page 12

6

File System Access Control Access rights are defined per file or directory: what

read

write

execute

owner

√

√

√

group

√

—

√

world

—

—

√

who

/usr/users/lubich/bin/test

lubich

staff

-rwxr-x--x

Rights are in octal notation, where read = 4, write = 2, execute = 1, e.g. “751”, the user command “chmod” also uses mnemonics such as “g+x” (man chmod). 2 – Kernel

© Hannes Lubich, 2003–2005

Page 13

Process and Kernel Data Structures to Manage File Access User File Descriptor Table

File Table

Inode Table

M.J. Bach, The Design of the UNIX Operating System, Figure 2.2

2 – Kernel

© Hannes Lubich, 2003–2005

Page 14

7

The Toolbox Approach of Unix Executable File

Standard Input stdin

Standard Output 0

Application Program

1

stdout

2 File Descriptors stderr Standard Error Output 2 – Kernel

© Hannes Lubich, 2003–2005

Page 15

Flexible I/O Redirection ls ls ls ls >> file file ls ls >! >! file file ls ls >> >> file file ls ls >&! >&! file file mail mail lubich lubich & errors errors prog1 prog1 > out_file out_file 2 – Kernel

© Hannes Lubich, 2003–2005

Page 16

8

Pipelining Between Processes ps ps -ax -ax || grep grep lubich lubich || wc wc -l -l

ps ps -ax -ax || grep grep lubich lubich || sort sort -u -u

Shell fork()

fork() fork()

stdout

wc -l sort -u

2 – Kernel

stdin

stdout

pipe()

grep lubich

stdin

stdout

stdin ps ax

pipe()

© Hannes Lubich, 2003–2005

Page 17

Pipelining Example ktik0{lubich}[lubich]34> niscat hosts | grep ktik 129.132.57.5 ktika 129.132.66.8 ktik6 129.132.66.12 ktik8 129.132.66.11 ktik7 129.132.57.6 ktikb 193.5.36.2 ktik8-is 129.132.66.5 ktik4 129.132.66.2 ktik1 129.132.66.1 ktik0 129.132.66.3 ktik2 129.132.1.155 ktik0-gw 129.132.66.4 ktik3 129.132.66.6 ktik5 ktik0{lubich}[lubich]35> niscat hosts | grep ktik | wc -l 17 ktik0{lubich}[lubich]36>

2 – Kernel

© Hannes Lubich, 2003–2005

Page 18

9

A Simple „copy“ Program #include char buffer[2048]; main (argc, argv) int argc; int fdold, fdnew;

char *argv[]; {

if (argc != 3) { printf ("need 2 arguments for copy program\n"); exit (1); } fdold = open (argv[1], O_RDONLY); /* open source file read only */ if (fdold == -1) { printf ("cannot open file %s\n", argv[1]); exit (1); } fdnew = creat (argv[2], 0666); /* create target file rw for all */ if (fdnew == -1) { printf ("cannot create file %s\n", argv[2]); exit (1); } copy (fdold, fdnew); exit (0); } M. J. Bach, The Design of the UNIX Operating System, Figure 1.3

copy (old, new) int count;

int old, new; {

while ((count = read (old, buffer, sizeof (buffer))) > 0) write (new, buffer, count); } 2 – Kernel

© Hannes Lubich, 2003–2005

Page 19

A File is a Byte Stream pegasus: cc copy.c pegasus: ls -l a.out -rwxr-xr-x 1 lubich 13129 pegasus: mv a.out copy pegasus: ls -l copy -rwxr-xr-x 1 lubich 13129 pegasus: copy copy.c t pegasus: ls -l copy.c t -rw-r--r-- 1 lubich 762 -rw-r--r-- 1 lubich 762 pegasus: copy copy t1 pegasus: ls -l copy t1 -rwxr-xr-x 1 lubich 13129 -rw-r--r-- 1 lubich 13129 pegasus: chmod +x t1 pegasus: ls -l t1 -rwxr-xr-x 1 lubich 13129 pegasus: t1 copy.c /dev/tty #include char buffer[2048]; int version = 1;

Dec 20 21:24 a.out Dec 20 21:24 copy Dec 20 21:21 copy.c Dec 20 21:24 t Dec 20 21:24 copy Dec 20 21:25 t1 Dec 20 21:25 t1

main (argc, argv) ...

2 – Kernel

© Hannes Lubich, 2003–2005

Page 20

10

A Directory is a Byte Stream pegasus: copy . t3 pegasus: od -c t3 0000000 \0 \0 # 0000020 \0 \f \0 0000040 c o p 0000060 c o p 0000100 d i r 0000120 s e c 0000140 \0 \f \0 0000160 t 1 \0 0000200 \0 \0 , 0000214 pegasus: od -c . 0000000 \0 \0 # 0000020 \0 \f \0 0000040 c o p 0000060 c o p 0000100 d i r 0000120 s e c 0000140 \0 \f \0 0000160 t 1 \0 0000200 \0 \0 , 0000214 pegasus: ls -l . drwxrwxrwt 2 bin pegasus: ls -l t3 -rw-r--r-- 1 lubich 2 – Kernel

366 002 y y e u 001 u `

\0 \f . . \0 023 . c n t r i t \0 \0 \0 \0 \f

\0 \0 @ \0 \0 t c 6 \0

001 005 200 200 200 y u \t 003

. \0 \0 \0 \0 \0 \0 \0 u

\0 \0 \0 \0 \0 H \0 \f e

\0 6 5 , , \0 6 \0 b

005 006 373 ( U = \b 002 \0

\0 \0 \0 \0 \0 \0 \0 t

\0 020 020 020 024 \0 \f 3

# \0 \0 \0 \0 6 \0 \0

350 004 006 006 \b 007 002 u

366 002 y y e u 001 u `

\0 \f . . \0 023 . c n t r i t \0 \0 \0 \0 \f

\0 \0 @ \0 \0 t c 6 \0

001 005 200 200 200 y u \t 003

. \0 \0 \0 \0 \0 \0 \0 u

\0 \0 \0 \0 \0 H \0 \f e

\0 6 5 , , \0 6 \0 b

005 006 373 ( U = \b 002 \0

\0 \0 \0 \0 \0 \0 \0 t

\0 020 020 020 024 \0 \f 3

# \0 \0 \0 \0 6 \0 \0

350 004 006 006 \b 007 002 u

1024 Dec 20 21:26 . 1024 Dec 20 21:26 t3 © Hannes Lubich, 2003–2005

Page 21

Special Files are Byte Streams pegasus: copy /dev/tty t5 sdfsdfs sdfsfsf sdfsfsf sdfsfsdf pegasus: cat t5 sdfsdfs sdfsfsf sdfsfsf sdfsfsdf pegasus: copy /dev/tty /dev/tty sdfsfsdf sdfsfsdf sdfsfsdf sdfsfsdf sdfsfsdf sdfsfsdf sdfsfsdf sdfsfsdf sdfsf sdfsdf sdfsdf sdfsf sdfsdf sdfsdf pegasus:

What Whatwould wouldhappen happenininthese thesecases? cases? copy copy /dev/tty /dev/tty /dev/disk1 /dev/disk1 copy copy /dev/disk1 /dev/disk1 /dev/disk2 /dev/disk2 copy copy .. /dev/memory /dev/memory

2 – Kernel

© Hannes Lubich, 2003–2005

Page 22

11

A Simple Shell for the „copy“ Program main (argc, argv) int argc; char *argv[]; { if (argc != 3) { printf ("need 2 arguments for copy program\n"); exit (1); } if (fork()== 0) { printf ("starting copy\n"); execl ("copy", "copy", argv[1], argv[2], (char *) 0); } printf ("waiting for copy to finish\n"); printf ("still waiting\n"); wait ((int *) 0); printf ("copy done\n"); }

M. J. Bach, The Design of the UNIX Operating System, Figure 1.4

2 – Kernel

© Hannes Lubich, 2003–2005

Page 23

Using the Simple Shell pegasus: ls copy spawn.c t1 t5 copy.c t t3 pegasus: more spawn.c main (argc, argv) int argc; char *argv[]; { if (argc != 3) ... pegasus: cc spawn.c -o spawn pegasus: ls -l spawn* -rwxr-xr-x 1 lubich 13082 Dec 20 21:53 spawn -rw-r--r-- 1 lubich 395 Dec 20 21:52 spawn.c pegasus: spawn spawn.c t6 waiting for copy to finish starting copy still waiting copy done pegasus: 2 – Kernel

© Hannes Lubich, 2003–2005

Page 24

12

Summary: Kernel Design • The kernel is a layered structure, isolating user programs & data from kernel code and data. • Unix processes are related to each other, they can be reused (overlay) or freshly created. • Each Unix process logically consists of a user mode and a kernel mode – transitions happen through interrupts or system calls / returns. • The file system is based on a logical tree structure, constructed from physical file systems. It handles all files as unstructured byte streams. • The file system accommodates plain files, as well as file representations of other objects (e.g. devices, processes). • Programs can use standard I/O mechanisms to allow interprocess communication. 2 – Kernel

© Hannes Lubich, 2003–2005

Page 25

13