Operating Systems

Operating Systems

“Operating Systems have two main functions. The first is to provide resource abstraction and the second is to manage resource sharing.”

Written by: Justin Browning 7 May, 2006

When we think of a computer operating system these days, Microsoft Windows comes to mind quite often. There are also other operating systems being used, the most common besides Windows being the Apple OS (OSX) and various ‘flavours’ of UNIX and Linux. We know computers seem to need special software to be able to operate, but what exactly is an ‘operating system’ and what does it do for us? What is an Operating System? An operating system (OS) is a very important piece of system software which manages the operation of application software (programs), hardware resources (storage devices, input devices, etc) of a computer and provides resource abstraction. Without an OS providing its users these functions, the computer may as well be just a tin box that lights up and blinks.

Figure 1 an Operating System is the ‘core’ of any computer system.1

The OS not only provides these ‘behind the scenes’ services such as resource management and hardware abstractions, but it also allows human users to interact with the computer2. A command line or graphical user interface helps the user utilize the abilities of the hardware without fully understanding the inner workings of the computer. The user may not know it, but whenever they use a computer they are manipulating the hardware in various ways with the help of the operating system. Each time they open a file from a folder, startup word and even perhaps saving that document back again, the user is utilizing an abstraction which the OS is providing to help them carry out these tasks.

1 2

http://www.webopedia.com/TERM/o/operating_system.html Operating Systems – 3rd Edition, Gary Nutt pg 4

So, I hear you saying what is abstraction? Well abstraction (or resource abstraction) is a function of the OS which simplifies the knowledge required by the user to be able to operate the computer. “System software hides (from the user) the details of how the underlying machinery operates”3. The user does not need to be fully aware of what specific hardware the machine is running or even how it works. Imagine if the only way to use a computer was to rewire circuits and swap memory chips just to move data around, that would be very difficult to use wouldn’t it. To understand resource abstraction a little better, we can liken it to driving a car. A car’s resources can be defined as the engine, transmission, tires, etc but as drivers, we do not need to fully understand how an engine works or how the transmission works for us to be able to move forward do we? All we have is a steering wheel, brake, clutch, acceleration pedals and gear shifter. With these ‘abstractions’ of the resources, we can manipulate the car to do exactly what we want, simplifying the process. An operating system has many levels of abstraction. Each level is responsible for managing certain classes of objects (files, devices, and so on) and providing services for the higher organizational levels, from which the details of these operations are hidden4. This is just like how our car is an abstraction, an OS being much more complex, has varying levels of abstraction which takes care of specific areas of the system. A big part of resource abstraction is the Application Programming Interface or API. The API is an integrated part of the OS which provides a base for software developers to write applications on one machine which will work consistently across many different machines regardless of the hardware setup, even if the amount of memory or the quantity of storage is different on the two machines5. So long as the computer is running an operating system, any software designed and written for that OS will work on any computer as the OS provides a ‘bridge’ for applications to access the system’s hardware resources. Speaking of hardware resources and applications utilizing these resources, this brings us to the next point, resource sharing. As I discussed briefly earlier, it is the job of the OS to manage the operation (or execution) of application software commonly known as programs. For programs to run on a computer, they must somehow be able to access resources to carry out a particular task. This is usually achieved by the program (or process) making a request to the OS for a specified hardware resource when say the program needs to read something from disk, or write back out to disk. The OS handles requests made by all processes in either of two ways, by space-multiplexed sharing or by time-multiplexed sharing. Multiprogramming as it is commonly called is a form of processing multiple programs in parallel using only the one central processing unit or CPU. “Since there is only one

3

Operating Systems – 3rd Edition, Gary Nutt pg 7 The New Encyclopedia Britannica – Volume 16, pg 634 5 http://computer.howstuffworks.com/operating-system2.htm 4

processor, there can be no true simultaneous execution of different programs. Instead, the operating system executes part of one program, then part of another, and so on”6. The OS can do this by finding which resources a program has requested, and sharing that resource amongst other processes by space-multiplexing or time-multiplexing that particular resource. The resource can be space multiplexed by segmenting it into multiple elements, then giving each process access to a specific piece, time-multiplexed by allocating the resource to a process for a given amount of time. Resource sharing is closely knit with the Operating system’s ability of multiprogramming because programs (or processes) require resources allocated to them for them to function. When multiple processes are running, the OS needs to be able to allocate different resources to different processes at different times or in different quantities, depending on the order of requests, process priority, how long a process requires a resource, etc. As you read on, you will better understand the importance of resource sharing and how it can affect a computer system. You will also become more familiar with the idea of resource abstraction and how it can help simplify some key areas of an operating system. The fundamental design of an operating system is very similar regardless of whether you’re looking at Microsoft Windows or UNIX/Linux. However, some of the finer details on how these two operating systems perform are very different in how each implement resource abstraction and resource sharing. Process Handling Previously discussed was the notion of a process and how it can make requests to the operating system for specific resources. The concepts of multiprogramming and resource sharing are very closely linked into how an operating system handles processes, but how does an OS deal with processes and how are these concepts related to the overall management of processes? Earlier I stated that a process can be described as a program in execution. This is true but more accurately, a process is made up of one or more threads (a process must contain at least one thread) which keep track of where the processor is in execution of the program. A process consists of the memory necessary to run the program (the process's address space)7 and a thread/s reside in the address space of the process, so one could say that a process acts as an execution container of threads; which help the operating system identify where to send requested resources for example, by use of the processes registered address space. With modern operating systems supporting multiprogramming and multitasking, an easier way to manage ‘x’ amounts of processes must exist, and it does by way of something called the process manager. The process manager is responsible for 6 7

http://searchvb.techtarget.com/sDefinition/0,,sid8_gci212615,00.html http://www.computerworld.com/softwaretopics/software/appdev/story/0,10801,66405,00.html

implementing the process, thread and resource abstractions using the underlying hardware8. More specifically, the process manager is in charge of the creation, termination, synchronization and resource allocation for processes.

Figure 2 Operating System Process Manager

The process manager has direct contact with the Operating System’s API to be able to provide resources to process and to have the ability to execute, pause or terminate a process. Hardware abstractions such as the fork() and createThread() system calls give the process manager these abilities mentioned. Different operating systems have different API’s hence different system calls, but they do have similar purposes. Sure the process manager has a big task of trying to manage many running processes at once and give them the resources they require, but how does the manager decide which process should get access to a resource, and for how long? “The process manager uses the state diagram to determine the type of service to provide to the process (process/thread)”.9 Processes can be in either one of three possible execution states, Ready, Running and Blocked. How the manager deals with each process and their states depend largely on CPU scheduling algorithms being used as the manager simply provides resources and controls the execution and termination of processes.

8 9

Operating Systems – 3rd Edition, Gary Nutt pg 205 Operating Systems – 3rd Edition, Gary Nutt pg 220

A process sitting in the Ready state is in a ‘war’ so to speak with all other Ready processes and are all fighting to be allocated to the CPU. Once a process has been allocated the CPU and switches into the Running state, it can do either of two things: continue and finish execution or request a resource such as an I/O or DMA operation. If the process continues through to the completion of execution, the process manager simply releases any resources in use by the process, and kills it freeing up space for other processes. If the process requires the use of a resource (depending on the CPU scheduling algorithm) and has enough time to finish work with that resource and finish its execution, it will remain in the Running state until it is terminated by the process manager, otherwise it will be moved to the Blocked state by the CPU and the manager will then move the next Ready process to the CPU while the Blocked process waits for the process manager to allocate it is requested resources.

Figure 3 Process States Diagram

This is a fundamental way an Operating System utilizes various aspects of its system such as the CPU and process manager in handling processes through various abstractions while managing resources. The process manager is a critical piece of any operating system (UNIX or Windows) and without it, execution of applications cannot happen. Memory Management Memory is an important part of any computer system. Memory, much like in humans can be classified into two groups; short term (i.e. Random access memory) or long term (i.e. hard disks). Without memory a computer system simply cannot function. So how does an Operating System handle memory and how is memory management related to the sharing of resources? Let’s find out shall we. As I’ve discussed earlier, and operating system has abstractions for the management of processes namely the process manager which helps out tremendously. It also has similar functionality for memory, more specifically the Memory Manager Unit or (MMU). It is the role of the MMU to allocate main system memory to running processes which is also closely tied in with the process manager. It can do this by using an effective memory management algorithm known as paging or memory paging.

The idea behind this algorithm is that it divides computer memory into small partitions, and allocates memory using a ‘page’ as the smallest building block10. One of the major benefits of paging is that it drastically limits the amount of fragmentation of system memory, which in turn means that less memory is wasted.

11

Figure 4The concept of memory paging.

Memory is addressed in two ways, either by a virtual address space or a physical address space. Both addresses are hexadecimal ‘hardware level’ addresses but refer to different things. Virtual memory which I will discuss shortly is addressed by virtual address spaces which can be linked to processes residing in system memory or in a physical address space. Figure 4 above illustrates a number of pages belonging to a particular process which reside in physical memory, and other pages which also belong to that process, but exist in virtual space or sitting in ‘virtual memory’. Virtual memory can be described as “An OS abstraction in which parts of a process address space are dynamically bound to primary memory addresses”12. Primary memory addresses in this case refer to physical (system memory) addresses. Paging and virtual memory work very closely together as virtual memory is almost always entirely implemented with memory paging. Most commonly, memory page sizes are around 4K with the lower bits of a page in a virtual memory address kept aside which 10

http://en.wikipedia.org/wiki/Paging http://en.wikipedia.org/wiki/Page_table 12 Operating Systems – 3rd Edition, Gary Nutt Glossary pg 871 11

directly correlate to the lower ordered bits in physical memory. The higher ordered bits in a virtual memory address are used as a key which uniquely identifies the memory address of the entire collection of physical addresses of information that are stored in this particular page. Memory management is a crucial aspect of the operating system as it helps increase the performance of executing processes/threads and also helps to keep the system clean by not over or under allocating memory to processes which require it. Input & Output How the operating system handles I/O is a very interesting area and hardware abstractions play a critical role in making life easier for both the OS (which provides them) and also for software developers who require the services of these abstractions when writing device drivers for example. It is important to understand that unlike process handling and memory management, how the OS handles input and output is a whole different story, yet some aspects remain similar. The input/output or I/O system of an operating system is just that, a system of devices working together to perform input & output operations. Devices such as a monitor, keyboard, mouse, video card, CD-ROM drives, are all examples of I/O devices. If all these devices are expected to work together, how does the operating system deal with input and output and how exactly can software ‘talk’ to hardware? Firstly, I/O devices can belong to either one of two groups. A device is either a block I/O device or a character I/O device. These devices are physically attached to the computer system by way of the system bus. The system bus is essentially a pipe or carrier of information which is used by hardware to ‘transport’ information from one device to another, whether it is from the CPU to system memory, an I/O device to CPU, etc. A block device reads and/or writes data in small ‘chunks’ along the system bus where character devices read and/or write only one byte (1 character = 1 byte) at a time. An example of how data is transferred along this bus will make things a little clearer.

Figure 5 I/O Data Transfer over the System Bus

An input device (i.e. keyboard) is attached to the computer via the bus. This device gives its input to the CPU which stores that information in a register. The CPU can store that registered data in system memory which it can then come back and collect at a later time,

read back into a register, then write out over the system bus for output on an output device such as a computer monitor. Input, Output and the handling of devices is made possible by the operating system though very important abstractions known as the device manager and device controllers. The device manager’s responsibility is to track and monitor hardware devices through communicating with each particular device’s controller. Imagine if it was the responsibility of application software to directly access hardware whenever it needed, we would soon find ourselves having to clean up and untangle one gigantic mess. If we were the operating system kernel, we would have no idea as to which devices were being used, for what purpose and if the system would ever get that device back. This is why these two abstractions are so very important to the operation of the OS. Direct Memory Access or DMA is another big part of input and output. As I discussed earlier, hardware uses the system bus to transfer data between the CPU which then can store that information in memory and/or write back through the system bus to an output device. How exactly does it do this? DMA is the answer.

Figure 6 Direct Memory Access

The DMA controller plays an important part in the transfer of data between the hardware and memory. Instead of the hardware having to access the CPU for it to store information from its registers into RAM, the DMA controller can essentially cut out the CPU from this process entirely, leaving it free to execute other commands. This means that through the DMA controller, hardware devices can directly access system memory, while not leaving the operating system vulnerable to the tangled mess I talked about earlier. File Management Files can be described as simply containers which hold data stored on a computer system. This data is stored typically on secondary memory devices such as hard disks and contain bytes of data stored in various memory addresses. A file can pull collect data stored in these addresses and organize it in a logical way, readable to human beings. The general concept of files is probably one of the most obvious and useful abstractions implemented in any modern operating system. Even early operating systems such as MS-DOS and first incarnations of UNIX had support for files.

A file can be an executable program, space for libraries to reside for applications (.dll – dynamic link library file) to use, or even word processing documents which hold character information. Files are a useful abstraction for both end users and programmers. Imagine if the only way to store information into a computer was to physically enter each byte of information into memory addresses, then somehow remember each address to even access that stored information. It is much simpler to create a file and let the OS handle it for us. The creation of files and saving information inside them is very closely related to input / output devices and the managers and abstractions associated with them. The organization of files within an operating system is one of the major variables which differ greatly depending on the OS. File systems are a data structure that translates the physical (sector) view of a disc into a logical (files, directories) structure, which helps both computers and users locate files13. This simply means that a file system is in charge of recording where files and directories reside on the physical disk, so both users and the OS itself doesn’t have to remember exactly which memory address data is stored in, in relation to a specific file. UNIX/Linux operating systems for example typically uses either EXT2/3 or Reiser file systems nowadays which organize their files into a hierarchical tree like format, whereas Windows products typically use FAT32/NTFS file systems, which still uses files and folders, but its design is slightly more complex, with drive letters being used to abstract the physical disk itself. Security Modern operating systems such as UNIX and Windows are designed with a high focus on security. When many of us think of operating system security, we typically think of user authentication, passwords, user names and file encryption, but if you think about it security measures are implemented in almost all aspects of the operating system design. UNIX and Windows are multitasking; multi-user operating system which allows many different people to share and access the same files on the system or computer network. With this ability to share files and resources for that matter, with it are inherent security problems which needed to be addressed. The idea of system policies and permissions was one of the first security measures implemented into an OS. Essentially, file permissions are a way to limit the use of particular files amongst users. In multi-user OS’s, users are typically segmented into groups (i.e. root(administrator), users, guests) and each group can be assigned a specific (pre-defined) security policy on accessing files.

13

http://www.cdmediaworld.com/hardware/cdrom/cd_cdr_glossary.shtml

14

Figure 7 Unix file permission notation.

Regardless of operating system, there are typically three different permissions one can assign to a user, or groups of users which are read (r), write (w) and execute (x). On a UNIX system, reading a files permissions is easy and looks something like: Mypage.html: -rw-rw-r--. Referring to figure 7 above, we can see that the file’s permission attributes are grouped into 3 sections of 3 characters (r-w-x). The first set of three refer to the read, write and execute permissions of a user, the 2nd 3 are permissions of an assigned group, and finally the last 3 refer to others not either in the user or group permissions list of that file. This is a very useful way of protecting files on the system from unauthorized use. Operating systems these days are geared heavily towards user authentication. When you sit down at a computer system, you will most likely be asked for a username and password to gain access to the system. This is the operating system’s primary method of restricting access to certain users. This aspect of protection is known as external or user authentication.15 Both UNIX and Windows implement user authentication and file permissions in slightly different ways, but it is important to understand that these security measures do exist and are an important part to the overall security of an operating system. Operating Systems have two main functions. The first is to provide resource abstraction and the second is to manage resource sharing. Aspects of the operating system such as process handling, file management, security, memory management and I/O all implement to some extent the ideas of resource abstraction and sharing in some way.

14 15

http://www.its.monash.edu.au/staff/web/slideshows/basicunix/slide7-0.html Operating Systems – 3rd Edition, Gary Nutt pg 576

Bibliography Nutt, Gary. 2004, Operating Systems, 3rd Edition, Pearson Education Inc. University of Chicago, The New Encyclopedia Britannica, 1991, Encyclopedia Britannica, USA. What is an Operating System? http://www.webopedia.com/TERM/o/operating_system.html

How Operating Systems Work http://computer.howstuffworks.com/operating-system2.htm

Multiprogramming http://searchvb.techtarget.com/sDefinition/0,,sid8_gci212615,00.html

Processes & Threads http://www.computerworld.com/softwaretopics/software/appdev/story/0,10801,66405,00.html

Paging http://en.wikipedia.org/wiki/Paging

Page Table http://en.wikipedia.org/wiki/Page_table

File Systems http://www.cdmediaworld.com/hardware/cdrom/cd_cdr_glossary.shtml

UNIX File Permissions http://www.its.monash.edu.au/staff/web/slideshows/basicunix/slide7-0.html