Design and Implementation of a Kernel Resource Protector for Robustness of Linux Module Programming Jongmoo Choi
Seungjae Baek
Sung Y. Shin
Division of Information and CS Dankook University Seoul, Korea
Division of Information and CS Dankook University Seoul, Korea
Department of EE and CS South Dakota State University South Dakota, USA
[email protected]
[email protected]
[email protected]
referred as supervisor mode or kernel mode). In a monolithic kernel, any modification of kernel, such as adding a new device driver or an enhanced file system function, requires not only all the related data structures and routines of kernel to be recompiled and relinked, but also the system to be rebooted. These troublesome jobs make it difficult to upgrade and evolve the kernel.
ABSTRACT Loadable kernel modules supported by Linux provides lots of benefits such as a small-sized kernel, on-demand loading, and easy software upgrading. However, since modules are executed in a privileged mode, trivial misuses in a module may cause critical system halts or deadlock situations. This paper presents a kernel resource protector which prevents kernel from faults generated by modules. The protector models the system in two objects: module object and resource object. By observing the interrelations between the two objects, the protector can detect misuses of modules and take actions to resolve the erroneous situations. Implementation study has shown that the protector can find out memory leaks wasted by modules and can reclaim leaks without degrading system performance. The proposed protector makes Linux more robust, which is required indispensably in the system equipped with NVRAM (Non Volatile RAM) such as FRAM and PRAM.
To overcome these difficulties, Linux supports loadable kernel modules [1]. Linux is organized as a collection of modules which can be loaded and linked into the kernel even while the kernel is executing in memory. Also, a module can be unlinked and removed from memory at any time. The dynamic linkage feature of modules gives lots of advantages, such as the ease of evolution, the efficient use of memory, and the small-sized kernel image. In essence, achieving many of the merits of micro-kernels without introducing performance penalties is the main goal of loadable kernel modules [2]. However, since modules are executed in a privileged mode, incautiously programmed modules could threaten the robustness of Linux. In a privileged mode, modules can execute any kinds of instructions and can access any address space, both kernel and user space. Hence, any trivial misuses in a module may bring about halting of the whole system. For example, an improper use of pointer variables in a module incurs the segmentation violation in a kernel mode, which may cause a system panic. For another example, if a module, which has already acquired a lock to access a resource exclusively, is unlinked from the kernel without releasing the lock, then any other process can not access the resource which may cause a deadlock situation. These days, as the employment of modules becomes popular, the possibility of these threats becomes more and more critical.
Categories and Subject Descriptors D.4.5 [Operating System]: Reliability—fault tolerant; D.2.5 [Software Engineering]: Test and Debugging—Error handling and Recovery; D.2.8 [Software Engineering]: Metrics— Performance measures
General Terms Robustness, Linux Kernel, Loadable Module, Fault Tolerance
Keywords Resource Protector, Slab Allocator, Memory Leak, Reclaim
1. INTRODUCTION
In this paper, we propose a kernel resource protector that can detect misuses of kernel resources possessed by modules and can recover the erroneous situations. In the protector, the system is modeled by two objects: one is a module object and the other is a resource object. When a module acquires a resource, a relation is established between the corresponding module object and the resource object. By observing the interrelations between module objects and resource objects, the protector can detect whether some misuses occur or not. Once misuses have been detected, the protector can resolve the situations by applying recovery mechanisms such as rollback and reclaim facilities.
Linux is based on a monolithic kernel architecture. It means all functionalities of the Linux operating system exist in a single kernel address space and are executed in a privileged mode (also This work was supported by the Korea Research Foundation Grant funded by Korea Government (MOEHRD, Basic Research Promotion Fund) (KRF-2005-214-D00148) Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. SAC’06, April, 23-27, 2006, Dijon, France. Copyright 2006 ACM 1-59593-108-2/06/0004…$5.00.
To verify the effectiveness of the proposed scheme, we implement the protector on Linux kernel 2.4.22. In this implementation, it is kernel heap space that the protector guards
1477
by other tasks. These misuses may lead the kernel to a deadlock situation. The protection of these misuses at the kernel level transparently is the main objective of this study.
against the misuses of modules. Modules can access kernel heap space by invoking the kmalloc() and kfree() interfaces. Since kernel heap space is managed by a memory manager called “Slab allocator” in Linux [3], we implement the protector within the Slab allocator layer. Experimental results show that the protector can detect memory leaks wasted by modules and can reclaim the space effectively.
We propose a kernel resource protector that detects the improper uses of kernel resources caused by modules and resolves these misuses. The protector manages two objects, one is a module object and the other is a resource object. A module object represents a module instance loaded into the kernel space, while a resource object represents a kernel resource guarded by the proposed protector.
The rest of the paper is organized as follows. In Section 2, we explain the kernel resource protector in details. Then, we describe the implementation of the proposed protector in Section 3. In Section 4, we show the experimental results. Finally, we conclude this paper with a summary and discussion of future work in Section 5.
The details of the object managements are as follows. When a module is loaded into the kernel, the corresponding module object is created. When a module reserves a kernel resource, a relation is established between the module object and the resource object. When a module releases a kernel resource, the protector deletes the established relation. Finally, when a module is unloaded, the module object is destroyed. The resource object is created statically per each kernel resource to be protected at the time the protector is initialized.
2. Kernel Resource Protector A module is an object file whose code can be linked to and unlinked from the kernel dynamically at runtime. Most of kernel components such as file systems, device drivers, executable domains, binary formats, and network layers, can be implemented as a module. Also, since a module can access any symbols exported by the kernel, many functionalities of operating system including system call wrapping and performance monitoring, can be implemented as a module.
By observing the interrelations between module objects and resource objects, the protector can detect whether some misuses occur or not. Assume that a module object has been already destroyed. However, if the protector still has some established relations between the module object and resource objects, it implies that some misuses have occurred by the corresponding module.
A module can be loaded either manually by using the Linux utility, called “insmod” or automatically by using the kernel module loader, called “kmod”. Also, it can be unloaded either manually by using “rmmod” or automatically. From a programming point of view, the loading and unloading of a module are manipulated by two interfaces, one is module_init() and the other is module_exit(). The module_init() interface is invoked at the time a module is loaded, while the module_exit() interface is invoked at the time a module is unloaded.
When a protector detects misuses, it solves the problems by applying recovery mechanisms. The recovery mechanisms are different depending on the types of resources. For the stateless resources such as kernel heap space and semaphores, the protector can simply reclaim the resources and recover them as available resources. For the stateful resources such as kernel internal tables and files, the protector employs the checkpoint and rollback facilities to go back to the original state.
The main role of the module_init() interface is reserving kernel resources for a module. For example, assume we make a character device drive as a module. To run the driver, it is necessary to reserve kernel resources such as an entry of the chrdevs table (character device switch table) to register the device name and the file operations structure of the driver [3]. The place where this reservation is done is in the module_init() interface (specifically, by invoking a kernel function named register_chrdev() in the module_init() interface). If a module supports an interrupt handler, reserving kernel resources such as an entry of the interrupt vector table and the irqaction data structure is conducted in the module_init() interface by invoking a kernel function named request_irq(). The reserved kernel resources are released in the module_exit() interface invoked when the module is unloaded.
3. Implementation We implement the kernel resource protector on Linux kernel 2.4.22. The main objective of this implementation is verifying the effectiveness of the proposed protector in real environments. Hence, in this implementation, the kernel resource guarded by the proposed protector is only one, that is kernel heap space. However, as we explained in the previous section, the protector can be extended to protect any kernel resources by creating resource objects and monitoring the accesses of modules to the kernel resources.
Also, a module could reserve some kernel resources dynamically while it is running. Acquiring a kernel semaphore for accessing a kernel resource exclusively or allocating memory from kernel heap space are typical examples of dynamic reservations. These resources are released dynamically when the module no longer needs them any more or when the module is unloaded. Now consider what happens in the kernel if a module does not release the reserved kernel resources when it is unloaded. If a module does not release an acquired semaphore, the related kernel resource can not be accessed by any other tasks even though the resource is available. If a module does not free an allocated kernel memory, the space becomes memory leaks that can not be utilized
Figure 1. Interface between Module and Slab allocator.
1478
The detected memory leaks are reclaimed by invoking the kfree() interface with the appropriate arguments recorded in the established relation. We implement the KRP_resolver() interface to be suspended by the task with a ready state so that it hardly interfaces the executions of normal tasks.
In Linux, kernel heap space is managed by a kernel component, called “Slab allocator” as shown in figure 1. When a module wants to allocate a piece of memory from kernel heap space, it invokes the kmalloc() interface provided by Slab allocator. When the allocated space is no longer needed in the module, it is freed by invoking the kfree() interface. Kernel heap space is splitted into caches, slabs, and objects where objects are the smallest unit of memory referenced by a returned kmalloc() pointer [2]. To increase performance, the objects of the same size are grouped into a general slab while the objects of the same type are grouped into a specific slab. Also, each slab is managed by cache objects with fast lookup capabilities. Hence, a module which repeatedly allocates objects of the same size or type can reuse a just freed object without paying much management overheads.
Table 1 summaries the files modified for implementing the kernel resource protector and their descriptions. You can find out the details of each file in [4]. Table 1. Lists of Modified Files in Linux Files include/linux/slab.h mm/slab.c
Figure 2 shows the mechanisms of our protector where KRP represents Kernel Resource Protector. When a module is loaded into the kernel, it creates a module object by invoking the
arch/i386/kernel/process.c kernel/module.c
KRP_register() interface (step ① in figure 2). When the module requests kernel heap space (step ③ in figure 2), the Slab allocator
arch/i386/kernel/ksyms.c
informs the KRP of this event (step ④ in figure 2). Then, the KRP establishes a relation between the module object and the kernel heap resource object. When the module frees the allocated
Descriptions Definitions of module objects and resource objects KRP_register(), KRP_unregister(), KRP_resolver() Invocation of KRP_resolver() in cpu_idle() Invocation of KRP_register() and KRP_unregister() in sys_init_module() and sys_delete_module(), respectively Exporting new kernel symbols for modules
4. Experiment Results In this section, we explain the experimental results. We first describe the results of the original Linux kernel. Then, we present the results of the modified Linux kernel and show the effectiveness of the proposed protector.
space (step ⑤ in figure 2), the Slab allocator notifies it to the KRP (step ⑥ in figure 2), which leads the established relation to be deleted. The unloading of a module is notified to KRP to destroy the created module object (step ② in figure 2).
(a) Before loading
Figure 2. Mechanism of Kernel Resource Protector.
(b) After loading
Figure 3. Original Kernel: Module loading.
The detection of kernel resource misuses is triggered by the
Figure 3 shows memory information of the system before and after a module is loaded to the original kernel. For the test, we make a module that allocates kernel heap space during its execution, but does not free the allocated space when it is unloaded. Although this scenario is one of extreme cases, we can find out from the survey of the existing modules that this is a feasible scenario if module programmers do not design their modules carefully and the kernel does not support any protection against these erroneous behaviors [5], [6]. Figure 3.a shows that the size of total memory space managed by the Slab allocator is
cpu_idle() function (step ⑦ in figure 2). The cpu_idle() is just an idle loop executed when there are no ready tasks to be scheduled. This choice makes it possible for the protector to detect misuses during the system idle time without degrading the overall system performance. When the cpu_idle() invokes the KRP_resolver() interface, it examines the list of created module objects and the established relations (step ⑧ in figure 2). If there exist any relations which are related with a destroyed module object, it implies that kernel memory leaks have occurred by the module.
1479
191,676KB and that of free space is 31,588KB. After the module is loaded, the size of free space is reduced into 6,880KB. Some of this space are used for the loaded area of the module itself, and others are used by the module internally when it invokes the kmalloc() interface.
(a) Before loading
(b) After loading
Figure 5. Modified Kernel: Module loading.
(a) Just after unloading (b) 20 Minutes after unloading Figure 4. Original Kernel: Module unloading. Figure 4 shows memory information of the system after the module is unloaded. As seen in the figure 4.a, the size of free space becomes 8,620KB. By comparing this value with the size of free space in figure 3.a and figure 3.b, we can find out that some portions used by the module are not freed (roughly 191,6768,620KB) while others are freed (roughly 8,620-6,880KB). This implies that memory leaks are actually occurred in the system. From figure 4.b, we can see that, after 20 minutes later, the size of free space remains almost same. It means that Linux kernel does not do anything about the memory leaks caused by the module. Recently, as the use of modules increases, this becomes critical deficiency of Linux system. Now, we have done the same test in the modified kernel incorporated with the proposed protected. Figure 5 shows memory information of the system before and after the module is loaded to the modified kernel. And, figure 6 shows memory information of the system after the module is unloaded.
(a) Just after unloading (b) 1.5 Minutes after unloading Figure 6. Modified Kernel: Module unloading. As we expect, the size of free space is not restored instantly to the initial state. However, in the modified kernel, the kernel resource protector detects memory leaks caused by the module. And when the system becomes idle, it tries to reclaim the memory leaks. The kernel log message of “GCONLINUX activated” shown in figure 6.b represents that the KRP_resolver() interface is invoked. As the results, the kernel regains the initial size of free space as shown in figure 7. If necessary, the KRP_resolver() can be invoked periodically or when the predefined number of modules is unloaded.
1480
common in real system and we expect that the swapping with call-back mechanism can be exploited usefully to handle this problem. Other directions of future research include applying the protector to various kinds of kernel resources and developing efficient recovery mechanisms for different types of kernel resources.
6. ACKNOWLEDGMENTS We would like to thank the anonymous reviews for their enlightening comments. Also, we would like to thank Prof. Jiman Hong who provided constructive feedback on this work.
7. REFERENCES [1] William Stalling, “Operating Systems: Internals and Design Principles”, 5th Edition, Pearson Prentice Hall, 2004. Figure 7. Modified Kernel: Memory Leak Reclaim.
[2] Daniel P. Bovet and Marco Cesati, “Understanding the Linux Kernel”, 2nd Edition, O’Reilly, 2002.
5. Conclusion
[3] Robert Love, “Linux Kernel Development”, Developer’s Library, 2003.
In this paper, we have designed and implemented the kernel resource protector for enhancing the robustness of Linux kernel. The proposed detector uses two objects: module objects and resource objects. By observing the interrelations between two objects, it can detect the misuses of kernel resources caused by loadable kernel modules and resolve them. Experimental results have shown that the detector can effectively reclaim memory leaks.
[4] Linux Cross Reference, http://lxr.linux.no. [5] Cal Erickson, “Memory Leak Detection in Embedded System”, http://www.linuxjournal.com [6] SecurityFocus, “Linux Kernel Netfilter Memory Leak Local DOS Vulnerability”, Error! Hyperlink reference not valid. [7] David Evans, “Static Detection of Dynamic Memory Errors”, PLDI 1996, http:// www.cs.virginia.edu/~evans/
Handling memory leaks is one of the topics studied extensively in academic and industrial arena [5], [6], [7], [8], [9]. Prevention of memory leaks is critical in the embedded system where available memory is very restricted. Also, the system equipped with nonvolatile memory such as FRAM and MRAM, robustness of the kernel is indispensable since the memory state is retained after the system is rebooted.
[8] Jim Patrick, “Handling memory leaks in JAVA programs”, http://www-106.ibm.com /developerworks/java/library/jleaks/ [9] Memory Leak Checker, http://www.thefreecountry.com/sourcecode/
In this paper, we have only considered the memory leaks caused by a module that has already been unloaded from the kernel. One direction of future research is to extend the protector to detect the memory leaks, which are allocated by the currently activated module but are not used any more. These live memory leaks are
1481
