Science Highlights
Embedded Systems’ Virtualization: The Next Challenge?
Alexandra Aguiar, Fabiano Hessel
Faculty of Informatics – PUCRS
Traditionally, virtualization has been adopted by enterprise industry aiming to make better use of general purpose multi-core processors and its use in embedded systems (ES) seemed to be both a distant and unnecessary reality. However, with the rise of each more powerful multiprocessed ESs, virtualization brings an opportunity to use simultaneously several operating systems (OS) besides offering more secure systems and even an easier way to reuse legacy software. Although ESs have increasingly bigger computational power, they are still far more restricted than general purpose computers, especially in terms of area, memory and power consumption. Therefore, is it possible to use virtualization - a technique that typically demands robust systems - in powerful yet restricted current embedded systems? In this paper we show why the answer should be yes. Laboratory: GSE Publication: To appear in IEEE International Symposium on Rapid System Prototyping, 2010
Funding: FAPESP, CNPq, CAPES, Ministério da Ciência e Tencologia
Embedded Systems’ Virtualization: The Next Challenge? Alexandra Aguiar, Fabiano Hessel Faculty of Informatics – PUCRS – Av. Ipiranga 6681, Porto Alegre, Brazil
[email protected],
[email protected]
Abstract Traditionally, virtualization has been adopted by enterprise industry aiming to make better use of general purpose multi-core processors and its use in embedded systems (ES) seemed to be both a distant and unnecessary reality. However, with the rise of each more powerful multiprocessed ESs, virtualization brings an opportunity to use simultaneously several operating systems (OS) besides offering more secure systems and even an easier way to reuse legacy software. Although ESs have increasingly bigger computational power, they are still far more restricted than general purpose computers, especially in terms of area, memory and power consumption. Therefore, is it possible to use virtualization - a technique that typically demands robust systems - in powerful yet restricted current embedded systems? In this paper we show why the answer should be yes.
1. Introduction Virtualization of computing systems consists on creating a logical group of resources that seems to be the physical resources of a given computing environment [9]. It has been widely adopted in the enterprise market, especially to make better use of multi-core processors, along with several other advantages: • allows several operating systems to run on a single machine; • provides isolation from one virtual machine to another, improving security; • increases system flexibility; • enhances workload management, and; • brings hardware independence. On the other hand, virtualization can be considered as a high consuming technique, since it usually requires a lot
of disk space and RAM use, besides adding a non-trivial management layer: the hypervisor. The hypervisor, also known as Virtual Machine Monitor (VMM), must allow that instructions ran by the virtual machine (or virtual board) can be executed correctly on the real hardware. In enterprise servers, virtualization allows a single physical server to perform as multiple logical servers and provides the host of multiple instances of different Operating Systems (OS), like Windows, Linux and others. Often, these systems use multi-core processors from Intel and AMD, which is a very strong trend, as most vendors have presented projects for beyond four cores to the near future. This trend of using multiprocessed platforms is also present in Embedded Systems (ES), where Multiprocessors System-on-Chip (MPSoC) are being increasingly adopted and have become each more a viable choice to implement embedded systems [5]. Thus, these new multicore devices force a true change in the way embedded developers are designing their systems, since there are several key differences in the way that ES developers face the use of multicore processors and virtualization techniques [7]. While virtualization provides advantages as the capability of running multiple instances of operating systems on a single processor (mono- or multi-core), embedded systems are far different from enterprise systems [14]. This means that in order to take the advantages offered by virtualization, much effort must be spent in understanding how to better adapt them to embedded special needs: they are usually constrained in terms of power consumption, amount of memory, timing restrictions and area. Therefore, the use of previously known solutions optimized for enterprise Windows and Linux systems are not well-suited. In spite of it, virtualization techniques must be adapted to fit into embedded systems. But then, a question arises: is it worth? In this paper, we discuss the most important concepts involving virtualization and exploit the possibilities of using this technique in the embedded systems world. To do so, we detail the most suitable ways of employing virtualization in ESs aiming to achieve certain goals. Also, we present the main expected drawbacks and possible ways of avoiding
them. The paper continues as it follows. The next section shows the virtualization background to be adopted throughout this paper. Section 3 discusses the advantages of using virtualization in embedded systems followed by Section 4, that shows the drawbacks of it. In Section 5 we discuss how virtualization can be applied in ES and finally, Section 6 concludes the paper.
the memory use, since modern embedded devices are desired to be cost effective and memory, besides being a high energy consuming device, is frequently a cost factor issue [4]. Finally it is possible to say that embedded systems are becoming more and more part of everyday life, being increasingly used in mission and life-critical scenarios [15]. Although these systems are far more restricted that so-called modern embedded systems, virtualization could bring advantages, by increasing their safety, reliability and security.
2. Basic concepts In this section, we present basic concepts about both virtualization and embedded systems. We highlight the concepts needed to better understand the remainder of the paper.
2.1
2.2
Virtualization
As stated before, virtualization allows a single computer to host multiple virtual boards (or virtual machines), each isolated from one another, with the possibility of running different operating systems. The main advantage is that, if a virtual board fails, the other ones are kept safe at a reasonable cost [2]. In enterprise IT, although it causes a single point of failure, since many servers can be placed at a unique hardware machine, it is considered as a safer approach because most of service interrupts are not caused by hardware failures, instead, the main problem usually is the use of a software which is not reliable besides being big enough so maintenance is harder. This piece of software usually is the operating system. Therefore, virtualization uses a single piece of software, that operates in kernel mode: the hypervisor, which is at least two orders of magnitude smaller than general purpose OSs and less likely to have failures [12]. According to [9], there are two main approaches to implement the virtualization technique, by using hypervisor either type 1 or type 2. In hypervisor type 1, also known as hardware level virtualization, the hypervisor itself can be considered as an operating system, since it is the only piece of software that works in kernel mode, like depicted in the left side of Figure 1. Its main task is to manage multiple copies of the real hardware - the virtual boards - just like an OS manages multitask. On the other hand, in type 2 hypervisors, also known as operating system level virtualization, depicted in the right side of Figure 1, the hypervisor itself can be compared to another user application that simply “interprets” the guest machine ISS. In both cases, the virtual board must behave the exact same way as the real hardware. More specifically, it should be possible to start and restart them as real computers, as well as install different OSs. Thus, the creation of this scenario is up to the hypervisor. The hypervisor (independently from its type) and the real hardware are responsible for dealing with instructions com-
Embedded Systems
In the past few years, embedded systems used to be relatively simple devices with severe hardware constraints, like memory use, processing power and battery life. Their functionality was also mostly determined by hardware, with software consisting largely of device drivers, a task scheduler and some control logic, resulting in a low complexity of the software. Besides, these systems used to be closed, which means that during its lifetime no change in the software in required [1]. This scenario, however, is under change. Increasingly, modern non-critical embedded systems are bringing characteristics typical of general-purpose systems. The main change is their growing functionality which dramatically affects and increases the complexity of their software. It is also very common to run general purpose applications on embedded systems as well as to use applications written by developers that have little or no acknowledge at all about the embedded systems constraints [16]. In this case, more and more high level application oriented OSs with common APIs are required. Moreover, device owners also want to load their own applications on their systems by using common and rather open APIs. It is also important to point that the more open and common develop APIs get, the more security it is against attacks that were present, previously, only in general purpose systems. Nevertheless, some of traditional differences between general purpose and embedded systems still last [6]. Even on high-end multimedia entertainment-driven embedded systems, some real-time constraints remain. The energy consumption is always a matter of concern, since these devices are supposed to operate through several hours (up to days) without any battery recharge. This impacts on the processor frequency allowed to be used: usually lower frequency rates are mandatory in order to accomplish energy consumption goals. Another common restriction regards 2
Figure 1. Hypervisors Type 1 and Type 2 Considering the options to virtualize systems: either OS or hardware level, although at hardware level we may need some support from the processor, at OS level the virtual boards share both the hardware and the host’s operating system. Since one of the most promising advantages of using virtualization is to allow several operating systems in a single hardware, this type of virtualization will no longer be considered for our analysis. Thereby, let’s detail virtualization concerns when done at hardware level without hardware support (like VT or SVM). Initially, since the hypervisor is at charge of translating instructions whenever the virtual board attempts to execute a privileged instruction (I/O request, memory write etc), which causes a trap into the hypervisor. This is known as pure virtualization and is often a very expensive way of dealing with virtual machines [13]. Another other option at hardware level is known as impure virtualization and requires that sensitive instructions (those that require a trap into the hypervisor) are removed from the code executing in the virtual machine. This can be done either at compile time, by a technique called previrtualization, or by binary code rewriting, where the executable code is scanned in order to replace such instructions. The main issue in this approach is that both of them can cause performance loss. Alternatively, para-virtualization can be used as it replaces sensitive instructions of the original code by explicit hypervisor calls (hypercalls). In reality, the guest OS is acting like a normal user application running on a regular OS. Instead of it, is the guest OS running on the hypervisor.
ing from the virtual board. It is important to highlight that since a virtual board “copies” the real hardware, it also has separate execution modes: kernel and user mode. In order to derive the virtualization requirements, classic studies of Popek and Goldberg [8] introduce a classification of instructions of an ISA (Instruction Set Architecture) into three different groups: 1. privileged instructions: those that trap when used in user mode and do not trap if used in kernel mode; 2. control sensitive instructions: those that attempt to change the configuration of resources in the system, and; 3. behavior sensitive instructions: those whose behavior or result depends on the configuration of resources (the content of the relocation register or the processor’s mode). Therefore, those researches first declared that in order to virtualize a given machine, sensitive (control and behavior) instructions must be a subset of privileged instructions. This is not a reality in many processors, as Intel’s x86 family and the common solution in this case, is to adopt processor’s hardware support. To name, Intel’s support is named as VT (Virtualization technology) and AMD’s named as SVM (Secure Virtual Machine). Hardware’s support probably is not an option for embedded systems, since the desirable solution must deal with existing hardware, especially to accelerate time-to-market. 3
When para-virtualization is adopted, the hypervisor must define an interface composed by system calls allowed to be used by the guest OS. Still, it is possible to remove all sensitive instructions of the guest OS, forcing it to only use hypercalls which makes the hypervisor to become more like a microkernel. Besides working on hardware that is unsuitable for pure virtualization, it can also bring performance boost. The difference between pure virtualization and paravirtualization is depicted in Figure 2. In part A of the figure, pure virtualization is showed. In this case, whenever the guest OS calls a sensitive instruction, a trap is caused to the hypervisor, which emulates the instruction behavior and return the proper results. In part B, para-virtualization is showed. The guest OS has been modified in order to make hypercalls instead of containing sensitive instructions. In this case, the trap is similar to the one that occurs in non virtualized systems, whenever a user application makes a system call on its OS.
The first use case for virtualization on embedded systems consists of allowing several operating systems to be executed concurrently. Here, the use of different operating systems can be specially indicated in two different situations: 1. when legacy software must co-exist with current and incompatible applications, and; 2. when it is desired to part real-time software from user interface applications, by using different OSs. When applying virtualization with this goal, the hypervisor must have total control of the hardware besides creating different virtual boards, one per OS. As depicted in Figure 3, this approach can be used in both single and multi-core architectures. It can strongly increase software development quality, since it allows the designer to choose among several OSs that one which is the most suitable for its application or even the one with the best cost/performance ratio. Moreover, the time required to develop an application can be reduced, since in the case that an ES has the virtual support for a given OS, any former application can be reused, without the onus of rewriting it [11].
Figure 2. Hypervisor control of pure virtualization (part A) and para-virtualization (part B)
Therefore, by using the basic concepts presented here regarding to embedded systems and virtualization, we discuss the advantages, disadvantages and how (and whether) virtualization should be used in embedded systems.
Figure 3. Hypervisor for separation with several OSs
Furthermore, this approach offers advantages to achieve a unified software architecture that can be executed on multiple hardware platforms. In this case, a current issue in embedded systems - software portability - could be widely affected and developers would be able to faster satisfy the increasingly restricted time-to-market. Besides that, security is another important issue to be managed by using virtualization, since it provides an envi-
3. Why should I want to use virtualization in ESs? In this paper, we consider ESs as being multiprocessed non-critical yet timing constrained systems, and in this section we highlight possible virtualization use cases considering such systems. 4
ronment that protects and encapsulates embedded operating systems and other software components. Initially, the idea of using an application specific operating system apart from the RTOS is again encouraged as user attacks will only be able to cause damages at the user OS, thus keeping the RTOS and specific system components safe. This approach is depicted in Figure 4, where the scenario containing separate OSs and an ongoing user attack is shown. Nevertheless, in order to actually guarantee that virtualization will improve security, the underlying hypervisor has to be significantly more secure that the guest OS. The most suitable way of achieving it, is by keeping the hypervisor as small as possible.
separate OS, responsible for scheduling its own tasks. AMP is a configuration that takes advantages of virtualization since it provides arbitration of resources use between the virtual boards, avoiding the user OS to cause unexpected behavior on the RTOS [3]. If no virtualization is used, the only way to achieve such separation is by doing it manually, which is a complicated and more likely to error manner. The hypervisor can map every virtual board on each core of the multi-core processor or even map a single OS onto multiple cores, creating an SMP subset of cores [10]. When working with SMP configuration, workload balancing is also affected, since information regarding system’s tasks are kept in OS’s tables. By migrating the memory footprint, everything related to a given set of tasks is also migrated, thus helping in this complicated matter. To embedded enterprise world, the reduction of costs brought by virtualization along with the possibilities of easier updates are also promising advantages that should encourage its use. Summing up, we have the following main advantages by using embedded virtualization: • the use of different OSs; • increased security and reliability of the system, and; • different configuration alternatives in a multi-core environment.
4. Why should I avoid to use virtualization in ESs? Whereas embedded virtualization can bring indeed a great number of advantages, it is important to clearly understand at what cost they can be achieved. Some limitations are already present when using virtualization in general purpose computing, while others arise from its use in severely constrained environments, as embedded systems. The first issue to be dealt with, when bringing virtualization to embedded systems, is how the hypervisor schedules system tasks. This occurs due to the fact that each virtual board is scheduled by the hypervisor as black boxes, while the guest OS is responsible for scheduling the tasks within a given board. This behavior should be strongly avoided in embedded systems. Consider the case where a given multi-core presents asymmetric multiprocessing behavior, by having two different OSs: a user OS and an RTOS. In this case, each OS is treated like a separate virtual board and, in embedded systems, is clear that the RTOS should be prioritized over the user OS, as well as real-time tasks that eventually are executed on the user OS (like media applications) must also gain different priorities. This priority based scheduling goes against the principles of virtual machines, where all
Figure 4. User attack blocked by virtual boards’ isolation Consequently, the use of previously known hypervisors, as those employed in general purpose computing, will probably not be possible. In order to keep a small hypervisor, specific constraints from embedded systems must be taken into account, what reflects in its whole development cycle. In multi-core architectures, there are different ways of utilizing the many processors of the system. A very common way of doing it so, is to run a single OS onto the processors, thus creating a symmetric multiprocessing (SMP) configuration. This approach brings the main advantage of making load balancing across the processors straightforward. However, it does not allow the use of different OSs in the same MPSoC, which we discussed previously to be an attractive option, besides consisting of a single failure point and - whenever the system crashes - all the cores must be restarted. In this case, we can adopt an asymmetric multiprocessing (AMP) configuration, where each processor has its own 5
virtual boards are supposed to share the underlying hardware at equal proportions. Another challenge to ES virtualization comes from the inherit ES heterogeneity. While in general purpose computing, where Intel x86 architecture is widely used, in ES the architecture range is wide enough to go from DSPs, to ARM cores, passing through MIPS and PowerPC architectures. This is a struggle for the hypervisor implementation, which must support the processor architecture of the ES. Still, the excessive and absolute isolation brought by virtual machines - while increasing security and reliability levels - can also mean that the various embedded subsystems will struggle in other to achieve its strong cooperation, a behavior that is highly present in ESs.
5. When and how could I use virtualization in ESs?
Figure 6. Scenario 2: improved reliability
After considering the previously explained concepts, we briefly discuss some scenarios where embedded virtualization seems to be more suitable to be adopted. In Scenario 1, we aim to reduce the number of processors in a system by consolidating them onto virtual boards in a single processor (single or multicore). Figure 5 shows a configuration where this is possible. Even in case the processor offers hardware support to virtualization (like VT or SVM), it is desirable to adopt para-virtualization, which allows the hypervisor to be independent from hardware virtual support besides bringing performance boost.
In Scenario 3, it is possible to migrate existing systems into a virtual board and to add more functionality in new virtual boards, providing the opportunity for reuse and innovation. This is showed in Figure 7.
Figure 5. Scenario 1: processor number reduction Figure 7. Scenario 3: migration between virtual boards
In Scenario 2, we can increase the reliability of AMP systems by guaranteeing resource (memory, devices) separation and the ability to restart virtual boards, as depicted in Figure 6. Note that we can use it also when an AMP subset of processors is present in the MPSoC.
In Scenario 4, the combination of real-time, legacy and general purpose operating systems in the same device is achieved thanks to virtualization, as Figure 8 shows. 6
[2] G. Heiser. The role of virtualization in embedded systems. In IIES ’08: Proceedings of the 1st workshop on Isolation and integration in embedded systems, pages 11–16, New York, NY, USA, 2008. ACM. [3] M. Hermeling. The key differences between enterprise and embedded virtualisation. Web, Available at http://ecenews.stc-d.de /custom/enews220709.htm. Accessed at 10 feb., 2009. [4] M. Hohmuth, M. Peter, H. H¨artig, and J. S. Shapiro. Reducing tcb size by using untrusted components: small kernels versus virtual-machine monitors. In EW11: Proceedings of the 11th workshop on ACM SIGOPS European workshop, page 22, New York, NY, USA, 2004. ACM. [5] A. Jerraya, H. Tenhunen, and W. Wolf. Multiprocessor systems-on-chips. Computer, 38(Issue 7):36– 40, July 2005. [6] L. Lavagno and C. Passerone. Design of embedded systems. In R. Zurawski, editor, Embedded Systems Handbook, chapter 3. CRC press, 2005. [7] G. Martin. Overview of the mpsoc design challenge. In DAC ’06: Proceedings of the 43rd annual conference on Design automation, pages 274–279, New York, NY, USA, 2006. ACM Press. [8] G. J. Popek and R. P. Goldberg. Formal requirements for virtualizable third generation architectures. Commun. ACM, 17(7):412–421, 1974. [9] M. Rosenblum. The reincarnation of virtual machines. Queue, 2(5):34–40, 2004. [10] J. Stoess, C. Lang, and F. Bellosa. Energy management for hypervisor-based virtual machines. In ATC’07: 2007 USENIX Annual Technical Conference on Proceedings of the USENIX Annual Technical Conference, pages 1–14, Berkeley, CA, USA, 2007. USENIX Association. [11] S. Subar. Virtualisation to enable next billion devices. Web, Available at http://www.embeddeddesignindia.co.in/ ART 8800576093 2800003 TA 7cb7532e.HTM. Accessed at 10 feb., 2009. [12] A. S. Tanenbaum. Modern Operating Systems. Prentice Hall Press, Upper Saddle River, NJ, USA, 2007. [13] C. A. Waldspurger. Memory resource management in vmware esx server. SIGOPS Oper. Syst. Rev., 36(SI):181– 194, 2002. [14] W. Wolf. Computers as components: principles of embedded computing system design. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2001. [15] W. Wolf. A decade of hardware/software codesign. Computer, 36(4):38–43, April 2003. [16] Y. Zorian and E. Marinissen. System chip test - how will it impact your design. In DAC’2000 - Design Automation Conference, Las Vegas, EUA, Jun 2000. ACM Press.
Figure 8. Scenario 4: legacy software use along with new user applications
All this scenarios are suitable enough to be used in noncritical embedded systems, as multimedia mobile devices. Specially, as consumer demand continues to grow, stricter time-to-market tends to be more present and virtualization can enable their achievement.
6. Conclusions Virtualization has been usually adopted by enterprise IT to better enjoy the multi-core processors computing power. Meanwhile, embedded systems (ES) used to be extremely restricted and single purpose systems. This scenario is under change. ESs are increasingly becoming part of people’s lives and their multiple functionalities lead to an non-linear growth of software complexity. In this context, many solutions are being studied, like virtualization. The great appeals of embedded virtualization, mainly, are: (i) to allow several OSs (RTOS and user OSs) to run in the same processor; (ii) reduce cost of manufacturing, since it can increase processor utilization; (iii) improve security and reliability, and; (iv) helps decrease ES software development complexity. The next challenge, thereby, is to adapt the hypervisor approaches with microkernel implementations, in order to enable the development of light virtualization techniques, suitable enough for modern and future embedded systems.
References [1] H. Hansson, M. Nolin, and T. Nolte. Real-time in embedded systems. In R. Zurawski, editor, Embedded Systems Handbook, chapter 2. CRC press, 2005.
7
