followed by a brief introduction to Intel's new Hardware. Virtualization Technology and its benefits. We will conclude by looking at some server management and.
Applications of Virtualization for Server Management and Security Naresh K. Sehgal Staff Architect, Enterprise Software Technology, Intel Technology India Pvt. Ltd., Bangalore, India
Naresh.K.Sehgalgintel.com
Mrittika Ganguli Engineering Manager, Intel Server Management Software, Enterprise Software Technology, Intel Technology India Pvt. Ltd., Bangalore, India
Mrittika.Ganguligintel.com
Abstract-Servers are often under-utilized in their computing capacity by the applications hosted on them. Unpredictable demands due to peak hour usage, and a need for dynamic allocation of computing resources in a server farm, necessitate the use of virtual hosted clients. This paper will review the IT scenarios in Indian industry, and then look at some popular SW virtualization solutions, followed by a brief introduction to Intel's new Hardware Virtualization Technology and its benefits. We will conclude by looking at some server management and security applications enabled by Intel's new Virtualization
32%
Technology.
IT Services Application
Management
I.
IT application growth in 2006
29%
Software Deployment and Support Services
24%
23%
Network ISOutsourcing Consulting and Integration Services
INTRODUCTION
High performance and efficient servers are the backbone of any solid IT infrastructure. Server sales growth in any geographical area depends on its strong economic growth outlook. India, in particular has seen more than 42% growth in shipment of volume servers in the year 2005 in comparison to yea shipments in Indiaincreased the yearbeore [. Serve r excess oof 200/o on a year-to-year basis, for 10 quarters eendingg in Indiasusurgedd more more than on 3Qy5. "Server revenues Indta 28eon year
Figure 1: IDC prediction of IT Applications growth in India
the~~~~~~~~~~~~~~~ beoe[].evrsimntnIdaicesdi ta0yearters 3Q05."Server rnuayesr-toi se says IDC tDC research. arc. On m ar for for servers serer inin APEJ pEinin 2006" 20 says market
.
T
. .
past several years, the services sector comprsig of
the IT and Business Process Outsourcing (BPO) services clocked the highestgrr growth and fuelled the growth for server demand in India. In addition, different state governments have implemented e-governance initiatives increasing the need for highly efficient and cost effective server environments. IDC nee oncin ol ge nt rdc htHm one side large corporations like public sector banks, financial goes on to predict that Home Internet Connections would likely grow more than in addition to growth in the anfacurng companies omanes have av grown grwnman fld oealBodadsbcie services,serics, manufacturing many folds aeiin 2006 06oe 05[] over 2005 [1]. Broadband subscriber base ...............overall . . . . . in to the server sales. On the other small hand, adding growth These growth areas will drive the demand for servers in the and medium sized businesses (SMBs) have deployed wide telecommunications segment. scale ERP and Business process applications. Such SMBs are segment ths growth comes from growth in different Although most comes of the turnig more and more to the x-86 and x86-64 servers for their enterprise applications. IDC predicts that in 2006, the domestic ' IT market in India will grow by at least 19 over 2005[l]. part of the process of IT consolidation, in turn, paving the way Figure 1 displays the IT applications growth in 2006 across for Dynamic IT. Servers are often under-utilized in their various sectors indicating a near-even split [7. This indicates a computing capacity by the applications hosted on them. healthy trend, as IT industry's growth is not overly dependent Unpredictable demands due to peak hour usage and dynamic allocation of resources necessitate the use of virtual hosted singl revenue revenue source. source, and thus thusisunlikelytobe iS unlikelv to be ~~clients. on on anvanysinale In such an environment, the client PC workloads '. ....... affetedby poentil dwntrn n ay iniviualsecor. migrate to a server which hosts multiple clients. The Bombay Stock exchange took part in a server consolidation exercise in 2005 wherein they reduced the use of 14 inefficient servers to
1000%
1-4244-0726-5/06/$20.OO '2006 IEEE
2752
640%
jFZ~ ~ samerv1-W
2, thus easing their IT management costs and also creating a highly efficient example of Dynamic IT [2]. This kind of server consolidation will drive additional growth in the server market in 2006. The purpose of this paper is to discuss the advances in software and hardware virtualization and an attempt to show how two major application areas impacting the growth of server sales, server security and server management, will be enhanced by Intel's hardware virtualization implementation.
Server consolidation refers to the practice of taking OS and application stacks running on several physical servers, with each OS and its apps contained within a virtual server, and then multiple virtual servers running on a single physical server, as shown in Figure 3. This practice allows better utilization of hardware resources and thereby allow more isolated applications to run in a given server farm infrastructure. The downside is that if a consolidated server crashes for any reason then all virtual machines running on it will be affected.
II. SOFTWARE VIRTUALIZATION In its broadest sense, Virtualization is the ability of a computer to run multiple guests, or Virtual machines, such that each guest is unaware of others, and thinks that it has the whole computer systems to itself. This technology was invented about 5 decades ago for mainframes to be able to support multiple users, running different tasks, isolated from
Server Consolidation
Vmln
Vrhin
.....
[5], with software assistance.
Figure 3: Consolidation enables multiple VMs to run on the
Without Virtualization
0$
.
balancing refers to the ability of an IT manager to take ~~~~~~~~~~~~~~Load
-
_
~~~~~~~~another server without stopping it. This becomes possible a virtual machine is nothing but a large binary file in ~~~~~~~~~~~~because
11
the memory, with no direct references to the underlying hardware. It can be paused and copied to the memory Of another identical physical server, especially if the first servers needs to be brought down for regular maintenance or is down due to too many tasks. Vmware has a tool called ~~~~~~~~~~~~~slowing ...
L,1jm jrdwrel;lvl
Wit=h Virtualization
_
_
~
~
~~
~ ~~
~ ~ ~~ ~ ~ ~~.
..
................
Failover reliability refers to the ability of preserving the state
a virtual machine and replicating it on another physical ~~~~~~of server, memory or OS other techniques, in aagrtot tn IguestT 1nsuitable eac gues ~o° 1Susingancsnte re mirroring eun an
il111 l=ill | nresources~~ 1 ~~~~~~~~~~~~~ | w
PlafformHardware
case the first machine suffers failure. applications for virtualization technology include ~~~~~~~~~~~~~Other ~~saving the state ofan OSloaded with applications and data, for quality assurance testing, and then replicating it to ~~~~~~~~~~~~~~software
| IJ -
improve rapid deployment and developer productivity.
Figure 2: A comparison of non-virtualized vs. virtualization 11W-sW stak ti In this scheme, as shown in the right hand side of Figure 2, a layer of software called VMM (Virtual Machine Monitor) suport mutipe O intanes,isoate inmemry nd ard disk is partitioned. A view of virtual CPU and other hardware ris ted to eac O state is saved between context switching. This technology has several applications for servers, such as consolidation, load balancing and failover reliability.
gti
However, it should be noted that the VMM in above
reserved for OS with privileged calls and thus it pushes the OS
t r
i r
no longer able to make privileged kernel calls, and the VMM traps these to replace them with an equivalent code. This technique for Software Virtualization is generally known as
Binary Translation.
2753
III. HARDWARE VIRTUALIZATION Intel Corporation decided to add new CPU instructions to help with virtual machine transitions, and saving or restoring the VM state between such transitions. This creates a new ring for VMM, allowing OS to remain in its originally intended ringO, without any ring compression. Thus, Intel's Virtualization Technology [3] allows physical hardware level partitions between VMs, enabling robust solutions, legacy OS and drivers to run without any modifications. Vruial Macidnes (VA&)
Other future products include further enhancements and performance improvements, description of which is beyond the scope of this paper. However, the overall goal is to make Intel architecture servers be the best platform for any virtualization based application. Since hardware will carry some of the work currently done by software VMM, it will tend to make the future VMMs to be thinner and more efficient, as well as improve their performance in addition to reliability due to hardware level partitioning. Software industry has already responded by rapidly adopting Intel's Virtualization Technology into multiple offerings, currently available in alpha and beta forms, and within this year as products.
Ring 3
IV. SERVER MANAGEMENT AND SECURITY
Ring 0
VMl E-dt
M Enfty | VI
Figure 4: Intel's VT doesn't require OS ring compression As shown in Figure 4, two new operating modes are added: a less-privileged mode (VMX non-root) for guest OSs, and a more-privileged mode (VMX root) for VMM. Two new transitions are defined: VM entry to non-root operation, and
One compellmg application for virtualization is to allow for a partition that is not user accessible. This can be used by IT managers to have a server management application continuously run, gathering hardware monitoring data (for example status of processor, memory, RAID cards, etc) and monitoring the CPU utilization and status of other applications. At any point te ad minstrator can remote ly log-in, perfm1 any maintenance tasks such as driver updates or patch installations and view the system alerts and take appropriate corrective actions. Thus an IT manager can remotely monitor and control multiple servers from a single console, as shown in the right side of Figure 5. The challenge will be to enable the management application to monitor the different VMs running on each monitored server and diagnose and correct issues eliminating the need to have IT personnel reach remote areas.
Management Partition
VM exit to root operation. Execution controls determine when exits occur, and state of each VM is saved in a Virtual Machine Control Structure (VMCS), and as CPU switches between different VMs on pre-defined events, causing an exit of control
...........by...
I
a
VMCS, loading the state of anlother VM from its VMCS into corresponding VM. This switching allows all Virtual Machines 1l Frwl llc to get a fair share of comnpute resources and improves the _f Intel's future roadmap for virtualization technology also allows direct DMA assignment for dedicated I/O to or from a virtual VM, increasing efficiency of certain transactions while ensuring that DMA accessing from any virtual machine will stay within its allocated memory. This is one area where an errant driver or virus with a software only virtualization solution can compromise the memory of another virtual machine, but Intel's chipset technology in near future will remap each DMA request from a 0-address based virtual machine to a new location in the system memory. This is to avoid any conflicts between multiple DMA devices allocated to different virtual machines.
Figure 5: Proposed IT management partition for Intel based servers and clients During a secure install some signature value of the operating system (OS), drivers and key applications can be stored in an area of hardware normally not accessible to the user software or virus software. Such a signature may consist of CRC (Cyclic Redundancy Check), time and date stamp of the installation or even the size of the file. Normally a virus attacks by attaching itself to the executable of an OS or driver or an application file.
2754
Then during the boot time an agent in the protected partition can compare the previously stored signature with the present signature. That can be flagged as a new signature. Additionally, if all the internet traffic is also directed through the management VM, it is possible to do virus detection and prevention with filters before the affected packets can reach the user VM. Such a partition secures all other applications running in the user VMM. Of course this will have a tradeoff with raw performance but the assumption is that future processor, memory and network speeds will improve making such filtering to be almost unnoticeable. Virtualization may prove to be the killer application that can effectively utilize multiple CPU cores in a single package, by scheduling different Virtual machines to simultaneously run diverse workloads on a single server. However, such task consolidation can also potentially create a single point of failure, should that hardware server go down for any reasons. Thus, Intel Architecture Servers [4] that already offer the best RAS (Reliability, Availability and Serviceability) features in the industry are best suited to host multiple virtual machines with maximum up-time possible. Through appropriate monitoring software, an IT manager can also remotely move the virtual tasks to other physical
servers
ensuring that loads
are
the management application can be notified to the management VMM if done in-band or broadcast by the management firmware to all the VMMs if done out-of-band so that they can gracefully shut down. Future innovations in virtualizing resources other than the CPU, like IO and NIC, will make the management partition more resource effective. V. DISCUSSION
In summary, although Virtualization technology has been around for decades, Intel's latest innovations bring this capability to microprocessors for seamless support of Virtual machines across servers, desktops and mobile computers. It makes the existing usage models to run more efficiently and opens up avenues for new applications in remote manageability and security domains, thereby improving productivity and reducing costs. REFERENCES [
poised to become the second largest market for servers in APEJ in 2006: IDC, March 2006, http://idcindia.com/Press/7Mar2006.htm. [2] "Powering India Inc - Indian enterprises believe in putting world-class server infrastructure in place. This is evident from the examples of The Stock Exchange, Mumbai, Goodlass Nerolac Paints and Sundaram-
in a balanced and optimized manner across a group of servers. Clayton.", Network Magazine India, July 2005. Currently, RAS capabilities are implemented in Intel servers [3] "Intel Virtualization Technology Specifications for Intel based http://www.intel.com/technology/computing/vptech/. as Intelligent Platform Management Interface (IPMI) protocol [4] Platforms", "Better Server Virtualization with Intel Architecture", http://www.intel.com/business/bss/products/server/virtualization.htm [6]. The most uncommon but potentially dangerous example of IT failure will be when one of the virtualized OSs crash or not [5] "VMware and CPU Virtualization Technology" by Jack Lo, boot up. Currently, an IPMI based management application can I[6]http://download3.vmware.com/vmworld/2005/pac346.pdf "ntel Platform Management Interface", http://www.intel.com/design/servers/ipmi/. get the health of the server elements out-of-band (without OS interference) by connecting to the monitoring firmware port on [7] "India Top 10 ICT Market Predictions 2006", the server. It can get server health elements like processor, http://idcindia.com/pdf/IDC.201ndiao20Top%2010o201To20Market memory, and information on voltage, temperature, RAID, memory mirroring and sparing, hot plugged cards and system health event logs. An example of remote management is when *Third party names and brands may be claimed as the property of others. the server temperature is exceeding a threshold limit. A Copyright © 2006, Intel Corporation. All Rights Reserved. predetermined action can be taken like reducing the speed of the processor or shutdown a sub-set of the CPU cores (or a multiprocessing system). This will reduce the power consumption and reduce the temperature of the system gracefully. Another example would be if the system administrators need to find out how the RAID (Redundant Array of Independent Disks) storage is being allocated between different VMMs so they can predict usage and plan accordingly. The management application can monitor and report that back to the system administrator periodically and historical data could be gathered and analyzed to deduce usage calculation. Such a management application can even take the action of shutting down the server without the operating system or VMM's knowledge. It would be ideal if the management application can signal the management partition to do an OS update or a VMM update to restore the failed VMM or failed OS. Intel is conducting more research on enhancing this management capability so that remote power control actions by
2755
