IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 11, NO. 3, JUNE 2015
763
Virtualization as a Way to Distribute PC-Based Functionalities Piotr Gaj, Senior Member, IEEE, Mirosław Skrzewski, Jacek Stój, Member, IEEE, and Jarosław Flak
Abstract—Virtualization theory is well known and successfully used in the computer domain. Personal computer (PC) workstations, as well as their virtual counterparts, are popular for general purposes. PC stations are also popular in networked control systems (NCSs). They are used as system components to deliver user interfaces and to run many important services of the data processing, communication, and database type. In this paper, the usage of virtual PC machines (VMs) is considered in the context of interoperability with NCS. This specific application area requests answers whether virtualization is applicable and secure, and what are the expectations from the temporal characteristics of running services. Index Terms—Efficiency, hypervisor, industrial communication, industrial distributed systems, networked control systems (NCSs), OPC, security, temporal characteristics, virtualization, virtual machine (VM), Xen.
I. I NTRODUCTION
P
ERSONAL COMPUTER (PC)-based software, which operates within applications of distributed industrial systems, is designed to deliver many functionalities and services directly to its users. They could be humans as well as processes (programs). To ensure the interoperability of manufacturing systems, security and maintenance issues of such activities should be available for groups of users which have various competences and rights. Therefore, restrictions of the software accessibility are strongly recommended. Restriction of accessibility may be achieved by isolation of the software functionalities and their separation in the context of user rights. It is possible by distributing the functionalities among different processes with access control, running either on a single machine or on separate ones. Contemporary operating systems (OSs) allow to control access to functionalities and resources. However, if only one physical PC station is used, the common OS becomes a significant weakness. Although users with different roles can have separate access with different rights, they are still able to affect the whole OS by executing undesirable processes, by unintended actions, by accidental impact of mistakes in the access rights configuration, or by general OS vulnerabilities. Such “interferences” can badly affect services not designated to be operated by a given user. A better approach would be to distribute among independent physical devices critical groups of functionalities related
Manuscript received February 09, 2014; revised May 26, 2014; accepted September 11, 2014. Date of publication September 26, 2014; date of current version June 02, 2015. Paper no. TII-14-0179. The authors are with the Institute of Informatics, Silesian University of Technology, 44-100 Gliwice, Poland (e-mail:
[email protected]; miroslaw.
[email protected];
[email protected];
[email protected]). Digital Object Identifier 10.1109/TII.2014.2360499
to different industrial-oriented applications, such as communication, supervising (SCADA, MES), databases (DB). The communication applications have services of the network protocols, interprocess communication, data routing, and schema execution. SCADA has visualization, local data processing, online, and historic charts, while DB has data storage, indexing, and queries. Unfortunately, when many physical PC stations are used for the distribution of functionalities, the issue of cost, power consumption, communication, and physical locations of computers and their infrastructure inevitably arises. Therefore, the best approach seems to be running many instances of the PC station on the same physical machine, which is possible by using virtualization. Virtualization technology provides techniques to run one or more virtual machines (VM) within a single physical host. The VMs are executed and managed by VM management software, usually named hypervisor, and run on the host machine. The VMs have their own OS with virtual resources that are mapped to the physical ones and controlled by a VM monitor (VMM). In this paper, virtualization is considered in relation to nonRT PC workstation providing a set of functionalities mentioned above and cooperating with NCS. The NCS is considered a typical one as described in [1] and [2]. It is assumed as a local system with horizontal RT exchanges between RT nodes, vertical communication with a non-RT SCADA, and a DB. Our research subject is the virtualized PC station with Windows type OS and industrial-oriented software. The possibility of an easy distribution of processes, NCS system functionalities, and management seems to be a good reason for considering PC virtualization as a useful technology while designing and developing NCS. It is important to mention, however, that the processes execution and its management are also not fully separated in this case, because the hardware (VMM host) is common. There are two issues to check. The first one is the question of potential security improvement. Since in industrial applications the operation in a timely manner is a crucial factor, the second issue is the temporal characteristics analysis of data processing. In this paper, both issues are discussed. The security analysis, included in the further part of the paper, is based on the literature; whereas the analysis of time characteristics was based on some research work oriented on execution of communication processes within a structure of real NCS devices and one or more VMs, conducted for the purpose of this thesis. Thus, the research deals with a typical behavior of a distributed industrial system, where applications and services are run on the user and kernel level of the OS, and cooperate with time-determined elements like controllers and industrial networks. Despite the
1551-3203 © 2014 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
764
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 11, NO. 3, JUNE 2015
fact that the temporal characteristics are not limited with critical parameters (soft- rather than hard-RT), their aggravation has direct impact on the cooperation ability of PC-side software. The tested VMs are based on Windows XP and Windows 7 due to the popularity of such OSs. The used hypervisor is Xen. The motivation for using virtualization while designing NCS is explained as a conclusion of this paper. In the authors’ opinion, it is worth considering in real applications. II. R ELATED W ORK The contemporary issues related to the security of industrial systems are presented in [3] and [4] while the NCS issues in [2]. The basics of the virtualization theory are presented in [5]. A few interesting thoughts about the virtualization development and technological advantages are summarized in [6]. The memory management issues are considered and a new method is proposed in [7]. IO virtualization technologies are presented in [8]. Well-known hypervisors are, e.g., Xen, VMware, VirtualBox, Virtual PC, KVM, and others. The latest works referring to hypervisor abilities are discussed in [9]. Quantifying performance research and comparison are presented in [10] and [11]. The VM efficiency issues related to Xen scheduler improvements are considered in [12] and enhancements for RT operation in [13]. Most of the recent works related to security issues refer to the access control mechanisms within VMs and the hypervisor, and reflect typical (office) domains of VM usage. A good general view of the security concerns is presented in [14] and [15]. VMspecific threats are mentioned in [16] and [17]. There are also some attempts to consider fault-tolerant issues as described in [17] and [18], where synchronization of two VMs and switching between the main and the backup one are used in order to keep the PC station alive. Some architectural modification designed to enhance the security of distributed resources are discussed by Lei Ren et al. in [19]. The interoperability aspects of NCS elements and PCdistributed functionalities are rarely considered. Most of the related research in this matter is concerned with the usage of virtualization in industry as a single-machine run, especially, when the hypervisor is executed within the environment of an OS running on a physical machine [20]. The available research presents some comparison either in the form of benchmark tests based on throughput or CPU efficiency [10] or of basic, timeoriented features referring to the resource access [9]. There is no holistic view that refers to running services and to typical automation-oriented applications. The presented research, with Xen as an example, is another approach. Such a class of hypervisors introduces a special layer between hardware and the OS which is dedicated only to resources management and OS execution. Many benchmarks of Xen-based VMs, e.g., [10] and [21], show good, near physical machine performance of the virtual systems. These results encouraged the authors to evaluate the performance of VMs in tests of communication with real NCS systems and to compare the results with those obtained in physical PC systems. Hence, in this paper, the temporal characteristics of industrial application reactivity in virtual environment are analyzed, measured, and discussed.
Apparently, the temporal characteristics issue in such a case is clear. Intuitively, as well as based on the existing references, if one tries to run a few VMs instead of a single physical machine, then one should expect some deterioration of the temporal characteristics of data processing in such a structure. The network, concurrent execution of OS and application tasks, and concurrent access to physical resources produce delays. It is a similar problem to the system reactivity issue presented in [22]. However, in this paper, the subject is not an RT-oriented data flow—it is the reactivity comparison of all non-RT functionalities associated with the PC station, considering them from the NCS point of view, both in virtual and real cases. The formal analysis of measured delays is hard to make but it is possible, e.g., in the way presented in [23] or in the more sophisticated way in [24]. However, such analysis is out of scope in this paper. Another approach to temporal characteristics of such processes can be found in [25] and [26]. There are statistical considerations of overall computer system activity, including interactions between OS and user processes and interprocesses data flow as well.
III. S ECURITY I MPLICATIONS The functional safety is the first issue when considering control systems. This issue is well known in literature, and many references are available on this matter, e.g., [27]–[29]. There are, however, some other threats besides the safety ones. Due to the intense development of IT technologies introduced in both NCS and supervisory nodes, new type of threats arrived, which can influence, e.g., the security of the resource and services access. Undesirable access leads the way to industrial espionage, sabotage, and terrorism as well as to any intended but unrequested actions, and also to nuisances caused by unintended activities of operators [30], [31]. Availability of the new technologies at the PC-side level as well as at the PLC level produces brand new maintenance requirements of such system nodes, which have never been existed. It especially refers to functions of data transfer and storage in applications of SCADA and MES types. As a result of a hasty development, many implementations have bugs directly impacting the device security and additionally the system safety [32]. There is a lot of news in the media each year, e.g., [33]–[36] about such cases. What is even worse is that the new functionalities, such as web services, data access servers, database maintenance, multiservices communication, remote and public access, and many others, need to have a highly qualified staff to maintain them. If one considers a SCADA operator as the right person to maintain all IT services associated with a supervisory station, then one should invest in thorough training or divide administrative duties between different employees. Thus, the threats should be pondered in a much wider context than only that of functional safety. To consider this, in recent years, the term of dependability has been commonly used to describe safety, security, and availability of an NCS [2], [27]. Following this term, threats can be classified as faults, errors, and failures. Generally, faults come from the run-time actions of system elements, users, and environment. They may produce errors in data processing, leading to failures
GAJ et al.: VIRTUALIZATION AS A WAY TO DISTRIBUTE PC-BASED FUNCTIONALITIES
of the system functionalities. The unavailability of required functionalities can impact the system dependability. If one considers functionalities belonging to a PC station, the safety level is not a subject of changes while migrating to the virtual environment. The used software and hardware are the same from the application point of view. All mechanisms increasing the safety level remain the same. The only new potential source of faults in this matter is the hypervisor. It can be, however, neglected due to maturity of the current solutions and its low vulnerability to faults. The dependability of the system depends also on the security level. The main advantage of using functionalities distribution within a virtualized environment is just reducing the risk of faults associated with the security of resources and services, following the idea of small trusted computing base (TCB) [37]. From this point, the distribution of key functionalities among different VMs entails separation of management duties and NCS functions among isolated run environments. VMs are logically independent and their dependence concerns only the physical platform of the host. Thanks to that, the functionalities are also independent when considering their run environment. A fault on the given OS does not directly impact the functions of the other. The benefits of this can be considered in case of both internal and external threats. Internal threats come from software malfunctions, hardware compatibility issues as well as from undesired local actions. External refers to remote activities and the external threats come from the software vulnerabilities to malicious attacks. The risk that the used software produces faults is the same both on the VM and on the real one, except for the hardware compatibility issues discussed later. The risk of hardware malfunctions refers to all started VMs and also to a single real machine. Generally, the occurrence of software and hardware faults does not depend on virtualization, but virtualization reduces the scale of the faults impact, e.g., the possibility of making snapshots of VM allows to keep the state of the whole PC station. In case of a failure, restoring the last good image is faster and simpler than running a new one. Moreover, the maintenance of the saved VMs is better than restoring the system from OS backups or disk images, mainly, due to the time needed for maintenance and independence of the host configuration. Using snapshots positively impacts the mean time to repair (MTTR) as well as the down time MDT of the given group of PC-side functionalities. The risk reduction factor (RRF) is increased due to reducing the risk of undesired actions. The risk of inappropriate influence of a user is reduced due to the separation of the OS. A VM introduces additional levels of access control related to security policy of the OS. Taking into consideration the external security threats, it is harder to break into the software protection mechanisms on the virtualized environment than on the real one [37], [38], especially, when considering remote access. Many of the existing bots are unable to attack the hypervisors. In this case, the RRF is also increased. Another security advantage is virtualization of hardware. The main task of the hypervisor is management of the physical resources access, ensuring isolation of environments and scheduling of threads. The hypervisor creates for VMs, an
765
Fig. 1. Organization of VM I/O data path of Xen.
illusion of availability of the same simulated types of resources, regardless of the physical platform hardware. To communicate with I/O devices, the Xen hypervisor uses drivers of the first, privileged VM (dom0), optimized to work with Xen. Meanwhile, for the subsequent virtual systems (domU), it provides drivers of virtual devices shared with dom0 (front-end/ back-end, e.g., block devices, network, MMU), communicating via shared memory blocks under the control of IO Memory Management Unit (IOMMU). Fig. 1 presents typical paths of Xen VMs data communication. The IOMMU is responsible for security of the VM operation and the speed of data processing. It is an area of many new ideas presented, e.g., in [9], [14], and [39]. The front-end part of the device drivers is built with QEMU (Quick EMUlator), which emulates various peripherals of computer systems, using binary translation or native code execution. Due to QEMU emulation, VMs use very limited set of available peripheral devices—single type of graphic card, network card, etc. Therefore, the hardware compatibility issues are reduced to minimum and the potential risk of the software faults caused by this issue is also reduced (increased RRF). Moreover, the configuration of VMs has little in common with the configuration of real hardware platform, which has a positive impact on the maintenance process. Finally, based on the above thoughts, using a virtualized environment in order to dispatch key functionalities of PC-side industrial-oriented software is desirable from dependability point of view. On the other hand, there is a question that if using the PC virtualization in NCS does not negatively impact the software responsiveness. The answer would be “yes” if the temporal characteristics of responsiveness in such structures are similar to the real PC. IV. T EMPORAL C HARACTERISTICS The temporal characteristics were collected by capturing the data exchange duration from an application running in a real NCS environment as described below. The capturing was performed for various configurations, i.e., for a system with a single physical PC machine and with one or more VMs with distributed functionalities that allowed comparing the system reactivity after virtualization was introduced to the system. The real-time implications are not considered. Certainly, the NCS is considered an RT system and it has some periodic tasks. It is reflected in distributions presented further. The tasks of
766
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 11, NO. 3, JUNE 2015
cyclic update are clearly observed and other periodic tasks as well. Unfortunately, there are a lot of aperiodic activities related to the OS, and the observed jitter of communication responsiveness is large. Observed tasks cannot be identified as well as analyzed in the scope of this paper and it is beyond its goal. A. Hardware Setup of the Host Platform All tests were run on a PC system with Intel Core 2 Quad CPU Q9550 at 2.83 GHz processor with hardware virtualization support (VT-x) enabled. Windows 7 and Windows XP (now deprecated but still in frequent use in industrial applications) were used as the OS of the physical machine and the VMs. The hypervisor was Xen (version 3.2.1) with Linux CentOS 5.10. The tests have been performed for both standard Windows drivers and special drivers designed for operation with Xen (paravirtual drivers—PVD) for the network adapter, RAID controller, SCSI, and PCI.
Fig. 2. Implemented measurement mechanism.
B. Test Bed: Single PLC System With No DB The test bed includes one PC station and one PLC controller. The communication between those two was based on Ethernet and OPC DA [40]–[42] in version 2.0. As the OPC-UA has become available and it is relatively mature, it is reasonable to use the UA version [43], mainly due to the fact that it eliminates many security threats [44]. Nonetheless, the OPC DA is still very popular for running systems. Moreover, the version can impact the given temporal characteristics but not the results of reactivity comparison. Additionally, the OPC communication standard is only an example. Any communication service based on processes executed by the OS could be used instead. Using the other standards can change only the absolute values of delays as well as the shape of their distribution, but the dependencies between real and virtual stations shall remain the same. It comes from the fact that in each case, the protocol stacks and all communication services are executed in the same OS and implemented by the same means. Their local management does not overcome the OS limitations. The real difference can be observed if the communication services are moved outside the OS. The PC-side program was based on SCADA software— Indusoft Web Studio IWS. It exchanged data with the PLC (GE Fanuc VersaMax) via an OPC Server (Top Server by Software Toolbox) and GE Ethernet communication protocol (SRTP). Just like the type of communication, the type of PLC does not impact the dependencies between real and virtual PC. The PC was used to build the real NCS RT system. The behavior of the PLC and SRTP is not the subject in question. The important thing is the vertical cooperation between RT applications at the NCS node and non-RT applications at the PC side. It has been decided to separate functionalities as follows. The OPC client was on the SCADA side, the OPC server was on the OS side, SCADA was connected to an SQL database by OLE DB, and PLC was connected via SRTP directly to the OPC server. The OPC update cycle was 10 ms. It allowed to measure the time needed for data transfer from the PLC to the PC station and
Fig. 3. Considered configurations. (a) Physical machine. (b) One virtual machine. (c) Separation of the functionalities on two virtual machines.
back with only some basic data processing on the PC side. The measurement was performed by the PLC side program, according to the typical “loop back” mechanism, which was realized as follows (see Fig. 2). 1) The PLC increments an integer value of the register R401 and stores the current time T1 locally. 2) The incremented value is read from the register by the PC via OPC, copied out to another register R402, and sent back to the PLC. 3) The PLC recognizes the new value of the R402 register, stores the current time T2, and increments the R401 register value again starting a new measurement cycle. The T2 − T1 subtraction result is stored in register R403—it is the time needed for the data exchange between the PLC and the PC station. 4) The PC station reads and records the values of the R403 in a log file, and stores it for later analysis. Several series of measurements were performed for three different configurations, i.e., with the SCADA application, and the OPC server running on a single machine (physical or virtual), and with a separation of those applications on two VMs as depicted in Fig. 3 with and without paravirtual drivers installed on the VMs. In every series of measurement, over 65 000 samples were taken. One sample is considered as one measurement of the transaction duration Tdur registered by the PLC in the R403 register. The research showed that the tested OS had no substantial impact on the results. Thus, the results presented below were obtained in the newer OS, Windows 7.
GAJ et al.: VIRTUALIZATION AS A WAY TO DISTRIBUTE PC-BASED FUNCTIONALITIES
767
Fig. 4. Transaction duration Tdur with SCADA system on the physical machine (10 000 representative samples with no virtualization).
Fig. 6. Transaction duration Tdur with SCADA station functionality separated on two virtual machines (10 000 representative samples).
Fig. 5. Histogram of the transaction duration Tdur with SCADA station functionality on the physical machine.
Fig. 7. Histogram of transaction duration Tdur with SCADA station functionality separated on two virtual machines.
C. Results: Single PLC System With No DB All samples of the transaction duration Tdur recorded during the experiment on the physical machine (with no virtualization) are presented in Figs. 4 and 5. It is clearly visible that the value of the Tdur value falls into two main groups concentrated between 60 and 70 ms (about one-third of the results) and between 110 and 130 ms (about two-third of the results). There were also some values from outside of these two groups, but their total count was less than 10% of all the samples count. The greatest recorded Tdur value was 138 ms. After tests on the physical machine, the same experiments were performed on the VMs. Interestingly, the results are quite similar for both considered cases—with and without separation of the functionalities on two VMs, i.e., cases (b) and (c) according to Fig. 3. In Figs. 6 and 7, the results for two VMs are shown. Comparing the histograms from Figs. 5 (the physical machine) and 7 (two virtual machines, 2VM), a considerable reduction of the Tdur may be noticed. Most of the samples for the 2VM case are between 50 and 70 ms (about 85% of the total results). This means that the same processes running on two separate VMs were executed with 40% greater efficiency than on the physical machine. This phenomenon was also observed by the authors during other research activities. There is also a significant similarity between the above histograms. On both histograms, the samples are concentrated in two groups—around the value of Tdur of 70 and 130 ms.
However, in the 2VM case, the 130-ms group included less than 2000 samples. The samples grouping may suggest that the duration of the cycle in which the processes taking part in the transaction operate is about 60–70 ms. The processes are the communication requests, local data processing, event logging, to name only the most important. What is worth noticing in the 2VM case is that the samples from the 130 ms group are concentrated in time. In Fig. 6, seven areas are visible with the transaction duration between 120 and 140 ms. In the authors’ opinion, the increased transaction duration is the result of some system task operations not associated with the measurement being performed. The presented research work is concerned with the behavior of the whole virtualized platform rather than a single process. Therefore, it is not important to identify such tasks and their contribution in the given measured value. The duration of data processing on the PC side needs to be considered as a composition of all activities of the PC-side structure. The presented comparison can answer how the usage of the VM is influencing the execution of PC-side functionalities. One more interesting feature may be revealed after zooming the y-axis of Fig. 7 in the bottom range as shown in Fig. 8. The distribution of the transaction duration values is characterized by strong variability. A kind of “bursts” are visible in the histogram every 10 ms. It is relevant to the OPC server operation cycle—it was defined that the read/write request may be sent to the PLC as frequent as once per 10 ms (shorter periods were not available in this case).
768
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 11, NO. 3, JUNE 2015
Fig. 8. Zoom in on the transaction duration Tdur histogram for the two virtual machines case. TABLE I AVERAGE T RANSACTION D URATION
Fig. 10. Histogram of transaction duration Tdur for a system with DB. (a) One PLC. (b) Four PLCs.
before, the communication with the PLCs was based on the SRTP protocol. E. Results: System With 4 PLCs and DB
Fig. 9. Modified configuration. (a) Physical machine with DB. (b) Three virtual machines with separation of the functionalities onto OPC, SCADA, and DB.
To make a final comparison of the results for all concerned architectures, in Table I, the average transaction duration is presented. On the VMs, PVDs were installed. Thus, the PVD cases do not apply to the physical machine. The installation of PVD had no significant influence on the results. The reason is that the network, hard disk, and other system devices were not intensively used during the experiment.
D. Test Bed: System With 4 PLCs and DB The test bed described above included only one PLC node. It refers to the simplest structure of NCS—single controller connected with some distributed I/O stations. However, the communication tasks on the PC machine may have much influence on the system reactivity. Therefore, during additional tests, the NCS was extended to four PLC nodes. Moreover, an SQL database server was started and the SCADA system was configured to store the exchange duration values to the DB. The modified test bed is presented in Fig. 9. During tests on VMs, the functionalities were distributed on three VMs. They included the OPC server, SQL server DB, SCADA with the OPC client, and OLE DB connection. As
The research results are presented in Fig. 10. They show significant improvement in the system reactivity after migration to virtualized environment and distribution of the functionalities onto three VMs. In both cases (test beds described in Sections V-A and V-C), the operation cycle time TPLC of the PLCs (sometimes described as sweep time) was kept as short as possible. Increasing the TPLC would of course increase the Tdur time but equally for both physical and VMs, while the measurement results would be more fuzzy (e.g., the cycles shown in Fig. 8 would not be visible). V. C ONCLUSION AND F UTURE W ORK Taking into consideration the presented results, it is all right to state that if one wants to virtualize a PC in order to increase security, separate functionalities and services, and use the same hardware platform to do so, then it is possible without temporal characteristics aggravation of communication between such a platform and other elements of NCS. Moreover, the OSindependent resource management, delivered with the hypervisor, is capable of scheduling access to hardware resources in a better way than the considered OS on the same hardware set. As a result, by virtualizing a PC station, it is possible not only to improve the security but also to improve the temporal characteristics of the PC-based software operation. It is a good enough motivation to use virtualization in practical applications. The question arises on what types of PC functionalities can impact the system reactivity on a negative way. Based on our
GAJ et al.: VIRTUALIZATION AS A WAY TO DISTRIBUTE PC-BASED FUNCTIONALITIES
research, accessing external devices simultaneously from many VMs produces increased delays. However, the delays impact all running VMs. The delay connected with a virtualized functionality accessing a HDD is bigger than the same on the real machine. But the influence of this activity on the other virtualized functionalities is lower than on the real PC. We do not expect the obtained characteristics to alter significantly even if more industrial-oriented processes are run. Certainly, the temporal characteristics of processes execution are expected to change in this case, but given enough resources, the comparison of the physical and virtual platforms brings similar results. Nevertheless, it would be interesting to investigate and prove such a statement. The temporal characteristics of data processing can have a direct impact on the user-oriented interfaces, data achieving, calculations, execution of database queries, and other key activities of industrial PC applications. Thus, it is the next step planned in our research on this matter. R EFERENCES [1] G. Pratl, D. Dietrich, G. P. Hancke, and W. T. Penzhorn, “A new model for autonomous, networked control systems,” IEEE Trans. Ind. Informat., vol. 3, no. 1, pp. 21–32, Feb. 2007. [2] P. Gaj, J. Jasperneite, and M. Felser, “Computer communication within industrial distributed environment—A survey,” IEEE Trans. Ind. Informat., vol. 9, no. 1, pp. 182–189, Feb. 2013. [3] M. Cheminod, L. Durante, and A. Valenzano, “Review of security issues in industrial networks,” IEEE Trans. Ind. Informat., vol. 9, no. 1, pp. 277–293, Feb. 2013. [4] A. Valenzano, “Industrial cybersecurity: Improving security through access control policy models,” IEEE Ind. Electron. Mag., vol. 8, no. 2, pp. 6–17, Jun. 2014. [5] M. Rosenblum and T. Garfinkel, “Virtual machine monitors: Current technology and future trends,” Computer, vol. 38, no. 5, pp. 39–47, May 2005. [6] G. Goth, “Virtualization: Old technology offers huge new potential,” IEEE Distrib. Syst. Online, vol. 8, no. 2, p. 3, Feb. 2007. [7] D. Le and H. Wang, “An effective memory optimization for virtual machine-based systems,” IEEE Trans. Parallel Distrib. Syst., vol. 22, no. 10, pp. 1705–1713, Oct. 2011. [8] N. Challa, “Hardware based I/O virtualization technologies for hypervisors, configurations and advantages—A study,” in Proc. IEEE Int. Conf. Cloud Comput. Emerg. Markets (CCEM), Oct. 2012, pp. 1–5. [9] J. R. Santos, Y. Turner, G. Janakiraman, and I. Pratt, “Bridging the gap between software and hardware techniques for I/O virtualization,” in Proc. USENIX Annu. Tech. Conf. Annu. Tech. Conf. Ser. (ATC’08), Berkeley, CA, USA, 2008, pp. 29–42 [Online]. Available: http://dl.acm.org/citation.cfm?id=1404014.1404017 [10] X. Xu, F. Zhou, J. Wan, and Y. Jiang, “Quantifying performance properties of virtual machine,” in Proc. Int. Symp. Inf. Sci. Eng. (ISISE’08), Dec. 2008, vol. 1, pp. 24–28. [11] L. Youseff, R. Wolski, B. Gorda, and C. Krintz, “Evaluating the performance impact of Xen on MPI and process execution for HPC systems,” in Proc. 1st Int. Workshop Virtualization Technol. Distrib. Comput. (VTDC’06), Nov. 2006, p. 1. [12] S. Xi, J. Wilson, C. Lu, and C. Gill, “RT-Xen: Towards real-time hypervisor scheduling in Xen,” in Proc. Int. Conf. Embedded Softw. (EMSOFT), Oct. 2011, pp. 39–48. [13] Y.-L. Chung and C.-Y. Tseng, “An enhanced CPU scheduler for Xen hypervisor to improve performance in virtualized environment,” in Ubiquitous Computing and Multimedia Applications, B.-H. Kang and J. Abawajy, Eds., 2012 [Online]. Available: http://onlinepresent.org/ proceedings/vol7_2012/10.pdf [14] S. Vaughan-Nichols, “Virtualization sparks security concerns,” Computer, vol. 41, no. 8, pp. 13–15, Aug. 2008. [15] L. Garber, “The challenges of securing the virtualized environment,” Computer, vol. 45, no. 1, pp. 17–20, Jan. 2012. [16] M. Price, “The paradox of security in virtual environments,” Computer, vol. 41, no. 11, pp. 22–28, Nov. 2008.
769
[17] C. Tan, Y. Xia, H. Chen, and B. Zang, “Tinychecker: Transparent protection of VMS against hypervisor failures with nested virtualization,” in Proc. IEEE/IFIP 42nd Int. Conf. Dependable Syst. Netw. Workshops (DSN-W), Jun. 2012, pp. 1–6. [18] J. Zhu, Z. Jiang, Z. Xiao, and X. Li, “Optimizing the performance of virtual machine synchronization for fault tolerance,” IEEE Trans. Comput., vol. 60, no. 12, pp. 1718–1729, Dec. 2011. [19] L. Ren, Y. Zhang, Y. Luo, and L. Zhang, “A virtualization approach for distributed resources security in network manufacturing,” in Proc. IEEE Int. Conf. Ind. Eng. Eng. Manage. (IEEM), Dec. 2010, pp. 1524–1528. [20] R. P. Goldberg, “Survey of virtual machine research,” Computer, vol. 7, no. 6, pp. 34–45, Jun. 1974. [21] B. Clark et al., “Xen and the art of repeated research,” in Proc. Annu. Conf. USENIX Annu. Tech. Conf. Ser. (ATEC’04), Berkeley, CA, USA, 2004, pp. 47–47 [Online]. Available: http://dl.acm.org/citation. cfm?id=1247415.1247462 [22] J. Greifeneder and G. Frey, “Reactivity analysis of different networked automation system architectures,” in Proc. IEEE Int. Conf. Emerg. Technol. Factory Autom. (ETFA’08), Sep. 2008, pp. 1031–1038. [23] J. Greifeneder and G. Frey, “Probabilistic delay time analysis in networked automation systems,” in Proc. 10th IEEE Conf. Emerg. Technol. Factory Autom. (ETFA’05), Sep. 2005, vol. 1, pp. 1065–1068. [24] A. Ghanaim, G. Borges, and G. Frey, “Estimating delays in networked control systems using colored petri nets and Markov chain models,” in Proc. IEEE Conf. Emerg. Technol. Fact. Autom. (ETFA’09), Sep. 2009, pp. 1–6. [25] S. Wideł, J. Flak, and P. Gaj, “Interpretation of dual peak time signal measured in network systems,” in Computer Networks, A. Kwiecie´n, P. Gaj, and P. Stera, Eds. Berlin, Germany: Springer, 2010, vol. 79, pp. 141–152 [Online]. Available: http://dx.doi.org/10.1007/978-3-642-13861-4_14 [26] S. Wideł, J. Flak, and P. Gaj, “Time domain measurement representation in computer system diagnostics and performance analysis,” e-Informat. Software Eng. J., vol. 7, pp. 53–64, 2013 [Online]. Available: http://www.e-informatyka.pl/attach/e-Informatica_-_Volume_ 7/eInformatica2013Art6.pdf [27] G. Buja and R. Menis, “Dependability and functional safety: Applications in industrial electronics systems,” IEEE Ind. Electron. Mag., vol. 6, no. 3, pp. 4–12, Sep. 2012. [28] L. Wang, “Get real: Real time software design for safety-and missioncritical systems with high dependability,” IEEE Ind. Electron. Mag., vol. 2, no. 1, pp. 31–40, Mar. 2008. [29] D. Dzung, M. Naedele, T. von Hoff, and M. Crevatin, “Security for industrial communication systems,” Proc. IEEE, vol. 93, no. 6, pp. 1152–1177, Jun. 2005. [30] K. Rauscher, “Writing the rules of cyberwar,” IEEE Spectr., vol. 50, no. 12, pp. 30–32, Dec. 2013. [31] D. Kushner, “The real story of stuxnet,” IEEE Spectr., vol. 50, no. 3, pp. 48–53, Mar. 2013. [32] R. Parks and E. Rogers, “Vulnerability assessment for critical infrastructure control systems,” IEEE Sec. Privacy, vol. 6, no. 6, pp. 37–43, Nov. 2008. [33] S. Aliya, “Hackers manipulated railway computers,” NextGov Newsletter, Jan. 2012 [Online]. Available: http://www.nextgov.com/cybersecurity/ 2012/01/hackers-manipulated-railway-computers-tsa-memo-says/50498/ [34] N. Ellen, “Water-pump failure in illinois wasn’t cyberattack after all,” The Washington Post, Nov. 25, 2011 [Online]. Available: http:// www.washingtonpost.com/world/national-security/water-pump-failure-inillinois-wasnt-cyberattack-after-all/2011/11/25/gIQACgTewN_story.html [35] H. Hal, “Hackers accessed city infrastructure via SCADA–FBI,” Information Age, Nov. 2011 [Online]. Available: http://www.information- age. com/ technology/security/1676243 /hackers-accessed-city-infrastructurevia-scada-%E2%80%93-fbi [36] I. Johnsrud, B. Haugan, G. K. Hegvik, L. M. Glomnes, and O. LarsenVonstett, “Kunne stoppet vanntilforselen med mobilen,” VG, Sep. 2011 [Online]. Available: http://www.vg.no/nyheter/innenriks/terrortrusselmot-norge/kunne-stoppet-vanntilfoerselen-med-mobilen/a/10098352 [37] C. Li, A. Raghunathan, and N. Jha, “A trusted virtual machine in an untrusted management environment,” IEEE Trans. Serv. Comput., vol. 5, no. 4, pp. 472–483, Nov. 2012. [38] K. Fraser, S. H. R. Neugebauer, I. Pratt, A. Warfield, and M. Williamson, “Safe hardware access with the xen virtual machine monitor,” in Proc. 1st Workshop Oper. Syst. Archit. Support Demand IT InfraStruct. (OASIS), 2004, 10 p. [39] N. Amit, M. Ben-Yehuda, D. Tsafrir, and A. Schuster, “Viommu: Efficient iommu emulation,” in Proc. USENIX Conf. USENIX Annu. Tech. Conf. Ser. (USENIXATC’11), Berkeley, CA, USA, 2011, p. 6 [Online]. Available: http://dl.acm.org/citation.cfm?id=2002181.2002187
770
[40] T. Sauter and M. Lobashov, “How to access factory floor information using internet technologies and gateways,” IEEE Trans. Ind. Informat., vol. 7, no. 4, pp. 699–712, Nov. 2011. [41] M. Son and M.-J. Yi, “A study on OPC specifications: Perspective and challenges,” in Proc. Int. Forum Strat. Technol. (IFOST), Oct. 2010, pp. 193–197. [42] R. Cupek, M. Fojcik, and O. Sande, “Object oriented vertical communication in distributed industrial systems,” in Computer Networks, A. Kwiecie´n, P. Gaj, and P. Stera, Eds. Berlin, Germany: Springer, 2009, vol. 39, pp. 72–78 [Online]. Available: http://dx.doi.org/10.1007/978-3642-02671-3_8 [43] Y. Chuanying, L. He, and L. Zhihong, “Implementation of migrations from class OPC to OPC UA for data acquisition system,” in Proc. Int. Conf. Syst. Sci. Eng. (ICSSE), Jun. 2012, pp. 588–592. [44] H. Renjie, L. Feng, and P. Dongbo, “Research on OPC UA security,” in Proc. 5th IEEE Conf. Ind. Electron. Appl. (ICIEA), Jun. 2010, pp. 1439–1444.
Piotr Gaj (M’10–SM’13) received the Engineering and M.Sc. degrees in informatics in 1994, and the Ph.D degree in industrial informatics in 2004 from the Silesian University of Technology, Gliwice, Poland. He is currently an Adjunct with the Department of Automatic Control, Electronic, and Computer Science, Silesian University of Technology. He has held a few professional, teaching, and research positions. He has authored or coauthored several dozen papers in the area of industrial systems. He has served as a member and reviewer for scientific conferences and journals. His research interests include industrial informatics, including industrial computer networks and systems. Dr. Gaj is currently the Organizing Chair of the International Science Conference on Computer Networks.
Mirosław Skrzewski received the Engineering and M.Sc. degrees in automation in 1972, and the Ph.D. degree in informatics in 1981 from the Silesian University of Technology, Gliwice, Poland. He is currently an Adjunct with the Department of Automatic Control, Electronic, and Computer Science, Silesian University of Technology. He has held a number of professional, research, and teaching positions related to the construction of computer systems and computer networks. He has authored or coauthored several dozen papers. He has served as a member and reviewer for scientific conferences and journals. His research interests include computer networks, network and systems security, systems virtualization, and performance testing.
IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS, VOL. 11, NO. 3, JUNE 2015
Jacek Stój (M’12) received the Engineering, M.Sc., and Ph.D. degrees in industrial informatics from the Silesian University of Technology, Gliwice, Poland, in 2005 and 2009, respectively. He is currently an Adjunct with the Department of Automatic Control, Electronic, and Computer Science, Silesian University of Technology. He has authored or coauthored several dozen papers in the area of industrial informatics. He has served as a reviewer for scientific conferences and journals. His research interests include industrial systems and industrial computer networks. Dr. Stój is currently a member of the Organizing Committee of the International Science Conference on Computer Networks.
Jarosław Flak received the Engineering and M.Sc. degrees in informatics in 1994, and the Ph.D. degree in multimedia in 2005 from the Silesian University of Technology, Gliwice, Poland. He is currently an Adjunct with the Department of Automatic Control, Electronic, and Computer Science, Silesian University of Technology. He has held a few research and teaching positions. His research interests include computer networks, web servers, performance of computer systems using virtual machines, time analysis of computer systems, data compression, and visualization.
