IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
Seamless Integration of Distributed Real Time Monitoring and Control Applications Utilising Emerging Technologies V. Kapsalis, A. Kalogeras, K. Charatsis, G. Papadopoulos Industrial Systems Institute University Campus, Building A University of Patras Rion Patras 26500, Greece
[email protected] services, such as RPCs [2], MOMs [3] and ORBs [4], include critical communications, program-to-program and data management services. Application enabling services, including transaction monitors and database services, give applications access to distributed services and the underlying network. Finally, middleware management services enable the continuous monitoring of applications and system functions. Among the above, distributed system services present the adequate middleware for the industrial environment, and are further presented in the following paragraphs. Remote Procedure Calls (RPCs) present a client/server infrastructure that increases the interoperability, portability and flexibility of an application by allowing its distribution over heterogeneous platforms. RPCs represent special function calls embedded within the client portion of the client/server application program. RPCs increase the flexibility of an architecture by allowing a client to employ function calls to access a server on a remote system. Both synchronous, request-reply and asynchronous implementations are available. Yet, RPC is not well suited for applications involving distributed objects or object oriented programming. Tools are available for the implementation of RPC applications over a wide variety of platforms. RPC infrastructures are implemented within the Distributed Computing Environment (DCE) [5], that was developed by the Open Systems Foundation (OSF), and comprises a set of integrated system services that provide an interoperable and flexible distributed environment with a primary goal of solving interoperability problems in heterogeneous, networked environments. The DCE is portable and flexible, independent of networks and operating systems, while supporting the integration of client/server applications. The DCE comprises different services such as Remote Procedure Call, Directory services, Time service, Security service, Thread service and Data-sharing services like Distributed file system and Diskless support. It can be used on different network hardware and transport software like TCP/IP, OSI, X.25 and uses standard operating system service interfaces like POSIX and X/Open. Yet, DCE increases complexity with reference to thread synchronization and shared data across threads, while it is not object oriented, further resulting in increased complexity, decreased productivity and maintainability. Message-Oriented Middleware (MOM) is a client/server infrastructure that allows the distribution of an application over multiple heterogeneous platforms. It resides in both por-
Abstract- The seamless integration of distributed real time applications is mandatory for such demanding environments as the industrial environment or building management systems. Several technologies presenting different degrees of maturity have been developed to support this goal. This paper presents work done at the Industrial Systems Institute, which resulted in the definition of a system architecture utilising different emerging technologies, and its pilot implementation.
I. INTRODUCTION The ubiquity of Information Technology (IT) infrastructure, makes it possible to seamlessly integrate distributed real time applications across Local Area Networks (LANs), Wide Area Networks (WANs) and the Internet. Such an integration becomes more compelling when real time applications are associated with time critical control data, as in the cases of industrial or building management systems. Such systems are traditionally handled with reference to a four layer architectural model comprising the Field and Shop Floor layers for the integration of applications relevant to control data, and the Plant and Enterprise layers for the integration of higher layer enterprise applications. The provision of appropriate technologies for the integration of all these applications over different platforms ranging from fieldbus control networks to LANs/WANs to the Internet, is therefore mandatory. In the framework of this paper, the different mature and emerging technologies, which may be used, as well as their advantages and disadvantages are presented. Furthermore, a system architecture, resulting from the utilisation of some of the presented technologies, is proposed to allow the robust, efficient and interoperable integration of distributed applications. Finally, a pilot implementation of the proposed architecture is presented. II. TECHNOLOGY ASSESSMENT Middleware [1] is connectivity software that comprises a set of enabling services, which make possible the seamless integration of diverse distributed applications across different heterogeneous platforms. Depending on the application problem that must be solved and the needed functions, different types of middleware services are required. Distributed system
0-7803-7108-9/01/$10.00 (C)2001 IEEE
176
IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
interNet Architecture (DNA). Although complexities of building distributed applications are not eliminated, the advent of DCOM resulted in a low cost, easy to implement environment for the seamless integration of distributed components. Java is an object oriented programming language addressing many of the issues of software distribution over a network, including interoperability, portability, scalability and reliability. When embedded in web pages Java programs represents the so called Java applets providing executable content via peer classes. Java RMI enables the programmer to create distributed Java technology based applications, in which the methods of remote Java objects can be invoked from other Java Virtual Machines (JVM) that execute the Java bytecode. RMI uses object serialization, extending the technology core Java Input/Output classes with support for objects. Object serialization supports the encoding of objects and the objects reachable from them into a stream of bytes and the reconstruction of the object graph from the stream. The serialized form must be able to identify and verify the Java class from which the contents of the object were saved and to restore the contents to a new instance. The ubiquity of the Internet infrastructure and the clear advantages that Java presents for the development of Internet based applications, have given Java RMI a considerable advantage over the other technologies and have led to interworkability with both DCOM and CORBA. COM objects can be created and manipulated, for instance, from Java code. Tools are provided for the creation of Java classes from COM type library information. On the other hand CORBA vendors support the development of ORBlets, meaning Java applets capable of invoking services of remote CORBA objects. The work presented in the following chapters of the paper, is based on an architecture combining the utilisation of such DCOM based technologies as OPC and HDA with the JAVA RMI technology. A pilot implementation of the system is also presented.
tions of a client/server architecture and is predominantly asynchronous and peer-to-peer while supporting synchronous message passing as well. It is typically implemented as a proprietary product, which leads to vendor dependence and results to a highly negative impact on system’s flexibility, maintainability, portability and interoperability. An Object Request Broker (ORB) presents a middleware technology that manages communication and data exchange between objects. Object communication details are hidden and isolated in the ORB, thus enhancing system maintainability. Furthermore, interoperability of object systems is also promoted by enabling the use of objects from different vendors, providing the framework for cross-system communication between objects, and object communication across platforms. Implementation details such as programming language, operating system, host hardware and object location are thus hidden to the clients. Three major ORB technologies exist: the Common Object Request Broker Architecture (CORBA) [6], the Distributed Component Object Model (DCOM) [7] and the JAVA Remote Method Invocation (JAVA RMI) [8]. CORBA is a specification of a standard architecture for ORBs, developed by the Object Management Group (OMG). Client invocations of server object methods are intercepted by CORBA-compliant ORBs, which are responsible for finding the object on the same machine or across the network, passing parameters to it, invoking its methods and returning results of the invocation. CORBA actually presents a refined generalization of RPC. The different objects defined in CORBA use the Interface Definition Language (IDL), presenting an object-oriented interface definition formalism. Several services are provided by CORBA such as asynchronous event management, transactions, persistence, externalisation, concurrency, naming, relationships and lifecycle. CORBA presents a general distributed platform-independent system architecture, but is a rather complex specification, lax regarding implementation details, and not very easy to use. CORBA uses the CORBA defined inter-ORB interoperability protocol (IIOP) as a transport protocol. Furthermore, it does not address real time issues, while the use of IDL results in a limited exploitation of the capabilities of the programming language that it is mapped to. DCOM provides a framework for integrating networkbased components, allowing processes to be spread across a network. This framework supports interoperability and reusability of distributed objects, while promoting their maintainability and adaptability. Interoperability between different software components written in arbitrary languages is achieved through the definition of a binary structure representing the interface between clients and objects. DCOM objects and interfaces are specified using the Microsoft Interface Definition Language (IDL). Both DCOM and associated components are not platform independent, and are best supported on Microsoft platforms. DCOM in conjunction with OLE, ActiveX and MTS constitutes Microsoft’s distributed and web-oriented strategy, which is referred to as Distributed
0-7803-7108-9/01/$10.00 (C)2001 IEEE
III. THE PROPOSED SOLUTION The proposed system architecture focuses on time critical applications and leverages advanced robust and emerging technologies, such as DCOM and Java RMI, in order to provide seamless integration across the control network, the backbone data network and the Internet. The system can be considered as consisting of two interconnected and cooperative functional layers, partitioned on the basis of network infrastructure and technology. These layers are: - the Control network layer interconnecting diverse distributed control system devices by means of the LonWorks network protocol. This network protocol provides the communication infrastructure at the Field and Shop Floor layers, interconnecting field devices such as sensors and actuators and controlling devices such as Programmable Logical Controllers and Computers.
177
IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
- the Enterprise layer comprising the operating domain, which is used as the communication backbone, enabling the interDataBase
connection of several heterogeneous subsystems and the inte-
MMI (SCADA)
Web browser
Monitor/Control Historical Data Viewer & Configuration
ADO/OLEDB Layer
Historical Data Access (HDA Client/Server)
Java-based Client
HTTP
DCOM HTTP
DCOM
RMI
I Internet nternet
TCP/IP (RMI)
Ethernet (Backbone network) DCOM
LonWorks OPC Server
Web Server ASP Java-COM Bridge Java Server
LonWorks subsystems
Fig. 1: System Architecture
works are the Java applets and the ActiveX controls. Both of these technologies may be used, in order to develop clients and are capable of providing graphical representations of real-time data fed through TCP/IP networks. The combination of the OPC technology with ActiveX controls is usually an inherent functionality to SCADA systems allowing both the creation of an advanced run-time environment and their embedding in Web pages so that they may be viewed by ActiveX-enabled browsers (e.g. Internet Explorer). Although pioneering an approach, ActiveX is not supported inherently by all Web browsers (at least without installing a plug-in). In this context, our proposal is for a system that uses an advanced SCADA application, with OPC client functionality, for real-time monitor and control across the LAN. Furthermore and in order to employ such features as platformindependence, portability and safety, the Java runtime environment was selected. Java is an advantageous choice whenever the information has to reach a number of diverse thin clients (Web browsers) through the Internet. In this context, the selected Java-based client extended the monitor/control functions across the Internet infrastructure. Different Java applets, one for each different application were developed. Apart for displaying real-time data, in raw, tabular and graphical formats, these applets, provide adequate functionality for alarm notification and logging of alarm events (for off-line viewing). In order to provide COM access from Java objects, a Java-toCOM bridge was used [10]. The utilisation of DCOM technology layered over RPCs, and in turn over TCP/IP, results
gration of the control networks to the LAN/WAN infrastructure. The system architecture that depicts the proposed solution for the interconnection of the control network(s) with the LAN and the Internet is illustrated in Fig. 1. It is a robust, flexible and scalable solution, providing an efficient mixture of several proven technologies, in order to exploit them in the most efficient way. The Object Link and Embed (OLE) for Process Control (OPC) specification [9] is an emerging proven and robust technology for the interconnection of control networks with LAN/WAN, being, virtually, a universal I/O driver for each different control network/fieldbus. The OPC architecture utilizes the OLE/DCOM technologies for remote OPC client/server communications. DCOM built-in features ensure that OPC clients and servers have a robust and reliable mechanism to exchange real-time information across the network. DCOM also undertakes the handling of retries and timeouts between an OPC client and a remote OPC server and the re-establishment of communications in case of disruption. The selection of the OPC specification for the interconnection of control networks and LANs in our proposal is due to its universal support by the majority of the automation suppliers. The best exploitation of control data made available across the LAN through the OPC technology should be coupled by appropriate technologies for their availability on the Internet. The most prevalent technologies for embedding real-time monitor and control within Web browsers, over TCP/IP net-
0-7803-7108-9/01/$10.00 (C)2001 IEEE
178
IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
in COM components looking like Java objects, while COM properties, methods and events are presented as Java proper-
Web Server
ties, methods and events. The communication between the
SCADA 90 80
Historical Data Viewer (DCOM client)
ASP
70 60 50 40 30 20 10 0 1st Qtr
2nd Qtr
3rd Qtr
4th Qtr
OPC Server 1
OPC Server n
Internet Table of values 10:00
10:05
10:10
10:15
10:20
parameter 1
104
123
132
133
145
parameter 2
122
111
135
154
130
parameter 3
121
125
134
144
129
parameter 3
135
130
109
132
143
parameter 4
146
137
145
134
132
parameter 5
157
168
168
179
165
HDA Interface
90 80
Buffering System
70 60 50 40 30 20 10
ADO/OLEDB Layer
0 1st Q tr
2n d Q tr
3r d Q tr
4th Q tr
Data Bases
Fig. 2. Historical Data Access (HDA) Architecture
OPC Historical Data Access (HDA) has recently been specified and HDA servers have been included in several SCADA software packets. These servers expose their functionality as a custom COM and/or OLE automation interface for use by several types of clients based on DCOM. However, in order to develop a system that provides universal accessibility to historical data through the Internet, which will also be platform independent, it is indispensable to embed the functionality of an HDA COM server to a Web server, allowing access to its services through legacy Web browsers. This kind of functionality presupposes that the system design will be based on ubiquitous standards. In our solution as illustrated in Fig. 2, historical data retrieval and trending/reporting at the LAN level is provided by the DCOM-based SCADA application, which includes a HDA client module. In order to provide accessibility through the Internet, our approach is based on the HTTP protocol and the HDA services are used in order to construct “on-the-fly” Active Server Pages (ASP) that include historical data information, based on queries constructed by combining query parameters of specific HTML pages. These pages are, subsequently, fed to the Web browser, which initiated the request. Historical data, together with Average, Max, Min values, etc., can be illustrated in table, Comma Separated values (CSV) and graphical formats. The approach of providing historical data access by all types of thin clients (Web browsers) and, at the same time, conforming to the HDA standard, is an innovative approach that extends the access of stored data to a huge number of platform-independent clients
Java server and the Java applets, that provide the graphical user interface for real-time monitoring and control, takes place through the Java Remote Method Invocation (RMI). In addition to the handling of the calls from the remote clients (Java applets), the Java server runs the RMI registry process for the registration of the remote objects that can be accessed remotely, as well. In order to provide secure communications over Internet, the server uses the SSL protocol, instead of the default sockets that included in the libraries of JDK 1.2. The procedure followed for obtaining the parameter values from the OPC servers to the Java applets (clients) is the following: The Java server invokes the services of each OPC client (each of them corresponding to a specific automation subsystem), which (each OPC client) communicates with the OPC server(s), through DCOM, in order to retrieve the current parameter values, and subsequently, transfer them to the Java server. The OPC client processes run in a continuous way and keep an updated image of all the parameter values to the Java server. In the client side, the Java applets make RMI calls to the Java server, periodically, in order to be updated with the current values, which cause the updating of their corresponding graphical representation. Further to the above a Historical Data Access module was used [11], for the provision of both a data collection/repository and a data retrieval mechanism for historical data to trending and reporting clients. Its functionality comprises the collection of real-time data from OPC Servers, data logging of this data to a database and data retrieval capabilities both local (within LAN) via DCOM and remote (Internet access) via HTTP.
0-7803-7108-9/01/$10.00 (C)2001 IEEE
179
IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
through Internet, without the need of any specific plug-in installation. ETHERNET
OPC Server
LONW ORKS Network Chiller System
Controller
Analyser
Boiler System
Controller
Oxygen Production System
Electric Generator System
Analyser
Controller
Analyser
Water Chlorinating System
Lighting Contol System
Controller
Controller
Pressure
Pressure
Presence Detector
Temperature
Controller
Pressure
Flow Level
Alarms
LuxM.
Temperature Temperature
REDOX Meter
A/M Switch
Petrol Level
Alarrm Detectors
Chlor. Pump
Dimmer
Pressure Ballasts Voltage Level
Fig. 3 Pilot Implementation
client (Java based) applications (through the Internet), providing a view of the whole system status (parameter values and alarm signalling). Furthermore, the monitored values and the alarm states (set/reset) are stored in the central database as historical data for off-line processing. Apart from the control algorithms, a Real Time Clock (RTC) is provided to the LonWorks network, for time stamping of measurement data and for the synchronization of functions, based on time schedules. Due to the vast power consumption of the chiller unit, especially during summer months, the system is equipped with an electrical power analyser, for continuous accurate measurement of electric loads. Oxygen production system: This system is responsible for the on time signalling of eight critical alarm signals, related to failures on the system operation. The alarms are signalled both at a local panel and through the network to the supervisor applications. Electric Generator. One of the critical systems in the hospital is the electric power generator unit, which requires monitoring and recording of many parameters. Among other parameters, the power line voltage, current, active power, and power factor, have to be constantly monitored both at the public mains circuit and at the electric generator side. Normally, during any power failure, the electric generator starts up. In order to pinpoint potential problems, the time between the power failure and the moment that the standby electric generator connects to the mains, is measured and stored to the database for further processing. Apart from the power analysers, one controller has been installed, as well, for the monitoring of critical operation parameters of the electric generator, such as oil pressure, engine temperature, fuel level and battery voltage level. Water Chlorination. The hospital is supplied with water through devoted water sources, thus requiring substantial
IV. A PILOT IMPLEMENTATION The pilot implementation comprises a control network that integrates a number of subsystems in a hospital, which are responsible for the uninterrupted monitoring and control of crucial units, such us the Boiler, the Chiller, the Oxygen Production Unit, the Electric Generator, the Water Chlorination and the Lighting control (Fig. 3). Each subsystem consists of a number of automation controllers attached to I/O devices, in order to perform sensor parameter measuring, data processing, actuator driving, local and remote alarm signalling. All controllers have networking capabilities and are able to communicate with each other in a distributed way, conforming to LonWorks (EIA-709.1) standard [12]. Each controller executes application code, in order to perform its specific functionality, as specified in the requirement analysis phase. Operational parameters, such as start and stop set-points and alarm limits, can be dynamically changed through the network, by using standard network variables (SNVTs), without a need to put any controller off-line, or to detach it from the network. A brief description of the systems mentioned above is presented in the following sections. Boiler and Chiller: For each system, one controller was installed, which was responsible for the monitoring of temperatures and pressures both at the inlet and outlet points, through appropriate sensors attached to them. The user can either pre-configure or configure on-line the acceptable operational limits over or under which the alarms are signalled. The controllers process the measured values and transmit them to the network on real-time, under an event-driven algorithm. These values are graphically presented both at the SCADA MMI (inside the hospital network) and at the thin-
0-7803-7108-9/01/$10.00 (C)2001 IEEE
180
IECON'01: The 27th Annual Conference of the IEEE Industrial Electronics Society
VI. REFERENCES
water processing before use. The system controls the water chlorination through dosimetric pumps and monitors the pressure, flow rate and water tank levels, as well. The control of the dosimetric pump, which is responsible for the water chlorination, is accomplished through a PID controller, which, based on the remaining chlorine concentration (parts per million) at the water tank, increases/decreases the injected chlorine in the tank. Furthermore, the system has the capability to operate in auto or manual mode. Lighting Control. This system is responsible for the lighting control in a number of hospital’s corridors, based on occupation status and external light level. In order to achieve this, each corridor is equipped with occupancy sensors, luxmeters, auto/manual switches, dimmers and ballasts, connected to one or more controllers. Auto, manual and time scheduled modes of operation were implemented, which were selected by the MMI SCADA application.
[1] Bernstein, P.A. “Middleware: A Model for Distributed Services.” Communications of the ACM 39, 2 (February 1996): 86-97. [2] Rao, B.R. “Making the Most of Middleware.” Data Communications International 24, 12 (September 1995): 89-96 [3] Steinke, Steve. “Middleware Meets the Network.” LAN: The Network Solutions Magazine 10, 13 (December 1995): 56 [4] Reddy, Madhu. “ORBs and ODBMSs: Two Complementary Ways to Distribute Objects.” Object Magazine 5, 3 (June 1995): 24-30. [5] Schill, Alexander. “DCE-The OSF Distributed Computing Environment Client/Server Model and Beyond,” 283.International DCE Workshop. Karlsruhe, Germany, October 7-8, 1993. Berlin, Germany: Springer-Verlag, 1993. [6] Object Management Group home page [online]. Available WWW [7] Microsoft Corporation. Distributed Component Object Model Protocol, [online]. Available WWW [8] Sun Microsystems Inc. Java Remote Method Invocation, [online]. Available WWW
[9] OPC Foundation, OLE for Process Control (OPC) Data Access Automation Interface Standard, Version 2.02, February 1999 [10] Linar Limited, J-Integra, [online] Available WWW [11] OPC Foundation, Historical Data Access Automation Interface Standard, Version 1.0, January 2001 [12] Echelon Corporation, LonTalk Protocol Specification Ver. 3.0, 1994
181
