High Speed Data Exchange Protocol for Modern ...

High Speed Data Exchange Protocol for Modern Distributed Data Acquisition Systems based on OPC XML-DA Suren Chilingaryan, Wolfgang Eppler

Abstract—The current work is inspired by the requirement to develop a data exchange protocol for the several distributed data acquisition systems. The primary objective of the project described here is to produce a universal middleware solution which can be used for almost any experiment independent of the data rates, real-time demands and metadata requirements. The possibility to reuse software components made by scientists from different groups around the world is a great advantage of such high level design over legacy proprietary data acquisition systems. However, conformity with the high level standards, which are mainly based on XML technologies nowadays, makes it difficult to sustain high data rates required by certain installations. The paper proposes a data exchange protocol developed on top of the industry accepted OPC XML-DA and OPC Complex Data specifications, while acceptable performance is achieved by using our extensions. The implementation of the protocol is mostly completed and first preliminary tests with benchmark results are presented. Index Terms—Protocols, Data acquisition, Distributed control, Real time systems

I. INTRODUCTION

T

HE investigation of operating data acquisition systems has shown that many of them use non-standard self-developed solutions. The wide range of different data exchange solutions based on DCOM (Distributed Component Object Model), CORBA (Common Object Request Broker Architecture), RMI (Remote Method Invocation), SOAP (Simple Object Access Protocol), MPI (Message Passing Interface) protocols were used. Furthermore, several industrial systems, using OPC (Open Process Control) or OPC XML-DA (OPC XML Data Access) are available. For example, the data acquisition system for the H.E.S.S. telescope uses the omniORB implementation of CORBA for inter-process communication [1]. The DVIN (Data Visualization Interactive Network), based on a legacy ASCII data representation and FTP data transfer protocol, is used in the ASEC (Aragats Space Manuscript received June 25, 2005; revised October 21, 2005; revised March 15, 2006. S. Chilingaryan, W. Eppler are with the ForschungsZentrum Karlsruhe, 76021 Karlsruhe, Germany (e-mail: [email protected]; [email protected]).

Environmental Center) experiment [2]. The KATRIN (KArlsruhe TRItium Neutrino) experiment is an excellent example for heavily heterogeneous system [3]. Besides the industrial solutions controlling subsystems of KATRIN, the experimental data is currently acquired using the LabView DSC (Data logging and Supervisory Control [4]) system from National Instruments. As a result of such different approaches data sharing and reusability of data analysis software becomes an inefficient and tedious task. Impaired effectiveness of scientific work, lack of cooperation between groups working on similar projects and slow downs in implementation of newer promising technologies, like the Grid Computing, is an evident consequence of such situation [5]. XML is today’s standard of choice for the representation and exchange of structured data, particularly where that data must be read and interpreted by different applications working in heterogeneous environments and written by different groups. Still the scientific data for most of the physical experiments is produced in structured binary format at high data rates. Often it consists of series of large vectors or matrixes. The XML representation of this data is significantly larger than the native one and requires much more computational resources for serialization of the numerical data, and is practically unusable for complex manipulations on matrices and vectors. For these applications the XML representation certainly should not be used. However, there is enormous value in data processing and publishing automation especially when scientists from different groups all other the world are involved into data analysis software development. This is achieved by associating XML metadata with the binary scientific data. The metadata describes properties like the data accuracy, how, when and by whom the data was produced (parameters, algorithms used etc.), so that every scientist can interpret the associated binary data. There are no industry approved solutions which combine high compatibility and flexibility with outstanding performance required for the real-time tasks. Nevertheless, the OPC Foundation maintains a number of specifications for the control automation. The decision was to take one of these specifications as a basis and provide missing capabilities by means of the self-developed extensions. There are several generations of OPC specifications. The

OPC DA (OPC Data Access [6]) is a nowadays standard for the control systems industry. Unfortunately, it is based on the proprietary Microsoft DCOM technology and, therefore, badly suited for the heterogeneous environments. The OPC XMLDA (OPC XML Data Access [7]) is a restatement of the industry accepted OPC DA specification in XML terms. The OPC XML-DA specification considers the SOAP protocol for the data dissemination. The SOAP messages are transferred by means of the HTTP data transport. The combination of the XML data representation with HTTP data transport perfectly fits the design of the heterogeneous systems with components distributed over the internet. However, the performance of the XML based solutions is approximately five to ten times slower compared with binary protocols [8]. Therefore, the OPC XML-DA is not able to provide the data dissemination capabilities for the high performance systems. The OPC UA (OPC Unified Architecture [9]) is a new promising specification from the OPC Foundation. It is based on the Microsoft .NET architecture and provides further development of the OPC data access interfaces. The specification consists of 13 (or more) parts. Currently, only two parts describing the basic concepts are available. Therefore, the OPC UA architecture still is not a good basis for the new data acquisition system middleware design. As stated above, the OPC XML-DA protocol is ideally suited for heterogeneous environments, maintains a rich set of the standard and user-defined metadata and in combination with the OPC Complex Data specification [10] is very close to our demands in most aspects. The only drawback lays in its XML nature, obviously prohibiting its utilization in high performance real-time systems. In order to achieve the acceptable performance a mixed XML / Binary approach based on the OPC XML-DA and OPC Complex Data specifications is proposed in this article. OPC XML-DA supports protocol and meta-data requirements while the scientific data is transferred in pure binary format. The XML message only carries a data type name and a reference to the binary stream. In the next section a short overview of the OPC XML-DA protocol is presented. Section III defines a HDR (High Data Rate) extension. Section IV and V give a brief overview of the implementation and the optimizations used to improve performance. The last section presents current implementation status and results of the benchmark. II. OPC XML-DA SPECIFICATION The OPC XML-DA specification is a restatement of the industry accepted OPC DA specification in terms of XML. It defines a Web Service [11] interface facilitating the exchange of plant data in heterogeneous environments across the internet. According to the specification, the server data space is divided into the set of OPC Items which are addressed using “ItemPath” and “ItemName” identifiers. Each of the OPC Items stores the data variable along with standard and userdefined metadata information. By default the OPC XML-DA

specification anticipates usage of variables and arrays of several basic data types (floating-point and integer numbers, enumerations, strings, time variables, etc…). However, the OPC Complex Data specification gives an idea how complex data types can be constructed, described and served to the clients. As well the OPC Complex Data specification defines a way how the data should be served to the clients using several different data representations. The Web Service provides interfaces for reading, writing current values and examining the metadata information. The variable value changes are reported to the clients using polled style subscription mechanism. The client initiates subscription to the group of OPC Items (OPC Group) and agrees to issue periodic refresh requests. The subscription behavior is controlled using several properties. The “EnableBuffering” demands buffering of the values-changes detected in between client polling requests. The “Deadband” specifies the percentage of a full engineering unit range of an item’s value that must change prior to the value being of interest to the clients. The “Holdtime” and “Waittime” are used to reduce latency time of reporting a value change to the client and minimize the number of round trips between the client and server. Despite the fact that the OPC XML-DA specification is designed mainly for control systems, the subscription mechanism used together with buffering makes the protocol usage in the data acquisition systems possible as well. III. OPC XML-DA HDR EXTENSIONS The XML nature of the OPC XML-DA protocol causes extra utilization of bandwidth and CPU resources in the case of scientific data. Especially the vector and matrix based data are affected. To handle this problem a HDR (High Data Rate) extension to the OPC XML-DA specification is introduced. The HDR extension scope is: • Allow separation of the scientific data from the protocol and metadata information. • Provide support for multicast data transfer to maintain several synchronous clients. • Define rules to exchange binary data in heterogeneous environments. • Revise security considerations. • Manage two-way compatibility with legacy clients and servers. The separation of binary data and metadata information is achieved using either the WS-Attachments (Web Service Attachments [12]) technology or a separate multicasting stream. The XLink (XML Linking Language) is used to link different parts of the compound message together. Thus an OPC XML-DA HDR enabled server will respond to the clients with OPC XML-DA message with all the binary scientific data replaced by XLink references. The binary scientific data can be sent along with a SOAP message as well as using a separate multicast stream in the case of multiple clients. To incorporate binary data into the SOAP message the

A. Binary Data Format However, the necessity to transfer data in heterogeneous environments still requires an agreement about a binary exchange format. Again, there are many widely used formats including XDR (eXternal Data Representation), CDR (Common Data Representation) and all of them require some data conversions even in the case of homogeneous environment. Assuming, that most PCs of the experiment have similar architectures this solution is sub-optimal and limits peek performance. This assumption is rather often true for modern data acquisition systems, since many of them, like ASEC, are based on the clusters of x86 Linux PC’s [2]. Another often used solution is a mixed Linux/Windows environment running on x86 based computers. In this case the systems also have similar data representations. The only difference is the alignment of the data. Therefore, the native data representation (NDR) of the server is used in our solution to transfer the binary data. The reference to the description of the data format is sent along with the data. The description is held on the server using a slightly modified OPC Binary notation from the OPC Complex Data specification. The modifications concern the concepts of data alignments which are unsupported by the original OPC Binary notation. In the process of data transfer the client requests this description from the server and makes a data conversion only in the case of format incompatibility. According to [13] such approach is faster than the common data exchange methods even in the case of heterogeneous environment. The comparison of the round-trip time using proposed native data representation with several widely used formats is presented in Fig. 1. The results confirm that in the mixed Linux/Windows environment the proposed solution is much more efficient than standard ones. B. Reliable Multicasting The original OPC XML-DA specification is not considering any special way to support multiple clients. The HDR extension assumes multicasting for that. Multicast communication allows a single transmission of the data from the OPC XML-DA HDR server to be received by multiple clients. Usually this capability is essential for the highly loaded data acquisition systems with the requirement to improve fault tolerance or implement some kind of loadbalancing. Fig. 2 represents typical scenario demanding usage of the multicasting approach. However, the multicasting is relying on the unreliable UDP protocol. So the data consistency can’t be guaranteed. To solve this problem IETF (Internet Engineering Task Force)

60000 50000 40000 time (us)

WS-Attachment technology is used. It is based on the DIME (Direct Internet Message Encapsulation [12]) encapsulation. Since the DIME encapsulation has binary nature, such solution is much better suited for real-time tasks than the alternate and more known “SOAP message with attachment”.

30000 20000 10000 0 4096

16384

65536

262144

size (KB)

NDR

XDR (Glibc)

CDR (XTL)

CDR (OmniORB)

Fig. 1. The chart represents the time required for a round-trip message exchange between Linux and Windows x86 PC’s depending on a portable binary format used for serialization.

working group on RMT (Reliable Multicast Transport) was organized. The RMT protocol uses multicasting to provide message dissemination capability and adds reliable delivery mechanisms. There is still no RMT protocol accepted as must use standard. However, the PGM (Pragmatic General Multicast) is accepted as experimental protocol by IETF. It is currently supported under the Windows 2003 Server operating system. Several implementations are also available for Linux platform. The PGM enabled routers are provided by the CISCO Corporation, the Nortel Corporation and perhaps some other vendors [14]. Therefore, the PGM protocol is assumed by our system to support multiple synchronous clients requesting the same data. The realization is oriented toward the high speed data acquisition systems supporting multiple intranet clients. Some of the supported OPC Items are selected by the OPC XMLDA HDR server for broadcast depending on their data rates and estimated number of clients. The dedicated multicast group is allocated for each of these OPC Items. The address of the multicast group is stored along with other metadata information. During the server operation, every value change of the data variable associated with the OPC Item is broadcasted into the corresponded group using native data representation. The OPC Item grouping is performed on the client-side. However, the OPC XML-DA HDR clients are not obliged to support PGM multicast. The data to the PGM incapable clients will be transferred using the standard “WS-Attachments” approach. C. OPC Group Handling Issues The OPC data access paradigm considers a group access to the server data space. The client by means of the “Subscribe” request specifies a group of OPC Items along with several configuration parameters defining the required dead band, caching and data representation. This group is called OPC Group and henceforth used to access the data. This design is used in order to parameterize subscription and perform partial data filtering and type mangling on the server side. However,

Data processing cluster

PC

OPC XML-DA HDR Server

PC

HTTPS/XMLSec OPC XML-DA Connection Encrypted using asymmetric-key cryptography

Client

PC Database

High data rate binary connection Encrypted with symmetric key

Multicast stream

Multicast Group

High data rate binary connection Encrypted with symmetric key

Symmetric key

Server Fig. 2. The figure represents an example of a typical multicasting system layout. The cluster of PC’s is used to process data acquired from the server using multicast connection.

for the high performance data acquisition systems based on the complex data entities the described group access is not required. The type mangling is only required by the OPC XML-DA specification for the simple scalar types (section 2.7.5 of the OPC XMLDA 1.01 specification) and, therefore, it is not performed for the complex vector and matrix based data anyway. The filtering options have sense for the control systems where the server is periodically queried to obtain current statuses of the certain sensors. However, in the data acquisition systems all data provided by the server usually should be mandatory obtained by the client application and processed. So the caching is mandatory enabled and dead band is not used at all. Obviously, the group access is not compatible with the multicasting extension. However, the multicasting is mostly required for the data acquisition systems considering complex fair-sized data items. Therefore, the item-level handling satisfies the requirements posed by the systems which may require the multicasting utilization. D. Security Considerations The OPC XML-DA assumes a HTTPS (Secured Hyper Text Transfer Protocol) security approach. However, for multicast data it is inapplicable and some other mechanism has to be implemented. The HDR solution assumes symmetric key cryptography for multicast data connections as seen in Fig. 3. An OPC XML-DA server generates a random symmetric secret key and encrypts by means of that all data passing to a multicast group. The client establishes a secure HTTPS control connection to the server, using asymmetric key cryptography as specified by the OPC XML-DA specification. Further, it confirms PGM protocol support and requests a data. The OPC XML-DA HDR server responds with an address of the multicast stream. The key used to protect data is send along with response. Therefore, the client can get data from the multicasting stream and decrypt it with the provided key. Another solution for a more XML adopted security approach is replacing the HTTPS protocol with an XML-only security protocol, as described in the XML Encryption and XML Signature specifications [15], [16]. This solution strictly

Fig. 3. OPC XML-DA HDR security layout used to protect data between the server and multicast clients.

utilizes the XML way and provides more control over authentication and read-write access. The possibility to manage access rights on the OPC Item level rather than on the server-wide level is the most important gain of the approach. E. Compatibility The two-way compatibility with legacy OPC XML-DA servers and clients is a strict requirement. The OPC XML-DA HDR enabled server stores information about all supported extensions along with their versions under the ‘/Extensions’ path. The client in the compatibility mode checks whether HDR extensions are supported. If this is the case it switches to the HDR mode. IV. DATA REPRESENTATION CONCEPT As is evident from the previous section the data has to be supported to the different clients in the corresponding representations (see Fig. 4). Therefore a procedure performing dynamic conversion between these representations is designed. The procedure is based on special XML descriptions of the binary data: content and system dependent layout. The OPC Complex Data specification proposes the usage of the XML based OPC Binary specification to provide such description [10]. However, the OPC Binary description is not applicable to data alignments which are often required to transport data in a native representation. Therefore the OPC Binary language is extended with several attributes introducing an HDR Binary data description. The attributes are added in a way that the description is compatible with the original OPC Binary. By this means the data can be represented by the legacy OPC Binary specification. To support such complex and time consuming conversions the concept of “Data Representation” is introduced. Each data item may have different representations, for example: server native representation, arbitrary client native representation, OPC XML-DA representation. Each representation, except the primary one, has a so called base representation with defined functions to convert data from/to this base representation. Conversions are performed with help of the HDR Binary description for binary data mentioned above and XSLT stylesheets in the case of XML. This design can be used more widely than just defining different representations of the same data. OPC

Data Format Description

Data in Server Native Format

Data in Server Native Format

Client

XML Representation

Client

Data in Client Native Format

Server

Fig. 4. Architectural overview of the data representation management. The leftmost white box represents the data originally submitted to the OPC server. White boxes in the middle represent two different data representations maintained by the server. The rightmost white box represents the data representation which is provided to the client by the client library.

Representations are able to provide data obtained by applying some data filters and transformations to the base data. This approach is very conventional in providing support to all kinds of the data filters, data masks and queries considered by the OPC Complex Data specification. The possibility to accept queries from clients combined with the “Data Representation” concept gives broad range of powerful options to the system designers. For example, it is possible automatically extract critical information for the operators, made server-side data preprocessing or filter parts of a data not important for the client. Even, it is possible to convert data in XHTML to simplify presenting it to the end user. A. Representation Caching The data conversion between the considered representations, especially XML ones, implies significant CPU usage. Therefore in the case of several clients a representation caching can preserve a considerable amount of computational resources. Client’s request processing time using such caching is compared in Fig. 5 against implementation without it. The figure confirms that representation caching becomes efficient starting from 3rd simultaneous client and can bring up to 20% of performance gain. V. MEMORY MANAGEMENT OPTIMIZATIONS Optimization of the memory allocation performance in a multithreaded environment is one of the most important aspects of the client-server architecture development. Standard memory allocators, like dlmalloc by Doug Lee, implemented in modern operating systems are very effective and in most cases custom implementations can not outperform them [17]. However, in the case of long-running heavily multithreaded 1st client

2nd and further clients

0

100

200

With Caching

300

400

Without Caching

500

600 time (ms)

Fig. 5. Impact of the representation caching on the request processing time. The chart represents time required for the client request processing with and without caching. The upper part represents the time required to process first request to the considered data. The time required for processing following requests is presented on the lower part of the chart.

server applications with big amounts of rapidly changing data considerable optimizations are possible, especially in the case when several threads should satisfy real-time requirements. The necessity of synchronization between multiple threads accessing the shared memory heap is a major drawback of standard memory managers. Especially this effect affects realtime threads running among the lower priority ones with active memory utilization. The OPC XML-DA HDR based data exchange system is greatly affected by this drawback, since high priority readout threads coexist with threads processing and generating the XML context, which require big amount of memory and high intensity of memory management operations. The requirement to convert data between XML and binary representations makes it difficult to determine the required size of the memory buffer to hold the whole data. Widely used solutions like the “growing buffers” are suboptimal since they consider multiple buffer resizing operations. These operations are very expensive and intensively utilizing the memory subsystem. In the case of fragmented memory, the displacement of already allocated memory may be required. Further complication is brought by the requirement of caching older values by the Subscription mechanism of the OPC XML-DA specification. This demand is suppressing the reuse of memory buffer and appears as another source of heavy heap utilization. Fortunately, the sequential nature of the OPC XML-DA protocol allows the assumption that older cached values will be freed earlier than newer ones. Making an additional assumption about successive submission of new values (i.e. allocation of any processing item should be completely finished before starting a new one) a described below custom approach can be used. The approach is based on the concept of the “dedicated ring buffers”. For effective memory space allocation each representation of a data item is associated with big enough dedicated ring buffer. The buffer is used to store all memory blocks related to the current data item representation and corresponding representations of all cached data item values. Fig. 6 illustrates the difference between “growing buffers” and “dedicated ring buffers” memory allocation schemes. In the case of the buffer being too small to hold all data required by the caching policy it will be replaced by a bigger one. The old buffer will be freed as soon as no utilized data will remain in it. To prevent overflow of a buffer during the memory block allocation, the memory usage statistic is collected. This statistical information is used to pre-allocate certain amount of space in the buffer and should prevent copying partly allocated blocks into new buffers. The approach gives the following benefits: • Most of the blocks are allocated in successive order, which is extremely efficient, since data can be manipulated as one big chunk, instead of many small pieces. • Storage of multiple cached values in the same ring buffer is optimal from both speed and memory utilization points of view.

•

VI. IMPLEMENTATION AND BENCHMARK RESULTS The protocol is implemented as a multiplatform C library. The library provides a set of 3 independent API’s (Application Programming Interface). The server API enables the management of the server configuration and run-time status. The backend API provides a way to configure data sources, allocate memory from the associated ring buffers and submits new data. The conversion between different data representations and coupling with the protocol related information is done using the frontend API. On top of the library an OPC XML-DA HDR server framework is constructed. The framework consists of the core data manager, the backend and frontend interfaces. The frontend interface accepts connections from the clients and prepares data in the user requested format in accordance with OPC XML-DA HDR specification. The backend interface maintains plugins which are communicating with physical hardware and submitting data to the server utilizing the backend API interface. The server is implemented using MVC (Model-ViewGrowing Buffer Current Value (412KB)

Cached Value (296KB)

512KB

Cached Value (192KB)

512KB

256KB

Dedicated Ring Buffer Current Value (412KB)

Cached Value (296KB)

Cached Value (192KB)

1024KB Buffer End

Buffer Start

free

Fig. 6. The figure illustrates the difference between “Growing Buffer” and “Dedicated Ring Buffer” memory allocation schemes. The schemes allocate storage for current and two cached values associated with a single OPC Item. The “Growing Buffer” scheme uses three independent buffers. The “Dedicated Ring Buffer “scheme uses single big buffer to store all values.

20000 15000

time (us)

Usage of pre-allocated dedicated buffers terrifically reduces memory fragmentation. • Statistical information helps to estimate required space before allocation and reduces amount of memory management system calls. The most important gain of the approach is elimination of the demand to lock / unlock the heap on every memory manipulation request. Combined with pre-allocation of buffers of certain sizes this resolves blocking of high priority threads by low priority ones due to memory allocation from the same heap. The evaluation results presented, in Fig. 7 show that performance of the proposed allocator is practically unaffected by the server load, whereas the growing buffer allocator reveals five-fold speed decrease under heavy loads. Another important improvement of the memory management subsystem is achieved by the ability to directly manipulate the ring buffers. It is possible for the data management threads to allocate blocks of memory directly from the associated ring buffers. Such approach completely eliminates the necessity of process-wide memory allocation routines and makes high priority data acquisition threads really independent from the system load.

10000 5000 0 0

1

4

16

64

concurrent clients Growing Buffer

Dedicated Ring Buffer

Fig.7. Comparison of “growing buffer” and “dedicated cyclic buffer” memory allocation techniques depending on the server load (X-axis). The charts represent the time required to allocate 512KB sized value, divided in 8 blocks. The server load is implied by multiple clients requesting data in OPC XML-DA compatibility mode.

Controller) style architecture. The frontend interface (view) is segregated from the data model by means of the core data manager. Such design allows using different frontend interfaces in conjunction with the same data handling core. At the moment, only the OPC XML-DA HDR frontend is developed. However, as soon as a complete set of OPC UA specifications will be available to the public a new frontend may be developed in order to handle the clients using simultaneously both OPC XML-DA HDR and OPC UA technologies. Currently the framework implements the OPC XML-DA specification and can support data in both XML compatibility and binary formats following the “WS Attachment” specification. The PGM multicasting support is currently under development. The framework is tested under Linux and Windows operating systems and also should work under FreeBSD. Porting to other POSIX compliant systems should be an easy task. Additionally, the bindings for the National Instruments Labview are developed to provide the possibility to use the OPC XML-DA interface in Labview based control systems. Both the Windows and Linux versions are supported. In order to support Labview Real-Time (running on the FieldPoint and PXI devices) the framework is ported to Phar Lap (Windows NT based real-time operating system) [18]. A. Benchmark Results To compare the performance of the developed system with other available solutions, the omniORB and gSOAP toolkits were used. The omniORB is one of the fastest CORBA implementations according to [19]. The gSOAP is the most featured SOAP toolkit available under Linux. However OPC XML-DA is a higher level protocol than the CORBA and SOAP ones. Therefore direct speed comparisons between OPC XML-DA implementation and these toolkits are impossible. Nevertheless an implementation of a simple OPC-like protocol using these toolkits was developed. Of course, the

VII. CONCLUSION

gSoap; 100% Corba; 28% XML; 65% Binary; 15%

20%

40%

60%

throughput (MB/s)

100 80 60 40 20 0 2

16

128

512

2048 item size (KB)

12 10 8 6 4 2

80%

100%

Fig. 8. Evaluation of the server load imposed by the 3 clients making one request per second. Each of the clients is requesting 512KB of floating point data. The Intel PIV Northwood (2.2GHZ, 1GB Ram) server connected to 100MBit network was used to perform the benchmark.

0 2

16

Binary

XML

128

Corba

512

2048 item size (KB)

gSoap

Fig. 9. The chart represents the maximal achievable throughput of a Linuxrunning server supporting a single windows client with arrays of floatingpoint numbers in HDR (binary) mode. The AMD Athlon 64 X2 4800+ (2.4GHZ dual core, 2GB Ram) server connected to a 1GBit network was used to get the top diagram, and the Intel PIV Northwood (2.2GHZ, 1GB Ram) server connected to a 100MBit network for the bottom one. 50 45 40 35 30 25 20 15 10 5 0

Throughtput (MB/s)

Throughput (MB/s)

In this paper it is shown that XML based protocols can be used to transfer data in a very efficient way if our extensions are added. The basis chosen for the developed universal data exchange protocol is OPC XML-DA, because this is an apparent successor of the de facto standard for automation industry and control systems, the OPC DA protocol. The performance is obtained with several self-developed extensions to the original specification. The inclusion of high data rates for many scientific applications as well as the compatibility with legacy clients is very important if OPC XML-DA should be used as a universal data exchange protocol. The separation of the data from protocol and metadata information, usage of the server native binary representation for data exchange and the opportunity to support multiple clients with the newest PGM multicasting standard are main concepts of these extensions. As well following ideas were used to improve implementation performance and usability. The “data representation” concept allows automatic conversion between different representations of a data and gives opportunity to create custom data filters and masks on the server side. Data tweaking with user supported queries is

0%

120

throughput (MB/s)

full OPC compatibility is not provided by these implementations, but it gives the possibility to make rough estimations of the system performance compared with the mentioned environments. Windows clients, connected to a Linux server using a 100MBit and 1GBit network links, were used to measure the performance and scalability. Both sides used x86 compatible hardware. As it can be seen in Figs. 8, 9 the developed framework in the binary mode (XML metadata and binary data) achieves the performance of pure binary CORBA system as soon as the transferred data frames have moderate sizes. Even in the XML mode (XML metadata and data) the framework is faster than competing solution, based on a standard SOAP implementation. The Fig. 10 proofs that the system is able to support multiple clients without significant performance lost.

1

2

4

8

32

128

1024 16384

Clients

5 4,5 4 3,5 3 2,5 2 1,5 1 0,5 0 1

2

4

8

32

128

1024

Clients

Fig. 10. The chart represents the performance dependence on the number of the clients. Each client continuously requests floating point arrays (16KB). The data was served to the clients in HDR binary mode for the left diagram and in OPC XML-DA compatibility mode for the right one. The performance is evaluated as a maximal data amount which can be transferred to all clients in a second. AMD Athlon 64 X2 4800+ (2.4GHZ dual core, 2GB Ram) server connected to 1GBit network was used to perform the benchmark.

another benefit of this approach. The memory management using “dedicated ring buffer” technique eliminates most of memory allocation delays on the heavily loaded servers and makes real-time data acquisition threads really independent from the system load. The results of the benchmark measurements demonstrate that the implementation achieves the performance of pure binary CORBA systems at the same time providing all XML metadata considered by the OPC XML-DA specification. Therefore, our solution can be used as a universal solution combining acceptable performance with high level industry approved standards.

REFERENCES [1]

[2]

[3]

[4]

[5]

[6] [7] [8]

[9] [10] [11]

[12]

[13]

[14]

[15]

[16]

[17] [18] [19]

Borgmeier, N. Komin, M. de Naurois, S. Schlenker, U.Schwanke, C. Stegmann, “The Central Data Acquisition System of the H.E.S.S. Telescope System,” Proc. of the 28th ICRC, Tsukuba, 2003. A.Chilingarian for the ASEC team, “Aragats Space-Environmental Center: Status and SEP Forecasting Possibilities,” Proc. of 22nd ISTC Japan Workshop on Space Weather Forecast in Russia/CIS, Nagoya, June 2002. W. Eppler, A. Beglarian, S. Chilingarian, S. Kelly, V. Hartmann, H. Gemmeke, “New Control System Aspects for Physical Experiments”, IEEE Transactions on Nuclear Science, Vol. 51, No. 3, pp. 482-488, Jun 2004. National Instruments Corp. Getting Started with Labview Datalogging and Supervisory Control Module. Available: http://www.ni.com/pdf/manuals/322946b.pdf W. Hoschek et al., “Data Management in an International Data Grid Project,” IEEE/ACM Workshop on Grid Computing, Bangalore, Dec. 2000. OPC Foundation. (2003, Mar 5). OPC Data Access 3.00 Specification. Available: http://opcfoundation.org OPC Foundation. (2004, Dec 21). OPC XMLDA 1.01 Specification. Available: http://opcfoundation.org Frank Iwanitz. XML-DA Opens Windows Beyond the Firewall. Available: http://ethernet.industrialnetworking.com/opc/articledisplay.asp?id=21 OPC Foundation. (2005, Sep 28). OPC Unified Architecture Release Candidate Specification. Available: http://opcfoundation.org OPC Foundation. (2003, Dec 10). OPC Complex Data 1.00 Specification. Available: http://opcfoundation.org Heather Kreger. (2001, May 1). Web Services Conceptual Architecture. Available: http://www306.ibm.com/software/solutions/webservices/pdf/WSCA.pdf H.F. Nielsen, E. Christensen, J. Farrell. (2002, Jun.). WS-Attachments. Available: ftp://www6.software.ibm.com/software/developer/library/wsattach.pdf F. Bustamante, G. Eisenhauer, K. Schwan, P. Widener, „Efficient Wire Formats for High Performance Computing,” Proc. of the 2000 ACM conference on supercomputing, 2000. J. Gemmel, T. Montgomery, T. Speakman, N. Bhaskar, J. Crowcroft, "The PGM Reliable Multicast Protocol," IEEE Network, January/February 2003. D. Eastlake, J. Reagle, D. Solo, M. Bartel, J. Boyer, B. Fox, B. LaMacchia, E. Simon. (2002, Feb.). XML – Signature Syntax and Processing. Available: http://www.w3.org/TR/xmldsig-core/ D. Eastlake, J. Reagle, T. Imamura, B. Dillaway, E. Simon. (2002, Dec.). XML Encryption Syntax and Processing. Available: http://www.w3.org/TR/xmlenc-core/ M. Masmano, I. Ripoll, A. Crespo, “Dynamic storage allocation for realtime embedded systems,” RTSS 2003, Cancun, Mexico, 2003. National Instruments Corp. Real-Time Measurment and Control. Available: http://www.ni.com/realtime/ P.H.F.M.Verhoeven, J.Huang, J.J.Lukkien, “Network Middleware and Mobility,” Proceedings of the 2nd workshop on Embedded Systems, p287-292, STW, Oct 2001.