An embeddable control system for astronomical instrumentation R. Cirami*a, M. Comaria, C. Cortea, D. Golobb, P. Di Marcantonioa, M. Pleskob, M. Pucilloa, P. Santina, M. Sekoranjab, C. Vuerlia a INAF-Osservatorio Astronomico di Trieste, Via G.B. Tiepolo, 11, 34131 Trieste, Italy b Cosylab Ltd. and J. Stefan Institute, Teslova ulica 30 1000 Ljubljana, Slovenija ABSTRACT Large experimental facilities, like telescopes and focal plane instrumentation in the astronomical domain, are becoming more and more complex and expensive, as well as control systems for managing such instruments. The general trend, as can be learned by realizations carried out in the most recent years, clearly drives to most cost-effective solutions: widespread, stable standards in the software field, COTS (commercial off-the-shelf) components and industry standards in the hardware field. Therefore a new generation of control system products needs to be developed, in order to help the scientific community to minimize the cost and efforts required for maintenance and control of their facilities. In the spirit of the aforementioned requirements and to provide a low-cost software and hardware environment we present a working prototype of a control system, based on RTAI Linux and on ACS (Advanced Control System) framework ported to an embedded platform. The hardware has been chosen among COTS components: a PC/104+ platform equipped with a PMAC2A motion controller card and a commercial StrongARM single board controller. In this way we achieved a very powerful, inexpensive and robust real-time control system which can be used as a general purpose building block in the design of new instruments and could also be proposed as a standard in the field. Keywords: Control Software – Linux – Real-time – CORBA – ACS – PC/104+ - StrongARM
1. INTRODUCTION Modern large experimental facilities (accelerators, telescopes, etc.) are highly sophisticated and complex systems, where the number of devices to be controlled and monitored may exceed tens-of-thousands. No well-established solution exists that would satisfy the requirements of flexibility, scalability, reliability and security needed for their control. Research institutions have to build control systems from scratch, which yields high costs, produces incompatible solutions and duplicates the effort. In particular, a growing number of telescopes requires now an automatic control system to be set up. So far no general purpose control systems are available that can be easily tuned for a particular telescope/instrumentation to be controlled. This led to a scenario where a new control system was set up from scratch every time a new telescope needed to be controlled. This scenario, together with the need to operate large scale reductions on the resources employed in the design, implementation and maintenance of such systems, strongly pushes towards a new generation of control systems for astrophysical instruments, both by the software and hardware points of view. The spreading of robust, free, open-source software, such as the Linux OS (even the real-time versions), Java and the socalled middleware solutions, like CORBA, greatly enhances the possibility to define clean and effective architectures for those systems, eventually approaching the goal of a solution which adopts a common environment from the high level user interfaces down to the embedded device control. Besides, the availability of faster networks, even of the wireless type, allows the progressive abandoning of mixed and complex communication solutions in favor, again, of a common environment. All these considerations naturally lead towards a lightweight, portable, adaptive system, based on the most diffuse standards, which shall be used as a general-purpose building block in the design of new instruments, or even when refurbishing old ones. *
[email protected]; phone +39 040 3199214; fax +39 040 309418; http://www.ts.astro.it
2. PROJECT’S REQUIREMENTS AND OBJECTIVES 2.1 Main control system requirements The role of a control system is to allow the operators of experimental facilities to extend their influence from the control room to the devices that they monitor and control. Modern control systems are expected to meet the following non-functional requirements: • • • • • • • • •
Flexibility: the control system should allow for easy reconfiguration of the facility, such as replacement of individual devices. Scalability: there should be no upper limit to the number of controlled devices. Performance: the control system must offer high performance for good usability, scalability and prompt reaction times. Certain subsystems are also subject to hard or soft real-time requirements. Usability: the operators should be able to efficiently monitor and control the facility via a consistent, appealing, user-friendly interface. Reliability: control systems of current experimental facilities offer 95% reliability, which ought to be increased for the sake of economic efficiency. Remote accessibility: the control system should be available remotely (e.g., via Internet), especially in cases where the experimental facility is at a location with limited human access. Security: the control system should be capable of countering malicious attempts to damage the controlled devices, which is of particular importance especially in the context of remote accessibility. Safety: misuse and accidents should be prevented, and also a basis for accountability should be provided. Reusability: a reusable control system allows cheaper and easier transfer of applications, knowledge and personnel between facilities using it.
The following non-technical requirements are also of crucial importance for use in science as well as in many industrial applications: • •
Inexpensiveness: the platform (both hardware and software), atop of which the control system is built, must be as inexpensive as possible, because its basic building blocks will be deployed in large numbers across the experimental facility. Openness: the control system must be open and based on accepted standards so that it can be maintained in the future and be made easily accessible to those who whish to extend it.
To cope with the stated requirements, architectures of typical control system solutions have three tiers: • • •
Field bus: input/output control boards, either embedded in the controlled devices or directly connected with them. Device servers: server computers connected to the field bus and responsible for exposing the devices to the Local Area Network (LAN). Workstations: workstations running the Graphical User Interface (GUI) applications, which allow the operators to interact with the control system.
Device servers communicate with the field bus using device drivers, whereas workstations and device servers exchange information using a solution-specific communication protocol. 2.2 Scientific and technological objectives In terms of the above requirements, the main objectives of our project are the following: •
The capability to auto-configure itself, on the basis of the underlying hardware (telescope and attached instrumentation). In this way it will be very easy to attach such a system to a new telescope to be controlled. In the same way the system will automatically react to any change involving the hardware under control (new
• • • • • • • •
telescope hardware setup, new instrument attached, etc.). This leads to a hierarchical tree structure, the leaves being the low-level hardware controllers and the root the main control system manager (see Fig. 1). Not to restrict it to any particular control system, but instead to allow reuse in all kinds of experimental facilities, as well as industrial applications. Achieve inexpensiveness by porting it to an embedded platform, while retaining its openness and flexibility. Produce input/output boards for synchronized hard real-time triggering. Improve the scalability of such a system by allowing hierarchical organization of controlled devices through the use of federation of several independent systems. Enhance usability by expanding the common look-and-feel of the GUI to a wider range of applications, supporting additional objects and data models. Make a control system remotely accessible via Internet. Implement security mechanisms based on the Public Key Infrastructure (PKI). Improve the quality level of existing and future software by further automating testing procedures and adjusting the development process, thus boosting reliability.
Fig. 1. General architecture of the control system
Applying such control system to a growing number of telescopes offers at least the benefits listed below: • • • • • •
The time required to bring a new telescope system to its operational state will be greatly reduced because no adhoc control system has to be built. The use of the same control system in several operational environments implies that its maintenance is greatly simplified and optimized (in terms of manpower, cost and time). The maintenance updates are cumulative: improvements made on one installation are applicable to all installations using the same system. The framework will provide consistent look and feel which will lower the learning curve for control system operation for new scientists, and foster scientific cooperation. The use of cheap embedded platforms will greatly reduce costs required to make a telescope and its subsystems fully operational. The possibility to use different network devices and physical transport means, ranging from cable connections to wireless solutions.
3. HARDWARE SOLUTIONS In order to satisfy the above requirements and objectives, we have adopted the following hardware solutions, chosen among COTS components and characterized by their inexpensiveness and low power consumption. We followed simultaneously two alternative paths: a PC/104+ platform equipped with a PMAC2A motion controller card adopted at the Astrophysical Technologies Group of the Astronomical Observatory of Trieste and a commercial StrongARM single board controller adopted at the Cosylab Ltd. and the J. Stefan Institute of Ljubljana. 3.1. PC/104+ PC/104 is a low cost PC compatible hardware environment for embedded applications. While the PC, PC/AT and PCI architectures have become extremely popular in both general purpose (desktop) and dedicated (non-desktop) applications, their use in embedded microcomputer applications has been limited by the large size of standard motherboards and expansion cards, internal dissipation and power consumption. Therefore a compact version of the PC, PC/AT and PCI bus has been designed for embedded system applications, the PC/104+ bus with the following key differences: • • •
Reduction of the form factor to 90 by 96 mm Elimination of the need for backplanes or card edges through its self-stacking bus Minimization of components and power consumption (to typically 12 Watts per module), by reducing the required bus drive on most signals to 4 mA
Fig. 2. PC/104+ control system hardware prototype
The PC/104+1 is backward compatible with the PC/104. The PC/104 specifies two module versions – 8 bit and 16 bit – which correspond to the PC and PC/AT bus implementations respectively. Low power consumption means low heat to be dissipated and consequently no fans (they usually have limited life), allowing to install the system inside the scientific instruments. The present prototype (see Fig. 2) is based on a “Digital Logic MICROSPACE PC/104 PLUS MSMP5SEN”, a miniaturized modular device incorporating the major elements of a PC/AT compatible computer, with the following characteristics: •
CPU PII-MMX, 166 MHz
• • • • •
128 MB RAM Ethernet adapter 10/100 Mbit HD 10 GB For the mass storage unit a 256 MB flash card disk drive connected via the AT-IDE interface, to avoid hard disk weakness such as vibrations and shocks For the motion controller card we used a PMAC2A-PC/104 base board
An Ethernet-based point-to-point system allows the network link between the distributed components of the control system to be established. 3.1.1. PMAC2A The PMAC2A-PC/1042,3 motion controller is a compact, cheap version of the Delta Tau’s PMAC2 family of controllers. The base board provides four channels of either DAC 10V or pulse and direction command outputs. The PMAC2 is a real-time, multi-tasking computer with its own stored programs and may run as a standalone controller or may be commanded by the host computer, either over a serial port or over the PC/104 bus. Optional boards can be added in a stack configuration. PMAC can hold up to 256 motion programs at one time. Any coordinate system can run any of these programs at any time, even if another coordinate system is already executing the same program. PMAC can run as many motion programs simultaneously as there are coordinate systems defined on the card (up to 8). The PMAC motion programming language is perhaps best described as a crossing between a high level computer language like BASIC or Pascal, and “GCode” (RS-274) machine tool language. These motion programs are not suitable for performing actions that are not directly coordinated with the sequence of motions. For these types of tasks, PLC programs have to be used; they operate in a similar manner as Programmable Logic Controllers, continuously scanning through their operations as fast as the processor clock allows. The PLC programs are very useful for any task that is asynchronous to the motion sequences. In the PC/104+ hardware solution, a PMAC2A card is connected to the PC/104+ bus. The PMAC2A card is connected to a DC motor equipped with incremental encoders that moves a linear axis provided with upper and lower limit switches. PMAC programs have been developed for axis initialization, absolute movements and limit switch management. 3.1.2. The PMAC2A Linux driver A Linux driver for the PMAC2A, based on a polling mechanism, has been developed by the Astrophysical Technologies Group of the Astronomical Observatory of Trieste. To interface this driver in a proper way, user applications must be based on the polling mechanism to check whether new data can be written on the PMAC card and/or output data coming from PMAC are available to be read. This driver is used to talk to the PMAC card and for the PMAC programs downloading. 3.2 StrongARM A family of I/O cards with onboard Linux and Ethernet has been developed. Combining those cards into a small casing and adding control software packages such as EPICS (Experimental Physics and Industrial Control System) or ACS (Advanced Control System), provides for compact software driven turnkey systems. It has always been our goal to reduce the number of hardware and software interfaces and the flavour of different field busses to a minimum. With the advent of low priced mass produced embedded controllers with high CPU power and an array of standard interfaces such as Ethernet, USB, RS-232, etc., it became possible to get rid of the field bus completely, even in highly distributed applications, bringing Ethernet and TCP/IP down to each single I/O card. We have opted for a Single Board Computer (SBC) with a StrongARM CPU and Ethernet interface (see Fig. 3), for which several low footprint solutions exist on the market. The SBC is positioned as a mezzanine module on our I/O cards. The CPU power and the available RAM allow for normal operating systems and control programs to run. We have customized a standard Linux port for the StrongARM to our SBC. This allowed us to port the CORBA-based Advanced Control System ACS onto the I/O card. An EPICS port has also been done. This allows to integrate the I/O card seamlessly into any control system, without the necessity for any extra drivers. As both Linux and Ethernet are indeterministic, we need another real-time component to handle timing and I/O synchronisation. This can be done with a CPLD (Complex Programmable Logic Device), which is built into each of our I/O cards. The CPLD can be programmed from Linux via a JTAG bus from the SBC for all realtime tasks that are not too computing intensive. It can serve as an on-board timing module, function generator and general digital interface.
Fig. 3. StrongARM control system hardware prototype
4. SOFTWARE SOLUTIONS We have developed a working prototype of a control system ported on an embedded platform. The software environment is based on the Linux operating system with the RTAI (Real Time Application Interface) extension and on the Advanced Control System (ACS), which provides a generalized common interface between applications and hardware in order to facilitate the implementation and the integration in the system of new hardware and software components. The Abeans framework has been adopted for building graphical user interfaces. It offers high-performance and common look-and-feel to the user, automatic installation and maintenance capabilities to the administrator, and rapid application developments to the developer. The choice of free, open source software, according to the state-of-the-art technologies has been adopted. 4.1. Real-time operating system The OS is an embedded Linux, with real-time extensions (RTAI). Such OS has been set up through the following procedure: •
•
•
Fig. 4. RTAI architecture
The embedded Linux OS has been created as a restricted version of Linux Red Hat 9.0 by selecting only those parts of the complete system that are strictly necessary to make it fully operative (e. g. the X window system has been tailored for our purposes, most of the services were not installed, etc.). The kernel of the embedded system has been updated to version 2.4.24. The kernel sources have been downloaded and compiled directly on the target machine running the full version of Linux Red Hat. The Real-Time extensions (RTAI) have been successfully applied to the embedded system: the appropriate RTAI patch has been applied to the original kernel sources, the patched kernel has been compiled and installed and finally the RTAI sources have been compiled and installed in turn as a kernel module.
RTAI is developed at the Dipartimento di Ingegneria Aerospaziale of Politecnico of Milan. Once applied to the Linux kernel, RTAI acts as an intermediate layer on top of the hardware (hardware abstraction layer – HAL). Such intermediate layer behaves mainly as an interrupt dispatcher, trapping the peripherals interrupts and, if necessary, re-routing them to Linux (see Fig. 4). It filters all the requests coming from real-time tasks and from the Linux kernel itself which are destined to the hardware. Because each real time task has a specific associated priority, requests coming from the various tasks are dispatched to the underlying hardware taking into account the associated task priority. The Linux kernel itself is treated as a special real-time task whose priority is the lowest one with respect to those of all other real-time tasks, and it executes only when there are no real-time tasks to run. A comprehensive description of RTAI is available4. 4.2. Advanced Control System (ACS) framework The ACS is a CORBA-based framework for the development of control systems and higher level data flow and coordination applications, developed by the European Southern Observatory (ESO) in collaboration with several institutes in Europe and USA5. The heart of ACS is an object model of controlled devices implemented as CORBA network objects. CORBA middleware provides the necessary infrastructure in exchanging messages among distributed objects, in a platform and language independent way and in an objectoriented context. Through the use of CORBA it is possible to automate many network programming tasks, such as registration, location and activation of objects. ACS uses several standard CORBA services such as notification service, naming service, interface repository and implementation repository. ACS hides all details of the underlying mechanism, which uses many complex features of CORBA, such as queuing, asynchronous communication, thread pooling, life cycle management, etc. In addition, ACS provides a powerful XML-based configuration database, synchronous and asynchronous communication, configurable monitors and alarms that automatically reconnect after a server crash, run-time name/location resolution, archiving, error system and logging system. Using standard CORBA middleware, ACS implements a Component/Container model that is language and platform independent. Any logical or physical device (e.g. power supply) in a control system is represented by a component. Component contains properties – entities that can be monitored and controlled, and characteristics which contains static data, such as name, units or description. Fig. 5. Embedded control system architecture
Containers written in C++, Java and Python manage the lifecycle of components implemented in the respective languages with the help of a centralized Manager, and provide them a very simple way to access common centralized services, at the same time hiding most of CORBA. The Manager is capable of deploying, managing the lifecycle and locating Components in appropriate Containers. Client written in any CORBA-aware language can access these Containers and Components, while the implementation of the server side in any other of these languages would be simplified. A full description of the ACS architecture is available5. 4.3. Abeans framework Abeans6 form an application and graphical user interface framework developed by the Cosylab Ltd. of Ljubljana. They allow simple development of complex applications, providing all the benefits of the underlying Java platform such as high speed graphics and easy installation and maintenance with Java Webstart. Abeans is a library that provides simple Java beans for connection with the control system. At the same time, it provides several useful services: logging, exception handling, configuration and data resource loaders, authentication and policy management. They also provide
clear and consistent visualization of dynamic data with standardized presentation of alarms and other events. Abeans act as mediators between the underlying remote data layer and top-level graphical user interface and decouple the presentation of data from the actual data sources. They use plugs to connect to different protocols. Through CORBA they can connect to ACS or some other middleware that uses it. At the time of writing this article the ACS and Abeans frameworks run on the PC/104+ platform, using the 10 GB hard disk with Linux-RTAI as operating system. The porting of these frameworks on the smaller flash storage unit (256 MB) is still in progress.
5. CONCLUSIONS At the Astrophysical Technologies Group of the Astronomical Observatory of Trieste an embeddable prototype control system has been set up (see Fig. 5, solid line). The choice of a PC/104+ platform and of a PMAC2A motion controller satisfies the requirements of a standard, low cost hardware environment for embedded applications. The software environment is based on free, open source software, namely the Linux operating system with real-time extension (RTAI) and CORBA, which provides the communication infrastructure in a distributed environment, guaranteeing flexibility and scalability to the control system. Moreover, the adoption of the Advanced Control System (ACS) and the Abeans provides the user with a common environment form the high level user interface down to the embedded device control, easing the learning curve for the control system operation, maintenance, and integration of new hardware and software components. Future plans will cover security and fault-awareness for the embedded control system. At the Cosylab Ltd. and the J. Stefan Institute of Ljubljana an alternative path has been followed, with the development of a commercial StrongARM single board controller (see Fig. 5, dashed line).
REFERENCES 1. 2. 3. 4. 5. 6.
“PC/104Plus Specification, Version 1.2”, PC/104 Embedded Consortium, 1060 North Fourth St. San Jose, CA 95112 August 2001, http://www.pc104.org “PMAC2A-PC/104 Hardware Reference”, Delta Tau Data System, 21314 Lassen Street Chatsworth, CA 91311, August 2002, http://www.deltatau.com “PMAC2A-PC/104 Installation Manual”, Delta Tau Data System, 21314 Lassen Street Chatsworth, CA 91311, January 2003, http://www.deltatau.com “RTAI Programming Guide 1.0”, Lineo, Inc., 390 S. 400 W. Lindon, Utah 84042, USA, September 2000, http://www.rtai.org G. Chiozzi et al., “CORBA-based Common Software for the ALMA project”, SPIE Advanced Telescope and Instrumentation Telescope Control Software II , SPIE 4848, pp. 43-45, Kona, HW, August 2002 I. Vertovsek et al., “CosyFramework: Application Development Framework for Java”, ICALEPS 2003, Gyeongju, Korea, October 2003
