USING RTLinux TO INTERCONNECT FIELD BUSES - CiteSeerX

International Conference on Manufacturing Engineering (ICMEN), Thessaloniki, Greece, 2002.

USING RTLinux TO INTERCONNECT FIELD BUSES: THE PROFIBUS CASE STUDY K. Thramboulidis, P. Parthimos, G. Doukas Electrical & Computer Engineering Dept. University of Patras, 26500 Patras, Greece

ABSTRACT Many different proprietary field buses supporting different standards have been installed in industry in different time periods under different circumstances. Working in the domain of distributed control applications we have defined the CORFU framework to address the interoperability issue and facilitate the development of distributed control applications. In this paper, a prototype implementation of the CORFU Interworking Unit using Real-Time Linux is presented. The Profibus was adopted as a case study and a wrapper was developed to abstract it to the IEC 61499 compliant Virtual Field Bus. The Rtnet, an RTLinux protocol stack was adopted and extended to provide TCP connectivity to fulfil the requirements imposed by the configuration management of field buses. KEYWORDS: Field bus, Profibus, IEC 61499, RTLinux, RTNet, Virtual Field Bus 1. INTRODUCTION For the development of industrial automation and process control information systems, the control engineer has to consider computer networks since they are an essential component of their architecture. Field buses are intelligent/smart transmitter protocols that ensure very reliable and self-correcting data communication. They provide control, alarm, trend and other services distributed to its devices. With their deterministic nature, field buses let system architects distribute a process over the network even for applications with hard real-time requirements. A wide variety of field bus networks are available in the market, each one with its own special capabilities on the different levels of the hierarchical structure of Industrial Manufacturing [1]. These networks come from many vendors, support different standards and target particular market requirements around the world. Among the most famous field buses are: Profibus, Fiedbus Foundation, CAN, WorldFIP, P-NET and LonWorks. In spite of the technological advances, process control industries lack the harmony that other areas of industry have. The final effort for an interoperable standard among the field buses, which was carried by the ISA'S SP50 standards committee, resulted in an 8-standards standard [2]. To address the interoperability issue recent standards like IEC-61499 [3] and IEC-61804 [4] have adopted the Function Block (FB) concept as the main building block for the development of distributed Industrial Process Measurement and Control Systems (IPMCSs). IPMCS applications can be defined in terms of logically connected function blocks that run on different processing resources. Complete applications, can be built from network of function blocks, formed by interconnecting their inputs and outputs. The IEC standards also define a development process in a format that is independent of implementation and the underlying field buses, and can be easily understood and applied by industrial engineers who are usually not specialists in the implementation of complex algorithms. Adopting the above standards, we were guided to the definition of the CORFU framework that facilitates the

1


development of distributed IPMCS applications [5]. This framework is mainly composed of the Virtual Field Bus (VFB) and the Industrial Process-Control Protocol (IPCP). For the prototype implementation of this framework we have defined the architecture of the CORFU Interworking Unit (CIU) [6]. The CIU is used to interconnect each field bus to the backbone, which is used to satisfy the real-time requirements imposed by the control application. We have considered ATM and fast switched-Ethernet for the real–time interconnection of Profibus and Lonworks and we have developed the corresponding test beds. In this paper, after a brief description of the CORFU framework, we present the wrapping of the Profibus to the IEC 61499 compliant VFB, and the use of RTLinux [7] for the development of CIU. For the wrapping of Profibus, we had to port the Softing’s Profibus Linux driver to RTLinux. For the implementation of the IPCP, the RTNET, an RTLinux protocol stack was utilized. However, since RTNET does not support TCP but only UDP, we had to develop a tiny TCP layer on top of IP to fulfil the requirements for TCP connectivity that are emanated from the configuration management of field buses. This last feature enables the CORFU framework to provide the necessary infrastructure that is required by modern Engineering Support Systems (ESSs) [8]. This paper is organized as follows: In the next section we briefly refer to CORFU framework. In section 3, we describe the architecture of the interworking unit and focus on these components that cover the operational phase of the control application. In section 4, we present the first draft implementation of the wrapper of the profibus to the VFB level. The architecture of the wrapper is presented and implementation issues are discussed. In section 5, the TCP implementation for the Rtnet protocol stack is presented and finally, in the last section, we conclude the paper. 2. THE CORFU FRAMEWORK As far as, field devices are interconnected with the same field bus, the distribution of the IPMCS application in terms od FBs, is a simple task. Data and event connection between FBs are mapped to the specific’s field bus communication mechanisms and real time constraints are satisfied by the field bus nature. However, things are going to become more complicated, when the application’s FBs must be distributed among devices assigned to different field buses of the same or different type. There is also a requirement, from the corporate level, to have integrated monitor and control of industrial processes, which are assigned to different field bus segments, through the enterprise’s intranet. The obvious solution to the above problem is to use Interworking Units (IU) to interconnect each field bus segment with the enterprise intranet. Although this solution, address the requirements for the integrated monitoring in corporate level, it does not satisfy the requirement for real-time interconnection between field bus segments. To satisfy this requirement, we were guided to adopt the network topology shown on figure 1.

Figure 1. Network topology for the CORFU Framework.

2


The selected communication subsystem, which is used as backbone, must provide the quality of service required to meet the timing requirements. ATM is one of the successfully used technologies for the interconnection of fieldbuses. However, to satisfy the key requirement of the simplicity of the communication system as well as the low cost of the equipment, Fast switched-Ethernet was also selected. One port of the Ethernet switch is dedicated, as is shown in figure 1, to connect the IPMCS to the enterprise intranet where part of the system’s HMI is supposed to run. Our approach to use IUs between each field bus and the backbone leaves unchanged the already defined legacy applications on each field bus and only requires the extra effort for the interconnection of FBs assigned to different field bus segments. This would result, in the least effort that the enterprise should spend over the configuration of the new system. On top of this network topology we have defined the layout of the CORFU framework for the development and operation of distributed IPMCS applications [6]. For the IU to be able to interconnect the many different commercial field buses to the selected backbone, a modular and highly expandable architecture has been defined. This architecture satisfies the need for the real-time interconnection among field bus segments as well as the need for the similes development of IPMCS applications over heterogeneous field buses. It also allows a uniform access form the corporate level through the many commercially available OPC-compliant SCADA client systems [5]. 3. THE ARCHITECTURE OF THE CORFU INTERWORKING UNIT The main components of the CIU that cover both the configuration and the operational phases of the system are: the VFB, the IPCP protocol and the wrapper, which undertakes the task of wrapping the specific field bus to the VFB. The VFB module, which is used to abstract any commercial field bus, to the IEC 61499 level so as the interoperability in field bus level may be achieved, is composed of: the Real-Time Entity (RTE), the Configuration Management Entity (CME) and the SCADA server, as is shown in figure 2. The CME provides the services required by the Engineering Support System for the configuration of IEC 61499 compliant field buses. The RTE is responsible for the real time interconnections between VFBs. For the implementation of the VFB module, a wrapper layer is defined to adapt to the specific field bus API. The architecture is complemented by the definition of the IPCP protocol that is implemented on top of the TCP-UDP layers.

Figure 2. Architecture of the CORFU Interworking Unit.

3


When an event occurs in the industrial process the wrapper is responsible to notify the Virtual Field Bus about it. This is done through two Real-Time FIFO queues; one of them is for high priority events while the other is for low priority. The VFB handles the events of the two FIFOs through the handle_event() function. This function discriminates the type of event and handles it appropriately by use of the two tables that correspond to event connections and data connections. These tables provide mainly the mapping of the events and data of the industrial process to the IPCP connections. The VFB interfaces with the IPCP with two real-time FIFOs to handle the two levels of priorities. The VFB passes to the IPCP, the connectionId of the corresponding event of the wrapper, along with a data buffer that contains the remainder of the attributes of the event. The IPCP resolves the connectionIds of the incoming events to the destination address of the target VFB, through the outConnectionTables. The pseudo code of the related post_event_with() function of IPCP is given in figure 3. The function creates the related IPCP packet and then it forwards it to the User Datagram Protocol (UDP). post_event_with(ConnectionHandler conId, IPCPData *dataBuf) { char * buff[IPCP_BUF_MAX_SIZE]; IP rem_ip; int rem_port; /* creates IPCP packet */ append connection to buff; append data to buff; rem_ip = outConnectionTable[conId].ip; rem_port = outConnectionTable[conId].port; len = outConnectionTable[conId].data_len + sizeof(conId); /* sends packet to UDP */ rt_udp_sendmsg(out_sock, rem_ip,rem_port,buff,len); }

Figure 3. Pseudo code of the post_event_with() function of IPCP. 4. WRAPING THE PROFIBUS TO THE IEC COMPLIANT VFB For the wrapping of Profibus to the VFB abstraction level we selected Profibus that is one of the most widely used field buses. For the installation of the two required Profibus segments we had in our disposal profibus slaves by Siemens and two PROFIboard ISA profibus master cards by Softing. Each Profibus segment should have at least one master card that is responsible for implementing the communication mechanism of the Profibus. For a more efficient implementation of the CORFU framework, all of the modules that constitute the CIU must be RTLinux modules to avoid the lack of determinism that is introduced by Linux processes. However, since an RTLinux version of the Softing’s driver is not available we had to undertake the task of porting the Linux version. Softing, kindly provided as with the source code of the PROFIboard linux driver v.1.2.32, for kernel software v. 2.2.x. Even though this driver is still under development, an ALPHA version is available that covers all the functionality needed to access the Profibus. A restriction of the driver is that it implements a hardware driver only and not a protocol one, so we have to deal with communication protocol ourselves and this means that we are responsible for sending requests and waiting confirmations while configuring and controlling the communication protocol. The PROFIboard driver supports the following 5 devices: • Board device: This is used for reading and writing the PROFIboard firmware. • Service devise: This device is used for configuration and control of the profibus services (FMS, DP, etc.) that the PROFIboard provides. It can be opened either in blocking or non-blocking mode.

4


•

•

Data device: This provides access to the PROFIboard DualPort Ram (DPRam). It supports 2 operational modes: normal and optimized. In normal mode a read call to the data device returns immediately with the requested data. In optimized mode a read call to the data device will only return immediately, if the data image had been changed since the last successful read call. Slave Input device, Slave Output device: These devices support the use of PROFIboard as a profibus slave input/output.

Porting the Profibus driver to RTLinux It is possible to use a Linux driver from RT-Linux, by registering a new set of entry points to the well-known Posix IO interface functions like read, write, open, close, ioctl, etc. [9]. These new entry points should be registered using the function rtl_register_chrdev() that is provided by RTLinux and may call portions of the Linux driver since the Linux kernel and RTLinux share the same address space. Although this seems a simple task, one must be very careful when calling from RTLinux tasks, functions written for Linux. Some actions, common in Linux programs, must be avoided from RT tasks. So RTLinux tasks should never: • Try to allocate or free memory, using functions like kmalloc and kfree. • Try to invoke the virtual memory manager. • Make calls to functions that may block. • Use printk or similar IO functions. • Use synchronization objects shared with Linux kernel. A first look in the source code of the driver version we have in our disposal shows, that many of the above restrictions are already satisfied. This makes the porting of the BROFIboard driver to RTLinux an achievable task. However to make things easier, we completely ignore the existence of the last 2 devices mentioned above and we concentrate on the service and data devices, which seem to be crucial for the operation of the test bed system. The Profibus wrapper The wrapper should provide services that concern both the configuration of the field bus and the correct and efficient transmission of real-time data and event from/to the field bus, so that various industrial control loops could be closed. It is obvious that the wrapper constitutes a basic part of the RT-path and therefore particular attention should be given while designing and implementing it. The architecture of the Profibus wrapper is given in figure 4. According to it the wrapper’s input path includes the following actions: 1. Read the field bus data and events. For the profibus case, read the PROFIboard data device. 2. Update the value of the items in Fieldbus_Item_Table_IN. 3. Convert the fieldbus-items that were updated, to IEC-items and write the corresponding values to the IEC_Item_table_IN table. For items that are declared as events, appropriate messages are send to RTE through the event fifo. The wrapper’s output path includes the following actions: 1. Read the IEC_Item_Fifo_OUT real-time fifo. This fifo is used from the RTE to pass IEC items to the wrapper. 2. Convert IEC-items to fieldbus-items and update the Fieldbus_Item_Table_OUT 3. Write the contents of the Fieldbus_Item_Table_OUT to the field bus. For the profibus case write to PROFIboard data device. By item we denote any data or event item that passes through the system and can be considered as input or output of a function block in the VFB level. The source of an item may be a sensor device or an intermediate item and the destination may be an actuator, of the same or another field bus segment. The RTE of the VFB has direct access to IEC_Item_Table_IN since RTLinux tasks share the same address space. IEC_Item_Table_IN is actually the exported DCT or just a part of it.

5


VFB RTE

CME

IEC_Item_Table_IN

Event Fifo

IEC_Item_Fifo_OUT Wrapper Configuration Manager

Fieldbus_to_IEC item converter

IEC_to_Fieldbus item converter Item Description & Fieldbus Configuration tables Fieldbus_Item_Table_OUT

Fieldbus_Item_Table_IN

read_Fieldbus

Fieldbus Wrapper

write_Fieldbus

Fieldbus Driver

Figure 4. The architecture of the Fieldbus Wrapper. The efficient operation of the wrapper is based on the detailed database (Item description & field bus configuration tables) that is constructed during initialization. This database holds all the information needed, to locate and retrieve items from the field bus and to handle them appropriately. The wrapper’s Configuration Manager is responsible of filling in these tables and configuring the field bus. We next refer to the initialization of the PROFIboard with more details: As communication protocol with profibus, we have selected to use DP since it is the most frequently used communication profile. DP is optimized for speed, efficiency and low connection costs and is designed especially for communication between automation systems and distributed peripherals. After opening service device and data device, we activate the DP protocol in class 1 and set all bus and protocol parameters. We continue by initializing the DP Master. In this step, the Address Assignment Mode (AAM) of the DPRam is set to IO-BLOCK. In this mode, the assignment of DPRam addresses to slaves is done automatically and there's no need of providing any offsets while locating DP slaves. After the configuration of the slaves, we can obtain the location of each slave in DPRam by sending an appropriate request to DP. The drawback of IO-BLOCK mode is that it uses a fixed DPRam layout so it is impossible to change slave configuration dynamically. Finally we configure all the slaves, by downloading specific information about them. The configuration is done using FMB protocol layer and through the service device of the driver. For our first implementation and in order to make things simple, we decided to implement the wrapper as a Linux process, but as close as possible to a stand-alone RT task version, so that it can be easily transferred to RTLinux when the porting of the driver is completed. The alternative to implement the wrapper as a RTLinux task, communicating, through rtfifos, with a simple linux process which makes the direct calls to the driver was rejected as more complicated.

6


5. ADDING TCP/IP TO RTLINUX Rtlinux comes with no networking capabilities. Even more we found only one protocol stack implementation, the RTNET one, which only offers UDP functionality. Rtnet is a separate implementation of networking protocols, for use with real-time Linux extensions. Rtnet provides direct access to IP-based networking from real-time code. Rtnet implements the IP, ARP, UDP, and ICMP protocols over ethernet. It also provides a sockets implementation for use by real-time tasks [10]. Rtnet was made from standard Linux networking source code, with the necessary changes to make it real-time. These changes add limitations among which we discriminate the following: • The user is unable to simultaneously share the network with existing Linux networking • Cannot be used by user-space applications • Drivers need to be converted from Linux to Rtnet • TCP/IP is not available For the last issue of the above list the Rtnet’s documentation states “TCP/IP is not supported, because it is not real-time, and there is no clear way to force it to be real-time.” However we are using Rtnet over switched Ethernet, which has no collisions and can support real-time communication. TCP was made to serve communication over many types of different networks and over the Internet. It has the capability to correct most types of errors that the underlying network may insert to the communication. These include lost, late, duplicate, disordered and destroyed packets and others. TCP is unable to provide real-time communication over a network of this type but no protocol seems to be able to do so. If we examine TCP over a network that guarantees real-time and reliable communication, we will notice that TCP itself does not insert any unpredictable delays. We can determine the time that TCP needs to process and forward an amount of data it receives. UDP is ideal for our real-time communication but we need a standard, reliable protocol for system configuration and monitoring at run time, as TCP. So we decided to extend Rtnet to support TCP. We utilized a small, embedded version, a miniature TCP/IP protocol stack implemented from scratch [11]. The advantage of it is that it is hardware-independent as well as operating system-independent, which makes it easier for insertion into the Linux kernel. The disadvantage is that it does not follow the OSI layering at all. All the networking code must be compiled with the application and it runs from its context. This made it difficult to isolate the TCP in order to insert it to Linux. Since TCP runs over IP we had to find a way to make out TCP layer to interface with the IP layer of the Rtnet. The task was not so easy, since no documentation could be found to describe the way that this could be done. We spent much time studying Rtnet source code trying to figure the way we could use Rtnet IP services from TCP. Looking at Rtnet’s UDP source code we managed to find the way it uses the IP services and to replicate it with the necessary modification for our TCP layer. Figure 5 shows the architecture of the extended RtNet module. Integrating our TCP layer into Rtnet includes: • Registering TCP into IP to receive data • Receiving data from IP • Sending data to IP

7


Figure 5: Architecture of our RTLinux TCP/IP protocol stack Registering TCP into IP to receive data. We used the rt_inet_add_protocol() function provided by Rtnet to register TCP. Register means providing the TCP protocol number, so that IP knows what datagrams to forward to TCP, and the rt_tcp_rcv() function that handles these packets. This function is called from the IP context every time we have a datagram, with the datagram as a parameter. The rt_tcp_init() function that is given below is used to register TCP. this function void rt_tcp_init(void) { rt_tcp_socket = rtsocket_alloc(); rt_tcp_socket->deliver = rt_tcp_rcv; /*********/ rt_tcp_socket->proto = IPPROTO_TCP; /* =6 */ rt_inet_add_protocol(rt_tcp_socket); } int rt_tcp_rcv(struct rtsocket * sk,struct sk_buff *skb) { #ifdef TMESSAGES rtl_printf("*****Packet received*****\n"); #endif tcp_storage[0] = skb; pthread_wakeup_np(tcp_thread); }

return 0;

Receiving data from IP. Since rt_tcp_rcv() runs within the IP context, it cannot include a lot of actions. It just passes to TCP a pointer to the datagram and wakes the TCP thread. Rtnet uses a struct called skbuff also used in Linux, to store network packets and pass them between protocols [12]. Skbuff is a very complex structure with many fields and is handled using special functions which are also complex, since skbuffs have many capabilities and can create lists and FIFOs. We did not utilized all of these capabilities since Lean uses its own structure that holds datagrams. However, understanding the structure of skbuffs took us some time. Sending data to IP. 8


The rt_ip_build_xmit() function is provided from IP for sending data to it. However, the use of this function is too complex. We would expect it to be called with source and destination IP and an skbuff holding the TCP packet. Then it should add the IP header and send it. Things are not so simple though. Source and destination are put into an rt_rtable struct using the rt_ip_route_output() function. We pass this struct and we also pass a void pointer and a function we call getfrag(). IP will make a datagram, add its header and then call our getfrag function giving it the void pointer and a pointer to the end of the IP header in the datagram. The void pointer points to a struct that holds the beginning of the TCP packet and its length. The getfrag uses it to copy the TCP packet after the IP header. Doing this is tricky but it was not easy to find the way out by reading the source code. int rt_tcp_getfrag(const void *pass, char * to, unsigned int offset, unsigned int fraglen) { struct IPpass *p = (struct IPpass *)pass; TCPKT * tcp = (TCPKT *)(to -20); if (offset != 0) { rtl_printf("Zois: Error. Ip required fragmentation.\n"); return 0; } memcpy(to,&p->tcp->t,p->len); tcp->t.check = 0; tcp->t.check = ~check_tcp(tcp, tcp->i.sip, tcp->i.dip, p->len); print_packet(tcp); return 0; } int put_frame(GENFRAME *gfp, int len){ struct IPpass p; struct rtsocket * sk = rt_tcp_socket; int ulen = len; struct ipcm_cookie ipc; struct rt_rtable * rt = NULL; u32 daddr; u32 saddr; int err; TCPKT * tcp = getframe_datap(gfp); ... err = rt_ip_route_output( &rt, daddr, saddr, 0, 0); ... p.len = len; p.tcp = tcp; err = rt_ip_build_xmit(sk,rt_tcp_getfrag,(void *)&p, ulen,&ipc,rt, 0); ... return err; }

We decided not to insert TCP into the Rtnet module. We created a separate module that uses Rtnet’s services ao the changes we made to Rtnet’s source code are minimal. We just exported a couple of functions. This way, we have two advantages. We can use our TCP with future versions of Rtnet with no additional effort and we have no risk of inserting any bugs to Rtnet itself that could make it unstable or limit its real-time performance. Our TCP is implemented as a single thread. It is usually sleeping and it wakes up when a packet is received or when one must be sent. It also runs periodically to check and handle timeouts. Data transfers between applications and the TCP take place within the application context but control returns immediately and all packet-handling processes take place within the TCP thread.

6. CONCLUSIONS 9


Currently, there is a great interest in the automation industry for distributed architectures, especially for these that give solutions to the problem of interoperability. In this paper, we presented the CORFU framework that is our approach for interoperability and a prototype implementation of the CORFU interworking unit for the interconnection of two independent profibus segments. Our approach attempts to facilitate the interconnection of two or more distant field buses over a common backbone. We selected Profibus as a case study and we proceed to the development of the wrapper that is required to abstract the profibus to the IEC compliant VFB. We used new technologies such as fast-switched Ethernet and RT-Linux and we examined how they perform in such environments. The final implementation was designed to have all of its processes in RT-Linux but this resulted in a lot of work for the profibus driver to be ported to the RTLinux and the TCP layer to be added to the Rtnet protocol stack. Working in this context proved to be very difficult since the only documentation was the source code. 7. ACKNOWLEDGMENTS The Authors would like to thank Sofing GmbH (Germany) for the source code of the Linux driver and Aslanis Stefanos and Chris Tranoris for their help in profibus understanding. 8. REFERENCES 1. Jim Pinto, "Fieldbus - Conflicting "Standards" Emerge, but Interoperability is Still Elusive", Design Engineering magazine, UK, Oct.99 2. Jim Pinto, "The great Fieldbus debate is Over!", proceedings of Industrial Controls Intelligence Nov.1999 3. IEC Technical Committee TC65/WG6, “IEC61499 Industrial-Process Measurement and Control – Specification”, IEC Draft 2000. 4. IEC SUB COMMITTEE No. 65C: DIGITAL COMMUNICATIONS, WORKING GROUP 7: FUNCTION BLOCKS for PROCESS CONTROL , “IEC1804 General Requirements”, IEC Draft 1999. 5. K. Thramboulidis, “Development of Distributed Industrial Control Applications: The CORFU Framework”, 4th IEEE International Workshop on Factory Communication Systems, Sweden, August 2002 6. K. Thramboulidis, C. Tranoris, “An Architecture for the Development of Function Block Oriented Engineering Support Systems”, IEEE International Conference on Computational Intelligence in Robotics and Automation, Canada August 2001. 7. Victor Yodaiken, “real-time Linux, A short position paper”, http://www.rtlinux.org 8. Κ. Thrampoulidis, “Towards a UML based Engineering Support System”, 9th IEEE Mediterranean Conference on Control and Automation, MED'01, Croatia 2001. 9. Alex Ivchenko, “Real-Time Linux”, Embedded.com, http://www.embedded.com/story/OEG20010418S0044 10. Lineo's OpenSource Development Repository Project Page, http://opensource.lineo.com/projects.html 11. Jeremy Bentham, TCP/IP Lean,Web Servers for Embedded Systems, CMP books 2000. 12. RT-net Documentation. Documentation/rtnet/rtnet.txt from the rtnet directory. 13. Jon Crowcroft, Iain Philips, “TCP/IP and Linux protocol implementation”, John Willey, 2002.

10