because different vendors vary in their interpretation of the standards. This paper ... layers of automation systems are increasingly dissolved, leading to more ...
Using automatic Topology Discovery to diagnose PROFINET networks Michael Jäger
Roman Just Oliver Niggemann
inIT - Institut Industrial IT Hochschule Ostwestfalen-Lippe University of Applied Science Lemgo, Germany {Michael.Jaeger, Roman.Just, Oliver.Niggemann}@hs-owl.de
Abstract Due to an increasing degree of automation, industrial communication networks grow in complexity and become more vulnerable to errors. Therefore a reliable network diagnosis is getting more and more important. Although modern industrial communication networks, such as PROFINET, have basic diagnosis mechanisms, they do not cover complex error cases. Furthermore, our work shows that a diagnosis of most error cases requires an automatic detection of a network’s topology. In general, PROFINET does support topology discovery by providing the necessary protocols. However, implementing a reliable automatic topology discovery is not a trivial task e.g., because different vendors vary in their interpretation of the standards. This paper presents our approach for discovering the topology of a heterogeneous network. Further, a first approach for using the topology discovery for a high-level PROFINET diagnosis is shown.
1
Key to the identification of such errors is knowledge about the current network topology. This knowledge allows for the identification of incorrect networks topologies or helps to interpret the relation between different error symptoms. So any high-level diagnosis will comprise two main steps: 1. The automatic identification of the current network topology and 2. the usage of this information to identify network errors. In this paper our approach is demonstrated using the popular PROFINET protocol (see PROFIBUS & PROFINET International (PI)). While PROFINET supports—in theory— means for topology discovery, this paper will outline several problems (and corresponding solutions) for heterogeneous networks, i.e. networks comprising components from different manufacturers. This is subject of section 3. In section 4, some first examples for high-level diagnosis functionalities are given. These diagnosis functionalities, of course, leverage the topology discovery algorithm mentioned above. Section 5 gives the conclusion and the outlook.
Introduction 2
Industrial communication networks become more and more complex. This increasing complexity is due to several reasons: (i) The plant complexities are increasing. (ii) Plant control is more often implemented by means of distributed control networks. (iii) The traditional vertical layers of automation systems are increasingly dissolved, leading to more intelligent PLCs (Programmable Logic Controllers), i.e. the networks traffic between PLCs and other IT systems increases. All these developments lead to a high demand for a high-level diagnosis of networks. Unlike build-in network diagnosis functionalities such as the detection of bit errors or connection interruptions, such high-level diagnosis functionalities focus e.g. on incorrect network configurations, on sporadic errors, or on the degradation of the network performance.
State of the Art
By default, PROFINET integrates a diagnostic concept based on alarms . However, this diagnostic concept covers only basic error cases [13]. Approaches for high-level diagnosis are hardly to find, e.g., in [9] T. Keane and H. Kaghazchi suggest using the functionalities of a PROFINET IO-Supervisor to retrieve diagnostic information. In general, topology discovery can be applied on three different levels: the link layer (layer 2), the network layer (layer 3) and the overlay network [6]. The subject of this paper is to discover the link layer topology, further details regarding the network layer and the overlay layer are given by B. Donnet and T. Friedman in [6]. Building a map of layer-3 topology is easy because routers must be explicitly aware of their neighbors, therefore most of the previous research focuses on discovering
layer-3 topology, e.g. [4], the Internet Mapping Project [1], or by means of active probing in [5]. Breitbart et al [3] introduced a method for discovering the link layer topology. Most protocols for discovering layer-2 topologies are vendor specific: Cisco Discovery Protocol, Extreme Discovery Protocol, Cabletron Discovery Protocol and Nortel Discovery Protocol . The initial algorithm introduced by Breitbart et al [3] relied on standard information stored in the SNMP Management Information Bases (MIB) [11]. However the algorithm required complete address forwarding tables (AFT) and switched domains consisting of single switched subnets. Based on Breitbart’s work, Lowekamp et al [10] developed a topology discovery algorithm which does not necessary need complete address forwarding tables. In [2], Bejerano et al criticize that previous attempts are either mainly theoretical, hard to implement or have high computational complexity. Hence Bejerano developed a novel algorithm for discovery of large heterogeneous Ethernet LANs with support for multiple subnets and dumb/uncooperative elements (e.g. hubs). The research described above deals with layer-2 topology discovery in multi-vendor-networks. However, in our work we discuss topology discovery in PROFINETnetworks. PROFINET makes topology discovery easier because it implements the vendor-neutral Link Layer Discovery Protocol (LLDP). In [14], Schafer and Felser introduce an approach for topology discovery in PROFINET networks. Their method is based on LLDP and SNMP. A similar approach is used in Hirschmann’s management tool, HiVision [8]. However this approach focuses on all Ethernet based devices, while our approach focuses primarily on PROFINET devices. PROFINET is the industrial ethernet standard developed by PROFIBUS International (PI). PROFINET is standardized in IEC 61158, IEC 61784 and based on IEEE 802.3. In PROFINET, the Link Layer Discovery Protocol (LLDP, standardized asIEEE 802.1AB-2009) is used to exchange information about directly connected devices on layer 2. This Information is stored in a LLDP Management Information Bases (MIB). The the simple network management protocol (SNMP) is used to query the MIBs. The reader may note that only class B and C devices support LLDP and SNMP; mainly these device classes are used in the field of automation. The Discovery and Configuration Protocol (DCP) must also be supported by each PROFINET device. DCP is used for the name/address resolution and for discovering the IP addresse of all connected devices [13]. While [14] and [7] focus mostly on LLDP, [13] states that a topology can be created by combining DCP and LLDP. Nevertheless, no detailed description of an automatic topology discovery algorithm is given.
3
Topology Discovery Algorithm
3.1 Overview PROFINET provides an automatic topology information exchange (LLDP). Thus, LLDP is supposed to serve as the foundation of topology discovery[13]. But almost no detailed descriptions of a precise algorithm for automatic topology discovery can be found in literature (see also section 2). Furthermore, several problems become apparent when a solution is implemented: Problem: Some class B and C devices do not support the LLDP MIB, i.e. topology relevant information can not be retrieved. Solution: If a device does not support the LLDP MIB, the other directly connected device is used to identify device connections. Problem: SNMP requires the IP address of a certain device to access the MIBs; often these IP addresses are unknown or must be taken from previous planing steps, i.e. manually. Solution: This problem is solved by using DCP to query all PROFINET devices for identity information (IP address is included). Problem: Within LLDP MIBs, different device identity types (MAC, NameOfStation) are used within the same MIB. With LLDP, the same identity types must be used for local and corresponding remote MIB entries. But several LLDP implementations fail to comply to this rule; leading to non-comparable MIB entries. Solution: To overcome this problem, for each device all identity information are collected and used to identify a specific device. This helps to cope with erroneous LLDP implementations. Problem: Within LLDP MIBs, entries are stored in an hierarchical database. Despite a standardization of the MIB structure, in practice different vendors vary in their interpretation of this standard. Solution: Here, all possible devices from different vendors have been acquired and the variations in the format are taken into consideration. In the following, the steps of the resulting algorithm are presented (see Figure 1 for an overview). 168.192. ..
168.192. .. 168.192. ..
168.192. ..
168.192. .. 168.192. ..
168.192. ..
Step 1: Collecting IP addresses by DCP
Step 2: Identifying local neighborhood by analyzing LLDP-MIBs via SNMP
168.192. ..
168.192. .. Step 3: Merging nodes to create the topology graph
Figure 1. Overview of the algorithm. Step 1 - IP Addresses are collected (using DCP) Because SNMP requires the IP address of a device, the IP address of each PROFINET device is identified by using
the DCP protocol. As a result, this step finishes with a collection of IP addresses that are used by SNMP in the next step. Step 2 - Topology information are found (using SNMP/LLDP) In the second step, the algorithm uses SNMP to retrieve local topology information from PROFINET devices. For this, SNMP accesses the LLDP MIB, which consists of entries modeling a device’s local neighborhood, i.e. it models the device itself and all other directly connected PROFINET devices. Step 3 - Topology graph is created In this step, a topology graph of the network is created. This representation of the network can be thought of as a graph where each device corresponds to a node. Based on the information contained in the MIB of each device, the connections among nodes are figured out: For this, the local information of each node v0 is compared to the neighbor information of each other node v1 . When v0 matches another node v2 in v1 ’s neighbor information, the nodes v1 and v2 are merged and v0 and v1 are therefore considered as neighbors. This process is repeated for every present node, resulting in a complete graph representing the network’s topology. 3.2 Some Experiments In order to test the algorithm’s viability in a heterogeneous environment we used several test setups. Our test setups consist of an IO-Controller, several managed switches and several IO-Devices from different manufactures like SIEMENS, PHOENIX CONTACT, WAGO, BECKHOFF and HILSCHER. Each of these devices are conformance class B devices which support SNMP and LLDP. We performed experiments using the devices in various topology arrangements, e.g. star topology, line topology and tree topology. Example Test Setup For demonstration purposes an example test setup will be outlined in the following. The real world test setup in Figure 2 shows a switch connecting three devices of different manufactures. In addition, two devices are linked with one of the switch port peers, expanding the setup into a tree topology. Our presented
Figure 2. Real System Setup.
algorithm currently operates on a standard PC, also connected to the considered network. After executing the discussed algorithm steps (see section 3.1) a graphical representation is generated by means of the graphviz tool(see Figure 3). et200s
fl-switch-mcs-16tx
wago-750-370
nxio-50repns-01
nxio-50repns-02
nxio-50repns-03
Figure 3. Example Topology Graph. Each node is labeled with the PROFINET specific NameOfStation while the edges have no label, since they are only indicating the connections among devices. In case of this rather small setup, the algorithm worked properly, but some error occurred during further tests. Statistical Overview In further tests we experimented with 81 topologies (see Table 1) arranged as line, star or tree. Referring to the results in Table 1, overall 66.7% of the 81 topologies could be fully recognized. Errors mostly occurred for two devices with missing LLDP MIBs or due to nonstandardized LLDP MIB entries.
4
Diagnosing PROFINET Networks
For most error cases, its necessary to have a precise map of the network topology in order to detect or locate errors. As an example, two typical PROFINET related error cases are explained which require high-level diagnosis and knowledge of the network topology. Initially, an error case which can occur during commissioning, that cannot be detected by standard PROFINET diagnosis mechanisms is chosen: Case 1: As described in the PROFINET Installation Guidelines the cascading depth of the network components is limited due to performance reasons, though a specific number is not mentioned. One advantage of PROFINET as it is based on Ethernet, non PROFINET devices e.g. Office Computers can share the same infrastructure. PROFINET distinguishes in this case between Real-Time (RT) and non Real-Time (NRT) traffic. As a feature of PROFINET, this mixed setup does not affect the network performance because PROFINET frames (RT) are prioritized. However, the combination of high cascading depth and high amount of NRT traffic leads in certain cases to disturbance/interruption of the RT communication. So a diagnosis is implemented as follows: Symptom: High frame drop rate for some IO devices d1 , . . . , d n Diagnosis Algorithm: (i) automatic topology discovery (see section 3),
Topology structure Line Star Tree
Table 1. Experiment results Topologies tested Topologies fully recognized 13 10 7 7 61 37
(ii) identification of d1 , . . . , dn in the topology, (iii) check whether d1 , . . . , dn are positioned in a high cascading depth (see Figure 4) high cascading depth
shown in percentage 77% 100% 60,66%
explanation of the topology discovery algorithm’s steps and their implementation. Tests were conducted to proof the algorithm’s functionality. Therefore, we used devices from various manufactures arranged in several topologies, to set up a heterogeneous test environment. Furthermore, first diagnosis functionalities are described. In a next step, further errors will be diagnosed and further protocols are considered.
References high cascading depth
Figure 4. Case 1: Diagnosing high cascading depth using automatic topology discovery.
Figure 4 shows a topology discovered by (i) and processed as in (iii) . Further, two paths indicate a high cascading depth. Case 2: PROFINET devices can be arranged in several topology structures, such as line, tree, star and ring. A ring topology is used to increase availability [12]. However, at least one device has to make sure that the ring logically remains a line [12]. This device-mechanism is controlled by the PROFINET redundancy manager [12]. Hence, it is important that the user sets up the redundancy manager during commissioning. Otherwise the ring configuration will lead to a broadcast storm causing a reduction of the network throughput, until 100% of the bandwidth is used. So a diagnosis is implemented as follows: Symptom: High frame drop rate for some IO devices d1 , . . . , dn ; No establishment or loss of Communication Relations (CR) between IO-Controller and IO-Devices d1 , . . . , dn Diagnosis Algorithm: (i) automatic topology discovery (see section 3), (ii) identification of d1 , . . . , dn in the topology, (iii) check whether two devices are connected in a ring structure
5
Conclusion & Outlook
In this paper we introduced first approaches for a PROFINET high-level diagnosis, i.e. for diagnosis functionalities not supported by the standard PROFINET protocol. Key to these diagnosis functionalities is a specific topology discovery algorithm, based on the protocols DCP, LLDP and SNMP. This paper covered a detailed
[1] The internet mapping project, 2010. Last accessed on 2010-05-17. [2] Y. Bejerano. Taking the skeletons out of the closets: A simple and efficient topology discovery scheme for large ethernet lans. Networking, IEEE/ACM Transactions on, 17(5):1385 –1398, oct. 2009. [3] Y. Breitbart, M. N. Garofalakis, C. Martin, R. Rastogi, S. Seshadri, and A. Silberschatz. Topology discovery in heterogeneous ip networks. In INFOCOM, pages 265– 274, 2000. [4] H. Burch and B. Cheswick. Mapping the internet. Computer, 32(4):97 –98, 102, apr 1999. [5] L. Colitti. Internet Topology Discovery Using Active Probing. PhD thesis, UNIVERSITA DEGLI STUDI ROMA TRE, Roma, 2006. [6] B. Donnet, U. Catholique, D. Louvain, T. Friedman, U. Pierre, M. Curie, and Cnrs. Internet topology discovery: a survey. In In IEEE Communications Survey and Tutorials, 2007. [7] P. M. Felser. Topologie-Erkennung in Profinet. [8] Hirschmann. http://www.hivision.de/, 2011. Last accessed on 2011-07-11. [9] T. Keane and H. Kaghazchi. Retrieval of diagnostic information from profinet networks. In Emerging Technologies and Factory Automation, 2007. ETFA. IEEE Conference on, pages 708 –711, sept. 2007. [10] B. Lowekamp, D. O’Hallaron, and T. Gross. Topology discovery for large ethernet networks. In SIGCOMM ’01: Proceedings of the 2001 conference on Applications, technologies, architectures, and protocols for computer communications, pages 237–248, New York, NY, USA, 2001. ACM. [11] D. Mauro and K. Schmidt. Essential SNMP. O’Reilly & Associates, Inc., Sebastopol, CA, USA, 2001. [12] R. Pigan and M. Metter. Automating with PROFINET: Industrial Communication Based on Industrial Ethernet. John Wiley & Sons, 2008. [13] M. Popp. Industrielle Kommunikation mit PROFINET. PROFIBUS Nutzerorganisation e.V., 2007. [14] I. Schafer and M. Felser. Topology discovery in PROFINET. In Emerging Technologies and Factory Automation, 2007. ETFA. IEEE Conference on, pages 704 –707, 25-28 2007.
