data rate we use RAMBUS instead of normal SDRAM chips and a RAID controller to store the pictures. 4.5 Illumination. For illumination a diffused infrared laser ...
Video based obstacle detection in catenaries of railways
H. Möller, B. Hulin, W. Krötz, B. Sarnes
Abstract The present paper describes the concept and first results of a system, capable to detect obstacles in catenaries as an onboard tool. To avoid a damage of the pantograph it must be retracted immediately after an obstacle is detected. The presented system is based on a triocular camera sensor system combined with smart image processing routines. The present paper describes the objectives of obstacle detection, realization concepts and its specifications, first. Then the system, the algorithms, the results and limits are presented and discussed.
1. Introduction The desire of European railways for a reliable and cheap obstacle detection system for catenaries is undisputed, because of obstacles (e.g. wooden knots, plastic films, crime acts etc.) and defects (e.g. down hanging, ripped off droppers, bondings, broken insulators, etc.) in catenaries of rail systems can result in long time delays and expensive damages. A system capable of retracting the pantograph automatically before colliding with an obstacle would avoid these damages and increase reliability, punctuality and therefore attractiveness of a railway system.
Presently there is no system available and only less research is done on this topic. Most of the applications of obstacle detection are in the area of automotive use (e.g. [1]) and only few publications exists that are related to obstacle detection for railroad use such as [2][3][4]. Only some activities are presently carried out for semi automatic video based diagnostics at catenaries for maintenance purposes [5][6]. But no publications of other groups are found which are explicitly dedicated to onboard obstacle detection in catenaries, although there are many significant differences which on the one hand require special algorithms but on the other hand enables simplifications which can easy lead to success. Therefore in the Research and Technology Center of Deutsche Bahn AG a demonstrator for obstacle detection in catenaries is presently under development.
2. Studies - Concepts
Concerning that obstacles and defects in catenaries induce high damage extensive pre-studies are performed by DB in order to evaluate the feasibility of different concepts.
2.1. Stationary system A stationary system based on vibration sensors in catenaries was tested. As results of this study we found out that stationary systems are impractical, because - it is too expensive to observe a 40000km large electrified net stationary - accuracy is low (e.g. it’s hard to distinguish between birds and obstacles) - it is only suitable for special obstacles and not for defects
2.2 Onboard systems Onboard systems are scanning the track about 50-100m in front of the pantograph. This gives enough time to retract the pantograph and avoid a collision. Different principles for that task are: Microwave Radar The resolution of microwave radars is too low for that purpose. Laser Radar The Laser radar concept has on the one hand a too small resolution for small obstacles. On the other hand it requires an extremely expensive laser scanner and moving mechanical components are incorporated which are always critical under operation at robust rail conditions. Concerning the high Laser power also safety criteria for human eyes are not easy to fulfil. Video based onboard systems with subsequent image processing algorithms No fundamental problems were found. Three cameras are necessary to fulfil this job reliably. Decreasing price of video technology and increasing functionality of the PC technology will promise a cheep system with a high functionality, flexibility, reliability and interoperability. Therefore we propose to develop a vision based three cameras obstacle detection system using PC’s and smart image processing algorithms.
The planned system and its requirements are described in detail in the subsequent sections.
3. Specifications The main feature of an onboard obstacle detection system in catenaries is that if an obstacle is detected the train has not to be stopped. Only the pantograph has to be retracted to avoid a collision with an obstacle. That requires a time of approximately one second. Assuming a processing time of 250msec and a train speed between 120 and 330km/h, the catenary has to be scanned about 35-115m before the pantograph. It must be mentioned that this distance is larger than the necessary surveillance distance. This is in dependence on the position of the pantograph on the train. For example: The first pantograph of ICE 3 - the high speed train of DB AG – is located on the second coach of the train. Therefore, even for highest speed, a surveillance distance of 80m is absolutely sufficient.
A schematic overview of this obstacle detection is given in figure 1.
Fig. 1: Proposed scheme for onboard obstacle detection on the rolling stock
In the context of the proposed obstacle detection system any object in the overhead contact system larger than 10cm is defined as an obstacle as soon as parts of it are positioned below the contact wire. In this case the object is in the so-called pantograph gauge. This height is limited to the overhead contact wire. This border is varying between 4.95m and 6.7m in Germany. Even collisions with small obstacles (i.e. at a size of 10cm) at overhead contact lines may cause big damages to pantographs and catenaries. Therefore the contact wire has to be determined by the software first, followed by a subsequent obstacle detection algorithm.
4. Hardware set up The hardware set up is based on a trinocular video camera onboard obstacle detection system.
Fig. 2: Block diagram of the hardware set-up
The reasons for the specific features are described below:
4.1 General aspects 1. The system is constructed with no mechanical, moving parts - especially important concerning rough conditions at railway vehicles. 2. Long live cycles of the components are expected. 3. It is capable to be a low cost system in the future.
4.2 Optics The scene is recorded by three progressive scan cameras (1024 x 1024 pel) in a distance of about 80m in front of the train. There are no special optical elements necessary. For the tests we use standard photo zoom objectives from Tamron (28-300mm). The cameras are mounted in a horizontal distance of 50cm on a torsion free bar in a height of about 2.5m (variable) above the track. The selection of the focal lengths and the position of the cameras are an optimization problem, determined by limiting conditions of the specific task:
Optimizing parameters and there interactions
1. Focal length: increasing focal length
decreasing focal length
increasing accuracy due to higher
stronger curves can be observed due to
resolution
increasing width of the visible area
increasing surveillance distance due to
light sensitivity of optic components
higher resolution
(objectives) is increasing
2. Height of camera position: near contact wire
near track
obstacles appears more clearly and
elements of the catenary appear more
separated of catenaries
clearly and are easier to detect soiling problems of sensors are increasing
Limiting conditions: -
possible location on trains
-
the location of pantographs
-
minimum of retraction time of pantograph
-
processing time of the computer system
-
maximum train speed
-
max. possible illumination distance
-
max. possible resolution of the camera chip
Therefore for each application the optimal operating parameters of the optics have to be determined.
4.3 Number of cameras Reasons for the use of a multi camera system are enumerated below: 1. Using more than one camera enables to reconstruct 3D Information from 2D pictures, necessary to distinguish if objects are in the clearance gauge of the pantograph or not. 2. Concerning staggering of the contact wire ( 0.55m at catenaries of DB) there exists no optimum of a horizontal position of the camera. Using 3 cameras the obstacle appears clearly in at least one camera pair. Fig. 3 shows the scene of three cameras. In
camera 1 and 2 the obstacle appears clearly while in camera 3 it is occluded by elements of the catenary. 3. Concerning obstacle detection it simplifies the stereo matching correspondence problem: finding corresponding objects in two images is not easy, because of the large baseline, the subsequent large disparity and therefore the different geometric appearance of the objects in the pictures of the cameras. Using three cameras can solve that problem in an adequate and fast way [7].
Fig. 3 View of the catenary and obstacles from the different cameras
4.4 PC System The data are transferred to two PC’s. One PC is used to store the images and the other is used for the image processing system. It is possible to transfer 10 frames / sec. from each image sensor to the PCs, therefore a data rate of max. 30 Mbytes / sec is necessary. To achieve this data rate we use RAMBUS instead of normal SDRAM chips and a RAID controller to store the pictures.
4.5 Illumination For illumination a diffused infrared laser beam (e.g. 820nm) shall be used in the future. This wavelength is not visible for the human eye but for standard video cameras it appears like daylight. We have not done many experiments with illumination yet and the hardware is only used at daylight scenes.
4.6 Results - Hardware Fig. 4 and 5 shows the outside and inside view of the hardware set up in the coach:
Fig. 4: Outside view of the coach, equipped with the obstacle detection system
Fig. 5: Inside view of the coach, equipped with the obstacle detection system
This system was tested up to a speed of 200 km/h. No problematic blur related to vibration modes were found. The frame rate is controlled in dependence on the train speed from 0.1 frames / sec to max. 10 frames / sec. Therefore in middle every 5.5m a frame is generated and stored. This is absolutely sufficient because of the visible range in one image is more than 30m. Therefore an obstacle appears in a sequence of images. This will increase the reliability of the system significantly.
5. Software architecture
In this chapter an overview of the obstacle detection algorithm will be given. This algorithm is not principle knowledge based. That means that there are no a priori data of tracks, catenaries or obstacles necessary. The decision if an object is an obstacle or not is influenced
by the height of the contact wire. Objects below and in a range next to the contact wire could damage the pantograph. Therefore the contact wire has to be determined as an important reference line. Concerning staggering, variation of height, and appearance (many other catenary elements look similar to it) of the contact wire an algorithm has to be developed which is capable to detect it precisely and automatically.
5.1. Contact wire detection Therefore we developed an algorithm based on Helmholz shears [8]. To use this the scene has to be recorded at least with 2 different located cameras. Then the images are warped in that manner, that only elements in the plane of the contact wire, appears in the same position in both pictures. After that only the contact wire is in the same position in the images of the different cameras and can then be abstracted as the darkest object – crossing the image from up to down - easily. For that algorithm the plane of the contact wire is necessary; it can be found online by methods combining possible heights – determined form pixels of arbitrary objects. One of them must be the contact wire. This is carried out in combination with information from previous pictures, which increase the speed and reliability of the detection. A detailed description of the algorithm can be found elsewhere [9][10].
5.2 Obstacle detection Once the pixels belonging to the contact wire are found other pixels can be tested to be parts of obstacles. To check the electrical clearance gauge of the pantograph the actual location in 3 dimensions of objects is constructed by matching at least two two-dimensional images of the cameras. A first approach is to consider only objects hanging on the contact wire as obstacles. It should be mentioned that the algorithm for finding obstacles is – unlike the contact wire identification- still in a very early stage and must be developed in detail.
The software architecture is summarized in fig 6.
Fig.: 6: Concept of software architecture for obstacle detection
5.3. Results - Software Using this method the contact wire can be detected (in daylight scenes) with a reliability of 80%. In another 16% the computer knows that the contact wire is not found in an image pair – which is no problem due to the overlapping of images mentioned above. The only crucial situation occurs when the computer falsely detects the contact wire and has actually found another object (4%). Using even a first approach of an obstacle detection algorithm, simple obstacles can be detected at the moment.
Fig. 7 shows the promising results.
Fig. 7: Detected contact wire (blue), elements of the catenary (green) and obstacles (orange) after image processing.
6. Conclusions and future aspects In the present paper we presented a concept and first results of an obstacle detection system. It shows that obstacle detection is on the one hand a very difficult topic. On the other hand it shows that there are possibilities to built up such a system, capable for the demands of railway use. The results of the chapter software shows, that it is possible for a smart algorithm to detect obstacles automatically. On the other hand it shows that much work has to be done in the software development to increase the reliability of the system. The next step is the generation of extensive statistic studies and testing on track after the implementation of optimised contact wire and obstacle detection algorithms. Concluding we have showed the demands for obstacle detection of railways and demonstrated the feasibility using an optical onboard system.
References
[1] T. Kashihara et al. Development of a road obstacle sensor combining image processing and a laser radar. Int. conf. on intelligent vehicles, Sumitomo Electric Industries, Ltd., (Japan ), 1998. IEEE.
[2] Joan Aranda et al. Obstacle detection in railroads using adaptive masks. Int. Conf. on intelligent vehicles, UPC (Spain), 1998. IEEE.
[3] H.C. Liesenkoetter, Obstruction detection for people movers operating on conventional small branch railways. Int. conf. on intelligent vehicles, 1998, Germany, IEEE.
[4] W. Enkelmann. An obstacle detection system for automatic trains. In WCRR’97, 4th World Congress on Railway Research, Florence, Italy, Volume C, pages 411–417., Nov 1997.
[5] P. Pohl; Der Hagener Video-Messtriebwagen. Der Eisenbahningenieur, 24–38, 6 1996 und 32-35, 7 1996
[6] U. Richter, R. Schneider ; Automatische optische Inspektion von Oberleitungen, Eisenbahningenieur (52), 2/ 2000, pp.18-23,
[7] M. Yachida, et al., Trinocular Vision : A new Approach for corresponding Problem, Proc. of Conference on Pattern Recognition, Paris, IEEE, pp1041-1044, 1986
[8] H. v. Helmholtz, Handbuch der Physiologischen Optik, Vol. 3, Verlag von Leopold Voss, Hamburg, 1910
[9] Video based onboard surveillance of the catenaries of railways , B. Hulin, M. Pfarrdrescher, B. Sarnes, W. Krötz and H. Möller, submitted to 2nd European workshop on advanced video-based surveillance systems, 2001 , London, UK
[10] Stereo recognition of the contact wire of railways for obstacle detection, B. Hulin and H. Möller, Proceedings of 6th Int. Conf. Pattern Recognition and Information Processing, Minsk 2001, ISBN 83-87362-37-9, Vol. 1, Chap. 7, pp.275-279
