BI-TRIGGER SYNCHRONIZATION METHOD TO ENHANCE THE ...

The Photogrammetric Journal of Finland, Vol. 21, No. 2, 2009

Received 22.8.2008, Accepted 4.12.2008

BI-TRIGGER SYNCHRONIZATION METHOD TO ENHANCE THE PERFORMANCE OF MOBILE MAPPING SYSTEM Yuwei Chen1, Constantin-Octavian Andrei1, Antero Kukko2, Ruizhi Chen1, Juha Hyyppä2 Harri Kaartinen2, Petteri Pöntinen3, Hannu Hyyppä3, Henrik Haggren3, Iisakki Kosonen4 1

Finnish Geodetic Institute, Department of Navigation and Positioning, P.O. Box 15, 02431 Masala, Finland 2 Finnish Geodetic Institute, Department of Remote Sensing and Photogrammetry, P.O. Box 15, 02431 Masala, Finland 3 Helsinki University of Technology, Department of Surveying, Institute of Photogrammetry and Remote Sensing, P.O. Box 1200, 02015 TKK, Finland 4 Helsinki University of Technology, Department of Civil and Environmental Engineering, Laboratory of Transportation Engineering, P.O. Box 2100, 02015 TKK, Finland [email protected] ABSTRACT Car-borne Mobile Mapping System (MMS) has become an irresistible trend used for transportation engineering, road survey and many other applications in the past few years. The Finnish Geodetic Institute (FGI) and Helsinki University of Technology (HUT) are now jointly developing a Mobile Mapping System named ROAMER. The system combines a terrestrial laser scanner, stereo cameras and a NovAtel SPAN system that integrates a tactical-grade Inertial Measurement Unit (IMU) and a Global Positioning System (GPS) receiver. All these sensors are synchronized to the GPS time via the NovAtel SPAN system. At a high driving speed, the synchronization pulses are emitted by the terrestrial laser scanner which then triggers the NovAtel SPAN system to log precisely the time-tags. Otherwise, the system performance would be degraded and the data availability would be reduced. The capacity of the NovAtel SPAN system to log synchronization pulses is 20 Hz under single port trigger mode whereas the laser scanner emits the synchronization pulses at 30 Hz as maximum. Therefore a bi-trigger synchronization method is designed, prototyped and tested. The bi-trigger approach separates the 30 Hz input pulses transmitted from scanner into two 15 Hz output pulses, which alternatively triggers the SPAN system. That enhances the capacity of NovAtel SPAN system to log the synchronization pulses up to 40 Hz and makes full use of its potential. The test results demonstrate that the approach can double the profile resolution of mobile mapping system and, therefore, it enhances the entire system performance by achieving an evener point distribution. Key words: Mobile Mapping System, Terrestrial Laser Scanner, Geo-reference, synchronizer, bi-trigger 1. INTRODUCTION Car-borne Mobile Mapping Systems (MMS) have been increasingly applied for a broad range of applications throughout the past years from transportation engineering and road survey (Hesse et al., 2006; Gräfe, 2006; El-Sheimy, 2005) to tourism (Manzoni et al., 2005). The state-of-the-art integrated MMS system is characteristics of fusing multiple sensors, providing sensor trajectory by using Inertial Measurement Unit (IMU) and GPS Real Time Kinematics (RTK), and inevitable

3

complex post-processing program. A typical car-borne laser scanner based MMS, also called as vehicle-based laser scanning, can be considered as a multi-sensor system that integrates various navigation devices and data acquisition sensors on a rigid, moving platform like a van or any other vehicle for determining the positions of the surrounding objects along the driving trajectory in a form of a helix. The integrated Global Position System/Inertial Navigation System (GPS/INS) provides the coordinates of the vehicle while the laser scanner measures the range from the scanner to the remote target with its reflectivity on laser band. Combined with the measurement of the angle coder of TLS’s scanning mirror, the coordinates of the target points can then be calculated. Precise measurements in accuracy of millimeter level are required for system calibration and data quality control. All these sensors in the MMS are synchronized to the GPS time via geo-reference system. High accuracy results require a precise synchronization of all sensors, otherwise the system performance is degraded and the data availability is reduced. Sorted with different synchronization frequency, the synchronizing methods are classified mainly into three types. The first method is the easiest synchronizing method, i.e. the Pulse Per Second (PPS) signal of the GPS receiver that suits for low frequency synchronization (≤1 Hz) (Tetsu et al., 2004). The second method is the most common method; the event marker that fits to medium frequency application (tens of Hz) (Cameron, 2001; Reulke et al., 2004; Pierre-Yves et al., 2001; Ellum, C., 2001).The third method, the centralized synchronization board method, is the best solution for high frequency application (hundreds to thousands Hz) (El-Sheimy, 1996; Xu et al., 2003; Xu et al., 2005) in which the GPS based clock guarantees the accuracy of the system. The ROAMER applies precisely the event marker synchronization method by solving the unmatched signal frequency between the output of the terrestrial laser scanner and the input of the geo-reference system. Therefore, a bi-trigger synchronization solution is designed, prototyped and tested. Accordingly, the performance of the car-borne mobile mapping system is enhanced by achieving a more uniform point distribution. 2. SYSTEM CONFIGURATION FGI and HUT has been prototyping a car-borne MMS, named ROAMER, which would acquire transportation data along the road and maximize the automation of feature extraction at the post processing phase since 2005. The system consists of a TLS sensor, backview stereo cameras and a geo-reference sub-system from Novatel, called SPAN (Synchronized Position Attitude and Navigation). That brings together two different but complementary technologies: GPS positioning and inertial navigation to extract the trajectory of the platform. The precise synchronization between geo-reference system and sensors is one of the major premises to assure the demanding accuracy of the MMS. Figure 1 presents the system configuration of ROAMER. For more detailed information, the reader is referred to (Kukko et al., 2007). The present prototype system applies FARO LS 880HE80 TLS operating at maximum 30 Hz profile rate. The laser scanner module combines a scanner with a 320° field of view (FOV) on profile direction. The laser beam moves in a plane at 120 k/s measurement rate and the outputs are the transversal profiles with a resolution of 0.06 degree and a precision below 3 mm for each measured distance.

4

Profile Direction

Stereo Camera

GPS Ant.

TLS Synchronizer

IMU Vehicle Direction

Stereo Camera

Figure 1. The Sensor pool of the FGI ROAMER. The SPAN system provides 3D position, velocity and attitude solution. Unlike GPS-only navigation systems, the solution is stable and continuously available by integrating a tactical-grade IMU, even through periods when GPS signals are blocked or under the full outrage. RTK mode enables the NovAtel SPAN system with millimeter scale position accuracy. (SPAN, 2005). Two AVT Oscar F-810C cameras are used to capture the backview stereo. The frame rate is up to 3.2 frames per second for full image size and 7.5 frames for external-trigger mode (AVT, 2007). 3. METHODS TO ENHANCE SYSTEM PERFORMANCE The along-track resolution of ROAMER is measured in meters under normal operation speed. Such disadvantage greatly impairs the system performance on extracting the thin traffic infrastructure such as traffic signs and traffic lights under normal work condition in which the speed of platform is about 20m/s. ROAMER applies the tilted scanner position to retrieve this problem (Kukko et al., 2007). Figure 2 illustrates how the tilted scanner position produces more echoes back from thin objects. That kind of installation brings also better point distribution than the vertical one by increasing cross-track resolution by a csc α factor, where α denotes the tilt installation angle. Conversely, the tilted position weakens the system performance on extracting the overbridges, due the fact that the base structure and tribatch of the instrument block the upward view.

5

Legend:

Vertical profile Tilted profile

P

STOP

Figure 2. Tilted position enhances the system performance on thin traffic infrastructure extraction. As it has been mentioned, ROAMER has sparse point distribution for the along-track resolution (1 meter at speed of 20 m/s at single port trigger mode at 20Hz) which is greatly lower than the cross-track resolution (2.6 mm to 15.7 mm from near field to far field). Defining RS as the ratio of the along-track resolution to the cross-track resolution of MMS, the RS of ROAMER is about 64~382. Such low point distribution degrades the system performance and impairs the system’s availability, which is not the case in airborne TLS that operates on hundreds meters height with less than 90o FOV normally. RS varies greatly from the near field to the far field in car-borne MMS system. One defines the near field RS as RS-n and the far field RS as RS-f. Accordingly, the RS-n is given by v Fp Cv R , (1) ≈ RS − n = a = Rc H × tg 2π 2πHFp2 C Fp while the RS-f is given by

RS − f =

v Fp

Cv Ra , = ≈ 2 π 2πFp2W Rc W × tg C Fp

(2)

where Ra is the along-track resolution, Rc is the cross-track resolution, v is the speed of the vehicle, C is the coverage rate of TLS, Fp is the profile rate of the scanner, H is the scanner height (to the ground), W is the horizontal distance between the scanner and target’s location on the road side. In order to obtain a uniformly-distributed point cloud, RS should be as equal to 1 as possible while the variations in interval between 0.1 and 10 are accepted in practice. Hesse and Kutterer (Hesse et al., 2006) recommended less than 10 cm along-track resolution for robust road infrastructure recognition and reconstruction. Although the uneven point distribution results on most prototype TLS based MMSs, which greatly impairs the availability of MMS, there are a few practical

6

solutions that are applied to improve the situation of bad point distribution: (1) Increasing the profile rate Fp is the most effective solution (Hesse et al., 2006), which has square relation to RS according to Equations 1 and 2; (2) Slowing down the vehicle’s speed is an acceptable solution, especially in empirical test (Hesse et al., 2006), but not suitable for practical road survey because a low speed (10-25 km/h) may cause an un-expectable dangerous situation to normal traffic flow on the road; (3) Installing the scanner as high as possible would increase the RS, but it is restricted by the environment and the platform in reality. Theoretically, such an unequal point distribution could be greatly improved by applying the following two approaches. First approach applies an array sensor with the parallel profile scan mode. Assuming that the n pixels line-array sensor is adopted, the RS is given by RS =

Cv . 2πn 2 Fp2 H

(3)

Obviously, RS is enhanced by n2 factor. This approach seems to be a practical solution for TLS design in further because lots of commercial array sensors are available nowadays (Hamamatsu, 2006). However, such a parallel scan mode is followed with corresponding optical beam splitter design (Chen, 2005), which increases the complexity of the TLS system. Second approach adopts the forward conical scan mode rather than the linear side-profile scan mode, which embraces three advantages: (1) Easy extraction of the thin traffic signs: the new scan mode catches more echoes from traffic signs since most of them face the driver, (2) 360o FOV: this approach solves the dilemma situation where the MMS operates along the road with both overbridges construction and thin traffic objects; (3) Evener point distribution: TLS scans the objects in a form of a forward helix that achieving lower cross-track resolution, which makes an evener point distribution in comparison with the linear profile scan mode. 4. BI-TRIGGER SYNCHRONIZATION MODE TO ENHANCE SYSTEM PERFORMANCE According to the illustration above, increasing the profile rate of the scanner is the most effective and practical solution to enhance the RS of ROAMER. In other words, in order to enhance the synchronization frequency as could as possible, which makes full use of the potential of geo-reference system and gets a better RS, a special trigger solution needs to be designed to resolve the unmatched signal frequency between the output of TLS and the input of SPAN system. The synchronizing pulses emitted by the scanner are inputted into the synchronizer as clock signals. The synchronizer applies to a bi-frequency divider to separate the input 30Hz clock pulses into two 15Hz pulses, called odd synchronization pulse and even synchronization pulse, then it triggers the geo-reference system by sending these two synchronization pulses to two event maker trigger pins of the I/O port of SPAN system alternatively. Meanwhile the synchronizer applies a quad-frequency divider to generate 7.5 Hz pulse to trigger the cameras. The approach increases Fp from 20Hz to 30Hz under GPS-only work condition and decreases RS from 64~382 to 28~171 under normal operation condition as shown in Table 1. Theoretically the Bi-Trigger mechanism can decrease RS to 16~96 if the TLS works on 40Hz profile rate, which reaches the SPAN system’s potential and moderate the bad point distribution of the ROAMER tremendously.

7

Table 1. Performance improvement by bi-trigger mode synchronization (GPS-Only). Along track resolution[cm]

Cross track resolution[cm] Near field (2.5m)

Vehicle 20Hza 30Hzb 40Hzc 20Hz 30Hz 40Hz 20Hz 30Hz Speed 5 m/s 25 16.67 12.5 (18km/h) 10m/s 0.26 0.39 0.52 1.57 2.36 50 33.33 25 (36km/h) 20m/s 100 66.67 50 (72km/h) a 20 Hz is the maximum frequency with single port trigger mode. b 30Hz is the frequency related to FARO LS 880HE80 with bi-trigger mode. c 40Hz is the bi-trigger-mode system’s potential.

40Hz

3.14

RS − f

RS − n

Far field(15m) 20Hz

30Hz

40Hz

20Hz

30Hz

40Hz

96.1

42.7

24.0

15.9

7.1

4.0

192.3

85.4

48.1

31.8

14.1

8.0

384.6

170.9

96.1

63.7

28.3

15.9

For the IMU-attached condition, the capacity of the SPAN system to log the synchronization pulses from each port is 5 Hz. In that case the synchronizer should apply a hex-frequency divider to generate the 5Hz from 30Hz clock, while such idea also works effectively to enhance the system performance. A test bench was set up to measure the SPAN system performance under both single trigger mode and bi-triggered mode before designing the hardware. In the test solution, an Agilent 33250 signal generator outputted one or two simulated synchronizing signals, and these simulated signals were sent into the event marker pins of the SPAN I/O (Input/Output) port. The logs were called MARKPOS and MARKPOS2 and they were recorded when the generator triggered the corresponding pins of the SPAN system. All the data from the GPS receiver was logged to a desktop through serial ports. The GPS got a stable RTK connection from the FGI server with a wide-band internet connection during the test and the RTK baseline length was measured in tens of meters level. Figure 4 shows the test solution system. Figure 3 presents the percentage of responded record against the frequency of inputted trigger for both single trigger mode and bi-trigger mode. As the frequency of imputed trigger signal increasing, the SPAN under the single trigger mode could not synchronized the inputted pulse, only 72.8% (43700/60000) of total trigger signal could be traced from the record. It is obviously to conclude that the SPAN could not catch up the inputted trigger signal since the frequency has exceeded the capacity of the port. While the bi-trigger mode still survived in this case since it separated the input trigger signal and trigger both event maker pins of the SPAN.

Figure 3. Percentage of responded record under single trigger mode and bi-trigger mode.

8

Figure 4. Test bench of bi-trigger mechanism experiment. The test solution collected the data through series tests which lasted minutes in a static mode. The trigger frequencies range was from 1 Hz to 20 Hz for each channel. In Figure 5 three subplots show the horizontal, vertical and 3D position errors of GPS antenna phase centre under 20 Hz trigger signals. The internal standard deviations of the solution along each axis are correspondingly 4.7 mm, 5.0 mm and 13.8 mm. It can be concluded that the standard deviation of the position is below 2.0 cm. Figure 6 presents the corresponding horizontal position accuracy. The test results show that the geo-reference system operates correctly with the bi-trigger input signal up to 20 Hz for each trigger channel, which doubles the system synchronizing resolution successfully and reaches the potential of geo-reference system. Positional error of GPS antenna phase center during the test

Error [m]

0.05

Horiz err

0.04 0.03 0.02 0.01 0

0

100

200

300

400

500

600

100

200

300

400

500

600

100

200

300

400

500

600

0.1

Error [m]

Vert err 0.05 0 -0.05 -0.1

0

0.1

Error [m]

3D err 0.075 0.05 0.025 0

0

T ime [sec]

Figure 5. Position error of RTK-GPS solution (20Hz).

9

Figure 6. Horizontal Position accuracy (20Hz). The core component of the synchronizer applies the complex programmable logic device (CPLD) with embedded firmware since the CPLD is more flexible to change if any new signal is needed in future. The firmware is written in VHDL (VHSIC Hardware Description Language), debugged and implanted within MAX PLUS II IDE (Integrated Development Environment). The simulating results are showed as the following Figure 7. Where the “clk” signal is the synchronizing pulse transmitted from TLS, while the “odd_syn” and the “even_syn” signals present the trigger signals to EVENT1 and EVENT2 pins of I/O port of SPAN system and the “camera_trigger” signal presents the trigger pulse to AVT Oscar cameras. It is clearly to conclude that the frequency divider and camera trigger’s functionality have been fulfilled successfully according to the simulation results. Based on the simulation result, the circuit of the synchronizer had been designed, prototyped and tested. And the first version of synchronizer with bi-trigger function has been applied to ROAMER for field surveying since 2008.

Figure 7. The simulating result Synchronizer (30Hz input and 7.5Hz camera trigger signal).

10

5. CONCLUSIONS The MMS method challenges traditional survey methods for road-side data collection by offering presumably comparable accuracy at higher speed, higher point spacing, lower costs and, to the last but not the least, lower security risks for both the surveyor and the driver. However, the uniformly-resolution point distribution impairs the practical applications and degrades the availability of the system. A few approaches are discussed to moderate such situation. The bi-trigger mechanism is the most practical method to retrieve that kind of condition, which not only doubles the system synchronizing resolution theoretically but also makes full use of the potential of the geo-reference system in this case, and should also be a suggestible design for other MMS solutions. 6. REFERENCES AVT Oscar Technical Manual V2.3.1 2007. Allied Vision Technologies GmbH. Chen Y., 2005. The Research on System Design of Pushbroom Earth Observation Laser Imager. Ph.D. Dissertation, Shanghai Institute of Technical Physics, Chinese Academy of Science, Shanghai China (in Chinese). El-Sheimy, N., 2005. An Overview of Mobile Mapping Systems. In: International archives of FIG Working Week 2005 and GSDI-8, Cairo, Egypt, 24 pages, http://www.fig.net/pub/cairo/ papers/ts_17/ts17_03_elsheimy.pdf, (15.10.2007). El-Sheimy, N., 1996. The Development of VISAT - A Mobile Survey System for GIS Applications. Ph.D. Dissertation, University of Calgary, Department of Geomatics Engineering, Alberta Canada. Gilliéron, P-Y., Skaloud, J., Brugger, D. and Merminod, B., 2001. Development of a Low Cost mobile Mapping System for Road Data Base Management. 3rd International Symposium on Mobile Mapping Technology, 3rd – 5th of January, 2001, Cairo, Egypt, 12 pages. Gräfe, G., 2006. Kinematic surveying with static accuracy. International Archives of Optical 3-D "Measurement Techniques VIII", Editors A.Grün and H.Kahmen, ISBN 3-906467-67-8, ETH Zürich, Switzerland, July 9-12, 2007,Vol. II, pp.142-149. Hamamatsu, 2006 Si APD array S8550 datasheet 2006, http://sales.hamamatsu.com/assets/pdf/ parts_S/S8550.pdf, (15.10.2007). Hesse, C. and Hansjörg, K., 2006. A mobile mapping system using kinematic terrestrial laser scanning (KTLS) for image acquisition. Proceedings of Optical 3-D "Measurement Techniques VIII",Editors A.Grün and H.Kahmen, ISBN 3-906467-67-8, ETH Zürich, Switzerland, July 9-12, 2007, Vol. II, pp. 134-141. Kukko, A., Andrei, C-O., Salminen, V-M., Kaartinen, H., Chen, Y., Rönnholm, P., Hyyppä, H., Hyyppä, J., Chen, R., Haggrén, H., Kosonen, I. and Čapek, K., 2007. Road environment mapping system of the Finnish Geodetic Institute - FGI ROAMER. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 36, Part 3/W52, pp. 241-247.

11

Ellum, C., 2001. The Development of a Backpack Mobile Mapping System. MSc. Thesis. University of Calgary, Department of Geomatics Engineering, Alberta Canada. Manzoni, G., Rizzo, R.G. and Robiglio, C., 2005. Mobile Mapping System in Cultural Heritages survey. International archives of CIPA 2005 XX International Symposium, Torino, Italy, 4 pages. http://cipa.icomos.org/fileadmin/papers/Torino2005/437.pdf, (12.10.2007). Ooishi, T. Yamada, K., Takeda, H. and Kawai, T., 2004. Development of Simple Mobile Mapping System for the Construction of Road Foundation Data. Archieve of XXth ISPRS Congress, ISSN 1682-1750, IAPRS, Vol.XXXV Part B2, pp. 67-72. Reulke, R., Wehr, A., and D. Griesbach, 2004. Mobile Panoramic Mapping using CCD-Line Camera and Laser Scanner with Integrated Position and orientation System. Proceeding of International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, ISSN 1682-1750 Vol. XXXIV, PART 5/W16, 6 pages, http://drops.dagstuhl.de/opus/ volltexte/2005/294/pdf/04251.ReulkeRalf.Paper.294.pdf.pdf, (12.10.2007). SPAN Technology System User Manual version 7, 2005 Novatel Inc. Xu, W. and Shu, R., 2003. Multi-sensor Synchronization Using GPS receiver for Airborne remote sensing application. Application of Electronic Technique, Vol. 6, No.10, pp. 55-58 (in Chinese). Xu, W., Wang, J., and Shu R., 2005. Synchronizer of multi-sensors for remote sensing application. Chinese Patent. No: CN200510026169.5 (in Chinese).

12