Statistical Modeling of Long-Range Drift in Visual ... - Semantic Scholar

Statistical Modeling of Long-Range Drift in Visual Odometry Ruyi Jiang1 , Reinhard Klette2 , and Shigang Wang1 1 2

Shanghai Jiao Tong University, Shanghai, China University of Auckland, Auckland, New Zealand

Abstract. An intrinsic problem of visual odometry is its drift in longrange navigation. The drift is caused by error accumulation, as visual odometry is based on relative measurements. The paper reviews algorithms that adopt various methods to minimize this drift. However, as far as we know, no work has been done to statistically model and analyze the intrinsic properties of this drift. Moreover, the quantification of drift using offset ratio has its drawbacks. This paper models the drift as a combination of wide-band noise and a first-order Gauss-Markov process, and analyzes it using Allan variance. The model’s parameters are identified by a statistical method. A novel drift quantification method using Monte Carlo simulation is also provided.

1

Introduction

Visual odometry uses camera(s) to incrementally calculate a robot’s motion between frames and to position the robot in all six degrees of freedom in a 3D world. Compared with other positioning sensors (e.g., odometry, GPS, IMU and so forth), visual odometry has its own characteristics. Classical odometry, installed on a robot’s wheel axis, is usually deceived by wheel slippage, especially in an outdoor environment. GPS is not always available for navigation, due to signals being missing or jammed. A typical example is the successful application of visual odometry for NASA’s MER missions [4]. Also, compared to GPS and IMU, cameras in visual odometry are relatively cheap. Visual odometry has been widely applied in many fields, such as driver assistance or autonomous driving [11], simultaneous localization and mapping (SLAM), helicopter navigation [10], and underwater navigation [6]. Many algorithms have been tested to implement visual odometry using monocular [13] or stereo [10, 14], perspective or omnidirectional [13] cameras. A popular framework for visual odometry is based on feature matching and tracking [11, 14]. While considering that a feature-based method is sensitive to systematic errors due to intrinsic and extrinsic camera parameters, appearance-based visual odometry uses the appearance of the world to extract motion information (e.g., [13]). Recently, a direct method was also tested for visual odometry with very accurate results [5]. Among all these algorithms, one intrinsic problem of visual odometry is its drift in long-range navigation. The drift is caused by error accumulation, as

2

Ruyi Jiang, Reinhard Klette, and Shigang Wang

visual odometry is based on relative measurements. Relative motion matrices between frames are concatenated to produce the final position. Small errors in these matrices accumulate during this process to a large amount, and the distance measurement drifts from its real trajectory after some long-time navigation. For feature-based algorithms, the sources for these small errors are mainly uncertainties of feature localization and triangulation. Section 2 reviews algorithms that adopted various methods to minimize this drift. Section 3 models the drift as a combination of wide-band noise and firstorder Gauss-Markov process, and analyzes it using Allan variance, named after David W. Allan [1]. The identification of model parameters using statistical methods is also introduced. Experiments and discussions are provided in Sections 4 and 5.

2

State of the Art

Before proceeding to drift-minimization algorithms, we discuss at first a method to quantify drift. Currently, the offset ratio (OR), ratio of the final drift value to the traveled distance, is the common choice to measure the drift when running a visual odometry algorithm over some distance, from tens or hundreds of meters to several kilometers. (Drawbacks of OR, and a better quantification method are discussed later in this paper.) Note that the following review of algorithms is not for visual odometry, but for drift-minimizing methods adopted in these algorithms. It has been proved that integrating visual odometry with other positioning sensors, such as gyro or GPS, minimizes the drift. But this is not the problem to be discussed in this paper. As visual odometry alone has its practical and theoretical meaning, the paper analyzes drift without any help from other sensors. A “fire wall” was inserted into sequences by Nist´er [11] to act against error propagation. With fire walls, relative poses, estimated before the application of the fire wall, only affect the choice of the coordinate system for subsequent poses, and relative poses after the fire wall are estimated as if the system was starting again. It is supposed that fire walls suppress the propagation of gross errors and slow down the error buildup. From the provided experimental results, visual odometry had an accuracy [compared to the ground truth from a Differential Global Positioning System (DGPS)] of 1.07%, 4.86% and 1.63% for three outdoor runs with traveled distances of 185.88, 266.16, and 365.96 meters, respectively. Bundle adjustment is another scheme that can be adopted to suppress error accumulation. It is widely used for solving the off-line structure and motion problem, and the SLAM problem. Full bundle adjustment is almost impossible for on-line long range navigation, as there is a huge number of poses and features to be optimized. A sliding-window sparse bundle adjustment was applied by S¨ underhauf [14] for visual odometry. A subset of several images (the number is fixed, or adaptive to the motion vector) is continuously selected to perform bundle adjustment. Experiments on simulation show that sparse bundle adjustment slows down the drift. Though many papers on visual odometry use various methods to suppress the drift, no work has been done to explicitly model the drift. Clark et al. [12]

Statistical Modeling of Long-Range Drift in Visual Odometry

3

Fig. 1. Position drifts after running a visual odometry algorithm (using ABSOLUTE algorithm, see the experiment section), with the same simulated motion vectors for the same time steps. Note that with the simulated motion vectors as the ground truth, camera’s pose can be estimated using visual odometry algorithm. It can be seen that drift values can be quite different. This results into incapability of the offset ratio.

analyze the contribution of position and orientation errors to the overall drift, and observed that the drift does not grow linearly in the distance traveled, but 3 super-linearly. The growth was regarded as O(dist 2 ), but no specific models and parameters were provided. As a new and promising sensor, visual odometry needs a methodology for systematic and comparative analysis of its drift, in order to quantify the performance of various algorithms. For this purpose, OR has its drawbacks. First, drift does not increase linearly with the distance traveled, as stated by Clark et al. [12] and further proved in this paper. Thus, OR, when running algorithms on some distance, changes with the travele different distances. Moreover, running the same algorithms on the same dataset repeatedly produces quite different ORs. The reason is that drift is a random process, and it does not always increase, but sometimes also decrease at some places, as errors in different motion vectors compensate to some extent. Thus, using end-point values (the final drift value, and the final traveled distance) is inappropriate to model the whole random process. An example is shown in Fig. 1. Considering these findings, a more accurate quantification method is introduced later.

3

Drift Model, Analysis and Quantification

In this paper, coordinate frame transformations are used to represent both poses and motions. Using general notations, a pose E is the transformation from the

4

Ruyi Jiang, Reinhard Klette, and Shigang Wang

world coordinate frame into that of the camera, and a motion M is a transformation of the coordinate frame of the camera at time t into that at time t + 1. Drift in orientation is limited to the range [−π, π], and it contributes to the drift in position; this paper only considers positional drift. We consider positional drift in world coordinates along x−, y−, and z−axes separately, and use the x−axis as an example in the following modeling and analysis. The concatenated camera pose at time t is Et , and the estimated motion from time t to t + 1 is Mt . Then Et+1 = Et · Mt . Note that the right multiplication with Mt is because the motion Mt is relative to the camera coordinate frame at t. The general structure of E and M is of the form [R3×3 T3×1 ; 0 0 0 1]4×4 , where R3×3 is a rotational matrix, and T3×1 = [x, y, z]T is a translational vector. Then the translational drift dxt+1 in x− direction is equal to dxt+1 = xt+1 − x ¯t+1

(1)

where xt+1 and x ¯t+1 are the estimated and the true position in x−direction at time t + 1, respectively. Drift Model. As drift increases unboundedly for an assumed unlimited time, modeling and analysis of drift is confined within a limited time region. This matches the analysis of drift for inertial sensors. For a limited number of time steps, the drift {dxi , i = 1, 2, . . . , N }, as established in Eq. (1), is modeled in discrete form as dxi = ωni + bi (2) where ωn is zero-mean wide-band noise with variance σn2 , and b is a first-order Gauss-Markov process. This process is given by bi = (1 −

1 )bi−1 + ωa τ

(3)

where τ is a constant called correlation time, and ωa is the driving noise modeled as zero-mean wide-band noise with variance σa2 . The variance of the GaussMarkov process σb2 equals 2 1 σb2 = σa2 /( + 2 ) (4) τ τ The model of Eq. (2) has been widely used as a static stochastic error model in drift analysis for inertial sensors [15]. In this paper we show that this model can easily also be introduced into drift analysis for visual odometry; we reveal some important findings. Identification of Drift Model Parameters. The parameters for the drift model of Eq. (2) are σn2 , τ and σb2 , and they can be estimated from experimental data using various identification techniques. Parameter σn2 can be specified as the value of the Allan variance corresponding to 1 second averaging time [15]. Parameter σb2 is the variance of experiment data after removing the high frequency components.

Statistical Modeling of Long-Range Drift in Visual Odometry

5

The time constant τ can be estimated from experimental autocorrelation data. This is because the first-order Markov process has an autocorrelation known as Rb (T ) = σb2 e−T /τ (5) ¯ b (T ) (normalization means R ¯ b (0) = 1), we For the normalized autocorrelation R −1 ¯ have that τ = T when Rb (T ) = e . In this way, the time constant τ can be estimated as being the value of T corresponding to the normalized autocorrelation value 0.368 (i.e., e−1 ). For an example of parameter identification, see the experimental section. Drift Quantification by τ and σb2 Using Monte Carlo Simulation. As σn is usually small and stable, the time constant τ and the variance of the Gauss-Markov process σb2 , from the drift model as established by Eq. (2), are characterizing parameters of drift. Parameters τ and σb2 of a drift sequence can be estimated as specified above. However, it can be expected that drift sequences, while running the same visual odometry algorithm, produce different τ and σb2 values. In order to model drift properties of a specific visual odometry algorithm, Monte Carlo simulation is used. Monte Carlo simulation is a technique that propagates uncertainties in input variables of a model into the output, depending on the given probability distributions. The procedure for parameter identification for a specific visual odometry algorithm is as follows: 1. Generate a random motion vector for every frame in visual odometry. Possible motion vectors are restricted by real situations. 2. Use the generated motion to simulate feature registration. 3. Estimate motion and concatenate for N frames. 4. Calculate τ and σb2 . 5. Run Step 1 to 4 repeatedly (say, n times). This procedure provides n possible τ and σb2 values. An analysis of these values, using some statistical methods (e.g., histograms, accumulated statistics, confidence intervals, and so forth), finally illustrates the distribution of τ and σb2 for a specific visual odometry algorithm. Drift Analysis Using Allan Variance. The Allan variance is a method of analyzing a time sequence to pull out the intrinsic noise in the system as a function of the averaging time. It was originally developed to analysis the stability of clocks, but can also be adapted for any other type of outputs. Full details of the construction of Allan variance can be found here [15]. A typical application of Allan variance is the analysis of navigation errors for inertial sensors [15]. Here, we use Allan variance to analysis the drift and validate the drift model as established above. A special capability of the Allan variance is that noise types can be identified by matching different curve slopes in an Allan variance chart. The different slopes on the plot indicate the unique time regions dominated by the sensor’s specific noise. As only wide-band noise and a first-order Gauss-Markov process

6

Ruyi Jiang, Reinhard Klette, and Shigang Wang

are considered, the specific curve slopes for time regions dominated by these two kinds of noise are -1/2 and 1/2 respectively. For the correspondence of other kinds of noise with some slope values, we refer to [15].

4

Experiments

Experiments are conducted to illustrate the validation of the established drift model, as well as the stability and robustness of the new drift quantification method. Moreover, some important facts of the drift in visual odometry are also revealed from the experimental results. Simulated data is more preferable than real data for drift analysis, as simulation controls the source of error (e.g., feature location uncertainty for featurebased visual odometry) and removes outliers, which are common for real data. We simulate a sequence of stereo pairs. Only feature-based visual odometry algorithms are considered, as they are more general compared to appearance-based and direct visual odometry, and they are also easy to be controlled by the errors in the estimated motion matrix. No feature matching and tracking failures are considered in the simulation; thus, robust regression is not adopted to remove the outliers. We apply a similar scheme as used by Badino [2] for generating matched and tracked features. Two typical feature-based visual odometry algorithms were implemented, to illustrate the behavior of drift. The first one, named ABSOLUTE here, estimates the motion matrix between frames as an absolute orientation problem. Motion matrices are directly concatenated to estimate camera poses. Thus, drift is not suppressed, and is expected to be large. The second algorithm, named SBA here, uses a sliding window sparse bundle adjustment to optimize the motion matrices as estimated by the ABSOLUTE method. The whole implementation is similar to the one reported by S¨ underhauf [14]. The number of features, tracked for both algorithms, is set to be 200, and the feature localization uncertainty is 0.5 pixel. For SBA, the number of stereo frames for bundle adjustment equals five. We assume that the stereo frames are taken one second apart in the following. Drift Analysis Using Allan Variance. This experiment illustrates the use of Allan variance for visual odometry drift analysis. Some facts can be observed from the Allan variance. Both cameras move the same way. For every simulation instance of running the algorithms, different drift curves are obtained. Results for ABSOLUTE and SBA algorithms are shown in Figs. 2 and 3. Though every running of ABSOLUTE algorithm gives a sequence of different drift, it produces a similar Allan variance chart. The main curve slope in Allan variance chart (Fig. 2, right) produced from ABSOLUTE algorithm is 1/2, which means that the noise modeled as first-order Gauss-Markov process dominates the whole time region. In other words, the estimated camera trajectory will obviously drift from its real one since the first step. For SBA there is an important point (see Fig. 3, circled point on the right), which separates the curve with slope -1/2 from the curve with slope 1/2. It

Statistical Modeling of Long-Range Drift in Visual Odometry

7

means that the dominating noise in the first ten seconds (the x−coordinate of the point) is wide-band noise, while for the time after ten seconds, noise modeled as a first-order Gauss-Markov process will take over. In our situation (a one second image sampling interval, five frames bundle adjustment), it states that among ten frames, the drift behaves as wide-band white noise, while for more then ten frames, first-order Gauss-Markov noise dominates.

Fig. 2. Drift analysis using Allan variance. Left: Five drift instances from running the ABSOLUTE algorithm for 500 time steps. Right: Allan variance of these five drift instances. Note that the vertical bar is for the uncertainties.

Fig. 3. Drift analysis using Allan variance. Left: A drift instance from running the SBA algorithm for 500 time steps. Right: Allan variance of the drift instance. The point circled by red is the point where the Allan variance curve changes slope from -1/2 to 1/2. The coordinate of this point is (10, 0.005908).

Drift with Static Camera. In this experiment we illustrate the behavior of visual odometry with cameras being static. The drift values (along the x−axis) from both the ABSOLUTE and SBA algorithm are shown in Figs. 4 and 5. It can be seen from Fig. 4 that ABSOLUTE has a large drift value, which is similar to the situation when the stereo pair is under motion. Also the drift error dominates the whole time region, which is illustrated by the Allan variance chart with a dominating slope of 1/2. The SBA suppresses the drift to a small value even after a long time. The Allan variance in Fig. 5 reveals that the main error source in the drift value is random noise, with no obvious drift. Drift Model Parameter Identification. Here we discuss a real example to illustrate the identification of drift model parameters σn2 , τ and σb2 . A drift sequence from the ABSOLUTE algorithm is used, as shown on the upper left of

8

Ruyi Jiang, Reinhard Klette, and Shigang Wang

Fig. 4. Drift with static camera using ABSOLUTE visual odometry. Left: Drift value along x−axis for 500 time steps. Right: Allan variance of the drift value.

Fig. 5. Drift with static camera using SBA visual odometry. Left: Drift value along x−axis for 5,000 time steps. Right: Allan variance of the drift value. Note that the number of time steps here is larger than that in Fig. 4, to illustrate the behavior of drift for long time.

Fig. 6. The whole procedure is similar for drift value estimations for other visual odometry algorithms. Parameter σn2 can be estimated from the Allan variance calculated from raw drift values. As shown in the lower left of Fig. 6, the Allan deviation σy , corresponding to one second, equals 0.139. Thus, σn2 = 0.1392 ≈ 0.019. After filtering the raw drift with a low-pass filter to remove the high frequency components caused by wide-band noise, σb2 can be estimated as the variance of the filtered drift. In this example, the filtered drift is shown on the upper right of Fig. 6, and σb2 = 18.8. The time constant τ is the time value corresponding to the normalized autocorrelation value 0.368. In this example, the autocorrelation curve using filtered drift is shown in blue; see lower left of Fig. 6. The model fitted curve by the expected autocorrelation function as Eq. (5) is shown in red; see also lower left of Fig. 6. Only the first 150 time steps are used to fit the model. As the time value for normalized autocorrelation 0.368 is 92 (the marked point; lower left of Fig. 6). Drift Quantification Using Monte Carlo Simulation This experiment calculates the parameters τ and σb2 for the ABSOLUTE algorithm. The number n of Monte Carlo iterations equals 1 000. The Monte Carlo simulation results are shown in Fig. 7.

Statistical Modeling of Long-Range Drift in Visual Odometry

9

Fig. 6. An example of drift model parameter identification. Upper left: The raw drift value for 500 time steps. Upper right: Drift values after low-pass filter. Lower left: Allan variance. Lower right: Autocorrelation values. Note that the blue curve is the raw autocorrelation value using the drift values as shown in the upper right, while the red curve is the model-fitted autocorrelation for the first-order Gauss-Markov process.

Fig. 7. Monte Carlo simulation results for ABSOLUTE algorithm. (Left: τ . Right: σb ). Note that the horizontal axis is the simulation number.

The analysis of the Monte Carlo simulation results can be conducted using histogram, summary statistics, and so on. For the results in Fig. 7, the mean of τ is about 94.8, while the mean of σb equals 27.6. It can be seen from this example that the parameters (τ and σb ) to quantify the drift for a specific algorithm take much more factors into account than offset ratio. Moreover, offset ratio is only for running of a visual odometry algorithm once on a trajectory, which means it will always change for different trajectory with various length. While, the new quantification method using Monte Carlo simulation provides an overall evaluation of the algorithm.

5

Discussion and Conclusions

Modeling and analyzing long-range drift in visual odometry is of practical and theoretical significance. This paper models drift as a random process, combining

10

Ruyi Jiang, Reinhard Klette, and Shigang Wang

wide-band noise and a first-order Gauss-Markov process. Model parameters can be identified from experimental data. The Allan variance, offering the possibility to separate between these two sources of error, is adopted to analysis the drift. Experimental results reveal several important facts: 1. Modeling drift as a combination of wide-band noise and a first-order GaussMarkov process is validated. This can be seen from the Allan variance of the drift. 2. Analyzing drift from various visual odometry algorithms using Allan variance is a feasible way to tell where drift, and not white noise, becomes dominating. 3. Quantifying drift from a specific algorithm by τ and σb2 is a more accurate way than the usual offset ratio method.

References 1. Allan D.W.: Statistics of atomic frequency standards. Proceedings of the IEEE. 54(2) (1966) 221–230 2. Badino, H.: Binocular ego-motion estimation for automotive applications. PhD thesis, Frankfurt/Main University (2008) 3. Calvetti, D.: A stochastic roundoff error analysis for the convolution. Mathematics of Computation 59 (1992) 569–582 4. Cheng, Y., Maimone, M.W., Matthies, L.: Visual odometry on the Mars exploration rovers. IEEE Robotics Automation Magazine 13(2) (2006) 54–62 5. Comport, A.I., Malis, E., Rives, P.: Real-time quadrifocal visual odometry. Int. J. Robotics Research 29 (2010) 245–266 6. Corke, P., Detwiler, C., Dunbabin, M., Hamilton, M., Rus, D., Vasilescu, L.: Experiments with underwater robot localization and tracking. IEEE Int. Conf. Robotics Automation (2007) 4556–4561 7. Flenniken, W.S.: Modeling inertial measurement units and analyzing the effect of their errors in navigation applications. Master thesis, University of Auburn (2005) 8. IEEE standard specification format guide and test procedure for single-axis laser gyros. IEEE Std 647T M − 2006 (2006) 9. Kelly, A.: Linearized error propagation in odometry. The Int. J. of Robotics Research 23 (2004) 179–218 10. Kelly, J., Sukhatme, G.S.: An experimental study of aerial stereo visual odometry. IFAC Symp. Intelligent Autonomous Vehicles (2007) 11. Nist´er, D., Naroditsky, O., Bergen, J.: Visual odometry. IEEE Conf. Computer Vision Pattern Recognition 1 (2004) 652–659 12. Olson, C.F., Matthies, L.H., Schoppers, M., Maimone, M.W.: Stereo ego-motion improvements for robust rover navigation. IEEE Int. Conf. Robotics Automation (2001) 1099–1104 13. Scaramuzza, D., Siegwart, R.: Appearance-guided monocular omnidirectional visual odometry for outdoor ground vehicles. IEEE Trans. Robotics 24 (2008) 1015– 1026 14. S¨ underhauf, N., Protzel, P.: Towards using sparse bundle adjustment for robust stereo odometry in outdoor terrain. Towards Autonomous Robotic Systems (2006) 206–213 15. Wall, J.H., Bevly, D.M.: Characterization of inertial sensor measurements for navigation performance analysis. Proc. of ION GNSS, Long Beach, CA (2005)