On Modeling Ego-Motion Uncertainty for Moving Object Detection from

On Modeling Ego-Motion Uncertainty for Moving Object Detection from a Mobile Platform Dingfu Zhou 2

1,2

, Vincent Frémont

1,2

, Benjamin Quost

CNRS Heudiasyc UMR 7253, France.

1,2

and Bihao Wang

1,2

.

1

Université de Technologie de Compiègne (UTC),

E-mail: {dingfu.zhou, vincent.fremont, benjamin.quost, bihao.wang }@hds.utc.fr

Experimental Results

System Overview

Introduction

 Dataset : KITTI vision benchmark suite, stereo vision color image sequences acquired in different environments.

Objective

To detect moving objects around the ego-vehicle using stereo cameras;

 Processing time: 30s per frame on a laptop (Intel Core i7) with Matlab R2013a environment.

 Difficulties

 Precision: P = 79.5%

Mobile cameras, no other sensors’ information, dynamic environments;  Solution 



Recall: C= 86.7%

Detection results in different environments

The motion likelihood of each image pixel is computed by modeling the uncertainty of ego-motion, pixel position and disparity value using first-order error propagation.

1:Inner City Sequences

2:Campus Sequence

The object segmentation is achieved via graph-based motion segmentation.

Step1: Ego-Motion Estimation and Uncertainty Computation

Step 2: RIMF and Motion Likelihood Estimation Matched feature point at frame t-1

3:Suburban Road Sequence

R,t

Predicted Optical Flow

Matched feature point at frame t

Residual Image Motion Flow (RIMF)

Real Measured Optical Flow

 Predicted optical flow:

Kt x t  KRK x t 1  z

 RIMF:

pRIMF ( m )  (ux  ux , vx  vx )( m )

 Mahalanobis distance

p

associated to RIMF:

 Ego-Motion:

ˆΘ  arg min F(Θ, x )  arg min x  f(Θ, x ) 2 t t1  Θ

Θ

1

 Covariance Matrix:

T

 g   g   g   g  Θˆ       x     Θ  x  x  Θ       

 Covariance matrix of

xt

RIMF:

T

ˆ Θ Θ

1

(m)

 RIMF



1 pmT RIMF pm

063  (Θ )66  T  J 2 2 2 J diag ( x ,  y ,  d )   036

Motion Likelihood Image

Step 3: Graph-Based Segmentation

Step 4: Bounding Box Generation and Verification

Conclusion and Discussion  Only two consecutive stereo images are required.  Arbitrarily moving objects (including partially occluded) can be detected.

Graph-cut (GC) cost function :

 Bounding boxes are generated from U-disparity map and region growing;

E ( L)  Er ( L)   Eb ( L) Moving pixel detection using a fixed threshold

 The region term is built using motion likelihood

Rough moving object clustering by depth

Er  L( x)motion ( x)  (1  L( x))static ( x)

 Object verification is achieved using the constraint below:



0.5m  h  2.5m

 The boundary term is built using the local depth Eb   Graph-based moving pixel segmentation result



 xˆN 4 ( x )

exp(  2( de( xi )  de( x j ) )) L( xˆ )  L( x) Bounding boxes generation

 Further accelerations could be achieved by C/C++ implementation with parallel/GPU computing.

 Robust multiple objects tracking algorithm can be used to obtain more stable results by reducing false alarms.