139
Y. Zhang, J. K. Paik, A. Koschan, and M. A. Abidi, "3D object representation from multi-view range data applying deformable superquadrics," Proc. Int. Conf. Pattern Recognition, Vol. III, pp. 611-614, Quebec City, Canada, August 2002.
3-D Object Representation from Multi-View Range Data Applying Deformable Superquadrics Y. Zhang, J. Paik, A. Koschan, and M. A. Abidi Department of Electrical & Computer Engineering The University of Tennessee, Knoxville, TN 37996-2100 fyzhang,
[email protected], fjpaik,
[email protected] Abstract
appropriate assumptions [3]. However, the confidence or the certainty of the models recovered from a single-view image highly depends on the viewpoint used to take scans. Especially when a scene contains complex, multiple occluded objects, for example, incomplete and/or incorrect superquadrics tend to be recovered from single-view information. To recover convincing and accurate superquadric models from a complex scene, multi-view information must be utilized. Whaite and Ferrie [6] investigated the model misfit problem caused by single-view data and proposed a gaze planning strategy to determine the next viewpoint based on evaluating the certainty of recovered superquadrics from each single view. Wu [7] recovered geons from multi-view range data. In this paper, a new framework is proposed to recover globally deformed as well as regular superquadrics from multi-view range data. The framework consists of initial model recovery, view registration, view integration, and final model recovery from integrated data. A new registration technique based on the recovered deformable superquadrics is also proposed. Although a lot of research has been conducted on recovering globally deformed superquadrics [5], [3], [4], the mathematical expression for bending deformation, proposed in [3], [1], is only suitable for recovering and visualizing bending superquadrics from surface points lying in the first quadrant in the 2-D Cartesian space, i.e., for x > 0 y > 0. This condition is satisfied by default when superquadric models are recovered only from 2.5-D range images as in [3], [3]. However, in order to recover a bending superquadric model in an arbitrary position, data points in all four Cartesian quadrants need to be explored. A new quadrant analysis technique is therefore proposed in this paper to recover bending superquadrics from calibrated 3-D range data points.
This paper presents a new framework for recovering superquadrics with global deformations from multi-view real range data. The framework aims at improving confidence and accuracy of recovered models by utilizing multiview information, and consists of the initial superquadric model recovery, view registration, view integration, and final model recovery from integrated data. A quadrant analysis technique is proposed to aid the recovery of bending superquadrics. A modified range data registration method based on recovered superquadrics is also proposed to handle tapered superquadrics. Experimental results indicate the proposed framework of multi-view representation significantly improved the accuracy and confidence of recovered superquadrics compared with existing recovery strategies which rely on single-view range images.
1. Introduction Object representation has many applications such as robotic navigation, object recognition in computer vision, CAD modeling in reverse engineering, data visualization in computer graphics, and volumetric representation in medical imaging. For example, in an application such as cleaning a hazardous environment, object representation is an indispensable step in many tasks including path planning, obstacle avoidance, object manipulation, and scene reconstruction in robotic navigation. Compared with surface elements and contours, volumetric primitives can better represent global features of an object with a significantly reduced amount of information. In addition, they have the ability to achieve the highest data compression ratio without losing accuracy of the raw data. For these reasons volumetric primitives are the most efficient features to represent objects using three-dimensional (3-D) models. As a volumetric primitive, superquadrics can successfully be recovered from a single-view range image under
1051-4651/02 $17.00 (c) 2002 IEEE
The remainder of this paper is organized as follows. In Section 2, the framework of recovering superquadrics from multi-view range data is proposed. In Section 3, experimental results on the recovery of deformable superquadrics are provided. Section 4 concludes the paper.
611
1051-4651/02 $17.00 (c) 2002 IEEE
2. A Framework for Recovering Globally Deformed Superquadrics from Multi-view Range Data A new framework is proposed to recover both regular and globally deformed superquadrics from multi-view range data, and to improve the confidence and accuracy of recovered superquadrics. Only single-part objects are considered under the assumption that each input scene is pre-segmented. The proposed framework consists of (i) initial model recovery of single-view raw data, (ii) view registration based on the recovered superquadrics, (iii) view integration, and (iv) final model recovery from integrated multi-view data. The diagram of this framework is shown in Fig. 1. Major steps in this framework are discussed in the following subsections. 3-D data in view #1
Model recovery
3-D data in view #2
Model recovery View registration
3-D data in view #N
View integration
Final model recovery
Model recovery
Figure 1. Diagram of the multi-view superquadrics recovery framework.
used in the Levenberg-Marquardt optimization. The objective function is defined as [3]
G() =
; + ;
X N
=1
(jr0 jj1 ; F ;
(xc yc zc )j)2 (3)
"1 2
i
2.2. View Registration Based on Recovered Superquadrics The computational cost of most existing image registration approaches is very high due to the significant amount of overlapping involved. The proposed superquadric modelbased registration technique, however, requires neither overlapping nor expensive searching and optimization. The proposed algorithm extends the existing registration technique of undeformed superquadrics proposed in [3] by handling tapered superquadrics. The essential difference between our approach and the approach proposed in [3] lies in the calculation of initial moments, which facilitates the construction of new frames. The object vector i and matrix I in the canonical coordinate system are defined as
2 6 i = 64 L i
3 77 5 and I
0 0 0
V
L i
2I =4 0
L xxi
0
i
I
0 L yyi
0
I
0 0 L zzi
3 5:
(4)
To form the object vector i, we evaluate the volume of a tapering superquadrics, Vt , by
is needed [3]. The first 11 parameters are for a regular superquadrics. Parameters Kx and Ky are for tapering, and k and for bending. Recovering tapered superquadrics is straightforward due to the linear operations involved. On the other hand, for bending superquadrics, a new quadrant analysis technique is proposed to recover bending superquadrics from points in all four Cartesian quadrants. The parameter needs to be calculated according to which quadrant the original surface point lies in, such as
=1
d2 =
where d represents the radial Euclidean distance between a point and the corresponding superquadric model.
To recover a deformed superquadric model, a set of 15 parameters
=
N
i
2.1. Initial Superquadric Model Recovery
= fa1 a2 a3 "1 "2 px py pz Kx Ky k g (1)
X
V
t
= 2
Z
a3
A (z )dz t
0
= 2a1 a2 a3 "1 "2 B (
"2 "2 + 2 ) B ( "1 " 2
2
K K B ( 32"1 "1 + 1) + (K B ("1 "1 + 1)]: x
y
2
x
1
+ 1) +
+ Ky ) (5)
To form the object matrix I , the second order inertial moments of a tapering superquadrics are derived as
for points in I and IV quadrants, for points in II and III quadrants. (2)
This equation demonstrates the limitation of the original definition of bending deformations shown in [3]. An objective function based on the radial Euclidean distance is
(Ixx )t
=
1 "2 "2 + 1) 2 2 a 1 a2 a3 "1 "2 a2 (Ixx)1 + 4a3 B ( 2 2 2 3"1 (6) B ( 2 "1 + 1)]
612
1051-4651/02 $17.00 (c) 2002 IEEE
where
(Ixx )1
=
Z
a3
;
a3
(Ixx 0 (z ))t dz
K K 3 B ( 52"1 2"1 + 1) + (3K K 2 + K 3) B (2"1 2"1 + 1) + (3K K + 3K 2) B ( 32"1 2"1 + 1) + (K + 3K ) B ("1 2"1 + 1): (7) Similarly, (I ) can be derived by substituting K with K , and vice versa. (I ) can then be derived as Z Z Z (I ) = (x2 + y 2 )dxdydz =
x
x
y
x
x
yy t
y
y
y
y
Figure 2. A range image and its superquadric representation. (a) A range image of a complex scene and (b) its superquadric representation.
x
y
zz t
zz
(b)
(a)
y
t
V
=
1 a1 a2 a3 "1"2 (a22 (Ixx )1 + a21 (Iyy )1 ): (8) 2
The remaining steps of the registration algorithm are similar to those suggested in [3].
2.3. View Integration and Final Model Recovery (a) View integration is performed in a straightforward manner by merging two registered views since the proposed registration approach does not require any overlapping between the two data sets. The proposed view integration approach is thus insensitive to overlapping, and the amount of misalignment is minimized during registration. After registering and integrating the data from two views, final model recovery is performed on the integrated data to obtain superquadrics with higher confidence and accuracy. Multiview range data can be treated pairwisely in the same way.
(b)
(c)
Figure 3. View registration and integration of real range data. (a) A 3-D data set of a small pyramid from view 1, (b) from view 2, and (c) registered and integrated 3-D data.
added to the object (not the background) shown in Figs. 4(a) and 4(b). The objects with additive noise are shown in Fig. 5. Table 3 shows the recovered parameters of superquadrics from both single-views and integrated data. It can be seen that the proposed framework recovers almost identical parameters with or without noise. In other words, the proposed view registration approach, together with the mult-view representation scheme, can handle this level of noise very well.
3. Experiments Real range data were captured by the IVP smart range sensor [2] in our experiments. To recover the two tapered superquadrics in Fig. 2(a) using the proposed framework, each was first recovered from two views, and then registered and integrated. The results for the two tapered objects are shown in Figs. 3 and 4. Final models were then recovered from integrated data. Recovered parameters for the objects in Figs. 3 and 4 are shown respectively in Tables 1 and 2. From Figs. 3(c) and 4(c), one can see that the data sets from two views are registered very well. Tables 1 and 2 indicate that parameters recovered from integrated data are significantly better than those recovered from single-view data. Combining recovery results of each object shown in Fig. 2(a), the corresponding scene consisting of recovered regular and deformable superquadrics is shown in Fig. 2(b).
Table 1. Recovered parameters of superquadrics from single-view and integrated data in Fig. 3(a). IT: parameters from integrated data. GT: ground truth parameters. View 1 2 IT GT
In addition, to investigate the noise issue of our approach, Gaussian noise at standard deviation of 0.1 was
a1 20.08 20.11 19.76 20
a2 21.13 20.79 20.45 20
613
1051-4651/02 $17.00 (c) 2002 IEEE
a3 39.69 39.78 45.07 45
"1 0.18 0.13 0.11 0.1
"2 1.99 1.98 2.01 2.0
Kx 0.88 0.85 0.89 0.9
Ky 0.90 0.86 0.90 0.9
(a)
(b)
(c)
(a)
(b)
(c)
Figure 4. View registration and integration of a tapered superquadrics from real range data. (a) A set of 3-D data from view 1, (b) a set of 3-D data from view 2, and (c) registered and integrated data.
Figure 5. View registration and integration of a tapered superquadrics from real range data. (a) A set of 3-D data from view 1, (b) a set of 3-D data from view 2, and (c) registered and integrated data.
Table 2. Recovered superquadrics parameters from single-view and integrated data shown in Fig. 4.
Table 3. Recovered superquadrics parameters from single-view and integrated data shown in Fig. 5.
View 1 2 IT GT
a1 19.23 17.64 19.55 20
a2 17.42 19.85 18.42 20
a3 29.50 23.20 29.85 20
"1 0.1 0.1 0.1 0.1
"2 0.97 0.99 0.98 1.0
Kx 0.39 0.38 0.39 0.4
Ky 0.39 0.40 0.40 0.4
4. Conclusions This paper proposes a new framework to recover deformable superquadrics from multi-view 3-D range data. In this framework, a quadrant analysis technique was proposed to recover single bending superquadrics from singleview range data. A modified version of an existing registration technique, which is based on recovered superquadrics instead of raw data, was developed. The new registration approach was able to register tapering superquadrics as well as undeformed superquadrics. Experimental results of real range data with additive demonstrated that the proposed registration technique was able to register twoview data correctly under a certain level of noise. Furthermore, this research indicated that superquadrics recovered from multi-view integrated data were significantly better than those from single-view data. Since the view registration algorithm used in this research highly depends on superquadrics recovered from single-view data, the registration result could be inaccurate if the superquadrics are misfit. In this case, the iterative closest point (ICP) algorithm proposed in [8] can be used to refine the registration result.
Acknowledgments This work was supported by the University Research Program in Robotics under grant DOE-DE-FG02-
View 1 2 IT GT
a1 18.56 18.14 19.74 20
a2 18.28 17.62 17.83 20
a3 28.62 25.96 28.95 20
"1 0.1 0.09 0.1 0.1
"2 0.96 0.97 0.97 1.0
Kx 0.38 0.39 0.39 0.4
Ky 0.37 0.38 0.40 0.4
86NE37968, by the DOD/TACOM/NAC/ARC Program, R01-1344-18, and by FAA/NSSA Program, R01-134448/49.
References [1] A. H. Barr. Global and local deformations of solid primitives. Computer Graphics, 18(3):21–30, 1984. [2] IVP Company, Sweden. MAPP2500 Ranger PCI System User Manual, 2000. [3] A. Jakliˇc, A. Leonardis, and F. Solina. Segmentation and Recovery of Superquadrics. Kluwer Academic Publishers, 2000. [4] D. N. Metaxas. Physics-based Deformable Models: Applications to Computer Vision, Graphics and Medical Imaging. Kluwer Academic Press, U.S, 1997. [5] D. Terzopoulos and D. Metaxas. Dynamic 3D models with local and global deformations: Deformable superquadrics. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(7):703–714, 1991. [6] P. Whaite and F. P. Ferrie. On the sequential determination of model misfit. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(8):899–905, 1997. [7] K. Wu and M. D. Levine. Recovering parametric geons from multiview range data. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 159–166, 1994. [8] Z. Zhang. Iterative point matching for registration of freeform curves and surfaces. International Journal of Computer Vision, 13:119–152, 1994.
614
1051-4651/02 $17.00 (c) 2002 IEEE
