In Proceedings of the First Asian Conference on Computer Vision, pages 454-457, Osaka, Japan, November 1993.
Similarity Invariants for 3D Space Curve Matching S. Z. Li School of Electrical and Electronic Engineering Nanyang Technological University Nanyang Avenue, Singapore 2263
[email protected]
Abstract. An invariant representation based on so-called similarity-invariant coordinate system (SICS) is presented for matching 3D space curves under the group of similarity transformations. In the SICS, the 3D geometry of a curve segment is unique. Thus, constraints on the curve can be fully explored for the matching. Experimental results with simulated data are presented.
1 Introduction Object recognition is an important problem in computer vision [1]. One of the approaches to solving it is through matching characteristic curves on objects. Although much work is done on representation and matching of 2D planar curves, less is seen for 3D space curves [2, 3, 12], especially non-algebraic curves, and even less has been reported for the situation where the scale of objects in 3D is an unknown factor. A frequently cited paper on 3D curve matching is [12]. There, the group of transformations dealt with is Euclidean, in which the scale is known; and connected curves are assumed to have been segmented by some other procedures. This is also the case in a recent paper [2]. The primary contribution of this paper is an invariant representation for 3D space curves under the group of similarity transformations for the purpose of matching. A similarity transformation consists of a rotation, a translation and an isotropic scaling. It has 7 degrees of freedom in the 3D space. Curves to be matched are allowed to consist of unsegmented (connected ) sub-parts of model curves. The availability of 3D curve data, such as curves derived from CAD/CAM object models, range images or MR medical images, is assumed. Curvature maxima and tangent discontinuities are located as invariant feature points. A similarity-invariant coordinate system (SICS) in 3D is then established for each curve segment bounded by a pair of feature points. The 3D geometry is unique in the SICS under the similarity group. Thus a full exploration of the curve geometry is possible. Based on the SICS, invariant coordinates and angles are derived for each of the curve segments and used in matching as bilateral relations between pairs of feature points. An attributed relational graph (ARG) [6, 7] is then constructed by taking the feature points as nodes and the relations as quantitative links between the nodes. The ARG gives rise to an invariant description of the entire curve. The matching is to nd the correspondences between a subset of nodes in the scene ARG and those in the model ARGs. The solution is de ned by the optimum of a gain function [6, 7] which measures the goodness of (partial) t between two (weighted) graphs. A relaxation labeling algorithm described in [4] is used to search for the solution.
2 Invariant Representation A curve is initially in the form of a sequence of 3D coordinates. It is nally represented by an ARG. There are four stages in this process: (1) Extracting invariant feature points, (2) Establishing SICS, (3) Computing invariant relations and (4) Constructing the ARG using the feature points and the relations.
2.1 Invariant Feature Points
Curvature maxima and tangent discontinuities are used as feature points1 , which are invariant under similarity transformations. Signi cant feature points can usually be detected as long as they are present. However, the set of useful feature points will be empty if all such points happen to be occluded. It is assumed in this paper that this never happens and an adequate number of feature points are present. The minimum number required for constructing an SICS is two; but the more feature points available, the more reliable the matching. Some regular curves, such as a planar circle or a helix, are not considered in this work because they cannot be characterized by the above features. The curvature estimation undergoes three steps. First, the input curve is repeatedly reparameterized by arc-length. This allows to determine the size of smoothing operators based on the arc-length. The repeated reparameterization has also an e ect of removing noise. The rst step results in a sequence of points evenly spaced in arc-length. In the second step, derivatives in three directions are estimated by convolving, respectively, the three coordinate sequences [9] with a Gaussian kernel of the size relative to the arc-length used for the reparameterization2. Compensation is done on the smoothed curve coordinates to reduce the e ect of curve shrinking due to smoothing [8]. The curvature is computed after that. A method proposed in [10] is used in detecting reliable curvature maxima from candidate maxima. The method takes into consideration the \region of support" for each candidate maximum and the perpendicular distance from the point to the line passing through the two end points of the support. Using information merely at these feature points has a disadvantage of that information about the curve between the points is e ectively wasted. To fully explore the 3D geometry of the curve, a similarity-invariant coordinate system (SICS) is formed as the next.
2.2 SICS
The SICS is established in two steps: (1) nding an orthogonal basis of three vectors invariant to rotation and translation and (2) normalizing the length measure to achieve scale invariance. Refer to Fig.1. Pick a pair of feature points p and p . Consider the curve segment p p between p and p and the straight line p p passing through the two points. Find on p p the mid-point, denoted by p , such that curve segments p p and p p have the equal arc-length. Obviously, p is unique. It is assumed that p does not lie on p p such that the basis to be constructed is not degenerated. Next, nd on p p the point, denoted by o, such that line op is orthogonal to op . The Point o is also unique. It will serve as the origin of the new system. The three orthogonal basis vectors are de ned as e01 = kpp ?? ook
d 0
i
d
d
j
i
j
i
j
j
i
0
i
i
0
i
0
d
j
0
i
j
0
j
j
j
j
j
e02 = kpp ?? ook 0
0
and
e03 = e1 e2
where k k and \" denote the Euclidean norm and the cross-product, respectively. The above three vectors are meaningfully de ned with the assumption that p does not lie on p p . The vectors are of unit length in the old system and are invariant, relative to the 3D curve segment p p , up to rotation and translation. Let 0
d i
l
0
= kp ? p k i
i
j
j
j
be the Euclidean distance between p and p . Invariance to scaling can be achieved by normalizing the above distance 0 in the old coordinate system into a unit of length in the new system. This is done by simply choosing e1 = 0 e01 e2 = 0e02 e3 = 0e03 Now = fo e1 e2 e3 g i
j
l
l
;
l
;
;
;
l
;
1 Torsion extrema and zeros are also invariants. They are not used because the 3rd order derivatives are involved which are less reliable. 2 Choosing the right size for a smoothing kernel is an unsolved problem. A xed size is not good for all scales. Multi-scale based methods may be used.
pi p j
p’
uk u n-1
e2
u k -p’’
pj
u2 pi p j
pi
o p’’
un
e1
e3
u1
Figure 1: Illustration of the SICS. forms the SICS. In this system, a point p is expressed as
p = o + 1e1 + 2 e2 + 3e3 where u = ( 1 2 3) is the coordinate of p in the new system. In the SICS, the coordinate of p is u = ( 0 0), that of p is u = (1 ? 0 0) and that of p is u = (0 0) u
u
u
u ;u ;u
j
j
d
a;
;
i
a;
i
0
;
0
; b;
where and are some real numbers. They are invariant under the similarity group. So are the coordinates of the remaining points on the segment p p . That is, a
b
i
j
u(p) = u(G (p))
d
8p on p p i
j
where G denotes the similarity group. This makes the 3D geometry of the curve segment unique. Like the invariant coordinate systems for matching under the planar ane group [5, 11], the SICS here is semi-local. In both cases, it is required that the curve under consideration should not break into too small segments. There should be at least one continuous segment in the scene containing at least a minimum number of consecutive feature points The minimum number is three in [5], four in [11] and two in this work. Finally, the basis for the SICS could be ane, like those in [5, 11]; however, the orthogonal basis may be better than general ane basis as the feature quantities expressed in the orthogonal basis are least dependent on each other.
2.3 Invariant Relations and ARG
Invariant coordinates and angles can now be computed and used as similarity invariant bilateral relations for matching, being bilateral relations because they are about curve segments bounded by two feature points. The coordinates are obtained as follows: Quantize each curve segment p p into equally spaced (in arc-length) points, denoted by u ( = 1 ) (see Fig.1). The position of the points gives coordinates. The angles, on the other hand, are de ned as follows: Consider the midpoint, denoted by p , of p p and obtain vectors u ? p . It is easy to see that the angles between each of these vectors and the three orthogonal axes e1, e2 and e3 are also similarity invariants3. Other quantities can also be explored. Consider tangents to the curve at u , for instance. Angles between the tangents and the three axes are also invariants. Additional useful quantitative relations include the ratio of the arc-length of p p over the length 0 and the ratio of curvature magnitudes at p and p . All above feature quantities constitute a vector of invariant relations between p and p for relational matching. An attributed relational graph (ARG) for the 3D curve can now be constructed. It is composed of the feature points as nodes and the relations as links between the nodes [6, 7]. Note that the links are generally quantitative rather than qualitative. Model curves and curves in the scene are nally represented in
d i
k
k
; :::; n
j
n
n
n
00
k
i
n
j
00
d i
j
k
l
i
i
j
j
3 The choice of the midpoint p00 instead of o as the reference point for the angles is for stability reason: angles between vectors uk ? o and e1 can change drastically as point o moves, due to noise for example, from within the line segment between pi and pj to the outside due to noise; but the angles between uk ? p00 and e1 are stable.
this form. There is a special model composed of a single node, denoted by NULL , to which spurious nodes in the scene should be matched (classi ed) [6, 7].
3 ARG Matching The matching process nds the correspondences between a subset of nodes in the scene ARG and those in model ARGs. Matching of the scene to one model is performed using an algorithm developed for matching of weighted subgraphs [6, 7]. The algorithm is aimed for matching under the circumstance of overlapping scene with missing and spurious features. The interested reader may refer to the cited references for details of the algorithm such as encoding invariant relations into the function of compatibility between matches and de nition of the goodness of partial t. Matching to multiple models is done by choosing the maximally supported matches over all models. This is not further discussed due to the space limit. Fig.2 demonstrates an example where the scene curve in (a) contains sub-parts of three of sixteen candidate model curves. (b)-(d) show the three matched model curves aligned to the scene curve.
4 Summary and Comments An invariant representation has been presented for matching unsegmented (connected) partial 3D space curves under the similarity transformation group. A similarity-invariant coordinate system (SICS) is established for each curve segments bounded by two invariant feature points. The geometry of the curve segment is unique in the SICS. The SICS enables a full exploration of the 3D geometry of the curve. Invariants thus computed in the SICS provide strong constraints for the curve matching. Some assumptions are made in this work. It is assumed that 3D data is available. In the scene, there is at least one continuous curve segment containing at least two invariant feature points. Curves under consideration exclude regular ones such as a circle and a helix. The present work performs successfully under the assumptions. Two factors have contributed to the success: First, the SICS allows a full exploration of the information about the curve; second, the matching algorithm used is capable of matching under the circumstance of inexact measurements and missing and spurious features.
Acknowledgement
The author is grateful to Cao En for his help in creating model curves. Credits are also due to Chan Kap Luk, Ching Wee-Soon, Ngoi Kian Peng and H. Wang for their comments on the manuscript.
658
658
270
z-axis
z-axis 270
398
275
x-
ax
398
y-ax
is 516
516 658
275
y-axis
270
z-axis
z-axis
658
658
658
is
x-axis
270
398 658
275
x-
ax
x-a
axis
xis
y398
516
is
516 658
275
y-axis
Figure 2: A result of matching. From left to right, top to bottom are: (a) the scene curve, (b) a rst model curve (in smooth curve) is matched with scale factor 2.0 and aligned with the scene curve (in noisy curve), (c) a second curve with scale factor 0.5 and (d) a third curve with scale factor 1. .
.
References
[1] H. Ballard and M. Brown. Computer Vision. Prentice-Hall, 1982. [2] P. J. Besl and N. D. McKay. \A method for registration of 3-D shapes". IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-14:239{ 256, 1992. [3] R. C. Bolles and P. Horaud. \3DPO: A threedimensional part orientation system". International Journal of Robotic Research, 5(3):3{26, Fall 1986. [4] R. A. Hummel and S. W. Zucker. \On the foundations of relaxation labeling process". IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-5(3):267{286, May 1983. [5] Y. Lamdan, J. Schwartz, and H. Wolfson. \Object recognition by ane invariant matching". In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 335{344, 1988. [6] S. Z. Li. \ Towards 3D vision from range images: An optimization framework and parallel networks".
[7] [8] [9] [10] [11] [12]
CVGIP: Image Understanding, 55(3):231{260, May 1992. S. Z. Li. \Matching: invariant to translations, rotations and scale changes". Pattern Recognition, 25(6):583{594, June 1992. D. Lowe. \Organization of smooth image curves at multiple scales". In Proceedings of Second International Conference on Computer Vision, pages 558{ 567, 1988. F. Mokhtarian and A. Mackworth. \Scale-based description and recognition of planar curves and twodimensional shapes". IEEE Transactions on Pattern Analysis and Machine Intelligence, 8:34{44, 1986. A. Rattarangsi and R. T. Chin. \Scale-based detection of corners of planar curves". IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(4), April 1992. C. Rothwell, A. Zisserman, A. Forsyth, and J. Mundy. \Canonical frames for planar object recognition". In Proceedings of the European Conference on Computer Vision, pages 757{772, 1992. J. Schwartz and M. Sharir. \Identi cation of partially obscured objects in two and three dimensions
by matching noisy characteristic curves". International Journal of Robotic Research, 6(2):29{44, 1987.
