3D Objects Coding and Recognition Using Surface Signatures Sameh Yamany Computer Science Dept. Old Dominion Univ., Virginia, USA Norfolk, VA23529
[email protected]
Abstract This paper presents a new concept for 3D coding of freeform surfaces. The proposed coding technique uses the surface signature representation scheme [1]. This representation scheme captures the 3-D curvature information of any free-form surface and encodes it into a 2-D image corresponding to a certain point on the surface. This image is unique for this point and is independent of the object translation or orientation in space. For this reason this image is called “Surface Point Signature” (SPS). Using SPS in 3D coding has many applications in 3D compression, 3D pose estimation and in 3D object recognition.
1. Introduction 3D object recognition has become an integral part in many computer and robot vision systems and still presents a topic of high interest in these fields [2, 3, ?]. In order for any 3D object recognition algorithm to perform accurately and efficiently, appropriate representation scheme for the surface is needed. Most of the surface representation schemes found in literature have adopted some form of shape parameterization especially for the purpose of object recognition. One benefit of the parametric representation is that the shape of the object is defined everywhere which enables high level tasks such as visualization, segmentation and shape analysis to be performed. Moreover, such representation allows stable computation of geometric entities such as curvatures and normal directions. However, parametric representation are not suitable to present general shapes especially if the object is not of planar, cylindrical or toroidal topology. Free-form surfaces, in general, may not have simple volumetric shapes that can be expressed in terms of parametric primitives. Discontinuities in the surface normal or curvature, and consequently in the surface depth, may be present anywhere in a free-form surface. We introduced a new free-from surface representation
Aly Farag Electrical and Computer Eng. Dept. University of Louisville, USA Louisville, KY 40292
[email protected]
scheme that matches and estimates the pose of free-form objects in cluttered 3D scenes [1]. This representation scheme captures the surface curvature information, seen from certain points and produces images, called surface point signatures (SPS), at these points. Matching signatures of different surfaces enables the recovery of the transformation parameters between these surfaces. Furthermore, we applied a selection process to select feature points on the surface to be used in the matching process. This reduction process solves the long registration time reported in the literature, especially for large surfaces. The proposed coding idea in this paper starts by generating SPS images for specific points on the 3D object surface. These images are then segmented into corresponding curvature clusters. These curvature clusters and their relations uniquely describe the structure of the 3D object and can be used to synthesize the 3D object.
2 Surface Point Signature (SPS) Generation Rather than just depending on the 3-D coordinates of the point on a free-form surface, the SPS framework obtains a “signature” image at each surface point. The signature, computed at each important point, encodes the surface curvature seen from this point, thus giving it more discriminating power than the “splash” [4] and the “point signature” [5]. Also using the curvature as a measure of matching is more discriminating than the point density used in the “spin image” [6]. As shown in Figure 1, for a specific point P defined by its 3-D coordinates and the normal UP at the patch where P is the center of gravity, each other point Pi on the surface can be related to P by two parameters: 1- the distance di = jjP ; Pi jj and ; 1 UP :(P ;Pi ) 2- the angle i = cos jjP ;Pi jj . This is a polar implementation of the SPS image and it can be easily converted into Cartesian form. Also we can notice that there is a missing degree of freedom in this rep-
Proceedings of the International Conference on Pattern Recognition (ICPR'00) 1051-4651/00 $10.00 @ 2000 IEEE
Figure 1. For each important point P
we generate an SPS image where the image axis are the distance d between P and each other point on the surface and the angle between the normal at P , UP and the vector from P to each other point. The image encodes the angle which represents the change in the normal at these points from UP .
resentation which is the cylindrical angular parameter. This parameter depends on the surface orientation which defies the purpose of having an orientation independent representation scheme. The size of the SPS image depends on the object size. However, in order to perform matching, normalizing the images is important. We chose to normalize each object to its maximum length, yet while doing matching and recognition we re-normalize the scene image to the maximum length of object in study, thus enabling scale independent matching. At each location in the image we encode the angle i = cos;1 (UP :UPi ). This represents the change in the normal at the surface point Pi relative to the normal at P . Due to the fact that we are ignoring the cylindrical angular degree, the same pixel in the SPS image can represent more than one 3-D point on the surface. We take the average of their angles i and encode it in the SPS corresponding pixel location. Figure 2 shows some SPS images taken at different important points on the statue and a phone handset.
Figure 2.
Examples of SPS images taken at different important points locations. Notice how the image features the curvature information. The dark intensity in the image represents a high curvature seen from the point while the light intensity represents a low curvature. Also notice how different is the image corresponding to a location from images of other locations.
Where t0s () is the segmented output at scale factor s, Vk (x) is the fuzzy decision function and c is the number of curvature clusters. The fuzzy decision function assigns the pixel to belong to a certain curvature cluster if its fuzzy membership is larger than a certain value (a value of 0.7 is used in our experiments), otherwise the value is computed from the average membership values of its neighboring pixels. This produces homogeneous segmented regions and reduces the effect of noisy pixels. Fuzzy membership functions were a-priori defined to the segmentation procedure. Figure 3 shows the result of segmenting signature images at different scale sizes.
3 Using SPS in 3D coding The idea is to use the generated SPS images to code the 3D object. This is done by segmenting the SPS images into homogeneous curvature clusters. The relation between these clusters will define a unique 3D code for the point location and the object in study. The segmentation was performed using fuzzy thresholding where each pixel is assigned to a curvature cluster according to its membership value. Each signature image is segmented as follows,
t0 (i; j ) = arg max(V (t(i; j=s))); k 2 [1; c] s
k
k
Figure 3.
(a) Two signature images at two different sizes. (b) The corresponding segmented images of the signatures in (a). (c) Another two examples of the same segmented signatures a two smaller sizes.
(1)
Proceedings of the International Conference on Pattern Recognition (ICPR'00) 1051-4651/00 $10.00 @ 2000 IEEE
4 Object Recognition using SPS Coding The signature representation was used in matching objects in a 3D scene with their corresponding models in a library. The proximity of the objects in the scene creates large amounts of clutter and occlusion. These contribute to extra and/or missing parts in the signature images. In [1] it was shown that using the signature polar representation, the effect of clutter, for many points, is only found in the third and/or fourth quadrant of the image. Since the size of the object is one variable in the matching, we normalize the signature images generated with the largest distance in the scanned scene. Matching is then performed between the scene signature images and the model signature images at different scale factors. Point correspondences are established once the correct scale is determined and the scene and object images are normalized to same scale factor. Figure 4 shows different 3D objects with their different SPS coding images. Notice how different each code for each object. Using these SPS image codes for each 3D object, the object can be localized in a 3D image and identified uniquely. Figure 5 shows an example of matching a model object with two 3D scenes each containing a different size of the object. Using the SPS codes the object is identified and localized in both scenes. Also, such a scheme could be used to compress 3D object libraries to include only the different coding images for each object, hence reducing the size of the library. Figure 6 shows more examples. Using the signature matching criterion, all of the models in the scene are simultaneously matched and localized in their correct scene positions. The models in the library and the 3D scenes are scanned using a Cyberware 3030 laser scanner with a resolution of 1mm. Some models (e.g. the duck, bell and cup) were obtained from a CAD/CAM library.
References [1] S. M. Yamany and A. A. Farag, “Free-form surface registration using surface signatures,” IEEE Int. Conf. Computer Vision (ICCV’99),Kerkyra,Greece , Sept 1999. [2] C. Dorai and A. K. Jain, “Cosmos-a representation scheme for 3d free-form objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence 19, pp. 1115–1130, October 1997. [3] C. S. Chua and R. Jarvis, “3d free-form surface registration and object recognition,” International Journal of Computer Vision 17, pp. 77–99, 1996.
Figure 4.
Examples of different 3D objects and their corresponding SPS coding at specific point locations.
[4] F. Stein and G. Medioni, “Structural indexing: Efficient 3-d object recognition,” IEEE Trans. Patt. Anal. Machine Intell. 14(2), pp. 125–145, 1992. [5] C. S. Chua and R. Jarvis, “Point signatures: A new representation for 3d object recognition,” Internation Journal of Computer Vision 25(1), pp. 63–85, 1997. [6] A. Johnson and M. Helbert, “Efficient multiple model recognition in cluttered 3-d scenes,” IEEE Proc. Computer Vision and Pattern Recognition (CVPR’98) , pp. 671–678, 1998.
Proceedings of the International Conference on Pattern Recognition (ICPR'00) 1051-4651/00 $10.00 @ 2000 IEEE
Intensity Image
Scene mesh
Results of Matching
Figure 5.
(a) Library object (b) One signature image from the library (c) The Corresponding segmented signature (d) Two 3D scenes with the object at different sizes (e) Corresponding segmented Scene signatures (f) Matched object segmented signature at the exact scale.
Figure 6.
Examples of using the signature representation in object matching. A library of 10 objects is used. Some of these objects were scanned using a Cyberware 3030 laser scanner with a resolution of 1mm. Others are obtained from CAD libraries.
Proceedings of the International Conference on Pattern Recognition (ICPR'00) 1051-4651/00 $10.00 @ 2000 IEEE
