PLANAR BACKGROUND ELIMINATION IN ... - Semantic Scholar

PLANAR BACKGROUND ELIMINATION IN RANGE IMAGES: A PRACTICAL APPROACH Alexandre Vrubel, Olga R. P. Bellon, Luciano Silva IMAGO Research Group, Universidade Federal do Parana Curitiba, Brazil - www.imago.ufpr.br ABSTRACT Separating data from objects of interest and background is a common procedure in range images applications. Most of the works presented in the literature use image segmentation, either automatic or supervised, to do that. We present a new method to automatically perform this separation without the need of using complex image segmentation techniques. In our approach, we consider that the object is always scanned over a supporting plane. Then, we assume that there is no information below the plane, and the object data is above it. By projecting all points into the supporting plane normal direction, the points on the plane would project at the same value, the points on the object would be spread with values larger than the plane, and there would be very few values below the plane value (due to noise). This allows us to quickly and reliably eliminate the background data from range images. Index Terms— Range images, plane detection. 1. INTRODUCTION Range images are used in several applications where precise measurements are needed, like the digital preservation of natural and cultural assets [1]. In these applications, a 3D model is built through the integration of several images, captured from different viewpoints around the object of interest. There are several different techniques to acquire depth information from an object: laser scanning techniques, multiview stereo, shape from structured light, shape from silhouette, contact digitizers, among others [2]. Whatever the technique employed, a range image is obtained containing points from both the object of interest and the background. Eliminating the background data is a necessary pre-processing step for applications using range images. The most used approach to eliminate the background is to perform the segmentation of the range images [3, 4, 5, 6, 7]. The drawback is that segmentation is an operation much more complex than a simple background elimination. Therefore, its execution time is costly. Besides, the problem of deciding which segmented regions represents the background still persists, and a posterior analysis of the segmented regions is still needed. All of these characteristics lead to the development Thanks to CAPES, CNPq and FINEP for funding.

of our approach, aiming for a practical, quick and reliable background detection and elimination technique. We considered several approaches to perform background elimination on range images. Manual elimination of the background is extremely laborious, so it is not applicable in all but the simplest applications. Another possibility is to perform segmentation by color, which is possible because the capturing devices usually return corresponding color images for each range image. Using a controlled acquisition background of uniform color allows the segmentation of the range image through the analysis of the corresponding color image. This approach has several drawbacks: it requires that the object does not contain the color used for the background; the image acquisitions can only be made in controlled environments, where the background color can be properly set; and illumination variations (e.g. specular highlights and shadows) may impose difficulties to the background color detection. Another approach is to perform two range captures, one with the background only, and the other with the object of interest over the background. The difference between the two range images correspond to the 3D data of the object. Although intuitive, this solution is not practical. Usually, for each view of the object, small adjustments of scanning parameters are necessary (e.g., when using laser triangulation scanner, the focus distance and laser power). When such adjustments are made, the acquired background changes; this forces us to recapture the “background only” image again. In practice, we need to capture two range images for each view, with and without the object. This excessive object manipulation, besides being laborious, in some cases is undesirable (e.g. sensitive objects that should be manipulated with care, like fossils and museum artworks). Avoiding the modification of scanner parameters can lead to sub-optimal capture (e.g. excessive noise or failure to capture out-of-focus data). Another possibility, applicable to laser scanning, is to capture the object placed over a black background, which usually is not captured due to its low reflectance. However, this is not a good option either, as dark regions of the object need higher laser power, therefore leading to the capture of the black surfaces used in the background. Besides, the need for a controlled background is another drawback of this approach. Our suggested method addresses the drawbacks of the previous approaches. We capture the objects placed over a

supporting plane (e.g. a table or the floor). The only restriction applied is that the object must be entirely above the plane, and the scanner should not capture data below the plane. This condition is easily attainable, both in a laboratory or in the field. Based on this pre-condition, we developed a method to automatically detect this supporting plane. Our planar background detector uses several techniques, like a variation of the Hough transform, robust methods (MSAC [8]) and our pre-condition of no information below the supporting plane (except for some outliers). Although we used our approach with laser scanning, it can be used with other capturing techniques. 2. AUTOMATIC SUPPORTING PLANE DETECTION Initially, we need to choose some parameters for the plane detection. Here we face a trade off solution, between detecting the plane with high precision (but eventually missing it), or with lower precision (but incurring the risk of false positives). In our implementation, we made all 5 parameters being selected by a single “generic” setting, the one for precision (see Table 1). In practice, we try to detect the plane with the highest precision, and if it is not found, we interactively reduce the precision until the plane is detected. The detected plane is eliminated in the range image, so that we have a visual feedback during the process. The individual parameters will be detailed in the following algorithm description. Also, Fig. 1 presents an overview of the algorithm. Step 1 - Populate buckets: We estimate the normal vectors for each sampled 3D point, taking into account its 4-neighbors and the grid-like structure of the range image. Then, the space of possible normal directions are divided in buckets. This division uses a structure similar to a cubemap [9], accumulating for each bucket the number of points whose normal vectors are close to the general bucket direction (see Fig. 2). This procedure is used to represent the distribution of normal vectors found in the captured scene; Step 2 - Sort buckets: We sort the buckets by the decreasing number of points accumulated in them. Each bucket represents a candidate normal direction for the supporting plane to be detected. As the points inside a plane usually have similar normal vectors, large planar regions of the range image are represented by buckets of higher values. Therefore, we examine the buckets by the decreasing number of points to accelerate the supporting plane detection; Step 3 - Generate histogram: For each analyzed bucket, we generate a normal vector that points from the cube center to the bucket. This is only an approximation of the candidate plane normal direction n, which will be refined later. Then, we project all 3D points x from the range image over this candidate normal vector, using the dot product operation: d = x · n. This generates a 1D distribution of distances d, which can be represented by a histogram. If the candidate normal n is close to the supporting plane normal direction, all points

Populate buckets with point normals (Fig. 2) Plane Not Detected N Are there other buckets?

Discard bucket

Sort buckets in decreasing order Y

Project points in the direction of the largest bucket, generating a histogram (Fig. 3)

N

Histogram represents a plane?

Y Calculate plane with MSAC using the points of the histogram peak Refine the plane through least-squares, using the inliers of MSAC plane

N

Calculated plane is valid? Y Plane Detected

Fig. 1. Supporting plane detection algorithm.

Fig. 2. Division of the normal directions space. The estimated normal vector of a 3D range point originates at the center of the cube, and points into a specific bucket inside one of the 6 cube faces; then the counter of this bucket is incremented. inside the supporting plane will have a similar distance value d, which will generate a peak in the distance histogram. What we are looking for is a histogram like the one of Fig. 3, with a great concentration of samples (a peak) in the beginning of the histogram, due to the supporting plane points. Besides, before this peak there should be only a few points (some outliers), and the remaining points (from the object of interest) should be spread after the peak. If the candidate normal of the bucket does not give such a histogram, it is discarded, and the next bucket is analyzed; Step 4 - Test histogram: To test the histogram conformity, we use two parameters. The first one (minNormal) is the minimum percentage of points on the histogram peak that should agree with the plane candidate normal direction . For a point to agree with the candidate normal, the angle between its normal and the candidate normal should be small. Our tests show that an angle below 30◦ gives good results, for different objects. The second parameter (minPeak) defines the minimum percentage of points belonging to the supporting plane, i.e. the histogram peak. We sum the distributions since

Num. points

Points of the support plane

Minimum Projected Distance

Maximum Projected Distance

Fig. 3. Example of a 1D distribution of range points over a candidate supporting plane normal direction. Precision mmTolerance (mm) maxBelow (%) minPeak (%) minAbove (%) minNormal (%)

0 4.0 3.0 1.0 5.0 77.5

1 2.5 2.0 1.0 6.0 80.0

2 1.5 1.5 1.0 7.0 80.0

3 1.5 1.5 5.0 8.0 80.0

4 1.0 1.0 10.0 9.0 82.5

5 0.5 0.5 20.0 10.0 85.0

Table 1. Parameters of the plane detection algorithm. Precision 0 correspond to the lowest precision, and 5 to the highest.

the beginning of the histogram, until we reach the percentage minPeak of the total points, and then we keep expanding the peak while the parameter minNormal is fulfilled. After this, we trim the start and end of the peak, and make a final check against the parameters to see if the histogram is accepted or not. If it is not, we go back to Step 3 with the next bucket; Step 5 - MSAC: If the histogram is accepted, it means we found a promising candidate normal. Now we use the MSAC algorithm [8] to refine the candidate normal, calculating the plane normal direction with higher precision. We must note that this step is essential, because the normal vectors derived from the buckets are only approximated. For this step, we consider only the points belonging to the histogram peak, calculated in Step 4. In summary, MSAC randomly chooses sets of 3 points, determines the plane that contains the 3 points and calculates the compatibility of all peak points with this determined plane. This process is repeated several times, and the best plane found during the process is selected. In the MSAC algorithm we use a new parameter, mmTolerance. This parameter specifies a maximum distance in millimeters between a point and the plane, to consider the point belonging to the plane. It is related to the noise present in the range image; Step 6 - Least Squares: Once we have a plane calculated by MSAC, it is time for a new data refinement. The main goal of MSAC is to classify points as inliers or outliers, so that only inliers are taken into account when determining the plane. Now that we know the inliers, we calculate the best plane that approximate them using least-squares. In our implementation, we built a covariance matrix of the inliers, and

calculate its eigenvectors with the Jacobi method. The plane normal direction is the eigenvector with the minor eigenvalue, and the plane origin is used as the center of mass of the inliers; Step 7 - Final validation: Finally, we perform a final validation of the plane found. We use two additional parameters: the maximum percentage of points below the supporting plane (maxBelow) and the minimum percentage of points above the supporting plane (minAbove). Given the plane calculated by least-squares and the parameter mmTolerance, we classify each point as above, below or inside the plane. The plane is accepted, and all points inside or below the plane are eliminated, if we have the following conditions: the number of points below the plane are less than the value defined by maxBelow; the number of points inside the plane is greater than the value defined by minPeak; and the number of points above the plane is greater than the value defined by minAbove. Otherwise, we return to Step 3 to try a new bucket. 3. EXPERIMENTAL RESULTS Practical results from our algorithm can be seen in Fig. 4. It is able to detect and eliminate supporting planes in difficult situations, like the one shown in Fig. 4(d), where only 5.6% of the total points belong to the supporting plane. The Step 4 of the algorithm, which tests the histogram conformity, may appear unnecessary but it is not; without it, the MSAC algorithm would fail, because it can only handle less than 50% of outliers in the data. If MSAC was to be used in the data from Fig. 4(d) directly, the algorithm would fail, because only 5.6% of the total points are inliers. The purpose of Step 4 is to increase the concentration of inliers (to an approximate level defined by the parameter minNormal), to ensure the correctness of MSAC. Furthermore, another benefit of Step 4 is that the execution time is greatly reduced. This happens because MSAC is too costly, and through Step 4 only promising normal directions lead to the MSAC calculation. Table 1 presents the 5 parameters that control the algorithm, and how they are selected by the precision setting chosen by the user. These parameters were empirically obtained through our experiments on more than 1200 range images, from more than 30 different objects. The parameters are fairly intuitive, and finding a good range of values that worked on all test cases was easy. In all these experiments, the plane was successfully found, although with different precision settings in each case. Therefore, the precision setting offers enough control over the algorithm to work with fairly different range images. Results from these experiments can be found at http://www.imago.ufpr.br/museu3d/en_index.html . In our tests, the supporting plane is usually detected in a few milliseconds; and when it does not exist (or if the precision setting is too high for the image in question), the algorithm takes less than 10 seconds to reach this conclusion, in an average PC (Pentium 4 2.4GHz with 1GB of RAM). Besides the separation of the object of interest from the

ners. It is a simple but very useful tool to work with range images, performing flawlessly on all our experiments to date. An improvement to the algorithm would be to develop heuristics to adjust the parameters automatically, according to the range image in question. If a plane is known to be present, the modified algorithm would work on this assumption, to return this plane with the highest precision possible. (a)

(b)

5. REFERENCES [1] K. Ikeuchi, T. Oishi, J. Takamatsu, R. Sagawa, A. Nakazawa, R. Kurazume, K. Nishino, M. Kamakura, and Y. Okamoto, “The Great Buddha project: Digitally archiving, restoring, and analyzing cultural heritage objects,” IJCV, vol. 75, no. 1, pp. 189–208, 2007. (c)

(d)

[2] G. Pavlidis, A. Koutsoudis, F. Arnaoutoglou, V. Tsioukas, and C. Chamzas, “Methods for 3D digitization of cultural heritage,” Journal of Cultural Heritage, vol. 8, no. 1, pp. 93–98, 2007. [3] P. F. U. Gotardo, O. R. P. Bellon, K. L. Boyer, and L. Silva, “Range image segmentation into planar and quadric surfaces using an improved robust estimator and genetic algorithm,” IEEE Trans. on Syst., Man, and Cyber., Part B, vol. 34, no. 6, pp. 2303–2316, 2004.

(e)

(f)

Fig. 4. Results from our plane detection algorithm: (a) and (b) color images of test objects; (c) and (d) captured range images, with points from the supporting planes; (e) and (f) range images with the supporting planes automatically eliminated. We note in (b) how color segmentation techniques would be hard to be applied in this case, due to the similarity between the object and the background. background, the supporting plane detected can be useful to other purposes. For example, in our 3D reconstruction pipeline, the plane defines a half-space where we know that the object is not present. This way, each range image helps to define the convex hull of the object, which is then used to discard outliers outside the convex hull. Other use would be for the space carving operation, as used in VRIP algorithm [10]. 4. FINAL REMARKS In this work, we present a practical algorithm to eliminate planar background from range images. This algorithm can be used in several applications, because the assumption of a supporting plane is easily attainable in a variety of scenarios. Our algorithm detects and eliminates the supporting plane automatically. This greatly reduces a laborious task of manually eliminating the background of range images, which happens in several commercial packages that ship with 3D scan-

[4] Y. Ding, X. Ping, M. Hu, and D. Wang, “Range image segmentation based on randomized Hough transform,” Pattern Recognition Letters, vol. 26, no. 13, pp. 2033– 2041, 2005. [5] A. Bab-Hadiashar and N. Gheissari, “Range image segmentation using surface selection criterion,” IEEE Trans. Image Proc., vol. 15, no. 7, pp. 2006–2018, 2006. [6] J. M. Biosca and J. L. Lerma, “Unsupervised robust planar segmentation of terrestrial laser scanner point clouds based on fuzzy clustering methods,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 63, no. 1, pp. 84–98, 2008. [7] I. Lavva, E. Hameiri, and I. Shimshoni, “Robust methods for geometric primitive recovery and estimation from range images,” IEEE Trans. on Syst., Man, and Cyber., Part B, vol. 38, no. 3, pp. 826–845, 2008. [8] P. Torr and A. Zisserman, “MLESAC: A new robust estimator with application to estimating image geometry,” CVIU, vol. 78, no. 1, pp. 138–156, 2000. [9] N. Greene, “Environment mapping and other applications of world projections,” IEEE Computer Graphics and Applications, vol. 6, no. 11, pp. 21–29, 1986. [10] B. Curless and M. Levoy, “A volumetric method for building complex models from range images,” in Proc. SIGGRAPH, 1996, pp. 303–312.