Vision Sensor Based Tracking and Focusing of 3-D

Proceedings of the IEEE International Conference on Mechatronics & Automation Niagara Falls, Canada • July 2005

Vision Sensor Based Tracking and Focusing of 3-D Singular Points∗ Tao Dai, Xiang Chen†

Robert Drake

Jun Yang

Dept. of Elect. and Comp. Eng. University of Windsor Windsor, Ontario Canada N9B 3P4

Proto Manufacturing Ltd 175 Solar Crescent Oldcastle, Ontario Canada N0R 1L0

Q-Tec Engineering Group Inc. 4525 Rhodes Drive, Unit 250 Windsor, Ontario Canada N8W 5R8

Abstract— In this paper, a laser scanning vision sensor based automatic focusing and positioning servo control mechanism is presented for 3-D singular points. The mechanism consists of two parts: position detecting and tracking/focusing control. In particular, Difference-from-Moving Average(DMA) algorithm is used to process the 3-D data from a circular laser scanning vision sensor for detecting position of singular points. The processed information is then fed back to the servo motion control and actuating system for accurate tracking and focusing of singular points. The result shows that this mechanism achieves desired tracking speed and accuracy with experimental validation on a 3-D vision tracking platform. Index Terms— DMA algorithm, 3-D vision, tracking

I. Introduction Industrial automation has observed more and more practical use of reliable computer vision systems capable of performing a wide variety of tasks such as inspection, recognition, and positioning of workpieces in a specified work area [5], [6], thanks to the tremendous progresses achieved in computing technologies in the last two decades. Indeed, computer vision based automation has its irreplaceable advantages in terms of non-intervention pattern measurements and versatile robust functions[3], [13], [11]. Three dimensional vision based measurements have come into reality in a long way, moving from purely visualization tools to serious measurement tools [1], [4]. The approaches applied in these 3-D systems include laser scanning[8], stereo viewing[10], and laser radar[12], etc. Each of these methods has its strong points and weak points for a given application in different industries. In general, most measurements in manufacturing industry[7] require that the measurement is conducted at high accuracy, such as resolution at millimeter or even micrometer level, and is completed in a range of almost real time to a few seconds, while capable of accommodating a wide range of geometric shapes. ∗:

This work is partially supported by Material and Manufacturing Ontario Enabling Grant to Xiang Chen. † : correspondence author, E-Mail: [email protected]

0-7803-9044-X/05/$20.00 © 2005 IEEE

Modern laser sensor technologies have made fast and accurate 3-D data acquisition possible as evidence by several commercialized 3-D laser sensors. Other than some existing 3-D technologies based on CCD 2-D image reconstruction[10], [9], these 3-D sensors are based on laser scanning and geometric methodologies such as triangulation[2] and have achieved more and more attentions in machine vision application due to its robustness and simplicity. However, to expand functional application of these laser sensors, powerful algorithm add-on is still needed to effectively process measured data from them, normally depending on individual application case. In this paper, a robust scanning vision sensor based 3-D tracking and focusing servo mechanism is presented for singular points on object surface. By singular point, we mean small convex or concave points deviated from smooth surface manifold, caused by either faulty or normal production/machining. This mechanism targets on solving a class of industrial problems requiring 3-D measurement for inspection, motion tracking control, etc. For example, an automated residual stress measurement using X-ray requires very accurate information about 3-D shape and location of objects for X-ray focusing purpose. This paper is organized as follows: in Section 2, a summary of signal processing algorithm DMA and its cascaded form is presented; in Section 3, the experimental platform and its core data acquisition device-circular scanning sensor (CSS)-are introduced; in Section 4, two experimental results are illustrated for the mechanism, while the paper is concluded in Section 5. II. DMA Algorithm In many data acquisition application for laser scanning sensor, scanned data is obtained in sequence: Xi = [x0 , x2 , · · · , xN −1 ] , i = 1, 2, · · · , where Xi indicates the i − th data sequence captured and N is the number of scanning times called the depth of the data sequence. This data sequence could

1312

Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.

Fig. 1.

A 1-D Circular Scanning Data Cycle

Fig. 2. A 1-D Circular Scanning Data Cycle with A Singular Point

represent data piece acquired by one sensor in different pre-set time period or by multiple sensors at same time. For example, a line or circular scanning laser sensor generates a geometric data sequence of objects in one scanning cycle. In this paper, only circular scanned data is considered, hence the depth N is also the scanning cycle period. An example of a 1-D real circular scanning data cycle is illustrated in Figure 1, which is conducted on a horizontally flat iron plate placed on a table. In ideal case, one would expect scanned data yields a horizontal line. However, this hardly happens in reality. Instead, as Figure 1 shows, a pseudo-sinusoid shape is presented reflecting either unevenness of plate surface or table leaning, or both. In the case that there is a hole-like singular point, caused by either normal or faulty machining, on the surface, the scanned data would normally reflect that point with a peak significantly deviated from the profile, unlike small peaks around the profile induced by noise. Such an example with a hole of depth 0.4 mm at the 100th pixel is shown in Figure 2 (1-D circular scanning data). From these two examples, it is clear that two major tasks have to be performed in order for us to obtain accurate information of targets from laser scanned data: reduction of effect of noise on the measurement and detection of singular peak. In practices, both noise and singular point induce peaks, just with different size. The difference between them is that noise is an extra external disturbance from the measurement instrument itself, the roughness and/or dusts on the object surface, or both, affecting the measured data, whereas singular points are inherent feature of the object. A real scanned data corrupted with noise can then be assumed to be decomposed into

where dj is the nominal data and nj is measuring noise satisfying certain distribution. A closer look at the circular scanned data sequence, the following facts are observed: 1. Signal component induced by object surface fluctuation is in the low frequency range. 2. Signal component induced by singular point is in the high frequency range. 3. Noise signal could spread over a wide range of frequencies. 4. Scanned data is N -periodic for a complete scanning cycle, that is, xk = xk+N , k = 0, 1, · · · , N − 1. These observations are behind the motivation to use the so-called difference-from-moving-average (DMA) algorithm to conduct signal processing in order to detect singular points, which is stated as follows. Definition 1: Given an N -depth data sequence representing a complete circular scanning cycle, {x0 , x1 , · · · , xN −1 } and 0 < n ≤ N2 , an n-augmented data sequence is defined as {xN −n , xN −n+1 , · · · , xN −1 , x0 , x1 , · · · ,

xN −1 , xN , xN +1 , · · · , xN +n−1 }, This is obvious due to the N -periodic property of the data sequence. Definition 3: Given an n-augmented data sequence in 1, the difference from the moving average (DMA) operation for the i-th scanning point is defined as

xj = dj + nj , j = 0, 1, · · · , N − 1,

x ¯i = DM A(x0 , x1 , · · · , xN −1 ) = xi − Win ,

xN −1 , x0 , x1 , · · · , xn−1 }. Proposition 2: The n-augmented data sequence defined in 1 can be re-indexed as {x−n , x−n+1 , · · · , x−1 , x0 , x1 , · · · ,

1313 Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.

i = 0, 1, · · · , N − 1, where Win

i+n  1 = xk , i = 0, 1, · · · , N − 1, 2n + 1 k=i−n

Fig. 3.

is called n-moving average window for the i-th scanning point and n is called radius of the moving average window. It is pointed out that this algorithm can be actually implemented in real time starting from the second scanning cycle. For detection of singular points, a threshold is applied, which is presented in the following definitions. Definition 4: Given an n-augmented data sequence in 1, let   ¯= x ¯0 · · · x ¯N −1 . X The standard deviation for the complete scanning cycle, ¯ is defined as denoted by std(X),  ⎛ ⎞2   N −1 n−1   ¯ =1 ⎝x¯j − 1 x ¯j ⎠ std(X) N j=0 N j=1 Note that the standard deviation is scanning cycle specific and has to be calculated for every cycle. With ¯ an α-threshold can then be defined std(X), Definition 5: Given an α > 0, the α-threshold is defined for the complete scanning cycle as: ¯ std(X) Tα = . α Like the standard deviation, the α-threshold is also scanning cycle specific. In the mechanism presented in this paper, the value of the constant α is chosen in the range of 2 to 5. Normally a smaller α is suitable for the case that the data sequence has high signal-to-noise ratio (SNR) and larger one is suitable for the case with low SNR. It is pointed out that this α-threshold is only an indication of the value range and practical threshold may be manually set in the experiment for computational simplicity according the value of Tα . Finally, the singular point can be qualitatively classified from the following definition Definition 6: For a given data sequence in 1, a scanning point xi is called singular if the following inequality holds x ¯i > Tα . Note that DMA algorithm can be also implemented in cascade to enhance the quality of singular point detection(Figure 3). The detection of singular points in a circular scanned data sequence with 2 cascaded DMA operation is shown in Figure 4 It is extremely suitable to apply cascaded DMA operation to the case that the object surface carries waveform

Cascaded DMA Operation

Fig. 4. Block Diagram of Singular Point Detection with Cascaded DMA Processing

shape, as in this case applying DMA once may not be adequate to extract singular points. In Figure 5, such an example is shown. III. Description of Platform for Experiment In this section, an experimental platform is described that is built to validate the tracking and focusing servo mechanism. This platform features a three degree of freedom (hence, accommodate 3-D) motion control through a MIT Handyboard, three servo motors and a circular scanning sensor (CSS), as well as a computing system. The schematic diagram is shown in Figure 6 and a picture of the platform is shown in Figure 7. An emulated head is attached rigidly with CSS to indicate the tracking and focusing actions. The circular scanning sensor (CSS) (Figure 8), with

Fig. 5. An Example of Singular Point Detection with Cascaded DMA Processing

1314 Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.

Fig. 6.

Schematic Diagram of Platform for Experiment

Fig. 9.

Fig. 7.

Platform for Experiment

embedded data acquisition device, is a laser scanning multi-axis machine vision device which produces 3-D data sequences of a target work piece for reconstruction of geometry characteristics. The specification of its scanning resolution is in Table 1, referring to Figure 9 for illustration of circular scanning. Stand-off Scanning Radius Lateral Resolution Depth Resolution

Illustration of Circular Scanning

Min (75mm) 32.5mm

Nominal (120mm) 52mm

Max (165mm) 71.5mm

200 microns

319 microns

439microns

75 microns

120 microns

165 microns

The CSS and the emulated head are driven by three

Fig. 10.

Flow Chart of DMA Based Operating Algorithm

servo motors(TS-69 S3K) to generate 3-D (x, y, z-axis) motion: arbitrary planar scanning route, up and down motion. The main purpose is to show that the emulated head could track any 3-d singular point on the surface of the work piece while conducting scanning and concentrate on it at specified focusing distance. The motors are controlled by a Motorola 68HC912B32 based HandyBoard which receives location information of singular points from a PC system. The loop is then closed by feedbacking 3-D geometric data of target points acquired by the CSS to the PC system which processes the data sequence continuously using DMA based algorithm to return the location information of singular points, if any, on the surface of work pieces. IV. Experimental Results

Fig. 8.

A Laser Circular Scanning Sensor

The DMA based data processing algorithm is implemented using C++ and is applied to the vision tracking and focusing platform. The operating flowchart is shown in Figure 10.

1315 Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.

Fig. 11.

An Iron Plat

Fig. 13.

Fig. 12.

Singular Point Detection on Iron Plat

A Cylinder-Like Can

To demonstrate the effectiveness of the mechanism, an iron plate and a cylinder-like can (Figure 11 and 12) are used as examples. On the iron plate, there are four holes of which the diameters are 2mm, 1mm, 0.5mm and 0.25mm, respectively, in different location, and the depth of which are the same as their corresponding diameters. The cylinder-like can has an aperture with width of 1mm approximately, between its cover and body. A cascaded two DMA algorithm is used to handle both cases and the results are shown in Figure 13 and 14 As the results show, where the upper plots show the raw data and the lower plots demonstrate the processed data through DMA algorithm, the singular point on both work pieces are clearly detected and positioned in pixels. After the detection, the servo motors accurately move the emulated head to position of the singular point and concentrate the head on the singular point with a focusing distance of 100mm.

Fig. 14. Can

Singular Point (Aperture) Detection on Cylinder-Like

control system. It is also concluded that the Difference from Moving Average (DMA) algorithm is a robust choice to conduct singular point detecting and positioning. The operation results of the mechanism have been successfully validated on an experimental platform.

V. Conclusion A 3-D laser circular scanning sensor based tracking and focusing mechanism is presented. The CSS applied in this system is demonstrated to be applied to real time motion

References [1] Bieman, L., “Three-Dimensional Machine Vision”, Photonics Spectra, p. 81, 1988. [2] Blais, F., Rioux, M., Domey, J. R., and Baraldin, J. A., “Very Compact Real Time 3-D Range Sensor for Mobile Robot Applications”, SPIE Proceedings, Vol. 1007, p. 330, 1988. [3] Gao, H., Chen, X. and Ren, Z., “Algorithm Design for a Camera Based Position Tracking Sensor Based on Pattern Recognition”, Proc. IEEE Industrial Electronics Annual Conference, pp. 2173-2178, 2002. [4] Hall, E. L. and McPherson, C. A., “Three Dimensional Perception for Robot Vision”, SPIE Proceedings, Vol. 442, p. 117-, 1983. [5] Harding, K., “Machine Vision Based Gaging of Manufactured Parts”, Proc. SME Gaging 95, 1995.

1316 Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.

[6] Harding, K., “The Promise of Machine Vision”, Optics and Photonics News, p. 30-, 1996. [7] Harding, K., “3D Machine Vision as a Shop Floor Metrology Tool”, General Electric Research Center, http://www.machinevisiononline.org/public/ articles/General Electric.pdf. [8] Harding, K. and Svetkoff, D., “3-D Laser Measurements on Scattering and Transluscent Surfaces”, SPIE Proceedings, Vol. 2599, p. 2599, 1995. [9] Kerkeni, N., Leroi, M. and Bourton, M., “Image Analysis and Three-Dimensional Object Recognition”, SPIE Proceedings, Vol. 449, p. 426-, 1983. [10] Luh, J. Y. S. and Klaasen, J. A., “A Real-Time 3-D MultiCamera Vision System”, SPIE Proceedings, Vol. 449, p. 400-, 1983. [11] Shahir, S., Chen, X. and Ahmadi, M., “Fuzzy Associative Database for Multiple Planar Object Recognition”, Proc. IEEE International Symposium on Circuits and Systems(ISCAS) , Vol. V, pp. 805-808, 2003. [12] Svetkoff, D. J., Leonard, P.F. and Sampson, R. E., “Techniques for Real-Time Analysis of Laser Radar Data”, SPIE Proceedings, Vol. 521, p. 302-, 1984. [13] Yang, J. and Chen, X., “A Real Time Algorithm for Planar Shape Recognition and Position Deviation Analysis Based on Statistical and Directional Flow Change Methods”, Proc. IASTED International Conference on Circuits, Signals and Systems, pp. 259-263, 2003.

1317 Authorized licensed use limited to: UNIVERSITY OF WINDSOR. Downloaded on April 21, 2009 at 18:57 from IEEE Xplore. Restrictions apply.