Eigen space approach for a Pose detection with ... - Semantic Scholar

Eigen space approach for a Pose detection with range images Toshiyuki AMANO Shinsaku HIURA Akashi YAMAGUTI and Seiji INOKUCHI Department of Systems Engineering Osaka University, Japan Abstract An application of the Pose-detection using Range-image usually uses characteristic matching of the geometrical model but this method has two problems. 1 Picking for characteristic from Range-images is difficult. 2 It is difficult to make geometrical model for a complicated shape. Recently Parametric Eigen-space Method is proposed to Pose-detect approaching such as Eigen-face. This method makes Object-Recognition, Pose-detection possible, but this Parametric Eigen-space Method has a problem that intensityimages depend on a variety of light condition therefor learning images must include variation of light condition. So we propose a Pose detection with range images by Eigenspace approach.

1

Introduction

In this paper, we propose a pose detection method using range-images. Most pose detection methods have to use a geometric model of the object, and it is necessary a simply model. So we use matching visual appearance for a pose detection. This matching visual appearance usually use intensity images, but it is difficult to learn all scene. Because the intensity image is changed by light condition [1][2]. To apply range images, a pose detection using matching visual appearance becomes simple. For a compress of learning range images and a retrieval method, we use Eigen-space. The use of range image for the Eigen-space Representation has one more advantage. Range-image expresses a geometrical object information. It is easy to make range-image from polygon data to using camera-parameter, so measurements of range images are not necessary to get many learning images. We call this method "Virtual Learning". Virtual Learning is easy method but it has a problem that range image from Active Range Finder has pixel lacks (pixel lack is a pixel that has no range value information). This pixel lacks occur by occlusion from projection light and/or texture of object surface. This pixel lacks can not project on Eigen-space, because those pixels have no range value. For that reason, we introduce a method using Eigen-space Representation to treat pixel lacks. We use range-image for the method of matching visual appearance for a pose detection. The out line of this paper as follows: In section 2 we establish

Parametric Eigen-space Method using range-image. Section 3 indicates problems to using Virtual Learn and shows a solution for problems to use a Robust Pose-detection. Section 4 shows results of the Experiment. Section 5 offers some conclusion and presents an idea for mulitiangle posedetection.

2

Parametric Eigen-space Method for Range Images

In this paragraph, explain about to applying range-images at Eigen-space Method [3][4]. Matching visual appearance for a pose detection is need to learn of the all scene for continuous movement. In the case of applying range-images to the pose detection, we use depth direct to camera view of the range-images. Because the range-image has position (X, Y, Z) but each pixel is direct to sight-line.Therefor it (X, Y, Z) expression is lengthy if regard Camera-Parameter as known. There for Image vector as range-image is make from Raster-scan of depth value.

[

x *i = x1 , x2 ,L , ximageSize

]

scene = i

for i =1,2,L ,sceneNum

, imageSize = sizeXsizeY

(1)

is the number of learning all images, is the number of pixels for one learning image (columns' rows' ), is range value of one pixel. In case of matching visual appearance using intensity images, distribution value for the all pixel value is important. In case of the range image, each pixel value is important because the pixel value of range image has geometrical information. So we don’t use normalize of the image vector. When use intensity images for the Eigen-space method the learning images must be contain variation of many light conditions but to use range-images it not depends light condition. Therefor in case of to use range-images are not need light conditions of many variations. Equation (2) is mean vector at learning image vectors.

c=

1 sceneNum

sceneNum * ∑ r r =1

x

(2)

Project to Eigen-space F

The learning images are express by Equation (3).

T

x 2 , L , x sceneNum

The covariance matrix

T

]

(3)

Input image has Pixel lacks Mask of pixel lacks (Pixel Number is x･y) as Matrix Sigma

is express by learning images .

Q = XXT

(4)

Eigen-vector is calculate by to apply orthogonal decomposition to matrix Q, and these learning images are able to compress to use dim piece of Eigen-vectors less than sceneNum.

[

e1 , e 2 ,L , e dim , ei = e1 , e2 ,L , eimageSize

]

T element = i

ｆ1

ｅ1

Generation of Eigen-space F to Using SVD method

In this paper we consider a pose detection of object rotation for one axis, so the locus for learning images to be closed line at dim-Dimension. We used linear interpolation for the Pose detection.

Ｙ

Ｙ

AAA AAA AAA AAA AAA AAA

Ｚ

AA A AA A

Ｘ

Projector

Camera

Projector

Ｘ

Camera

Occlusion from projection light

The object has texture

Figure2. A couse of the Pixel lacks

[

) x i = x1 , x2 ,L , 0,L , 0,L , ximageSize

Robust Pose-detection

In this paragraph explain a problem of Virtual Learning and establishes a method of “Robust Pose-detection” that is not depends pixel lacks (fig 1). The virtual learning has a problem. In case of active range finder, it is easy to get pixel lack information from the number of space encoding but it is difficult to forecast these pixel-lacks from the range finder model (The range finder model has camera and projectorparameter, model situation of texture of object surface). There for we have to consider a method that ignores the pixel lacks to treating range images of actual measurement. So we propose a “Robust Pose-detection”. In Equation (3) if the element value of image vector is zero, it means applying to mean value to the element of image vector. To applying mean-value at one lack pixel is make independent of Eigenspace encoding is given by

Addition of Pixel lacks

Figure1. A concept of the Robust Pose-detection.

(7)

(8)

AA AA AAA AAAAA AA AA AA AA

Approximate image-vector from Eigen-space E

This liner-Combination is expressed by projections of learning image vector to Eigen-space and Eigen-matrix E is transpose Matrix, so image vector is an approximate value reappeared by Equation (7) is expressed

3

Ｎ

(5)

(6)

1 2 3

Liner-Combination of Eigenmatrix E and Location P

Positions of Eigen-space are locate by image-vectors shown Equation (7), Eigen-vectors. location Pi = [ p1 , p2 ,L , pdim ]scene = i = Xi E

1 2 3

Ｎ

is singular value at Eigen-vector and Eigen-matrix shown as E at Equation (6). E = [e1 , e 2 ,L , e dim ]

Pose-detection from the locus of Eigen-space F

Pixel lacks

ｆ2

for i = 1, 2,L , sceneNum

Ｚ

[

X ≡ x

T 1 ,

ｅ2

xi = x*i − c

[

= x1 , x2 ,L , ximageSize

1  1         0  

]

]

scene = i

, imageSize = sizeXsizeY

0 0 0

     =xΣ i      1

(9)

To use this expression, Eigen-space and Eigen-vector E for one image have pixel lacks is reconstruct. Equation (10) shown Covariance matrix for one image has pixel lacks. ) T Qscene = i = Σ i T [ x1T , x 2 T ,L , x sceneNum T ][ x1T , x 2 T ,L , x sceneNum T ] Σ i

(10)

Eigen-space Ｅ Image Vector

Mask of pixel lacks Pixel lacks

Generation of Eigen-space F

Input image

Locus of Eigen-space F Function of Pose-detection at Eigen-space F Liner Combination of Eigenmatrix E

Pose information Invert Function of Posedetection at Eigen-space E

Remaked Image

Picture of the Model Polygon-Mogel Figure4. Object

Figure3. Remake non pixel lacks image from Robust Pose-detection

4 To operate orthogonal decomposition, Eigen-vector E is reconstructed to Eigen-vector F. Eigen-vector F is express in Equation (11) which image vector has pixel lacks.

[

f1 , f2 ,L , fdim , fi = f1 , f2 ,L , fimageSize F = [f1 , f2 ,L , fdim ]

]

T element = i

(11)

Positions at Eigen-space F are express in Equation (12). (12) When operate orthogonal decomposition to Equation (10) is a lengthy expression, because rank of Eigen-vector E is ) ) ) (13) rank [ x1T , x 2 T ,L , x sceneNum T ] ≤ dim

(

)

So we use SVD method. ) ) ) Σ i [ x1T , x 2 T ,L , x sceneNum T ] ≡ [ x1T , x 2 T ,L , x sceneNum T ] = UDλ V T U imageSize × dim

U T U = I dim × dim

V T dim × sceneNum

V T V = I dim × dim

Dλ

dim × dim

(14)

traceMatrix for Singular _ Value

Eigen-vector F and positions at Eigen-space F are express by Equation (15) to use of SVD. U=F

) Dλ VT = P

(15)

This Eigen-space F is the best conditions for the pixel lacks image and this Eigen-space F is orthogonal space. It is correct distance space for each projection point of learning images. To use of Eigen-space F it able to doing pose-detection at images has pixel lacks, and it is possible to get pose information. So it is possible to make non pixel lacks image by this pose information that approximate at rank of Eigenvector Matrix E (shown in figure 3).

Experimental Results

4.1. Computer Simulation In figure 4 shown a picture of the object and Polygon model expressed by wire frame of the this object. Rangeimages are able to make from Polygon model of the objects and Eigen-vector, if regard Camera-Parameter as known. We called this method as Virtual-Learning. We get a CameraParameter by the Calibration, and generate range-images by Simulation. Resolution of generate image is 24*24 pixel, and these range images are not consider pixel lacks. These range-images are generate by the object rotation at vertical axis of the object and rotational step is 64 step (one step is 5.625 degree). Figure 5 is the Eigen-vector and it generated from Orthgonal decomposition of these range images. Figure 6 is Eigen-space representation for the object (called Eigenspace E). We generate Eigen-space in 16 Dimension and figure 6 is a locus of the learning images expressed in 3 Dimension. We gave any pixel lacks patterns show figure 7 (random, half of vertical, half of horizontal, checked), to these learning results and simulated a pose-detection by Robust Posedetection. The results shown in figure 8.1 to 8.4. Pixel lacks patterns must be generated by occlusion from projection light or texture of object surface, but this lacks pattern is difficult to make from simulation. The accuracy of the pose-detection is not depend on location of the lack pixel it only depends the total number of pixel lacks. So these patterns are enough to evaluate this method. Figure 9 shown Eigen-space representation from the Back Projection. This results by Robust Pose-detection expressed on the Eigen-space E is in case of pixel lacks pattern shown figure 7.1 (random). This locus is not sharp compare with locus shown in figure 6, but these loci is resemble each other. For that reason Robust Pose-detection seem working well. Figure 10 shown range images that use to pose-detection and remade range images. This result shows this method is able to restore to use Eigenvector, locus of Eigen-space. In figure 10.2, part of roof edge at remade image is not sharp that caused by approximate Dimension is not enough to remake range images. It looks likes Gibbs’ phenomenon. In case of using SVD method

360

Y X Axis

315 270 225 180 135 90 45

315 270 225 180 135 90 45 0

0

Eigen Vector e1

Mean Vector

Detect Angle of Pose [deg.]

is Ax


Depth Z

360

0

0

45 90 135 180 225 270 315 360

45 90 135 180 225 270 315 360 Input Angle of Pose [deg.]

Input Angle of Pose [deg.]

Figure8.3. Pose-detection (horizontal pattern 7.3)

Figure8.4. Pose-detection (checked pattern 7.4) 20

Eigen Vector e2

Eigen Vector e3 Eigen Vect or ｅ3

Figure5. Eigen Vector E

Angle Pose

.]

0

Eigen

Vector

ｅ1 0

tor

Figure9. Back Projected Locus

ge

tor ｅ1

on the Eigen-space E

20

tor

ec

0

nV

Vec Eigen

2 ｅ

-20

0

0

-20

0

ec

Angle [deg

20

nV

Pose

0

Ei

20

0

ge Ei

or ｅ3

Eigen Vect

[deg

.]

20

2 ｅ

Figure6. Locus on the Eigen-space E

20

calculation time is about 2 seconds (Calculate by Silicon Graphics Indy).

4.2. Apply to S.R.F. Normal pixel: It have range value information (depth Z). Lacking pixel: It have no range value information. Axis X

Axis Y

7.1 random 7.2 virtical

7.3 horizontal 7.4 checked

Figure7. Pixel lacks patterns 360



360 315 270 225 180 135 90 45 0

315 270 225 180 135 90 45 0

0


0


Figure8.1. Pose-detection

Figure8.2. Pose-detection

(random pattern 7.1)

(virtical pattern 7.2)

To confirm usefulness of this Robust Pose-detection, we attempt to apply this method to real time range-finder SRF[5]. SRF is a real time range finder that used slit laser with rotational mirror shown in figure 11. This range-finder is measures time-stump and transforms to space encoding. The range image is able to calculate to using the Systemparameter after the measurement. The System-parameter is make up Camera-Parameter and Projector-Parameter. To begin learning we assumes pixel lacks for only texture of object surface (shown in figure 12). Figure 12.1 is learning term that has no pixel lacks for texture of object surface. After doing pose-detection that has black mask, and this mask gives pixel lacks of head part shown in figure 12.2. In case of figure 12 get a result of pose-detection shown in figure 13. In figure 13 this locus of the detected angle is not straight it is seems the number of pixel lacks are very much. Actually, the number of pixel lacks is about 70% of all pixels (all number is 24*24) around at set pose (110 degree). Pixel lacks of other part is less than about 50% of all pixels. So we think this method depends on only pixel lacks number and testing at case of range image has non pixel lacks. In figure 12.2 we removed mask, and put the Unit Matrix to matrix . This result shown in figure 14, and this result is get a smooth locus.

Object coordinate set (X,Y,Z)

AAAA AA AAAA AA

is X

Black Mask

10.2. remade range image

Figure10. No pixel lacks image from the Robust Pose-detection

Ｘ

Turn Table

Rotational Direction

Ｚ

Y

Y

is

Axis

X

10.1. input range image

Ｙ

Object

Ax

Ax Axis

AA AA AA AAAA A AAAAA

Rotational Axis

ＳＲＦ

12.1. Learning term

.

Rotational Axis Object Ｘ

Turn Table

Rotational Direction

Ｚ

Depth Z

Depth Z

Ｙ

Object coordinate set (X,Y,Z)

ＳＲＦ

12.2. Pose-detection term

Figure12. Experimental set-up Object

SR

F

LS

IS

Refrection of the Slit light

Lens

en

so

r

AAAAAA AAAAAA AAAAAA AAAAAA AAAAAA

Slit laser projector

A

B Time

O

O Compare output voltage

Figure11. Structure of the S.R.F.

5

180 135

Conclusion

In this paper we proposed the method for a Pose-detection using range images. This method is one of the matching visual appearance expressed in the Eigen-space. To using rangeimages for this method the cost of learning image variations is reduce than to using intensity images. Range-images have geometrical information. If Polygon model of the object is known, learning is possible without actual measurement. This imaginary measurement is useful for time cost of the measurement but it has a ploblem.The actual measurement contains pixel-lacks but it is difficult to generate pixel-lacks by the imaginary measurement (called Virtual learning). For that reason, we developed the method it not depends pixel lacks at the Eigen-space. We called this method Robust Posedetection. In the computer simulation, this method has an enough robustness if the number of pixel lacks is less than 50%. After we apply to the Real time range finder, “Silicon Range Finder” calls SRF. In this SRF we attempt Robust Pose-detection for pixel lacks by the texture. In this time it method has an enough Robustness less if the number of pixel lacks is less than 50%. In this paper we attempt a posedetection for the one Axis rotation only but in the FA and CV applications are need to multi-angle and Position detection any where. So we consider 2 points of ideas against

360 315 270 225 180 135 90 45

0

-

are

225

0

+

mp

270

45

Rotational mirror

Co

315

90

B A


Sight line


360

45

90 135 180 225 270 315 360 Input Angle of Pose [deg.]

Figure13. Pose-detect result

0 0

45

90 135 180 225 270 315 360 Input Angle of Pose [deg.]

Figure14. Pose-detect result(non pixel lacks)

to this problem. 1 Using pan-tilt stage and tacking. 2 Make many Eigen-spaces for depth of the camera view. Pick up two Eigen-spaces by the mean of range value and detect poses to using Eigen-space generated by the interpolation. This 6 axes pose-detection is degenerate 3 axes pose detection to using 1,2. In this case assume learning step is 32 at one rotation then all steps of 3 axes are about 33,000 steps. This number is too big and difficult to learn all scene, and this problem is a theme of the future.

Reference [1] Sirovich, L. and Kirby, M.: Low Dimensional Procedure for the Characterization of Human Face, Journal of Optical Society od America, Vol.4, No.3, pp.519-524 (1987). [2] Turk, M.A. and Pentland, A.P.: Face Recognition using Eigenface, Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp.586-591 (June 1991). [3] Murase, H. and Nayar, S.K.: Learning Object Models from Appearance, AAAI-93, pp.836-843, American Association for Artifical Inteligence (July 1993). [4] Murase, H. and Nayar, S.K.: Illumination Planning for Object Recognition in structured Environments, Proc. of IEEE Conference on Computer Vision and Pattern Recognition (June 1994). [5] Kosuke, S., Atsushi, Y. and Seiji, I.: Silicon Range Finder --A Realtime Range Finding VLSI Sensor--, CCIC '94, pp.339-342, Proc. of IEEE Custom Integrated Circuits Conference (May 1994).