Sensor Planning in 3D Object Search: its Formulation and ... - CiteSeerX

Sensor Planning in 3D Object Search: its Formulation and Complexity 1 Yiming Ye and John K. Tsotsos

Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 1A4 [email protected] [email protected]

Abstract

Object search is the task of searching for a given 3D object in a given 3D environment by a robot equipped with a camera. Sensor planning for object search refers to the task of how to select the sensing parameters of the camera so as to bring the target into the eld of view of the camera and to make the image of the target to be easily recognized by the available recognition algorithms. In this paper, we study the task of sensor planning for object search from the theoretical point of view. We formulate the task and point out many of its important properties. We then analyze this task from the complexity level and prove that this task is NP-Complete.

1 Introduction

The research described in this paper conforms to the complexity level analysis of the sensor planning task for object search. Complexity considerations are commonplace in the biological and computational vision literature. For example, Tsotsos [8] shows that the general problem of visual search (search for a target within an image) is computationally intractable in a formal, complexity-theoretic sense. He [9] also ties the concept of active perception to attentive processing in general and to his complexity level analysis of visual search and proves that active unbounded visual search is NP-Complete. Kirousis and Papadimitriou [4] show that the problem of polyhedral scene labeling is inherently NP-Complete. Many other vision researchers ([3],[6], etc.) routinely provide an analysis of the complexity of their proposed algorithms. Complexity level analysis of robotics and vision problems is important because it can reveal basic insights into the structure of the problem and delimit the space of permissible solutions in a formal and theoretical fashion. Object search is the task of searching for a given 3D object in a given 3D environment. Sensor planning for object search refers to the task of how to select the sensing parameters so as to bring the target into the eld of view of the sensor. This task is very important if a robot wants to interact intelligently and eectively with its environment. Connell [1] constructs a robot that roams an area searching for and collecting soda cans. The planning is very simple since the robot just follows the walls of the room and the sensor only searches the area immediately in front of the robot. Rimey and Brown [7] use composite Bayes net and utility decision rule to plan the sensor in their task-oriented system TEA. The indirect search mechanism proposed by Garvey [2] is to rst direct the sensor to search for an \intermediate" object that commonly participates in a spatial relationship with the target and then direct the sensor to examine the restricted region speci ed by this relationship. Wixson and Ballard [10] present a mathematical model of search eciency and predict that indirect search can improve eciency in many situations. It is interesting to note that the operational research community has done a lot of research on optimal search [5]. Their purpose is to determine how to allocate eort to search for a target, such as a lost submarine in the ocean or an oil eld within a certain region. Although the results are elegant and beautiful in a mathematical sense, they can not be directly applied here because the searcher model is too abstract and general and there is no sensor planning involved in their approach. There is no previous research within the computer vision community that attempts to formalize the sensor planning task for object search in general and to analyze this problem at the complexity level. This 1 This paper is submitted to Annals of Mathematics and Arti cial Intelligence. It is the full length version of of the paper with the same title which is accepted for presentation by \Fourth International Symposium on ARTIFICIAL INTELLIGENCE AND MATHEMATICS", Florida, U.S.A., January 3-5, 1996

1

paper is an attempt along this direction. By combining the probability distribution of the target and the detecting ability of the recognition algorithms, we formulate the sensor planning problem for object search, discuss several properties of this task, and prove that this task is NP-Complete. The theoretical result provided in this paper has been used as a guideline in designing the practical sensing strategies (see [11] for detail).

2 Problem Formulation Although it is important to examine dierent aspects of object search individually and in some degree of isolation, it is even more important to study their relationship and how to integrate them into a whole search system. The search region can be in any form and it is assumed that we know the boundary of exactly but we do not know its internal con guration. In practice, we will tessellate the region S into a seriesTof little elements c (here, we assume that those elements are in the form of little cubes), = =1 c and c c = 0. The searcher is a mobile platform equipped with a camera that can pan, tilt and zoom. The state of the searcher is uniquely determined by 7 parameters (x ; y ; z ; p; t; w; h). (x ; y ; z ) is the position of the camera center (the starting point of the camera viewing axis), (p; t) is the direction of the camera viewing axis (p is the amount of pan, t is the amount of tilt). w; h are the width and height of the solid viewing angle of the camera. (x ; y ; z ) can be adjusted by moving the mobile platform. (p; t) can be adjusted by the motors on the robotics head. w; h can be adjusted by the zoom lens of the camera. Only nite number of platform positions is allowed. An operation f = f (x ; y ; z ; p; t; w; h; a) is an action of the searcher within the region , where a is a recognition algorithm. An operation f entails: take a perspective projection image according to the current camera con guration and then search the image using the given recognition algorithm. The number of total dierent operations is big, but is not in nite. This number is determined by hardware properties of the mobile platform, the robotics head, the zoom camera, and the available recognition algorithms [11]. The target distribution can be speci ed by a probability distribution function p. p(c ; t) gives the probability that the center of the target is within cube c at time t. The detection function on is a function b, such that b(c ; f ) gives the conditional probability of detecting the target given that the center of the target is located within c and the operation is f . For any operation, if the projection of the center of the cube c is outside the image, we assume b(c ; f ) = 0; if it is too far from the camera or too near to the camera, we also have b(c ; f ) = 0. In general [11], b(c ; f ) is determined by various factors, such as intensity, occlusion, and orientation P etc.. It is obvious that the probability of detecting the target by applying action f is given by P(f ) = =1 p(c ; tf )b(c ; f ): Where tf is the time just before f is applied. The reason that the term tf is introduced in the calculation of P(f ) is because that the probability distribution need to be updated whenever an action is failed. Here Bayes' formula is used to incorporate the new recognition results into the old distribution. Let  be the event that the center of the target is in cube c ,  be the event that the center of the target is outside the region, let be the event that after applying a recognition action, the recognizer successfully detects the target. Since the above events 1; : : :;  ;  are mutually complementary and exclusive, we can get the following formula )P( : j  ) P (1) P( j : ) = P( )P(: j P(  ) + =1 P( )P(: j  ) ; i = 1; : : :; n; o Thus we have the following updating rule (Note: tf + refers to the time just after the action f is applied): p(c ; tf +) p(c ; t ) p+(cP; tf )(1p(c? ;bt(c)(1; f ))? b(c ; f )) ; i = 1; : : :; n; o (2) =1 f f P P Since p(c ; tf ) + =1 p(c ; tf ) = 1 and P(f ) = =1 p(c ; tf )b(c ; f ), the updating rule becomes ? b(c ; f )) ; i = 1; : : :; n; o (3) p(c ; tf +) p(c ; tf1)(1 ? P(f ) n

i

i

i

c

c

c

c

c

c

c

c

j

i

c

c

c

c

i

i

i

i

i

i

i

i

n

i

i

i

i

i

o

n

i

i

i

n

o

o

i

i

o

j

j

i

n

j

j

n

o

j

j

j

n

j

j

j

i

i

i

2

j

o

The cost function t(f ) gives the time required to execute action f . This include the time needed to adjust the camera con guration to the status speci ed by f and the time needed to take a picture under current con guration and run the recognition algorithm. We assume that the cost for a given operation is xed and it is not in uenced by the previous actions. Let O be the set of all possible operations that can be applied. The eort allocation F = ff1; : : :; f g gives the ordered set of operations applied in the search, where f 2 O . Let tf represent the time just before the action f (1  i  q) is applied. It is clear that the probability of detecting the target by this allocation is: X X X [F] = p( f1 )b( f1) + [1 ? p( f1 )b( f1)]  [ p( f2 )b( f2 )] q

i

i

i

n

n

P

ci ; t

i=1

n

ci ;

ci ; t

?1 Y

n X

j

i

q

ci ;

i=1

ci ; t

n X

f [1 ? p( f )b( f )] g  [ p( =1 =1 and the total time for applying this=1allocation is (following X [9]): +

+

:::

ci ; t

ci ; j

j

ci ; t

i

F] =

T[

iq

1

ci ;

i=1

f )b( f )]

(4)

ci ; q

q

t(f )

(5)

i

Suppose K is the total time that can be allowed in the search, then the task of sensor planning for object search can be de ned as nding an allocation F  O , which satis es T(F)  K and maximizes P[F].

3 Some properties of the object search process We list some properties of the sensor planning task in this section, proofs are omitted. S  g. For any operation f 2 O , we de ne its in uence range Suppose = =1 c , O = ff1 ; f2 ; : : :; fm

as (f ) = fc j b(c; f ) 6= 0g. Its complement form is: (f ) = ? fc j b(c; f ) 6= 0g. For any eort allocation F = ff1; : : :; f j f 2 O g. The initial probability distribution is denoted as p[0] (c1); p[0](c2 ); : : :; p[0](c ); p[0](c ). After the application of the operation f1, the distribution is denoted by p[1] (c1 ); p[1](c2 ); : : :; p[1](c ); p[1] (c ). Generally, after the application of the operation f , the distribution is denoted by p[ ] (c1); p[ ] (c2 ); : : :; p[ ](c ); p[ ] (c ), where 1  i  q. Let P(f ) represent the probability of detectingPthe target by applying the action f with respect to the allocation F. Then of course we have P(f ) = =1 p[ ?1] (c )b(c ; fi). Let P [0] (f ) represents the probability of detectingPthe target by applying the action f when there is no action been applied before, then we have P [0](f ) = =1 p[0] (c )b(c ; fi). n

i

i

q

n

o

n

i

n

i

j

i

o

i

i

i

i

n

o

i

i

i

j

j

i

n

i

i

Lemma 1. For allocation F = ff1; : : :; f j f 2 O g, we have [0] (c)t1 (c)t2 (c) : : :t (c) p[ ] (c) = (1 ? P(pf ))(1 1 ? P(f2)) : : :; (1 ? P(f )) where  t (c) = 1 ? b(c; f ) if c 2 (f ) ; i = 1; : : :; k q

j

j

i

k

k

k

i

j

i

1

otherwise

i

Proof (1) When k = 1, from update rule 3 we have [0] f1)) p[1] (c) = p[0] (c )p+ P(c)(1 p?(cb(c; )(1 ? b(c ; f )) =1 [0] ? b(c; f1)) P = P pp(c )(c)(1 ? =1 =1 p(c )b(c ; f )) [0] ? b(c; f1)) = p (c)(1 1 ? P(f1) n

o

n;o i

i

i

i

n

i

i

3

i

i

(6)

Since b(c; f1) = 0 when c 2 O ? (f1 ), we have

[0] f1 ) p[1](c) = p 1 ?(c)g(c; P(f ) 1

So, the result is true when k = 1. (2) Suppose when k = r, we have

[0] f1)g(c; f2 ) : : :g(c; f ) p[ ] (c) = (1 ?pP((c)g(c; f ))(1 ? P(f )) : : :; (1 ? P(f )) r

(3) When t = r + 1,

1

r

2

r

[] p[ +1] (c) = p[ ] (c ) +pP(c)(1p[?] (cb(c;)(1f ?+1b))(c ; f )) +1 =1 [ ] (c)(1 ? b(c; f +1)) p = P p[ ] (c ) ? P p[ ] (c )b(c ; f )) +1 =1 =1 [ ] ? b(c; f +1)) = p (c)(1 1 ? P(f +1 ) (c; f +1)) = p[ ] (c) (11??bP( f +1 ) [0] f1)g(c; f2) : : :g(c; f ) (1 ? b(c; f +1)) = (1 ?pP((c)g(c; f1))(1 ? P(f2)) : : :(1 ? P(f )) (1 ? P(f +1 )) [0] f1)g(c; f2) : : :g(c; f )(1 ? b(c; f +1)) = (1 p? P((c)g(c; f1))(1 ? P(f2)) : : :(1 ? P(f ))(1 ? P(f +1 )) Since b(c; f +1) = 0 when c 2 O ? (f +1 ), we have p[0] (c)g(c; f1)g(c; f2) : : :g(c; f )g(c; f +1 ) p[ +1] (c) = (1 ? P( f1))(1 ? P(f2)) : : :(1 ? P(f ))(1 ? P(f +1)) So, the property is true when k = r + 1. Thus the conclusion is true for k  1. 2. r

r

r

r

n

o

r

i

i

i

r

n;o i

r

r

r

n

i

r

i

r

i

i

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

r

De nition 1 For any group of actions ff1; : : :; f g, we de ne: \ \ Q[0] (f1 : : : f ) = (f1) : : : (f ) k

k

k

k

k

i

k

k

::: \ \ \ \ \ \ \ \ [ ] Q (f1 : : : f 1 : : : f 2 : : : f : : : f ) = (f1) : : : (f 1 ) : : : (f 2 ) : : : (f ) : : : (f ); ::: \ \ [ ] Q (f1f2 : : : f ) = (f1) : : : (f ); r

i

k

i

ir

i

ir

k

k

k

Where Q[ ] (r  k) means that there are k sets (f1 ), (f2), : : :, (f ) that are taken into the consideration, r of them are in their complement form. For a given k, it is easy to see that the intersection of any two dierent Q[ ] is . The uni cation of all the possible Q[ ] is , as stated in Lemma 2. r

k

k

r

k

r

k

Lemma 2. For the above de ned Q[ ] , we have: r

k

[n

[

k

r

=0 1 1

2

i