Robot Vision - CiteSeerX

NASA Technical Paper 1819

Marking Parts To Aid Robot Vision

John W. Balesand

APRIL 1981

L. Keith Barker

1

TECH LIBRARY KAFB, NM

NASA Technical Paper 1819

Marking Parts To Aid Robot Vision

John W. Bales TzlskegeeIllstitUte Tzlskegee, Alabama

L. Keith Barker LullgleyResearchCellter Huttzptolz, Virgiuia

National Aeronautics and Space Administration Scientific and Technical Information Branch

1981

SUMMARY

The premarking of parts for subsequent identification by a robot vision system appears to be beneficial as an aid in the automation of certain tasks, such as construction in space. Accordingly, a simple, color-coded marking system is presented which allows a computer vision system to locate an object, calculate its orientation, and determine its identity. Furthermore, such a system has the potential to operate accurately, and, because the computer shape analysis problem has been greatly simplified, it has the potential to operate in real time. INTRODUCTION

The use of robots in industry has increased dramatically inlast the several years, particularly in Japan but a l s o in this country. A robot is usually a programmable, general purpose arm dedicated to specific tasks. For the most part, these arms are blind.That is, the software controlling most arms makes no useof visual inputs. How to incorporate visual inputs into such software is a problem of much current interest. One of the most basic tasks of a robot vision system is to locate and identify parts for handling. Four primary problems which must be overcome in order to carry out such a task are noted in the succeeding paragraphs. The first problem is to distinguish the part from its background. Many sophisticated approaches exist for distinguishing the image of a sought object from its background. However, most approaches require too much computation to allow real-time operation and, in addition, require specially controlled lighting. Any system can be said tooperate in real time if it permits the robot arm to operate at an adequate rate. An adequate rate for one situationor operation may be hopelessly slow for others. Since robot arms will, for the immediate future,be operating alongside human workers, any rate which slows down the workers would be inadequate. The second problem is to determine the orientation of the part. In a precisely controlled situation, the sought.part may be prepositioned in the correct orientation. This approach limits the versatilityof the system. An adequate vision system should end the necessity of precise placement of parts for the robot arm. However, software for determining the orientation of complex parts can involve excessive computation, slowing real-time operation. The third problem isto determine the position of the part. In many manufacturing situations this problem is simplified by the fact that the part is on a conveyer belt, the position of which is known. In this situation, a single camera is adequate to provide the information needed to ascertain the position of the part. If the part is noton a conveyer belt and is, in fact, randomly positioned and oriented, some other technique is required. It is

natural to consider the use of two cameras to provide the position information to a robot vision system in this more complex domain. However, the extensive computation required for the cross correlation of two images severely handicaps any two-camera system because of the real-time requirement. Currently, rangefinding systems involving a single camera are more promising. Of particular interest is a system first described by Shirai (ref. 1 ) which uses asingle or plane camera and a plane of light as a range finder. In this system a sheet of light is projected toward an object to be identified. The intersection of this planeof light with the object results in a visible light stripe being drawn across the object's surface. This stripe is essentially moved across the surface by panning the plane-of-light generator. The resulting series of light stripes are recorded by aTV camera. Computer software is used to classifysets of straight line segments in these stripes into planes. The object is then recognized by the relationship between these planes. Other plane-of-light of systems appear in references2 and 3 . Investigators at the National Bureau Standards ( G . J. VanderBurg, J. S. Albus, and E. Barkmeyer) performed experiments usedsuccessfully acquire in 1979 in which a plane-of-light vision system was to an object randomly placed on a table. At present, this type of range-finder system is tooslow for real-time operation in three-dimensional space.

The fourth problem is to track the part if it is in motion. Unfortunately, the time requiredto distinguish the part from its background, determine the orientation of the part, and calculate its position restricts the rate at wh the system can track a moving object.

The purpose of this paper is to report on investigations into a vision system capableof handling the previously mentioned problems while requiring only one camera and a minimum of software. For faster execution time, a simpl color-coded marking system eliminates the time-consuming shape analysis requir by plane-of-light systems to operate in a three-dimensional environment. SYMBOLS

point on observed

object

constant point on observed

object

exact center of circular image centroid of illuminated pixels diameter of cylinder, cm distance from lens center to front image plane, cm diameter of disc image, cm constant 2

M

w i d t h of image p l a pn iex, e l s

m

mean v a l u e

n

image diameter , p i x e l s

P

c e n tleern s

n.

P

p1 rP21P31P41P5

R*

S

P

p o i nl ot c a t i o nocsny l i n d e r

r a n gfer o lme ncse n t e r

r 1 r ~ , r 2 , r ~ , r ~ , r 5r a n g e r (A) , r (B)

point

h y p o t h e t irceaf l e c toi of n

to o b j e cptr o j e c t i oonpnr i

from l e n sc e n t e r

r a n g e from l e n cs e n t e r

ncipal

to p o i n to no b j e c t ,

to points

A

and

cm

B, r e s p e c t i v e l y , cm

diameter o f disc, c m

dotlocationsonobserved

object

unitvector u n ivt e c t o r s

from

P

toward

and

A

B, r e s p e c t i v e l y

w i d t h of image p l a n e , c m

spacingbetween

m a r k s o no b s e r v e d

r e c t a n g u l a rc o o r d i n a t e si n e x a c tc e n t e r

image p l a n e

of c i r c u l a r image

c o o r d i n a t e s of c e n t e r o f centroidcoordinates a n g l e su s e d

i t h i l l u m i n a t e dp i x e l

of i l l u m i n a t e d p i x e l s

t o compute r o t a t i o n a n g l e

a n g u l a r f i e l d of viewmeasured

$

for c y l i n d e r ,d e g

on d i a g o n a l of image p l a n e , d e g

t i l t a n g l e away f r o mp e r p e n d i c u l a r

t o lineofsight,deg

a n g u l asr e p a r a t i o nb e t w e e nr a y es m a n a t i n g B', deg

from

81 , 8 2 , 8 3 , 8 4a n g u l a r 0

object, cm

P

through

A'

and

separation of s u r f a c em a r k i n g sd, e g

l e a s t s q u a r ef is t

of

On,

p i x e l s or cm

3

-

on

standard deviation of x - xc computed from random data of c i r c u l a rd i s c diameter n, p i x e l s

Or

standard deviation

of error i n

r , cm

06

standard deviation

of error i n

6 , deg

4

angle between two range vectors to points

9

rotation angle

of cylinder about

on observed object,

i t s longitudinal axis,

deg

deg

Mathematical notation:

II

length or

magnitude

dotproduct ”*

vector from

AB

A

to

B

approximation A prime ( ’ ) over a symbol denotes a proje ction of that point

i n the image

plane.

ANALYSIS

Mar king System A l l vision systems t o d a t e , a t l e a s t t h e ones of which the authors are aware, i d e n t i f yp a r t s by analyzingtheirshapes. I f theshape i s complexand if the part is presented to the camerarandomly positioned and oriented, the process of analysis is complicated and computationtime i s increased. One s o l u t i o n t o t h i s problem is to place an identifying mark or symbolon each p a r t a t a s u f f i c i e n t number of l o c a t i o n sf o rv i s i b i l i t y fromany side.Furthermore, because the geometry of the mark is known and is l e s s complex than the geometry of the part, less software is required to deduce the range and o r i e n t a t i o n of the part. The most parsimonious markwouldbe one from which could be deduced range, orientation, and a t h i r d parameter t o i d e n t i f y which part the mark is on. Additionalparameters would require more complex marks. However, there is s t i l l the problem of distinguishingthe markfrom the background. T h i s problem could be simplified by u s i n g marks of s p e c i f i c complementary colors (two colors whose combinationproducewhite) which could be enhanced w i t h color f i l t e r s . For example, denotethe complementary colorsascolor 1 and color 2. If t h e mark is a color 1 d i s k and color 1 and color 2 f i l t e r s a r e placed a l t e r n a t e l y i n f r o n t of the camera lens,thecolor 1 spot w i l l appearas a flashingdiscwhiletherest of the image remains r e l a t i v e l y unchanged. By s u b t r a c t i n g a l t e r n a t e images, the disc is g r e a t l y enhanced r e l a t i v e to t h e r e s t of the image.

To i l l u s t r a t e t h i s concept, an investigation has been made into possible marks for two generalclasses of objects:cylindricalobjects and objects w i t h 4

I a f l a t s u r f a c e . The m a r k s i n v e s t i g a t e dw h i c hg i v er a n g e (R) a n do r i e n t a t i o n ( O ) , p l u s a g i v e n number of e x t r a parameters (N), w i l l b e r e f e r r e d t o a s RO + N m a r k s . RO

+

1 Mark f o r C y l i n d e r s

two stripesplacedradially The RO + 1 m a r k f o r a c y l i n d e r c o u l d c o n s i s t o f stripes c o n n e c t e d by a t h i r d which s p i r a l s o n c e a r o u n dt h ec y l i n d e rw i t hb o t h a r o u n dt h ec y l i n d e r( f i g .1 ) . The s p i r a l s t r i p e andone r a d i a l s t r i p e a r e t h e same color, w h i l e t h e o t h e r r a d i a l stripe i s t h e complement of t h a t color t o r e s o l v ea n ya m b i g u i t yi nt h ed i r e c t i o n .T h i sm a r k i n g scheme is u s e di na n of t h e c y l i n d e r , p l u s t h e a p p r o x i m a t e way t o f i n d t h e r a n g e a n d o r i e n t a t i o n The diameter is e n c o d e db ys p a c i n gt h e two c i r c u l a r d i a m e t e r of t h e c y l i n d e r . stripes so t h a t t h e w i d t h of t h e space times t h ed i a m e t e r of t h e c y l i n d e r i s a p r e d e t e r m i n e dc o n s t a n t .

Image p l a n e . - I n f i g u r e 2 , t h e image p l a n e i s d e p i c t e d a s a m a t r i x o f l i g h t t h e image p l a n e shown i n f i g u r e 2 s e n s i t i v ee l e m e n t sc a l l e dp i x e l s .A c t u a l l y , is a n i m a g i n a r y f r o n t image p l a n e i n t r o d u c e d f o r c o n v e n i e n c e by workers i n computergraphicsandmachineperception t o d i s p l a y t h e image i n a n u p r i g h t p o s i t i o n . The r e a l image p l a n e i s located b e h i n dt h el e n sc e n t e ra n dg i v e sa n t w o p l a n e s are r e l a t e ds i m p l y by a i n v e r t e d image.Image c o o r d i n a t e si nt h e m u l t i p l i c a t i v ef a c t o ro fm i n u su n i t y . The l e n s c e n t e r is located a t a known is perpendicular t o d i s t a n c e from t h e image p l a n e o n t h e p r i n c i p a l r a y , w h i c h t h e imageplane. A n g u l a rs e p a r a t i o n of image p o i n t s . - If t h e c o o r d i n a t e s of two p o i n t s A ' image p l a n e (shown i nf i g . 3 ) a r e known a n dt h ec o o r d i n a t e so f and B ' i nt h e are known, t h ea n g u l a rs e p a r a t i o n 8 b e t w e e nt h e r a y se m a n a t i n g from P be found from t h ed e f i n i t i o n o f t h e d o t p r o d u c t as t h r o u g h A ' and B ' can

I n a camera, P is t h el e n sc e n t e rw h i c h i s known i na d v a n c ea, n d are d i r e c t l y measurable i n t h e image p l a n e c o o r d i n a t e s y s t e m ( f i g .

A'

and

B'

2).

Diameter andrange.Shown i n f i g u r e 4 ( a ) is t h e p o r t i o n o f a c y l i n d e rw i t h t h e t w o c i r c u l a r s t r i p e s a n dt h es e p a r a t i o nd i s t a n c e w b e t w e e nt h e s e stripes. The image p l a n ei nf r o n to ft h el e n sc e n t e r P is n o t shown. The p o i n t s P1 and P2 are p o i n t sw h e r e P would be located i f it were s h i f t e d p a r a l l e l t o thecylinder'saxis to a point lying in the plane of t h e c i r c u l a r stripes. Assuming t h a t P i s located a t P1 leads t o t h eg e o m e t r yo f i g u r e4 ( b ) . However, t h ec i r c u l a r s t r i p e and P are n o tr e a l l yc o p l a n a r ;t h e r e f o r e ,u s e o ff i g u r e4 ( b )i n s t e a do ff i g u r e 4 ( a ) r e s u l t si nt h ea p p r o x i m a t i o n

rl = 2 csc

(2) 5

P

where d is the momentarily unknown diameter of the cylinder. The angle 01 can be found using equation ( 1 1 , where the points A' and B' are the end points of the visible circular arcon the image plane. In a similar manner, the approximate equation for r2 is

d

Referring to figure 4(c), notice that by the law of cosines

By assumption, the product of the width w and diameter d is a predetermined constant k, so that

k w = d

Substituting equations (2), ( 3 1 , and ( 5 ) into equation (4) gives

(w + $

k2 - " d2 csc 2 d2

csc2

(>) g (>) (2) -

csc

csc

cos @

in which only d

is unknown. Solving for d yields

This value of d for rl andr2.

( 3 ) to solve may then be substituted into equations (2) and Hence, both range and cylinder diameter are found.

Direction cosines of cylinder's axis.- In figure 5, A and B+ are points in-P the field of view whose ranges3re known. Let r (A) = ]PA1 and r (B) = I PBI Then, the orientation of AB is specified by the unit vector

.

6

AB Z G +=

I 4

whose coordinates are the direction cosinesof

-+

AB.

Let

and

then

so

This procedure is applied to figure6 to obtain the direction cosines of the cylinder's axis. Rotation o f cylinder ab ut^ its axis.-The orientation the of cylinder can be specified by the direction cosines of the cylinder axis and by an angle between Oo and 360°, to indicate the amount of rotation about that axis. Figure 7 is used to examine rotation. The imaginary line 2 in figure 7 is parallel to the cylinder axis and indicates that pints P1 and P3 are lined up with some distinctive feature of the cylinder, such as the notch shown. Traveling from P1 to P3 via the spiral stripe represents a rotation of 360°, so traveling from P1 to P2 represents some fraction of 360°, which is equivalent to the amountof rotation. The angle $ indicates the amount of rotation about the axis with respect to an imaginary 1 ,line which is parallelto the cylinder axis. For example, if the pints P2, Pgr and P-) are coincident, then $ = Oo; and if P2, P4r and P3 are coincident, then 9 = 360O. The angle Q is given exactly by ~~~

" "

~

7

I

I

1

I

where s s and s 5 a r e l i n e s e g m e n tl e n g t h s .I nt h i sd e v e l o p m e n t , e q u a t i o n ( 1 2 ) is approximated by

The a n g l e s a1 and a 2c a n be approximated as follows. The a r c p o r t i o n o ft h ec i r c u l a r s t r i p e 1 w i l l appear o n t h e image p l a n e ,w h i c h is n o t shown i n f i g u r e 7. The p o i n t P; is approximated by t h ep o i n ti nt h e image p l a n e whichcorresponds t o a p o i n th a l f w a yb e t w e e nt h e two e n d p o i n t s o f t h e arc. P o i n t P4 is s i m i l a r l ya p p r o x i m a t e d L . ine 1 is t h e na p p r o x i m a t e d by t h e s t r a i g h tl i n ec o n n e c t i n gt h e s e t w o a p p r o x i m a t i o n so ft h ei m a g ep o i n t sP i and P i . N m , t h e s p i r a l stripe w i l l c a u s e a s e t o fp i x e l s t o be i l l u m i n a t e d i n t h e imageplane. Common p i x e lp T i n t s of + i n e 1 a n dt h e s p i r a l s t r i p e locate P iS. i n c et h e image p o i n t sP i , P2, and P5 a r e known, t h ae n g u l a r s e p a r a t i o n as 1a n da 2 are computed as 8 was i ne q u a t i o n (1 )

.

RO

+

N Mark for F l a t S u r f a c e s

Shown i n f i g u r e 8 ( a ) is a w i d g e t w i t h t h r e e c i r c u l a r spots e q u a l l ys p a c e d i n a s t r a i g h tl i n e . One o ft h ee n d spots is thecomplementary color o ft h e o t h e r t w o , t o e s t a b l i s hd i r e c t i o n .I nf i g u r e 8 ( b ) , S1 , S2,andS3denote spots o nt h ep a r t . The a n g l e s 81 a n d8 2 a r e found using equat h et h r e e t i o n (1 ) The t r i a n g l eP s i s 3 i s e x t e n d e d t o a p a r a l l e l o g r a m PS1%3 to i l l u s t r a t et h ee q u a t i o n sw h i c hf o l l o w .

.

"-*

I

I

"-j

r l = IPS1 , r 2 = PS2I , and23 between spots, t h e law o fc o s i n e sa p p l i e d Let

a n du,s i n g

is t h ew i d t h

PS2S3,

By a p p l y i n gt h ei d e n t i t ys i n (180° i t follows t h a t t r i a n g l eP S I 5 8

+ I PS31 .

Then, i f w1 t o t h et r i a n g l eP s i s 2g i v e s =

-

8) = s i n 8

a n dt h e

law of s i n e so nt h e

In equations (1 4 ) and rl I r2, and

,

(1 5) , and (1 6), the quantities 81 , 82, and r3 are unknown. From equation (16),

r1 =

2r2 sin 8 2 sin (81 + 8 2 )

r3 =

2r2 sin01 sin (01 + 82)

w1

are known,

and

Substituting equation (1 7) into equation (1 4 ) , or substituting equation (18) into equation (15), leaves r2 as a function of w1, 81, and 82. The resulting equation in either situation can be expressed as

Then, 1-1 and r3 are found by substituting equation (19) into equations (1 7 ) and (18). The direction cosines can now be found by the,same,procedury used in equations ( 8 ) to (11 ) , remembering that the images S1 , S2, and S3 are directly measurableon the image plane.

There is insufficient information to deduce the rotationof the part in figure 8(b) about the S1S3 axis. A set of three spots perpendicular to the original three could resolve this ambiguity and, coincidentally, allow the encoding of an additional parameter.One -way to construct suchROa+ 1 mark is shown in figure 9(a) with the orthogonal sets of spots sharing a center spot. In figure 9(b), the product of the distances w1 and w2 from the center spot to alternate corner spotsis a prespecified constant k, that is, wlw2 = k. Spots S1 , S2, and S3 in figure 9 lead to equations (1 7), (1 8) , and (1 9) Similarly, spots S 4 r S2, and S5 in figure 9 lead to the following equations:

.

'4

=

2r2 sin 84 sin (83 + 84)

9

r5 =

where

r4,

2 r 2 s i n 83 sin

(e3 + 8,)

r5,

83, and 84

are shown i nf i g u r e9 ( c ) .

(1 9 ) a n d( 2 2 )g i v e sa ne q u a t i o nf o r rZ2 i n terms M u l t i p l y i n ge q u a t i o n s o ft h e known p r o d u c t w1w2 = k a n dt r i g o n o m e t r i cf u n c t i o n so ft h ea n g l e s 81, 8 2 , 83, and 8 4 . T a k i ntgh e square root g i v ers2 . Then, w1 and w2 c a n

W1

- r epr ew2 s e n t s an a d d i t i o n a l parameter whichcan be used t o i d e n t i f y a particular part. F u r t h e r m o r e ,t h ed i r e c t i o nc o s i n e sc a n now b ef o u n df o r two o r t h o g o n a l v e c t o r s and S S , l y i n go nt h ef l a t surface c o n t a i n i n gt h e m a r k . T h i s is enough information t o completely specify the orientation of the part. befound

by u s i n ge q u a t i o n s( 1 9 )

The r a t i o

a n d( 2 2 1 r, e s p e c t i v e l y .

The c o n f i g u r a t i o n of a RO + 2 m a r k for f l a t surfaces is shown i n f i g u r e 1 0 . mark f r e e i n g t h e The c o r n e r spot a u t o m a t i c a l l y p r o v i d e s t h e d i r e c t i o n o f t h e complementary colors for u s e as a b i n a r yc o d ef o re a c ho ft h ef i v e spots. T h i s parameter w i t h 32 d i s c r e t e v a l u e s ( e a c h o f t h e f i v e p r o v i d e sa na d d i t i o n a l

spots can be e i t h e ro f

two c o l o r s ) , i na d d i t i o n

to t h e

W1

-

w2

parameter.

An

e q u i v a l e n t "T" shaped m a r k c o u l d a l s o be used.

ERROR ANALYSIS From a g e o m e t r i c a l s t a n d p o i n t t h e RO marks d i s c u s s e d h e r e are f u l l y purpose o f f i n d i n g t h e d i r e c t i o n , r a n g e , a n d o r i e n t a t i o n of a d e q u a t ef o rt h e a g i v e n part. Hawever, i n a c c u r a c i e s of measurement m u s t be t a k e ni n t oa c c o u n t i na na c t u a ls y s t e m . The d e t e r m i n a t i o n of r a n g ei ne q u a t i o n s( 2 ) , ( 3 ) , and accurate d e t e r m i n a t i o no fs e p a r a t i o na n g l e s from (1 7 ) t o ( 2 2 )i n v o l v e st h e e q u a t i o n ( 1 ) . B u t , s e p a r a t i o na n g l e sc a n n o t be d e t e r m i n e de x a c t l y , because i t i s n o tp o s s i b l e t o f i n dt h ee x a c tc e n t e ro fa n image i n t h e i m a g ep l a n e .I n order to investigatethisproblem it was assumed t h a t t h e image p l a n e c o n s i s t e d of a r e c t a n g u l a ra r r a yo f picture e l e m e n t s , or p i x e l s .I nf i g u r e1 1 ,t h e image of a c i r c u l a r d i s c is s e e no nt h e image p l a n e . If a g i v e n . p i x e 1 is i l l u m i n a t e d s u f f i c i e n t l y t o exceed some p r e d e t e r m i n e d t h r e s h o l d v a l u e , i t is s a i d t o be " i l l u m i n a t e d . "I no r d e r t o s i m p l i f y t h i s aspect of t h ep r o b l e m i t is assumed t h a t a g i v e n p i x e l is i l l u m i n a t e d i f i t s c e n t e r f a l l s w i t h i n t h e b o u n d a r y of t h e image;otherwise, i t is assumednot t o b ei l l u m i n a t e d .I nf i g u r e1 1 ,t h e s h a d e ds q u a r e sr e p r e s e n tt h ei l l u m i n a t e dp i x e l s . The c e n t e ro ft h e image i s c a n dt h ec e n t e ro ft h ec o l l e c t i o no fi l l u m i n a t e dp i x e l s i s D. L e t t h es e q u e n c e ( x k , Y k )d e n o t et h eC e n t e r s of t h ei l l u m i n a t e dp i x e l s . (x1 , y 1 ) ,( x 2 , y 2 ) ,

. . .,

10

If k

i=l

and

then then

D = (X,y)

is t h ec e n t r o i d

of t h ei l l u m i n a t e dp i x e l s I. f

is t h e d e v i a t i o n o f t h e c e n t r o i d f r o m t h e

t r u e c e n t e ro ft h e

C = (xc,yc),

image.

n p i x e l sa n d a g i v e n locaFor a c i r c u l a r image w i t h a diameter g i v e ni n t i o no f its c e n t e r C o n some p i x e l , D - C c a n be e x a c t l yd e t e r m i n e da n a l y t f a c t , t h o u g h t, h ec e n t e r C is known o n l yi n a statistical sense, i c a l l y .I n a g i v e np i x e l .T h u s , D - C b e c a u s e i t is randomlyplacedsomewherewithin would be a random v a r i a b l e . F o r t h e purpose o f t h i s s t u d y , t h e random variable -x - xc f r o me q u a t i o n (25) was i n v e s t i g a t e d f i r s t a n d t h e results subsequently applied to - yc and D - C. The c i r c u l a rd i s c s u s e d a s images were assumed t o have diameters w h i c hv a r i e df r o m 2 to 50 p i x e l s .F o re a c hs p e c i f i c image d i a m e t e r , 1 0 0 c h o i c e so f C were s e l e c t e du n i f o r m l y a t random w i t h i n a g i v e n n, 1 0 0 samples of Z - xc were computed. The p i x e lF. o er a c hc h o i c eo f un of t h e s e 1 0 0 samples are shown i n mean m a n dt h es t a n d a r dd e v i a t i o n actual d i s t a n c e s , t a b l e I. The u n i t s of t h e s e statistics c o u l db em e a s u r e di n such as c e n t i m e t e r s ;h o w e v e r ,t h e S t a t i s t i c a l v a l u e sw o u l dv a r y ,d e p e n d i n g upon m and 0 are thp e a r t i c u l adr i m e n s i o n os tfh iem a g i n gd e v i c eT. h e r e f o r e , m e a s u r e di np i x e lu n i t s . The n i nt a b l e I d e n o t e s image diameter i np i x e l s . Because t h ev a l u e s of m were c o n s i s t e n t l y small a n dn ot r e n d was a p p a r e n t , it w a s assumed t h a t m = 0. The assumption is made t h a t 0 is i n v e r s e l y prop o r t i o n a l t o t h es q u a r e root o fn ;t h a t is,

11

The best v a l u e of

i n=2

a =

5 n=2

is

a, i nt h el e a s t - s q u a r e ss e n s e ,

-

*n n 1n

A p p l i c a t i o n of e q u a t i o n( 2 7 )

t o t h ev a l u e s

of

for

is

0,

i nt a b l e

I yields

IT=- 0.222

For l a t e r r e f e r e n c e t, h e q u a t i o n

20 =

20

0.444

fi

I nf i g u r e1 2 ,t h ev a l u e s of On i nt a b l e I a n dt h ea p p r o x i m a t i o n 0 in are p l o t t e d i n p e r c e n t a s € u n c t i o n so fc i r c u l a r imagediameter'n. e q u a t i o n( 2 8 ) B e c a u s et h e f i t is a p p a r e n t l y a goodone,equation(28) i s t a k e n as a good e s t i mate of t h es t a n d a r dd e v i a t i o no f E xc as a f u n c t i o no f imagediametern. I n a g r e e m e n tw i t hr e f e r e n c e 4 , t h e a c c u r a c y of p r e d i c t i n g t h e c e n t e r o f t h e circ u l a r imageimproves as t h e image s i z e i n c r e a s e s .

-

error v e c t o ri ne q u a t i o n( 2 5 )

The l e n g t ho ft h e

ID

-

CI =

Substituting t i o n( 2 9 )g i v e s

ID

-

?I

C( =

F-

xc)2

-

xc = 20 and

yc) 2

-

yc = 20

i n t oe q u a t i o n( 3 0 a) n du s i n ge q u a -

0.628 fi

as anapproximation 12

+ (y -

is

t o t h e maximum l e n g t h of t h e error v e c t o r .

The l o c a t i o n error o ft h e c i r c u l a r image c e n t e rf o re q u a t i o n s( 2 9 )a n d (31) i s shown i n f i g u r e 13 a l o n g w i t h n u m e r i c a l s o l u t i o n s f o r t h e a p p r o x i m a t e maximum error f o rd i s c r e t ep o i n t s computed i nr e f e r e n c e 4. E q u a t i o n( 2 9 )r e p r e s e n t s a 20 error i ne i t h e rt h e x- or y - d i r e c t i o n . E q u a t i o n ( 3 1 ) r e p r e s e n t sa na p p r o x i m a t i o n t o t h e maximum error for 20 v a r i a t i o n s i n b o t h t h e x- a n dy - d i r e c t i o n s . In g e n e r a l ,e q u a t i o n (31) g i v e s a c o n s e r v a t i v ee s t i m a t e of t h e maximum error for t h ed i a m e t e r sc o n s i d e r e di nf i g u r e 13. As e x p e c t e d ,t h e estimate g i v e n by equat i o n ( 2 9 ) is l o w due t o l i m i t i n g to o n l y o n e d i r e c t i o n . Then i ne q u a t i o n (28) c a n be related t o p h y s i c a lp a r a m e t e r si nf i g ures 14 and 15. F i g u r e 14 shows t h e disc w i t hd i a m e t e r s i nc e n t i m e t e r sa n d its image w i t hd i a m e t e r I i nc e n t i m e t e r s . By similar t r i a n g l e s ,

I - S " 2f

2R*

OK

fS

I = R*

(33)

The r a t i o of t h e disc image diameter n i n p i x e l s t o t h ew i d t h of t h e image p l a n e M i np i x e l s is equal t o t h e r a t i o of t h ed i s c image diameter I i n centimeters t o t h ew i d t h of t h e image p l a n e W i cne n t i m e t e r sH. e n c e ,

"-

I

M

W

(34)

or

M

n = I w

S u b s t i t u t i n ge q u a t i o n

fS M n = - R* W

(35)

(33) i n t oe q u a t i o n

(35) g i v e s

(36)

13

The f in equation (36) can be expressed as a function of the field-ofview angle y and width W of the image plane in figure 1 5 as

cot

(;)

=

;p

(37)

or

f =

E

cot

(f)

where y is the field ofview measured on the diagonal of the image plane. Substitute equation ( 3 8 ) into equation ( 3 6 ) to obtain

n=-

(39)

Equation (39) can be used to express equation( 2 8 ) as

To convert 0 from pixel units to centimeters, multiplyby the scale factor W/M to get

CJ = ( 0 . 2 6 4 ) -

M

-

In figure 11 , it is assumed that D - C = (X - xc, y - yc) is a random variable satisfying a bivariate normal distribution with each variable having mean m = 0 and standard deviation 0 satisfying equation ( 4 1 ) . For a given be charge-coupled-device (CCD) imaging chip, the values of M and W may different in thex- and y-directions, giving different values ofCI in the x- and y-directions.

14

The a c c u r a c y w i t h w h i c h t h e r a n g e a n d o r i e n t a t i o n may be computed i s determined for a sample problem byassuming t h a t ,i ne q u a t i o n (41), W = S = 0.625 and M = 256 i nb o t ht h e x- a n dy - d i r e c t i o n s I. no t h e rw o r d s t, h ei m a g i n g s u r f a c e is s q u a r ew i t h sides W e q u a l t o 0.625 c e n t i m e t e r ,t h ei m a g i n gs u r face is composed of a m a t r i x of 256 by 256 p i x e l s (M = 256), a n d t h e d i a m e t e r of t h e disc m a r k S is 0.625 c e n t i m e t e r T . h u s e, q u a t i o n (41) becomes

T h et h r e e - d o tm a r k i n gi nf i g u r e 8 s u f f i c e s for t e s t i n g r a n g e a c c u r a c y f r o m e q u a t i o n s (1 7 ) , (la), and (1 9) a n d tests t h e aspect o f o r i e n t a t i o n most s u s c e p t i b l e t o errors; namely, t h e amountof t i l t 6 away from a p e r p e n d i c u l a r t o t h el i n eo fs i g h t( f i g . 16). To f u r t h e rs i m p l i f yt h ea n a l y s i s , i t was assumed t h a t t h e imageof t h e m a r k l i e s h o r i z o n t a l l y i n t h e image p l a n e ( f i g . 1 7 ) Errors were c a l c u l a t e d i n r a n g e a n d t i l t for r a n g e v a l u e s from 50 c m t o 150 c m i n 25-cm i n c r e m e n t sa n d for tilt v a l u e s from 5O t o 85O i n 20° i n c r e m e n t s . Widea n g l e (45O) andnarrow-angle (5O) f i e l d s of view were used.Foreach choice of parameters, 1000 u n i f o r m l y random c h o i c e s were made w i t h i n t h e e n t i r e image p l a n e for t h ec o o r d i n a t e s of t h e image of t h ec e n t e r spot ( f i g . 1 7 ) . Foreach of t h e s e 1 0 0 0 c h o i c e s , t h e t r u e p o s i t i o n s of t h e t h r e e i m a g e s were a l t e r e d i n v a r i a b l ew i t h mean m = 0 a n d t h e x- and y - d i r e c t i o n s by a normalrandom (5 d e t e r m i n e db ye q u a t i o n (42). Of c o u r s e ,t h et h r e e image spots w i l l n o t windowassumed f o r t h e c e n t e r spot, always be s i m u l t a n e o u s l y v i s i b l e i n t h e e s p e c i a l l yf o rt h e smaller f i e l d of view (Y = 5O); however, t h i s d i f f i c u l t y i s separate from theproblem of d e t e r m i n i n g t h e a c c u r a c y o f t h e m e a s u r e d t i l t when i t is m e a s u r a b l e .

.

The rangeand t i l t were t h e n c a l c u l a t e d u s i n g t h e a l t e r e d p o s i t i o n s of t h e imagesand t h e d i f f e r e n c e sf r o mt h et r u ev a l u e s were c a l c u l a t e d . The mean and s t a n d a r d d e v i a t i o n of t h e s e d i f f e r e n c e s were c a l c u l a t e d for e a c hc o m b i n a t i o n of r a n g e , t i l t , a n df i e l d - o f - v i e wp a r a m e t e r s .T h er e s u l t s a r e shown i n t a b l e s I1 and 111. The r e s u l t s f o r t h e n a r r o w (5O) f i e l d of view are g o o d ,s u g g e s t i n gt h a t of a p a r t can be d e t e r m i n e d f a i r l y a c c u r a t e l y when therangeandorientation is n o t more t h a n 125 cm. The t h e t i l t is n o t more t h a n 45O a n dt h er a n g e r e s u l t s for t h e w i d e f i e l d of view (45O) a r e less e n c o u r a g i n ga n ds u g g e s tt h a t a w i d e - a n g l el e n ss h o u l do n l y be used t o locate t h e m a r k , a f t e r w h i c h t h e camera s h o u l d s w i t c h t o a n a r r o w - a n g l e l e n s . Inherentinthe error a n a l y s i s a r e t h e a p p r o x i m a t i n g a s s u m p t i o n s t h a t t h e disc projects p l a n e of e a c h d i s c is p a r a l l e l t o t h e image plane a n d t h a t e a c h as a circle o n t h e imageplane. A more t h o r o u g h error a n a l y s i s is needed t o e x a m i n et h ei m p l i c a t i o n s of t h e s es i m p l i f y i n ga s s u m p t i o n s .

15

I

CONCLUDING REMARKS

Having a robot view the three-dimensional world as man views itis only a dream at present; however, much headway has been made in this direction a current research efforts are attacking the problems involved. While generic is proceeding, there are foreseeable appliwork in automated computer vision or simplified cations, suchas construction in space, which can use intermediate vision systems. In general, computer vision systems identify parts by analyzing their shapes. However, the premarking of parts for subsequent identification by a robot vision system appears be to beneficial as an aid in the automation of certain tasks. Accordingly, a simple, color-coded marking system is presented in this paper which allows a computer vision system to locate an object, calculate its orientation, and determine its identity. Such a system has the potential to operate accurately, and, because the computer shape analysis problem has been greatly simplified, it has the potential to operate in real time.

Langley National Hampton, February

16

Research Center Aeronautics and VA 23665 9,

1981

Space

Administration

REFERENCES 1 . Shirai, Yoshiaki: Recognition of Polyhedrons With a Range Finder. Pattern

Recognition, vol. 4, 1972, pp. 243-250. 2. Nevatia, Ramakant; and Binford, Thomas 0.:

Curved Objects. Artif. Intell.,

Description and Recognition of v o l . 8, 1977, pp. 77-98.

3. Popplestone, R. J.; and Ambler, A. P.: Forming Body Models From Range Data. D.A-1. Res. Rep. No. 46, Dep. Artif. Intell., Univ. Edinburgh, Oct. 1977. 4. McVey, E. S . ; and Jarvis, G. L.: Ranking of Pabterns for Use in Automation. IEEE Trans. Ind. Electron. & Control Instrum., vol. IECI-24, no. 2, May 1977, pp. 211-213.

77

TABLE: I.- VARIATION OF STATISTICS OF

n, p i x e l s 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

18

m, p i x e l s

_""_ -0.01 5 -.005 .002 .009 .002 ,008 002 002 .003 008 .005 004 001 001 01 0 002 007 006 006 002

-.

-.

-.

-.

-. -. -. -. -. -. -. -. .ooo -.001 -.01 1 -.001

un, p i x e l s

"_" 0.1 68 .133 .lo3 .lo9 .091 .082 .085 .064 .065 .061 .062 .067 .055 .061 .052 .059 .052 .048 .053 .043 .048 .048 .042 .042

x-

Xc

WITH PIXEL DIAMETER OF DISC

n, p i x e l s 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

m, p i x e l s

on, p i x e l s

-0.003 003 002 .004 001

0.041 .050 ,035 .041 .039 .036 .039 .032 .044 .036 .033 .036 .033 .041 .033 .032 .033 .032 .035 .030 .029 .031 .030 .032 .026

-.

-. -. .OOl -.004 -.004 -.003 -.001 .003 .01 0 ,002

.ooo

-.002 -.001 .004 -.002 .OOl -.002 -.003 -.001

-.004 .005

.ooo

TABLE 11.- ERROR I N CALCULATED TILT ANGLE AND RANGE FOR So FIELD OF VIEW (y = So) -

Actual t i l t a n g l e , 6 , deg

5

tilt

Error i n c a l c u l a t e d angle

A c t u a l ran( r r cm

m g r deg

06, deg

0.0 ;0

0.0

.o .o .o

.o .1 .2 .3

Error i nc a l c u l a t e d range

mr,

cm

0.00

50 75 1 00 125 1 50

0.00 -00 00

.

.oo .oo

.oo .oo .oo .oo

50 75 1 00 1 25 1 50

0.00

0.00

.oo .oo .oo .oo

.oo .oo

50 75 1 00 1 25 1 50

0.00

0.00

50 75 100 1 25 1 50

0.00

"

25

0.0

0.0

.o .o .o .o

.o

0.0

0.0

.o .o

.o

.1 .2 .3

.25 .25

~.

45

65

85

-0

.1 .2

.o

.3

0.0

0.0

.o .o .o .o

.o

0.0

0.0

.o .o

.o

.o

.2

.o

.1 .2 .3

.1 .3

50 75 1 00 125 1 50

.oo .oo .oo .oo

.

.oo .oo .oo .oo

0.00

.oo .oo .oo .25

.oo -25 .25 .75 0.00 .25 .25 .75 1.50

0.25 .75 1.75 4.00 7.75

-

19

TABLE 111.- ERROR I N CALCULATED TILT ANGLE ANI:

Actual t i l t a n g l e , 6 , deg

T

ZANGE FOR

45O FIELD OF VIEW (y = 450)

tilt

E r r o r i nc a l c u l a t e d

angle

Actualrange, r r cm

06,

deg

E r r o r i nc a l c u l a t e d range

m,,

m

Oyr

5

0.0

0.4 1.2 2.4 4.4 6.9

.o .o .a .o

I

"

50 75 1 00 ? 25 1 50

~~

25

,o. 0

-.50 .oo

-1

-

-

0.00 00

.o

0.0 -0 -.l -.4 5

0.4 1.2 2.4 4.3 6.9

50 75 1 00 125 1 50

0.00

-1

-.

0.0

.o

-.2

-.4

-1.7

-

0.25 -75 2.00 4.25 7.75

.

.oo .oo .25

-

.oo

.00 .50 .25

-

0.25 1 .50 4.00 9.00 17.00

-

___

65

0.00 -25 -50 1 .oo 2.25

.oo .oo

50 75 1 00 1 25 1 50

-.3 45

0.00

0.4 1.2 2.4 4.4 6.7

-.l -.l

cm

-

"

"

0.4 1.2 2.5 4.7 7.8

50 75 1 00 125 1 50

0.5 1.4 22.0 49.3 66.4

50 75 1 00 125 1 50

0.00

.oo .50 1.25 8.00

0.75 3.25 9.25 21 -25 39.75

~-

85

-0.1

-.3

-5.2 -20.5 -36.5

0.75 4.00 49.75 108.00 95.50 -

20

4.25 21 .oo 168.50 280.75 305.75

Visible portions of spiral stripe

Circularstripes Figure 1

.- RO

+

1

1 mark for cylinders.

Principa I ray

/L/ illuminated

P [lens

center

Figure 2.- Array of pixels (light-sensitive elements). 21

A

B

Lens center

Image plane

F i g u r e 3.- A n g u l a rs e p a r a t i o n

22

of image p o i n t s .

I

STRIPE 2

Stripe 1

( a ) Location of lenscenter

(b) Assuming t h a t

P

P

r e l a t i v et os t r i p e s .

is l o c a t e da t

PI.

W

P

(c) Separation of s t r i p e s . Figure 4.-

Coded stripes on cylinder.

23

Figure 5.- Orientation of two points with known ranges.

Stripe 2

Stripe 1

P Figure 6.- Orientation of cylinder. 24

Visible portions of spiral stripe ,-Circular

stripe 1

Figure 7.- Rotation of cylinder.

25

( a ) Three-discmarkingon

a widget.

N

d

\ p

w

/ c

/

\ \ \

( b ) Geometry of t h r e e - d i s cm a r k i n g .

F i g u r e 8.26

I l l u s t r a t i o n andgeometry t h r e e - d i s c mar king.

of

( a ) RO + 1 mark onwidget.

P W

s1 0

1

. s3

(b) Geometry of RO F i g u r e 9.-

+

1 mark.

I l l u s t r a t i o n andgeometry RO + 1 m a r k .

of

27

( c ) P r o j e c t i o no n

image p l a n e of RO

F i g u r e 9.-

28

Concluded.

+

1 mark.

I

Figure 1 0 . - RO

+

2 mark €or f l a t s u r f a c e s .

C : Center of circle D : Centroid of illuminated pixels Figure 11.-

Pixel image of c i r c u l a r d i s c .

29

n ‘ percent of pixel

0

20

10 \

Figure 1 2

30

.- S t a n d a r dd e v i a t i o n

of

50

30

n, pixels -x component

of c e n t r o i df r o m

xc.

I

1

2

"

1

-

I

.I

3 4 5 Circle diameter, n, pixels

F i g u r e 13.-

6

L o c a t i o n error of c i r c u l a r image c e n t e r v e r s u s s i z e .

Disc I I

S radius 2

I I

I I

I

F i g u r e 14.-

I l l u s t r a t i o n of d i s c and i t s image. 31

P

F i g u r e 15.- I l l u s t r a t i o n o f a n g u l a r f i e l d of view measured on diagonal ofimageplane.

P F i g u r e 16.- Range and t i l t of three-disc marking. 32

. .. . ..

-. .

d lI

XImage pla ne

P F i g u r e 17.- Image of three-disc marking i n t h e imageplane.

33

~

.

-

.

"

"

1. Report No.

NASA TI?-1 81 9 ""

__

.

" "

4. Title and Subtitle

MARKING PARTS

I 2:&2;nrn1?

.

" "

-

-

"

.

-

-. ...

3. Recipient's C a t a l o g No.

Accession No. -

c

~. -

~.

TO A I D ROBOT VISION

-

5. ReportDate

April 1 981 6. PerformingOrganizationCode

506-54-1 3-02 -.

J o h n W. Bales and L. K e i t h Barker -- -

"

.

"

No.

L-14055IO. Work

-"

..

~. -

. .

8. PerformingOrganizationReport

7. Author(s)

..

Unit No.

9. PerformingOrganizationName and Address

NASA L a n g l e y R e s e a r c h C e n t e r Hampton, VA 23665 ... . .

"

11.ContractorGrant

- -- - -- -

.-

"

. ..

--

-

.-

.

-

"".

. . ..

~.

-.

" "

- -

"-

-~

t

and Period Covered

T e c h n i c a lP a p e r ~

-- -

.- . - ..

.

.

14 Sponsoring Agency Code

-

-

J o h n w. Bales: T u s k e g e eI n s t i t u t e ,P a r t i c i p a n ti nt h e1 9 8 0 Summer R e s e a r c hF e l l o w s h i pP r o g r a m . L. K e i t h Barker: L a n g l e yR e s e a r c hC e n t e r . ."

- .

~~

13. Type ofReport

NationalAeronauticsandSpaceAdministration Washington, DC 20546 -

-

J-

"

--

No.

" "

..

L -

.

~-

.

.

NASA-Hampton I n s t i t u t e

-

."

~

.

..

" "

-

"

The p r e m a r k i n g o f p a r t s f o r s u b s e q u e n t i d e n t i f i c a t i o n by a robot v i s i o n s y s t e m a p p e a r s t o be b e n e f i c i a l a s a n a i d i n t h e a u t o m a t i o n o f c e r t a i n t a s k s , s u c h as c o n s t r u c t i o ni n space. A c c o r d i n g l y , a s i m p l e ,c o l o r - c o d e dm a r k i n gs y s t e m is p r e s e n t e dw h i c ha l l o w s a c o m p u t e rv i s i o ns y s t e m t o locate a n o b j e c t , c a l c u l a t e i t s o r i e n t a t i o n ,a n dd e t e r m i n e i t s i d e n t i t y .F u r t h e r m o r e ,s u c h a s y s t e mh a st h e p o t e n t i a l t o o p e r a t ea c c u r a t e l y ,a n d ,b e c a u s et h ec o m p u t e rs h a p ea n a l y s i sp r o b l e m h a sb e e ng r e a t l ys i m p l i f i e d , it h a s t h e p o t e n t i a l t o operate i n r e a l time.

~

17.Key

"

~

Words (Suggested by Author(s))

-

.. .

- -.

.

.

"~

I

.. .

Unclassified - Unlimited

Automation Robot vision Computer v i s i o n C o l o r - c o d e dp a r t s Mar k i n g~. p a r t s

..

19. Security Classif. (of this report1

Unclassified UnS-lassified

-

18.DlstributlonStatement

~

."

..

S u b j e c t C a t e g o r y "59 "

. ~

.

.

20. Security Classif. (of this page) - .

.

~

-

~

For sale by the Natlonal Techntcal lnformatlon Servlce,

Sprlnefleld.

Vlrglnla

22161

-

~.

"~

National Aeronautics and Space Administration Washington, D.C. 20546

THIRD-CLASSBULKRATE

Postage and Fees Paid National Aeronautics and Space Administration NASA451

Official Business Penalty for Private Use, $300

m

POSTMASTER:

If Undeliverable (Section 158 Postal Manual) Do Not Return