A syntactic method for object recognition from a color ...

A syntactic method for object recognition from a ailor image

a

Xianyi ZENG and Christian VASSEUR GEMTEX ENSAIT 2, place des Martyrs de la R6sistance 59100 Roubaix, Fra&x

Abstract:This paper presents a method for object lpcoBlljfjon composition operations is somtimes called the pattem &om a cobr image by using syntactic paanalyds. From a digital image, the coptoum of the input objects UT segmented and encoded into a s l r h g of Freeman codes. A m n a ln aand a circular arc approprc used to gke a dgnkative and concke shape representation of the analyzed objects. T6 mcognize the s h p e of the ob*, we propose a mew “ i based 011 the idea of the atLribuded grammar. By using this new method, the strw of prim#ives representlag an input object can be matched agaht strings of prhnltlves representing each model or lrferenee obJect Finally, the input object Is dsasilkd in the same class as the model that is tbe “best” match to the input Amy exbthg model can be genemlized &om a set of input objects as examples, which have the same duuacterbticr in some sense.

I INTRODUCnON

d e s q t i o n l a v a g e . The d e s governing the composition of primitives mto patterns are u s d l y spci6ed by the ge. so-called gnmmar of the pattem dcscnption la After each primitive within the pttm is i d e n t i c t h e recognition process is acconplished by pe&orming a syntax analysis of the sentenoe descdbiug the given pattern to determine whether or not it is syntactically coned with respect to the specified grunnrmr. The syntactic appraoch to pattem recognition provides a capacity for desaibing a large set of conykx patkms by using small se$ of simple peltems primnivcs a d of gr0mmrtkd Nk.

In this paper, we give a new syntactic recognition method for the objects extraded from an image in order to match with the models in the knowledge base If the matching is succ:essll and the sentence desaiiing the given object is syntsdictlly CoSTect with mpect to an existing model, we consider that t h i object is recoglljzed by this model.

The decision-theoretic approach and the syntactic or structural approach are two general techniques to solve pattern recognition problems [l]. In the decision-theoretic approach, a set of characteristic measurements, called This paper gives a procedure of syntactic recognition features, am extracted from the patterns. The reco for the objeck of a cobr ima This procedure is divided of each pattern is usually made by partitiomng the into several mapr parts as fo ows: space [Z].

r.

In som pattern recognition problems, the structural information that describes each pattern is important. A typical example of these recognition problem is scene analysis. In this case, the patterns under consideration are usually quite complex and the number of features required is often very large, which makes the idea of describing a complex pattem in terns of a hierarchical composition of simpler subpatterns very attractive. In order to represent the hierarchical structure information of each pattern, the syntactic or structural approach has been proposed 131. This approach draws an analogy between the structure of patterns and the syntax or grammar of languages. In this approach, the simplest subpatterns selected, called pattern primitives, should be much easier to recognize than the patterns themelves. The language that provides the structural description of pattem in terns of a set of pattern primitives and their

a) 4 ’ to ncognize the shape of objects, the analysis of contours is frequently used. A contour is generally defined as a border between regions of different and relatively homogene brightness. These contours we discuss here represent the borders separating the different objects or separating the objects and the background which compose the image. ’be contours can be detected by an examination of local variations of brightness. ‘bere elrisk numrous local operators very sensible to the variations of the huge function [4]. In this paper, we use the padhut calculation method to detect the object contours m a color image. Finally, a set of pixels mpmenting the borders of objects are extracted from all the pixek of this image.

-

b) Codinn of CQIPteum ’ ’ in this procedure, chains of the pixels= formed. Each chain is a string of connected pixels such that there exists for each pixel of the chain no more than two pixels

71

......an

ala2

connected to itself. In this manner, all the pixels of contours are then ordered. Each chain of ordered pixek or each contour is segmnted and encoded into a string of Freeman codes. This slring is then approximated by a new symbolic representation: a string composed of polygons and circular a m . c) Q&ct

- 1

..

-

...., 7)

where ai€{O, 1, .i 12

on the . the attriited grammar is proposed

by You and F m shape recognition. In this grammar, each prhnitive or nontglninal has a symbol part and a value art that may have several set of attributes. Each synrb~lc production d e has a corresponding set of altribute ~ l e sthat process the attributes related to the production le in parsing.

In this paper, we propose a recognition structure which integrates symbolic processing and calculations of values for *rent property fields.Tbe sentence representing each object, which wlrsisls of two pa-: symlmk and values, can be conducted to match with a model in the knowledge base of image. This cedure is decomposed into two par+s: 1) matching o E e symbols between object and model. 2) matching of the values between object and model, which consists of two main operations as follows: assigning kmwn values to the variables in the ponding fields of model and detecting if the values coriMpponding filds of object and model are equal. If none of models in the base can match with the analyzed object, a new model is built according to the strudure of this object.

3z

. . d) Gend” of PMdels in the k: a model

6 Fig-1 Ffecmrn d i q scheme of contour

To simplify the object recognition, we define the direction of the Freeman code chain of the analyzed object as clockwise sense. me object syntactic expression (sentence) presented below adopts also this sense. In shape analysis and recognition procedure, polygonal approximation and circular arcs approximation are often used to cany out a s e p n t a t i o n of the contour of the analyzed ob’ d. From these appmmaxitions, we can get primitives o r t h e contour for o b p t “ p i t i o n [l], [7]. In this paper, we propose a new approximation method for the object coutour by using lines and circular a m . Tbis method is illustrated as follows:

......,

Let Po, Pi, Pm be consecutive points on the contour C where pi=(Xi,yi) (i=O,l, ......,m) and the Freeman code chain of P& ......Pm be ala2......am. Evidently, the initial point is Po=(xo,yo). For any Pm (mE{1,2, ......,n}), we have:

stored in the knowledge base is obtained from a set of objeds by using a learning procedure. This procedure to canserve common characteristics of these objects into the models and to adapt this model to s p i f i c characteristics of each object.

m

e$

In this paper, we are-interested in problems b), c) and d), which are respectively stated in detail from Section II to Section IV.

#n

(3) Next, we study the conditions which permits Po, PI, line segment. Assuming this line segment can be expressed by P#m, we get its equation as follows:

......, Pm to comtitute a ah

y-yo=fg$X-xO)

(4)

Il CODING OF CONTOURS AND APPROXIMATION

By using the well known gradiant method, we get the closed cohtour of the object as follows: C={(xi, yi)

I i=12,......,n}

(1)

For simplification, we suppose that the contour of one image object constitutes one closed chain. Each pixel in this chain coinc** with one of the eight pixels that sumund the previous pixel. If we label these eight neighboring pixek from 0 to 7 in a counterclockwise sense starting from the positive x axis, the chain of the object can be represented by a sequence of octal digits. This d i n g scheme is p p e d by Freeman [5].

Tbe Freeman code chain of a contour is expressed as follows:

m

m

i- 1

i- 1

where dn/4 is the angle between P@m and the positive x axis. The e m r of each point Pj @E{12,......,m}) to this line segment is expressed below:

The average of these e m t s for the points Pi, P2, Pm is expressed by:

72

......,

By noting p=(xo-n) and q=(yo-n,, we get the following conclusions from the equations &I, (9) a d (10):

......,

In this

we calculate successively D1, &, line segmnt if the following conditions are verified

Dm,

...... POP1......I")constitute a

DiE

21/2

R=@ +q )

The approximation procedure of contour points by a line segment is illustrated in Fig-2.After approximating P& Pm by a line, we carry out the same procedure for the rest points PmPm+l Pa in order to fmd the next line segment This procedure is repeated until the extreme points PD and Po are integrated into approximated line segmen6. Finally, the Freeman code chain of the contour is transformd into a chain of line segmnts (a closed polygon) which simplifies largely the recognition procedure.

......

(11)

A ( ~ ) ~ + B ( ~ ~ ~ ~ ( ~ ) + (12) B ~ ( ~ ) ) 2

for j€{l, ......,p)

where p is a integer defined a priori and E is the threshold of position error permitting to form a line segment.

......

A(m)p+B(m)q=*A2(m)+B2(m))

(13)

p, q are two variables which can be easily soived from

the equations and the center and the radius of the circle can be calculated from p and q. The error of each point Pi to this circle is defined by:

dI(peA( S)2+(4+8(9)2-$1

(14)

The average of these ermm to the cirde for Po,

......,

Pm is expressed by: m

m

The conditions permitting these points to form a circular arc are given below:

I

1) ~ ] : < e ' for i~{2,3,..,.,m} and &+,?e' for j€{l, ......,p} where E' is a threshold defined a pnori.

1 2

......

1

mm+l

...... m+p@

2) The number of line segments generated from the previous polygon approximation procedure for Po, ......, Pm is large enough (>5).

Fig-2 approximation by a line *went

On a object contour, there often exists certain regions whose curvatures are high enough and many small line segmenfs are gathered in these regions after the previous appromaxition procedure by lines. This makes the recognition procedure very complicated. To o v e m m this difficulty, we propose an approximation procedure which replaces these small tine segments by smooth circular a m .

The approximation procedure by circular arcs is repeated for the rest points Pm+l, ......, PD, Po. Finally, we get a closed objed contour which is represented by a chain of line segmenls and circular arcs segments.

We give below the procedure which permits the consecutive points of the contour Po, Pi, ......, Pm to be approximated by a circular arc.

In this paper, we propose a new syntactic repmion of object based on the idea of attributed p n n n a r presented in [6]. In our system, by integrating symbol part and value part, we give below the line segment expression:

The circle defined by Po, P d 2 and Pm can be expressed by: (x-ri)2+(y-Ir)2=R2

(7)

with Rt=(xo-ri+@(m))2+@o-Ir+B(m))2 R2=(xo-ri) +(yo-n), R =(xo-ri+A(m/2)) +(yo-n+B(~nD))~

(8)

m

(9) (10)

m

U;n

U g

where A ( m ) = x c o s q and B(m)=Esini- 1

i- 1

4

A(m) and B(m) can be calculated from the following recursive procedure: A(m)=A(m-l)+cos-

amn:

4

and B(m)=B(m-l)+sin-

4

111 SYNTACTIC STRUCTURE FOR IMAGE OBJHX

(a-m)(c-l)(d-O)

(16)

In this expression, a, c, d are symbols for identification and m, 1, 8 are values or variables. (a-m) means that the segment m is a line and it is the identification of thk segment, (c-1) means that the lengtb of this segment represents I percent of the penmter of the object (d-e) means that the angle from the positive x axis to this oriented line segment is 8. In this paper, the an@e diredion between vecteun is also defined as cbclnvlse sense. The syntactic representation of an aTc AB, which is expressed by its two radius AC et BC c o ~ e c t i n g respedively the extremes A and B to the center C, can be defined as follows:

where b, c, d are symbols and nl, values or variables with m=nl+l.

m, 1,

81,

eZ

are

Fig-3 A accular IVC AB

In this expression, @ni), @n2) represent the lines AC and BC. 1 is the percentage of AB in the object perimeter. 81 and eZ are respectively angles of the positive x axis to AC and BC.

In an object recognition procedure, we are more interested in relative positions between different segments of the object than their absolute mrdinrtts in the scene. merefore, we modify the previous expressions by changing e, 81, eZ into the a$es f" the vecteurs of these segmenls to the followmg neighbor segmnts. Next, we give an example to show the proposed synhctic expmssion.

w

n

4u2

This property is also a lied to line sc.gmenp3. It is satisfied by two s nts ik%cir lentb are equal with a precision e2 d e f i i x p r i o r i . The syntactic expression of this is (f-n) where f is the symbol for identification a n IS the numba of length equality class.

P p

Thk property is applied to the angle fields (symbal d) of object expression. T w o se nts validate this p r o m if they have the s1m value a precision e3) in them angle fields. The syntactic expression of this properly is (g-n) where g is the s 1 for identifiation a d n is the number of angle equa IQP Oclass.

It is also applied to the angle fields of object expression. This property is v d e d by a segment if i& value is equal to 90" or to 270" with a precision a defined a priori. Its expression is @-O) where h is the symbol for identification and its value field is 0. By inserting these properties into Example 1, we get a complete syntactic expression of the object Fig4 as follows:

H

3

Fig4 Object as a example

we give its syntactic expression as follows:

In this expression, there exists 5 s nt identification filds @2), (a-3), (a4), (a-Twhich represent respectively one circular arc and three lines. Each s identification field is followed by certain p m p e r t y E ! These property fields characterize the pmtrk form of the obJed and they are very useful in the recognition

el),

procedure.

before angle modification

after angle modification

In the object recognition procedure, the segment identification numbers are not very impoltant. Therefore, it is possible for us to make further simplification on this syntactic expression by integrating the segment identification field and the length fEld. Finally, we get the simplified expression as follows:

To recognize effectively the object form, we introduce s o m properties about the geometric form of the analyzed object into the syntactic expression. These geomtric propetties reflect the relations between different object segmenb. We illustrate these properties as follows:

(b-20 d-180)(d-270)(h-0) (a-30fd-90) e-l)(f-l)(g-l)@-O) (a-30)(d-90) e-1) f-l)(g-l)(h-O) (a-2O)(d-W)~-l7h-O)

IV OBJECT RECOGNlTlON PROCEDURE

This property is uniquely applied to line segments. Two segmnts are parallel if the scalar product of their

e

unitary vecteurs is equal to 1 with a recision EI defined a priori. The syntactic expression of t 's property is (e-n) where e is the symbol for identification and n is the number of parallel class.

In the object syntactic expression presented previously, the contour direction is defined as clockwise sense in order to identify easily the analyzed object. In the recognition procedure presented below, we consider that the syntactic

74 __

expression, after circular permutation from its original form, should represent the sa= object. For example, one circular permutation of the above expression is (a-30) ...@-20)...(a -3O)...(a -20) It represents the sa= object as its original form.

....

After the procedure of symbol matching, we get a set of syntactic e ressions of models M=(Mi,M2 Mg} so that Fw)=F;pO) (z{l3 q}). one model can be resented by several expressions in M, which are results ?different circular permutations.

,......

,......,

Next, we start the procedure of values matching by In this paper, we propose two processing subprocedures in order to recOgnize an introduced object according to the models in a existing knowledge base.

In the first sub-procedure, we carry out the matching of the symbok.

......

Assuming that the analyzed ob&{ is O=ClC2 Cn where Ci (E{1,2......,n}) represents the i segment (circular arc or line) in the syntactic expression. Next, we defiie a function F on the set of object syntactic expressions: F(O)=cla

......Cn

(18)

is the where 0 is the analyzed object and Cia......Cn chain of a and b which constitute the identification symbols of the object 0. n is the number of segments in the object 0. In Example 1, we have F(O)=baaa and n=4. M e r a circular permutation of the segments, we get F(O)=abaa. In the symbol matching procedure, we try to find a model M in the knowledge base so that F(M) is equal to F(0) in the sense of circular permutation. A model M can be expressed By M=Gi=......Gm where Gi (E{1,2 ,......,m}) is the it segment in the

syntactic expression of M. The syntactic expression sttucture of the model M is similar to that of objects. The unique difference is that specific values are replaced by variables in certain property fields of M. A model in the knowledge base is the generalized form of specific objeds. When M is used for recognition of an object 0, we make symbol matching between 0 and M and assign the values of the property fields of 0 to the variables in the corresponding fields of M.

comparing the ptoperty fiilds between the expression of

MIand that of 0. By noting M~GiiGiz ......Gin, we make matching between the corresponding property f i b of Gij and Cj.

If there does not exist a property fiild in %xhnging to Gij, the matching between Gij and Cj 6 If two different values are found in the sa= property field of Gij and Cj, the matching Eails ago.

.

If one variable is found in a proprty field of Gj., we assign the value of the corresponding fiild of Cj to it. The matching between Gj and Cj succeeds if the values of all the corresponding fields of Qj and Cj are equal. The object 0 is recognized by the model Mi if all the matchings between Qj and Cj succeed for j=1,2......,n. If the object 0 is not recognized by any of models in M, we consider 0 as a new model and put it into the knowledge base. The transformation from an object into a model form needs some human intervention so as to turn the values of certain property fiilds describing a specfic object into variables characterizing a generalized model. For example, the object in Example 1 can be generalized into a model a m d i n g to the re uiremnts of users. The corresponding model is given as Iollows:

By supposing that M and 0 have the same number of segments in syntactic expression (m=n) and F(M)=mi ma, we give below the procedure of symbol matching between F(M) and F(0):

where vari, with Vaq+.

......

0) Assign k:=l.

1) Assign i:=l.

2) If Ci=mi, then i:=i+l; goto 4). Eke, we carry out a circular permutation on C SO that Z=Cn, Cn:=Cn-l, ......, C2:=Ci, Ci:=Z and we get a new expression of 0 and the value of the function F(0) is also renewed. Assign k:=k+l. 3) If k=n+l, then stop. The matching with M fails. Else, goto 1). 4) If i=n+l, then stop. The matching with M succeeds. Else, goto 2).

......, varq

are variables

.....+varq=1

In this model, the proportions of length between different segments are variables. All the objects having the same form (composition of a semi circle and 3 edges of a rectangle) can be recognized by this model. A model can also be generalized from several objeds. In these objects, the corresponding property fields having the same values can be considered as common characteristics. These values should be also conserved in the corresponding fields of their model. If there exists different values in corresponding fields of the objects, we should associate variables to these fields of the model. Finally, the generalized model can represent common characteristics of the objects and it can be adapted to the specific characteristics of each object.

75

V SIMUIATION RESULT3

REFERENCES [ l ] K.S. FU Syntactic pattern recognition and applications PrenticeHall, I n c . Englewood Cliffs, New Jersey, 1951 [2] K S . FU Sequential metho& in pattern recognition aod machine leaning Academic Press, New York, 1968. [3] T. PAVLIDS Structural pattern recognition Springer-Verlag, New York, 1977. (41 J.G. POSTAIRE De I'image B la dCdsion Paris, Dunod, 1987. [SI H. FREEMAN "Computer procssing of linedrawing images" Computer Surveys, vo1.6, no.1, M a d 1974. [q F.C. YOU and K.S. FU "A syntactic approach to sbape reognition using attributed grammars" IEEE TransSMC-9, pp.334-45. 1979. [7JW.A. PERKLNS "A model-based vision system for industrial parts" IEEE Trans. Comput C-27, 126-43, 1978.

We give below a simulation example to illustrate the recognition procedure: Fxample 7: A square to be. recognized

Its syntactic expression is:

(a-26)(d-90)(e-l)(f-l)(g- l)(h-0) (a-24)(d38)(e-2)(f-l)(g-l)(h-O)

(a-25)(d-9 l)(e- l)(f-l)(g- l)(h-0) (a-25)(d-9l)(e-Z)(f-l)(g- l)(h-0) It is easily recognized by the model "ANGLE which exists in the knowledge base. RECTANGLE (a-varl)(d-90)(e-var3)(f-vars)(g- l)(h-O) (a-van)(d-90)(e-var4)(f-var6)(gl)(h-O)

(a-vari)(d-90)(e-va.a)(f-vars)(g-l)(h-O) (a-var2)(d-90)(e-var4)(f-vara)(gl)(h-O) After the matching procedure between the object and the model, the values of the object are assigned to the

variables in the model. We have: varl=255 var2=24.5 var3=l var4=2 vary=l varg=l VI CONCLUSION

This paper provides a new syntactic method for recognition from a color scene. The proposed approximation scheme permits to get a simplified form (lines and circular arcs) from Freeman code chain of object contours. The syntactic expressions presented in Section III, integrating the identification symbol part and the the value part, make the recognition procedure less complicated and more flexible. The objects which are not recognized can be entered into the knowledge base through a laming procedure which turns these objects into models. A good performance of this system is shown in the above simulation example. ACKNOWLEDGEMENTS

The proposed object recognition systeni was implanted by Mr C.CLAIRFT and Mr N.GALLIET in the preparation of their D E 3 memories.

76