Shape Recognition based on Shape-Signature ... - IEEE Xplore

Shape Recognition based on Shape-Signature Identification and Condensibility: Application to Underwater Imagery Jeet Banerjee1 , Ranjit Ray2 , Siva Ram Krishna Vadali3 , Ritwik Kumar Layek4 , Sankar Nath Shome5

Electronics and Electrical Communication Dept, Indian Institute of Technology Kharagpur, Kharagpur, West Bengal-721302 1,4 Robotics and Automation Div., CSIR-Central Mechanical Engineering Research Institute, Durgapur, West Bengal-713209 2,3,5 Email: {jeetbanerjee1 , ritwik4 }@ece.iitkgp.ernet.in, {ranjitray2 , srk vadali3 , snshome5 }@cmeri.res.in, Abstract—In this paper, a shape recognition method is proposed for a few common geometrical shapes including straight line, circle, ellipse, triangle, quadrilateral, pentagon and hexagon. In the present work, two indices namely Unique Shape Signature (USS) and Condensibility (C) are employed for shape recognition of an object. Using the USS index, all the above mentioned non-circular shapes are neatly recognized, whereas, the C index recognized the circular objects. An added advantage of the proposed method is that it can further differentiate triangles, quadrilaterals and both symmetric and non-symmetric shapes of pentagon and hexagon using distance variance (V ar(dsi )) parameter calculated from USS. Applying the proposed method on above mentioned shapes, an overall recognition rate of 98.80% is achieved on several simulated and real objects of different shapes. Proposed method has also been compared with two existing methods, presents better result. Performance of the proposed method is illustrated by applying it on underwater images and it is observed to perform satisfactory on all the images under test.

I.

I NTRODUCTION

Owing to the recent technological advancements in industrial automation, design of novel shape recognition methodologies has become a common requirement. A wealth of research is available in the field of object shape recognition and understanding. Zai Kai Ku et al.[1] used width to height ratio (WTHR), Base to Abdomen ratio (BTAR) and shape boundary test (SBT) for differentiating among humans, animals, vehicles and other object shapes. Object shape recognition using shape context has been done by S. G. Salve et al. [2]. The main idea in shape context is to describe all boundary points of a shape with respect to single boundary point which is independent of translation, rotation and scaling, is used for object shape recognition and matching with database images for image retrieval purpose. Object shape recognition using tactile images has been done by Amit Konar et al. [3]. Shape recognition of planar surfaces, objects with one edge and two edges, cylindrical objects is emphasized using segmentation of high pressure regions and by feature extraction from different tactile images. Other related research on automatic object recognition using image morphology is carried by Shih and Mitchell [4]. Fuzzy logic based object shapes classification is also available in [5]. In general, objects are identified based on properties of images like texture and color [6, 7]. However, shape of an object is a dominant property which helps in near accurate

object recognition [8]. In the present work, object shape recognition is performed using rotation, translation and scale invariant parameters [8, 9] i.e. USS and C. Standard shapes like straight line (ST), circle (CR), ellipse (EL), triangle(TR) (equilateral (ET), isosceles (IT) and scalene (SCT)), quadrilateral (QR) (rectangle (RT) & square (SQ)), pentagon (PT) and hexagon (HX) (symmetrical (SY) & asymmetrical (ASY)) are tested with the proposed shape recognition approach. The proposed method has also been tested on underwater objects and observed to provide satisfactory results. The rest of the paper is organized as follows: proposed method is presented in Section 2. Consequent experimental results and comparison are discussed in Section 3. Section 4 discusses the performance of the proposed method on underwater imagery. Section 5 gives conclusion on proposed method. II.

P ROPOSED M ETHOD

In the proposed approach to shape recognition, two rotation, translation and scale invariant parameters i.e number of corners or sides calculation from USS and C have been considered. USS, is basically a graphical representation of the distance between edge points of the shapes and its centroid as shown in Fig. 1, is unique for a given shape category. xi , y i d ( x, y )

Where, ( xi , yi ), i 1,........,N are ' N' edge points. (c x ,c y ) is the centroid of the edge detected image.

cx , cy

d(x,y) distance between centroid and edge points (DBCEP)

[(c x  xi ) 2  (c y  yi ) 2 ].

Fig. 1: Centroid, edge points and DBCEP representation

It is observed that, corner detection of edged objects is not quite accurate using standard methods. Hence USS has been adopted for detecting the number of corners or number of sides in a given shape. To illustrate the advantage of using USS, an edge detected image with an equilateral triangle (Fig. 2 (a)) is considered. First, the distance of each of the edge points from the centroid of the image is calculated and represented graphically (Fig. 2 (b)).

Distance between edge points and centroid

200 180

P1 A

160

E F

C D

140

1, 2, ........., (i/2) − 1 and m2 = (i/2) + 1, (i/2) + 2, ........, i (1) can be written as:

P1

P3

P2 B

i/2−1

120

L2

100 80 0

L4

V1 200

L5

V2

L6

400 600 Index of edge points

(a)

μs1 = E(ds1 ) = 2

L1

L3

V3 800

= (1/i)[4(d1 + d2 + ...... + di/2−1 ) + di/2 ]

Fig. 2: (a) Triangular shape; (b) USS of triangle

USS can also be used for further differentiating QR (RT and SQ); TR (ET, IT and SCT); symmetric and asymmetric PT, HT by applying the concept of variance to the set of distances between edge points and centroid of a particular side. Mathematical essence of this concept is described below by considering a rectangular figure as shown in Fig. 3. Here Ce = centroid, di = distance between Ce and ith edge point= max{d1 , d2 , ......, di } − di . di/2

dn d(nk) / 2

ce

Side4 (s4)

dj Side3 (s3)

dk

d( jk) / 2

Fig. 3: Rectangular figure with mentioning the distances Expectation of set of distances from edge points to centroid, considering the side 1: = E(ds1 ) =

i 

dm p(dm )

m=1 i/2−1

=



m1=1

dm1 p(dm1 ) + di/2 p(di/2 ) +

i 

V ar(ds1 ) =

i 

(dm − μs1 )2 p(dm )

m=1

= (1/i)[4{(d1 − μs1 )2 + ...... + (di/2−1 − μs1 )2 } +(di/2 − μs1 )2 ]

dm2 p(dm2 )

m2=i/2+1

(1) In (1), p(dx )= probability of occurrence the distance value dx in Ωs1 . Ωs1 = (d1 , d2 , ........., di−1 , di ). Now, it is very obvious that (d1 = di ), (d2 = di−1 ), ........, (di/2−1 = di/2+1 ) and p(dm1 ) = p(dm2 ) = 2/i; m1 =

(3)

Similarly (i+j)/2−1

μs2 = E(ds2 ) = 2



dm1 p(dm1 ) + d(i+j)/2 p(d(i+j)/2 )

m1=i

=

1 [4(di + di+1 + ...... + d (i+j) −1 ) + d (i+j) ] 2 2 (j − i)

(4)

and V ar(ds2 ) =

j 

(dm − μs2 )2 p(dm )

m=i

(5) 1 [4{(di − μs2 )2 + ...... + (d (i+j) −1 − μs2 )2 } = 2 (j − i) +(d (i+j) − μs2 )2 ] 2

For rectangle, V ar(ds1 ) V ar(ds2 )  V ar(dS4 )

d1

Side2 (s2)

d (i  j ) / 2

(2)

and corresponding variance:

From this knowledge, it is easier to calculate the number of peak points and number of valley points which represent the number of corners and the number of sides respectively. As corners or number of sides are the dominant characteristics for differentiating among triangle, quadrilateral, pentagon and hexagon, USS has been used to evaluate corners or number of sides. Using the present approach, for a triangle, a quadrilateral, a pentagon and a hexagon 3, 4, 5 and 6 valley points are obtained respectively. Detection of peak and valley points from USS is invariant to rotation, translation and scaling and is therefore extremely helpful to identify the number of corners or sides accurately in all circumstances.

di

dm1 p(dm1 ) + di/2 p(di/2 )

m1=1

1000

(b)

Side1 (s1)





V ar(ds3 )

and

By generalizing the above concept for remaining shapes it can declare that For Square,V ar(ds1 )  V ar(ds2 )  V ar(ds3 )  V ar(ds4 ) For equilateral triangle, V ar(ds1 )  V ar(ds2 )  V ar(ds3 ) For isosceles triangle,V ar(ds1 )  V ar(ds2 ) or V ar(ds1 )  V ar(ds3 ) or V ar(ds2 )  V ar(ds3 ) For scalene triangle,V ar(ds1 ) = V ar(ds2 ) = V ar(ds3 ) For symmetric pentagons and hexagons, V ar(ds1 )  V ar(ds2 )  V ar(ds3 )  V ar(ds4 )  V ar(ds5 ) and V ar(ds1 )  V ar(ds2 )  V ar(ds3 )  V ar(ds4 )  V ar(ds5 )  V ar(ds6 ) respectively and otherwise it signifies asymmetric pentagons and hexagons respectively. Variance parameter can be used for sub-categorical differentiation of QR, TR, PT and HX due to its invariant property considering translation, rotation and scaling. In the present approach, another parameter C is used for identification of circular objects defined as a ratio between area multiplied by 4*pi and square of perimeter for a particular shape, shown in (6). C=

4π × Area P erimeter2

(6)

Geometrically, C is equal to 1 for a circle. However, in an image processing point of view, perimeter and area are the

number of pixels at the outer periphery and total number of pixels contained in a particular shape respectively, so the value of C remains very close to 1 as depicted in Fig. 4.

Fig. 4: Range of C for different shapes

On the other hand for non-circular shapes, C is less than 0.9 and is different for different object shapes. It is also seen that the range of values the parameter C for non-circular shapes overlap. Hence it is difficult to recognize non circular shapes only through C, which necessitates the need for USS as described above. The following section presents simulation methodology, discusses the results obtained thereby and compares with existing methods. III.

between a rectangle and square is that, opposite sides are equal in the case of rectangle and whereas in the case of square all the four sides are equal. So, in case of rectangle, distances between two consecutive peak points are equal in an alternative fashion. However for a square all sides are equal and hence distances between all the consecutive peak points are equal as shown in the Fig. 5 (f1, f2 and g1, g2 respectively). An inference that can be made out of these two illustrations is that if any shape is having four valley points then it should be a quadrilateral and it is possible to further differentiate them into a rectangle and a square with help of V ar(dsi ) calculated from USS. In similar fashion the variants of triangles can also be distinguished ( Fig. 5 ( c1, c2; d1, d2 and e1, e2)). Moreover, if all peak-to-peak distances are same then it should be an equilateral triangle, if alternative peak-to-peak distances or two consecutive peak-to-peak distances are equal then it would be an isosceles triangular shaped object. TABLE I: OVERALL SHAPE RECOGNITION RATE (in %) Manuel & Joaquim [5]

Proposed

95.80

98.8

TABLE II: SUB CATEGORICAL SHAPE DIFFERENTIATION RATE (in %)

E XPERIMENTAL R ESULTS AND C OMPARISON

The proposed signature-condensibility method has been tested on different types of images - i) objects drawn in AutoCAD, real standard shapes of ii) objects captured using Nikon Coolpix L610 having 14x zoom, 16.0 megapixels resolution, and iii) underwater objects from a camera attached to AUV150 [10,11]. In the present analysis, C is used to distinguish a circular shape from non-circular shapes like ST, EL, TR, QE, PT, and HX. For a perfect circle, the index USS is a straight line. However, edge detection of circular objects usually comes with pixel discontinuity and USS is not an useful approach to shape detection. Now a non-circular shapes can be neatly distinguished from USS and its other properties, identifying the number of valley points or number of peak points, which represents the number of sides and number of corners respectively. USS follows unique structure for a particular shape category as shown in Fig 5. From Fig 5, it is obvious that USS signifies a particular shape in representing the number of corners or number of sides present in a shape. From Fig 5 (a1, a2) it is clear that USS for straight line possesses a single valley point. This is because the centroid of a straight line is its edge pixel. In Fig 5 (b1, b2), it may be seen the USS for an ellipse contains vital information i.e. length of semi major and semi minor axis and also contains two peak points and two valley points, which is helpful for ellipse identification using USS. Similarly the TR, QR, PT and HX contain three, four, five and six sides and corners respectively, which is evident from USS of TR (c1, c2; d1, d2; and e1, e2), RT (f1, f2), SQ (g1, g2), PT (h1, h2) and HX (i1, i2), as shown in Fig. 5. Usefulness of USS is that, it is possible to distinguish between rectangles and squares and among different triangles like equilateral, isosceles and scalene triangle. The main difference

Ehsan & Erfan [8]

94

Shapes

Manuel-Joaquim [5]

TR (ET, IT, SCT) QR (RT & SQ) PT & HX (SY & ASY)

83.33 85.71 84.52

Ehsan-Erfan [8] 92.85 89.28 90.47

Proposed 96.82 97.61 97.61

It needs to be mentioned here that the proposed method has been tested on more than 500 objects of different shapes (few glimpses are shown in Fig. 5, 6, 7) and the success rate is 98.8%. From table I and II it is cleared that proposed method gives better results in shape recognition as compared with other two existing methods. IV.

A PPLICATION TO U NDERWATER I MAGERY

The present work was initiated as a part of post-trial (visual image) processing involved in project AUV- 150 [10,11]. The basic requirement of the work was to identify and classify objects observed in the underwater images/ videos (with a standard resolution of 640 x 480) captured by the AUVonboard underwater camera (manufactured by Kongsberg Simrad, model No. OE14-110/111). For this purpose, initially available shape recognition methods were implemented in MATLAB environment. In view of the limitations of available methods the present USS based method was devised and employed in the application. The present section validates the shape recognition performance of the proposed method when applied over underwater images. Since underwater images commonly suffer from Color Cast problem [12], a preenhancement method was also an essential operation [13]. Before applying the proposed shape recognition algorithm a few further steps were also employed namely., (i) Sobel Edge detection ; (ii) Cubic interpolation of edge detected image; (iii) Freeman chain coding on interpolated image [14]; (iv)

Dilation-closing-filling operation; and (v) CG calculation. The proposed USS-(C) approach has been applied to both structured & unstructured underwater objects and were observed to be neatly recognized, as shown in Fig. 7. V.

d1

d2

g1

g2

e1

e2

f1

f2

h2

i1

i2

C ONCLUSION

This paper presents a signature identification cum parameter evaluation based method to automatic shape recognition of an object. From the different shapes recognized using the proposed method, it is observed that the object shape recognition is performed to a 98.8% certainty. Since the operations including edge detection, calculation of DBCEP and generating USS are less time consuming, it can be deployed for real time applications as well. The proposed approach has also been employed on underwater images (structured and unstructured) and it is observed that it gives competent recognition results which partially fulfilling the aim of applying it to AUV -150.

h1

Fig. 5: (a1,a2) Straight lines and their USS; (b1, b2) Ellipses and their USS; (c1, c2) Equilateral triangles and their USS; (d1, d2) Isosceles triangles and their USS; (e1, e2) Scalene triangles and their USS; (f1,f2) Rectangles and their USS; (g1, g2) Squares and their USS; (h1, h2) Pentagons and their USS; (i1, i2) Hexagons and their USS.

R EFERENCES [1]

[2] [3] [4]

[5] [6]

[7] [8] [9] [10]

[11]

[12]

[13]

[14]

Z.K. Ku, C.F. Ng and S.W. Khor, ”Shape Based Recognition and classification for common objects- An Application for Video Scene Analysis,” ICCET, vol. 3, pp. 13-16, 2010. S.G. Salve and K.C. Jondhale, ”Shape Matching and Object Recognition using Shape Contexts,” ICCSIT, vol. 9, pp. 471-474, 2010. G. Singh, A. Jati, Konar, A. et. al, ”Object Shape Recognition from Tactile Images Using Regional Descriptors,” NaBIC, pp. 53-58, 2012. F.Y. Shih and O.R. Mitchell, ”Automated Fast Recognition and Location of Arbitrarily Shaped Objects by Image Morphology,” CVPR, pp. 774779, 1988. . M.J. Fonseca and J.A. Jorge, ”Using Fuzzy Logic to Recognize Geometric Interactively,” FUZZ-IEEE, vol. 1, pp. 291-296, 2000. D. Smith and G. Lu, ”Review of shape representation and description techniques,” The Journal of the Pattern Recognition Society, vol. 37, pp. 1-19, 2004. J. Wu and Z. Xiao, ”Video Surveillance Object Recognition Based on Shape and Color Features,” CISP, vol. 1, pp. 451-454, 2010. E. Moomivand and E. Abolfazli, ”A Modified Structural Method for Shape Recognition,” ISIEA, pp. 332-336, 2011. O.C. Hamsici and A.M. Martinez, ”Rotation Invariant Kernels and Their Application to Shape Analysis,” PAMI, vol. 31, pp. 1985-1999, 2009. S.N. Shome, S. Nandy, D. Pal et. al, ”Development of Modular Shallow Water AUV: Issues & Trial Results,” Journal of Institute of Engineers (India), Series-C, (Journal: Large Springer India), vol. 93(3), pp. 217228, 2012. S.N. Shome, S. Nandy, S.K. Das et. al, ”Autonomous Underwater Vehicle for 150m DepthDevelopment Phases and Hurdles Faced,” Trends in Intelligent Robotics, Communications in Computer and Information Science, vol. 103, pp. 49-56, 2010. K. Iqbal, M. Odetayo, A. James et.al, ” Enhancing the low Quality Images using Unsupervised Color Correction Method,” SMC, pp. 17031709, 2010. J. Banerjee, R. Ray, S.N. Shome, and S.R.K. Vadali, ”An Unsupervised Underwater Image Enhancement Method Based on RGB and YCbCr Color Space Processing for Online Applications,” The Imaging Science Journal, 2013 (Submitted). J. Banerjee, R. Ray, S.N. Shome, ”A Novel Approach for Freeman Chain Coding with prior modification using Cubic Interpolation,” ICCIC, pp. 1-4, 2012.

a1

a2

b1

b2

c1

c2

a

b

c

d

e

Recognised as (a) Circle with C=0.9038; (b) Triangle with number of sides (NOS) =3, C=0.5625; (c) Rectangle with NOS=4, C=0.7215; (d) as Pentagon with NOS=5, C=0.7478; (e) as Hexagon with NOS=6, C=0.8036. Fig. 6: Real images of different shapes

a1

a2

a3

a4

a5

a6

a7

a8

b1

b2

b3

b4

b5

b6

b7

b8

Fig. 7: (a1, b1) Original Underwater Images of ractangular shaped object and a hammer respectively; (a2, b2) Enhanced images with URYPr method [14]; (a3, b3) Sobel edge detected images; (a4, b4) Cubic interpolated images; (a5, b5) Applying Freeman chain coding on interpolated images; (a6, b6) Employing Dilation-Closing-Filling operation on images: (a5, b5); (a7, b7) Sobel edge detection of filled images and calculating the CG point; (a8, b8) USS of particular extracted shapes.