FREEMAN CHAIN CODE AND. MODIFIED NEEDLEMAN-WUNSCH. Ema Rachmawati, Masayu Leylia Khodra, Iping Supriana. School of Electrical Engineering ...
SHAPE BASED RECOGNITION USING FREEMAN CHAIN CODE AND MODIFIED NEEDLEMAN-WUNSCH Ema Rachmawati, Masayu Leylia Khodra, Iping Supriana School of Electrical Engineering and Informatics Institut Teknologi Bandung
The 8th 2016 International Conference on Information Technology and Electrical Engineering (ICITEE)
OVERVIEW REPRESENTATION
SHAPE REPRESENTATION BY POLYGONAL APPROXIMATION based on FREEMAN CHAIN CODE
MATCHING
SEQUENCE ALIGNMENT ALGORITHM
CONTOUR BASED SHAPE DESCRIPTION
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
2
FREEMAN CHAIN CODE Definition of Freeman code [15]. sequence of n integer-coordinate points describe a closed curve 𝐶, 𝐶 = 𝑝𝑖 = 𝑥𝑖 , 𝑦𝑖 , 𝑖 = 1, … 𝑛 , where 𝑝𝑖+1 is a neighbor of 𝑝𝑖 (modulo n). The Freeman chain code of C consists of the n vectors, 𝑐 = 𝑝𝑖−1 𝑝𝑖 , each of which can be represented by an integer 𝑓 = 0, … , 7. The chain of C is defined as 𝑐, 𝑖 = 1, … , 𝑛 and 𝑐 = 𝑐𝑖±𝑛 . All integers are modulo n.
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
3
POLYGONAL APPROXIMATION
Modifying the breakpoint concept of [14], we give the pseudocode to extract line segments from string of chain code
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
4
0 0 6 1 1 7 1 0 0 0 0 0 0 0
7 0 0 0 0 0 0 0 0 0 0 0
1
7
1
7
1
7
1
7
1
7
2
7
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
2
6
3
5 3
5 3
5 3
5 3
5 3
5 3
5 3
5 3
5 4 4 4 4 4 4 4 4 4 4 4 4 4 4 5
0-0-0-0-0-0-0-7-0-0-0-7-0-0-0-0-0-7-0-0-0-0-0-0-7-7-7-0-0-7-07-7-7-6-7-6-7-6-7-6-6-7-6-6-6-6-7-6-6-6-6-7-6-6-6-6-6-6-5-6-55-6-5-5-5-6-5-5-6-5-6-6-5-5-6-5-6-5-6-5-6-5-5-4-4-5-4-5-5-4-55-4-5-4-5-4-4-4-4-4-4-4-4-4-4-3-4-4-4-3-4-4-3-4-4-3-4-3-3-3-33-2-3-3-3-2-3-3-3-2-3-2-3-3-2-3-2-2-2-2-2-2-2-2-2-2-2-2-2-2-22-2-2-2-2-1-2-2-2-2-2-2-2-2-2-1-1-1-1-1-1-1-1-1-1-0-1-0-0-0-00-0-1-0-0-0-0-1-0-5
Length = 194 characters
0: 0.122 - 7:0.015 - 0: 0.02 - 7: 0.046 - 6: 0.107 - 5: 0.056 - 6: 0.01 - 5: 0.056 4: 0.02 - 5: 0.046 - 4: 0.112 - 3: 0.102 - 2: 0.153 - 1: 0.061 - 0: 0.066 - 5: 0.005
Length = 16 characters (of line segment)
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
5
SHAPE SIMILARITY MATCHING USING NEEDLEMAN-WUNSCH [10] SCORING FUNCTION (σ) 𝑖𝑓 𝑝1 𝑖 = 𝑝2 𝑗 𝑡ℎ𝑒𝑛 𝜎 𝑝1 𝑖 , 𝑝2 𝑗 = 𝑤1 𝑖 × 1 𝑒𝑙𝑠𝑒 𝜎 𝑝1 𝑖 , 𝑝2 𝑗 = 𝑤1 𝑖 × −1 GAP PENALTY RECURRENCE RELATION
𝑇 𝑖 − 1, 𝑗 − 1 + 𝜎(𝑆1 𝑖 , 𝑆2 (𝑗) 𝑇 𝑖 − 1, 𝑗 + 𝑔𝑎𝑝𝑃𝑒𝑛𝑎𝑙𝑡𝑦 𝑇 𝑖, 𝑗 = max 𝑇 𝑖, 𝑗 − 1 + 𝑔𝑎𝑝𝑃𝑒𝑛𝑎𝑙𝑡𝑦 The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
6
WORKFLOW
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
7
EXPERIMENT
Fruit images of RGBD Object Dataset [18]. - 7 species; 32 variety. - 10 images per fruit variety 320 images in total.
No Fruit Species
Fruit Variety
1
Apple
Apple_1, Apple_2, Apple_3, Apple_4, Apple_5
2
Banana
Banana_1, Banana_2, Banana_3, Banana_4
3
Lemon
Lemon_1,
Lemon_2,
Lemon_3,
Lemon_4,
Lemon_5, Lemon_6
4
Lime
Lime_1, Lime_2, Lime_3, Lime_4
5
Orange
Orange_1, Orange_2, Orange_3, Orange_4
6
Peach
Peach_1, Peach_2, Peach_3
7
Pear
Pear_1, Pear_2, Pear_3, Pear_4, Pear_5, Pear_6
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
8
EVALUATION 𝑇𝑃 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛 = 𝑇𝑃 + 𝐹𝑃
𝑇𝑃 𝑅𝑒𝑐𝑎𝑙𝑙 = 𝑇𝑃 + 𝐹𝑁
Each image was used as a query, and the number of similar images (which belong to the same class) was counted using some threshold value. true positive(TP) = the similarity value is in accordance with particular threshold and the image being compared is having the same fruit variety with the query image false positive (FP) = the similarity value is in accordance with particular threshold, but the fruit variety of image being compared is different to query image, the image is not relevant to query image false negative (FN) = the similarity value is not in accordance with particular threshold, but the image being compared is having the same fruit variety with the query image The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
9
Weighted Line Segment in Recognition involving weight segment in the similarity calculation, might increase the TP, with the number of FP also increases. Hence, the precision is decreased.
It can be inferred from the recall value that the detection performance seems promising, tough still many false positive detected.
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
10
The smaller the threshold, the bigger the recall. This condition is caused by the true positive becomes higher and the false negative become smaller. Meaning, the number of retrieved images having the same fruit class with the query image, is increasing.
the smaller the threshold, the smaller the precision. In this case, the false positive become higher. Meaning, the number of retrieved images recognized as having the same fruit class with the query image, is increasing, tough those images did not fall into the same class as query image.
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
11
Detection Rate/False Positives Per Images (DR/FPPI) curves
Detection rates at the standard 0.3/0.4 FPPI that range from 0.3 to 1
The best recognition comes from lime_1 and pear_6, and it is probably due to the fact that the approximated polygon belonging to these classes is very distinctive (and hence easy for discrimination) compared with other fruit species. The worst result (in terms of AR) are from lemon_6. The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
12
CONCLUSION ■ A novel shape representation and matching method based on Freeman Chain Code and Needleman-Wunsch concept is presented.
■ We represent the shape of an object into segment-based representation using Freeman Chain Code – each segment having a particular weight in accordance with its length in its polygonal approximation of the object shape ■ We modified the scoring function used in Needleman-Wunsch algorithm for calculating the similarity between shapes. – by defining a new substitution matrix for the purpose of scoring function, which considers the characteristics of set of polygon segment as the representation of the object shape – we successfully shown that the weight of each segment of the object shape has positive impact in the similarity calculation
The 8th International Conference on Information Technology and Electrical Engineering (ICITEE)
13
1. 2.
3. 4.
5. 6. 7. 8. 9. 10. 11.
12. 13.
14. 15. 16. 17. 18.
O. Van Kaick, H. Zhang, G. Hamarneh, and D. Cohen-Or, “A survey on shape correspondence,” Computer Graphics Forum, vol. 30, no. 6. pp. 1681–1707, 2011. S. Belongie, J. Malik, and J. Puzicha, “Shape matching and object recognition using shape contexts,” Pattern Anal. Mach. Intell. IEEE Trans., vol. 24, no. 4, pp. 509–522, Apr. 2002. V. Ferrari, L. Fevrier, F. Jurie, and C. Schmid, “Groups of adjacent contour segments for object detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 1, pp. 36–51, 2008. C. Yang, C. Feinen, O. Tiebe, K. Shirahama, and M. Grzegorzek, “Shape-based Object Matching Using Point Context,” in ICMR, 2015, vol. 1, pp. 519–522. K. Kapach, E. Barnea, R. Mairon, Y. Edan, O. Ben-Shahar, and K. K. E. B. and R. M. Y. E. O. Ben-Shahar, “Computer Vision for Fruit Harvesting Robots: State of the Art and Challenges Ahead,” Int. J. Comput. Vis. Robot., vol. 3, no. 1/2, pp. 4–34, Apr. 2012. F. Attneave, “Some Informational Aspects of Visual Perception,” Psychol. Rev., vol. 61, no. 3, 1954. R. Neumann and G. Teisseron, “Extraction of dominant points by estimation of the contour fluctuations,” Pattern Recognit., vol. 35, no. 7, pp. 1447–1462, 2002. H. Freeman, “On the Encoding of Arbitrary Geometric Configurations,” IEEE Trans. Electron. Comput., vol. EC-10, no. 2, pp. 260–268, Jun. 1961. G. Navarro, “A guided tour to approximate string matching,” ACM Comput. Surv., vol. 33, no. 1, pp. 31–88, 2001. S. B. Needleman and C. D. Wunsch, “A general method applicable to the search for similarities in the amino acid sequence of two proteins,” J. Mol. Biol., vol. 48, no. 3, pp. 443–453, 1970. L. Chen, R. Feris, and M. Turk, “Efficient partial shape matching using Smith-Waterman algorithm,” in 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops, 2008, pp. 0–5. J. Sinapov, T. Bergquist, C. Schenck, U. Ohiri, S. Griffith, and A. Stoytchev, “Interactive Object Recognition Using Proprioceptive and Auditory Feedback,” in Proceedings of the 2009 IROS Workshop: Semantic Perception for Robot Manipulation, 2009. M. N. V. Muzameel Ahmed, “Adopting Biological Sequence Alignment Tools in 2D Shape Recognition,” Int. J. Innov. Res. Sci. Eng. Technol., vol. 4, no. 8, pp. 6969–6975, 2015. W. Wu, “Dominant point detection using adaptive bending value,” Image Vis. Comput., vol. 21, pp. 517–525, 2003. B. K. Ray and K. S. Ray, “A new split-and-merge technique for polygonal approximation of chain coded curves,” Pattern Recognit. Lett., vol. 16, no. 2, pp. 161–169, Feb. 1995. M. Dayhoff and R. Schwartz, “A Model of Evolutionary Change in Proteins,” in In Atlas of protein sequence and structure, 1978, pp. 345– 352. S. Henikoff and J. G. Henikoff, “Amino acid substitution matrices from protein blocks.,” Proc. Natl. Acad. Sci. U. S. A., vol. 89, no. 22, pp. 10915–10919, 1992. 14 K. Lai, L. Bo, X. Ren, and D. Fox, “A large-scale hierarchical multi-view RGB-D object dataset,” in ICRA, 2011, pp. 1817–1824.
