Intelligent System for Persian Calligraphy Learning - Science Direct

Available online at www.sciencedirect.com

ScienceDirect Procedia Computer Science 102 (2016) 555 – 561

12th International Conference on Application of Fuzzy Systems and Soft Computing, ICAFS 2016, 29-30 August 2016, Vienna, Austria

Intelligent system for Persian calligraphy learning Fahreddin Sadikoglua*, Behzad Soroushb a

Department of Electrical and Electronic Engineering, Near East University,P.O.BOX:99138,Nicosia,North Cyprus, Mersin 10, Turkey b Department of Information Systems Engineering, Near East University, P.O.BOX:99138, Nicosia,North Cyprus, Mersin 10, Turkey.

Abstract Persian Nastaliq calligraphy is one of the most famous oriental arts. This study proposes for the first time an intelligent tutor system for automation of the Persian calligraphy learning process. The system incorporates image processing and machine learning technologies. Digitization, filtering, segmentation and feature extraction are image-processing techniques that prepare the appropriate input for the training phase of the SVM machine learning phase. Displaying suitable feedback on screen for learners is another aim of the study. In this regard, the system provides facilities for Persian Nastaliq calligraphy learners to reduce errors that are inherent in traditional education methods, makes the process more efficient and allows people to take advantage of learning possibilities whenever and wherever they choose. © Published by Elsevier B.V. This © 2016 2016The TheAuthors. Authors. Published by Elsevier B.V.is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Organizing Committee of ICAFS 2016. Peer-review under responsibility of the Organizing Committee of ICAFS 2016

Keywords: Image Processing; Machine Learning; SVM; Intelligent Tutor; Persian Nastaliq Calligrapy.

1. Introduction Persian Nastaliq is one of the best1 and most beautiful2 Persian calligraphy styles. In the traditional method of Nastaliq calligraphy teaching, learners are required to learn in advances how to use ink and a specific calligraphy pen called “Galam” on glossy coated paper and then they must practice the sample isolated character called “Mashq” several times as assigned by the teachers. Learners’ Mashqs are carefully evaluated by the teacher one by one using specific rules and feature structures that are specifically related to each isolated character stroke proportion. This requires more time and effort to assess the work and give

* Fahreddin Sadikoglu. ;Tel.: +90-392-223-6622. E-mail address: [email protected]

1877-0509 © 2016 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer-review under responsibility of the Organizing Committee of ICAFS 2016 doi:10.1016/j.procs.2016.09.442

556

Fahreddin Sadikoglu and Behzad Soroush / Procedia Computer Science 102 (2016) 555 – 561

feedback on the learners’ Mashqs3,4. The lack of specialists in this area who can act as teachers is another insufficiency in the traditional Nastaliq teaching method, which leads to learners becoming discouraged and leaving courses3. In this paper, the Persian Nastaliq Calligraphy Intelligent Learning (PeNCIL) tool is presented as an intelligent standard tutorial tool to overcome the traditional method’s insufficiencies. The developed system provides facilities for learners to practice individually and, in addition, the PeNCIL is capable of evaluating and providing appropriate feedback automatically on the learners’ work. 2. Related Work Xu et al.5 designed and implemented a Chinese calligraphy system involving Artificial Intelligence techniques and machine learning algorithms. Existing calligraphy strokes extraction was carried out by a two-phase semi-automatic mechanism. The developed system was useful for “aesthetic calligraphy decomposition, visual quality evaluation, automatic calligraphy generation, and calligraphy writing tutoring and correction”. Bezine and Alimi6 developed an automated Arabic handwriting system. The developed system is capable of checking for mistakes in the inserted handwriting and provides immediate feedback to the students in order to correct these mistakes. This system was developed by applying the Attributed Relation Graph (ARG) method by using a graph matching algorithm to detect differences between the inserted pattern and the template in order to determine optimal graph matching. Taele and Hommond7 presented an intelligent sketch educational tool called BopoNoto, based on the Zhuyin system of phonetic notation, in order to provide a platform for Zhuyin language students. The developed system included two main parts: a suitable interface that provides facilities to students for practicing the symbols, and a recognition and assessing inserted sketch subsystem facilities to assess technical and visual correctness. Albu, Hagiescu, Puica and Vladutu8 developed an English language intelligent tutoring system for pupils. The developed system included two main sections. The first section of the system evaluated students’ handwritten symbols quality automatically, which was acquired by using a digital-pen. A binary image representation technique called Dynamic Time Warping (DTW) was carried out to compare the inserted symbol with the template. They claimed that teaching handwriting improves pupils’ personality, communication skills and basic coordination abilities. Therefore, emotion recognition consisting of speech and facial emotion evaluations were conducted in the study. Positive, negative and neutral are the three basic emotional expressions recognized by the developed system, which revealed the pupils’ interest level for the lesson. Han, Chou and Wu9 developed a fuzzy inference and image processing system targeted at primary school students to evaluate their Chinese calligraphy skills, which is capable of scoring their handwriting automatically. Accurate location and appropriate width and size were used as assessing statistical features. 3. The Aim of the Study The development of an intelligent platform for Persian calligraphy learners to practice individually and receive suitable feedback is the main aim of the current study. In this regard, the PeNCIL tool was developed to overcome the shortages in the classical learning method. 4. The Study Framework Persian calligraphic characters are written on glossy coated paper3 with cream or tan color. In order to identify and evaluate the students’ handwriting, the authors used image processing and machine learning

557


methods. The current study framework consists of six main sections as shown in Fig.1: Input data, image processing, feature extraction, character recognition, assessment and feedback. 4.1. Input data Written Isolated Calligraphy Character (ICC) images were applied as input data in the current study. The Offline method10 was carried out to complete the input process by converting the written paper-based ICCs into images (JPG and BMP formats), either by scanning or using a camera.

Proces singIn put

Pre-

ICC Image

Image Processing

Feature Extraction

Character Recognition

Assessment

Feedback

Machine Learning SVM Fig. 1. Framework of the Study

4.2. Pre-Processing The main result of the pre-processing step was the preparation of a suitable image for the feature extraction phase, the steps of which are shown in Fig. 2. Pre-processing consists of the Grayscale conversion, Binarization, Noise Removal, Dilation and Normalization steps. The Grayscale conversion function converts the inserted image from Red-Green-Blue (RGB) format to Grayscale. The OTSU method as a global Binarizition11,12,13 operation converts the Grayscale image to a binary format of pixels xi ࣅ{0,1}, where zero refers to a white pixel and 1 refers to a black one. The Grayscale filter and Binarizition operations were used to decrease the size of the input image12. In the Noise-Removal step, many pixels are lost and image clarity decreases. Morphology offers a set of image processing functions, including Dilation, which was applied in the next step. Afterwards, Noise-Removal by using a Median filter14 removed speckle, salt and paper noise and provided a clear image for the next phase. The last phase of pre-processing called Normalization was applied to resize the input image to 128 × 128 pixels by15 recognizing the Region Of Interest (ROI)9 and mapping it to standard template size. As can be seen in Table 1, there are a variety of ROI sizes.

Fig. 2. Snapshot of Pre-processing Steps a) Grayscale b) Binarization c) Noise Removal d) Dilation e) ROI f ) Normalization

558


Table 1.The ICCs’ ROI Template. Isolated Persian Characters

Calligraphic Character Group

ҫ ӮҬ үҮ ӶҰ Ҳұ Ӂ Ҹҷ

H×W

H/W Ratio

Isolated Persian Characters

50 × 10

5.0

Ҵҳ

0.37

Ҷҵ Ԉ

30 ×80

ӆ

Ӈ

30×30

1.0

H×W

H/W Ratio

40×30

1.33

100×60

1.66

60×40

1.50

50×95

0.52

80×50

1.60

ӄ Ҽһ

Ӆ Թ

ROI

Calligraphic Character Group

Ӄ

ҹ Һ Ҿҽ

ROI

Ӏ ԟԙ

4.3. Feature extraction Feature extraction was used to convert the pre-processing phase results’ segments to a relevant feature vector by using structural, statistical and topological features9, shown in Table 2. In order to determine the number of black pixels along the columns and rows, the Projection profiles method16 based on structural features was conducted in the study. The 256 future vector elements consist of 128 horizontal and 128 vertical projection profiles. The details of the feature extractions are provided in Table 2. Table2. Feature Extraction Features

Methods

Descriptions

Vector Elements

1.Structural

Projection profiles

Giving the number of ON pixels along columns and rows

256

2.Statistical

Hu’s moment invariants

Effective in image representation

7

3.Topological

Height to width ratio

Giving a character general proportion

1

4.4. Recognition Phase The Support Vector Machine (SVM) method, which includes the training and testing sections, was carried out in the recognition phase. The main reason that the SVM method was chosen is because of its high recognition rate in comparison with other classification methods17. The current study training and testing phase block diagram is shown in Fig.3.

559


1) Training

SVM Training

Feature Extraction

Sample Characters

2) Testing

SVM Model

Feature Vector

Feature Vector Testing Character

Feature Extraction

SVM Testing

Result

Fig. 3. SVM Training and Testing Block Diagram

Training phase: All 400 valid possible forms of each of the 18 isolated characters’ image were created in the training database. Afterward, the classification method18 was applied to classify the samples database and generate a final model to be used in the testing phase in order to recognize the inserted ICCs. The data was divided into 50% training and 50% testing data by choosing the images randomly. The Radial Basis Function (RBF) kernel function was used to implement a 5-fold cross validation technique to gain the optimum C and ȟ parameters. Implementing the SVM on a larger sample database resulted in more accurate results17. Testing Phase: The main role of the study’s testing phase was identifying the inserted isolated character. SVM18,19 was applied to identify the character by using the inserted character feature vector. 4.5. Assessment and Feedback phase Providing a suitable practice platform for Persian calligraphy learners is the main aim of the PeNCIL. The last phases of the current study were assessing and displaying appropriate feedback on screen. In this study, the histogram of character contours distributions in polar coordinates was used as a feature descriptor. The contours are well-known shape representations that embody visual information by using a limited set of object points. In addition, the contours are widely used to reduce the number of points dramatically; moreover, the contours preserve the visual information shape20. Features were extracted by calculating the distances and angles of pixels inside a circular layout located at the shape centroid Fig.4.

Fig. 4. Feature Extraction Layout

560


The similarity between the identified template (I1) and the ICC (I2) is expressed by the Histogram Intersection measure: ெିଵ ேିଵ ଵ ǡ ‫ܪ‬௜௝ଶ ሻ ܵ ൌ ෍ ෍ ݉݅݊ሺ‫ܪ‬௜௝

(1)

௜ୀ଴ ௝ୀ଴

Where H1 and H2 are the histograms corresponding to template (I1) and the ICC (I2), and M and N are the histogram dimensions. The corresponding histogram of identified template (H1) is fetched from the template dataset, where all templates’ features had previously been extracted and saved. This descriptor is efficient and global, which makes it conform more to human perception. This result creates feedback for the learners, which is displayed as a progress percentage on screen. Moreover, the correct identified template and inserted ICC image are shown immediately on screen in order to guide users to find out their mistakes. 5. Conclusion The PeNCIL was developed in this study as the first intelligent Persian Nastaliq calligraphy tool. In this study, the two main methods applied to create the intelligent tool were the image processing and SVM machine learning techniques. In addition, the feature matching technique was used to assess the compatibility of the learners’ work with standard Persian Nastaliq calligraphy features. The developed system is capable of identifying and assessing the learners work automatically, which leads to a reduction in human errors that occur in the traditional method and also saves time. By using structural, statistical and topological features, the PeNCIL provides a teacherless standard platform for Persian Nastaliq calligraphy enthusiasts and learners to use and practice wherever and whenever they like.

References 1. Mohsen S. The problem of stretching in persian calligraphy and a new type postscript Nastaliq font. Master Thesis, University of Waterloo, Canada; 2009. 2. Sattar SA, Haque S, Pathan MK, Gee Q. Implementation challenges for Nastaliq character recognition. Communications in Computer and Information Science 2008;20:279-288. 3. Rouhfar A. A new, easy, and effective method to learn Persian calligraphy (Nastaliq). Amazon: CreateSpaceIndependent Publishing Platform; 2014. 4. Hamidi B, Rashvand Z. Visual capacity of Siah-MashqNastaliq calligraphy, in contemporary typography posters of Iran. Int J Arts and Commerce 2015;4(9):124-131. 5. Xu S, Jiang H, Lau FCM, Pan Y. An intelligent system for Chinese calligraphy. In. Proc. 22nd national conference on Artificial Intelligence; 2007. p.1578-1583. 6. Bezine H, Alimi AM. Development of an Arabic handwriting learning educational system. Int J Software Engineering & Applications 2013;4(2):33-49. 7. Taele P, Hammond T. BopoNoto: An intelligent sketch education application for learning Zhuyin phonetic script. In. Proc. 21st International Conference on Distributed Multimedia Systems; 2015. p.101-107. 8. Albu F, Hagiescu D, Puica M, Vladutu L. Intelligent tutor for first grade children’s handwriting application. In. Proc. 9th International Technology, Education and Development Conference; 2015. p.3708-3717. 9. Han C, Chou C, Wu C. An interactive grading and learning system for Chinese calligraphy. Machine Vision and Applications 2008;19:43-55. 10. Patel U. An introduction to the process of optical character recognition. Int J Sci and Research 2013;2(5):155-158. 11. Otsu N. A threshold selection method from gray-level histograms. Syst. Man Cybern 1979;9(1):62-66. 12. Gupta MR, Jacobson NP, Garcia EK. OCR binarization and image pre-processing for searching historical documents. Pattern Recognition 2007;40:389-397. 13. Chen X, Li S, Hu J, Liang Y. A survey on Otsu image segmentation methods. J Computational Information Systems 2014;10:4287-4298.

Fahreddin Sadikoglu and Behzad Soroush / Procedia Computer Science 102 (2016) 555 – 561 14. Srinivasan KS, Ebenezer D. A new fast and efficient decision-based algorithm for removal of high-density impulse noises. IEEE Signal Processing Letters 2007;14(3):189-192. 15. Wang Z, Bovik AC, Sheikh HR, Simoncelli E P. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 2004;3(4):600-612. 16. Gazzah SS, Amara NB. Arabic handwriting texture analysis for writer identification using the DWT-lifting. In. Proc. 9th International Conference on Document Analysis and Recognition,Parana, IEEE; 2007. p.1133-1137. 17. Kazemi M, Yousefnezhad M, Nourian S. A new approach in Persian handwritten letters recognition using error correcting output coding. J Advances in Computer Research 2015;6(4):107-124. 18. Zhang H, Berg AC, Maire M, Malik J. SVM-KNN: Discriminative nearest neighbour classification for visual category recognition. Computer Vision and Pattern Recognition 2006;2:2126-2136. 19. Sani MM, Ishak KA, Samad SA. Classification using adaptive multiscale retinex and support vector machine for face recognition system. J Applied Sciences 2010;10:506-511. 20. Belongie S, Malik J, Puzicha J. Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence 2002;24(4):509-522.

561