Published in the Proceedings of the Biennial Conference of North American Information Proecessing Society (NAFIPS’96), Berkeley, IEEE, pp. 147-151, 1996.
Multi-Script Handwriting Recognition with FOHDEL* Ashutosh Malaviya, Christoph Leja and Liliane Peters German National Research Center for Information Technology (GMD) Schloß Birlinghoven, 53754 St. Augustin, Germany
[email protected] http://set.gmd.de/EIA/fohres.html
Abstract Until now handwritten character recognition systems used script specific methodologies. In this paper we present a unified syntactic approach for multi-script recognition. The fuzzy pattern description language -FOHDEL- is used to store fuzzy features in the form of fuzzy rules. First we briefly describe the proposed recognition methodology for Latin, Devanagari and Kanji scripts by analyzing their characteristic properties. Further we present the main features of FOHDEL by a comparative rule generation of three scripts under experiment. Finally the system integration of the proposed multi-script recognition scheme in the existing Latin recognition system is presented.
1
Introduction
The world is full of distinct and unique sets of handwritten symbols. In today’s business world mostly used script is the Latin. But with the increasing communication between the different world communities more scripts are getting integrated into the information technological systems. This also include scripts like Kanji, the major script of Chinese speaking countries and Devanagari, the major script of the indian subcontinent. In recent years various syntactic handwriting recognition methods have been developed for both on-line and off-line
* Fuzzy On-line Handwriting Description Language
applications for all three scripts (Latin, Kanji and Devanagari)[1][2]. Until now their success has been limited due to the large number of rules needed to cover the different handwriting styles. These difficulties partially originate from the crispness of the definition of the patterns, which in turn hinders the description of complexity and style variations of handwritten characters. Some research areas, e.g. on-line handwriting recognition of Devanagari script, were neglected as there was no potential market for the development of such products. But with the development of telecommunication and chip miniaturization, a large market for online handwriting application is opening in the asian continent also. Another aspect is the development of a unique recognition scheme, that enables a multi-script classification. In this paper we present a possible multi-script online recognition scheme based on our newly developed fuzzy language FOHDEL [5]. The paper is structured as follows. First we present the main feature of the recognition scheme, followed by a brief description of the feature characteristics of the different scripts. Section 3 deals with the basics of the fuzzy language FOHDEL exemplified by a comparative rule generation. Some concluding remarks related to the integration of the recognition scheme in FOHRES system are presented in the last section.
Published in the Proceedings of the Biennial Conference of North American Information Proecessing Society (NAFIPS’96), Berkeley, IEEE, pp. 147-151, 1996.
Level 4
Rule generation
>VH#VL_L & >VH#O_BR
HL_T
VL_L Level 3
HL_T
Aggregation O_BR
Level 2
Feature Extraction
Left Vertical line Bottom Right O-Like Segment S1
Level 1
Segmentation
HKL_R
VL_ML
C_ML
VL_C
VL_R MidLeft C-like
Positive Slant at Top
Top Horizontal line Right Vertical line
MidLeft Vertical line
Segment S1
Segment S3
Leftward Hook to Right Vertical line at Center Segment S1
Segment S2 Segment S2
Level 0
>H#HL_T & VH#VL_MR & VH#HKL_R & >M#VL_C
VH#HL_T & >H#C_ML & VH#VL_R
Segment S3 Segment S2
Segment S4
Acquisition
multi-level structure
Latin[be]
Devanagari[wa]
Kanji[bi]
Figure 1 Rule generation of multiple scripts.
2
The Recognition Scheme for Multiple Scripts
We will mainly show that FOHDEL is powerful enough to facilitate the handwriting recognition of various scripts such as Latin, Devanagari, and Kanji. The characteristic features of the proposed scheme [7] include the utilization of the fuzzy linguistic descriptors to describe the character features, the on-line adaptability to cope with the handwriting variability and finally the multi-level modular structure for extensions. The fuzzy linguistic descriptors have the property to cover a wide range of possibilities with a single fuzzy linguistic term enhanced by attributes, e.g. the character ‘b’ comprises of “very straight (attribute) vertical line (fuzzy geometric term)” in the “beginning (fuzzy positional term)” followed by another fuzzy linguistic complex term “almost (attribute)circular curve (fuzzy geometric term)” in the “end (fuzzy positional term)”. On-line adaptability of the method refers to the ability to incorporate new handwriting features of a writer during the recognition process itself. No matter how good the training phase is, there are always some unexpected features to which the given rules don’t fit. To overcome this problem, there must be an automatic training method which changes or extends, the
knowledge base [8]. The developed multi-level fuzzy rule based pattern recognition system [4] enables us to change parameters of the various levels of recognition (Fig. 1) and extend the system with new modules. After the acquisition of a symbol at Level 0, Level 1 represents the segmentation step, which follows the higher recognition levels of feature extraction, aggregation, rule generation and classification. While at Level 2 simple geometrical features are extracted, at the Level 3 the aggregation of various features gives a very compact but meaningful linked list of features which can be converted into a syntactic rule at Level 4. The next level (not shown in Fig. 1) is the context independent classification module. This hierarchy can be extended by adding additional levels of contextdependent word classifiers. Every script has a unique structure. This structure is normally evolved from the phonetic and linguistic characteristics. The distinct nature of a script requires a modified feature set, a suitable segmentation criterion and a separate rule-base for classifying various symbol sets. The initial development of the proposed methodology has shown that although these scripts are so different they have some common properties. These enable us to use the same classification approach of initially devel-
oped approach for the Latin script. In earlier works we have presented [4] the possibility of describing latin characters in the form of linguistic features. The extracted features (global, geometric, positional) were aggregated in a further step [6] and formed a fuzzy rule-base which enabled a syntactic classification of the symbols. The most of the FOHDEL elements used in the recognition of the Latin script are used in the proposed method, these include the aggregated features, fuzzy operators and the linguistic terms. About 5000 of more than 50000 characters of the Kanji character set are commonly used in China [1]. In our study we took a small subset of Kanji characters to test the proposed methodology. In the selection of characters we took mainly into consideration the most frequent Kanji characters. The basic features of Kanji script are simple straight lines (strokes) or various combinations of these. While for the Latin script the first level (the segmentation) was done mainly on abrupt curve changes rather than on pen-ups, for Kanji script the identification of pen-ups build the major segmentation parameter. Most of the selected Kanji symbols can be described by the available geometric (strokes) and positional features (position of the strokes) which are then combined with some global features (aspect-ratio, number of strokes, etc.) characterizing the handwritten symbol as a whole. The basic alphabet of Devanagari consists of 13 vowels, 34 consonants and 4 diacritical remarks. The ‘Matras‘ (Vowels) of the Devanagari script are protruded upwards and downwards. The complexity of Devanagari characters and off-line recognition strategy is discussed by Sinha in [2]. Similar to the Latin script the geometric features of Devanagari script are combinations of basic curves and straight lines (Fig. 1). Their relative position, e.g. diacritic signs, give additional information. The segmentation criterion in Devanagari is a combination of pen-ups and abrupt curve changes. The Devanagari test set contained the 192 mostly used combinations of Devanagari consonants, vowels and other diacritical remarks. Based on the script specific features we have developed FOHDEL rule-base for each script. It consists of around 50 shape features which are aggregated
with the positional and global features (see example described in the next section).
3
Fuzzy Handwriting Description Language (FOHDEL)
FOHDEL language was initially designed to describe Latin handwritten characters in a fuzzy linguistic manner. It incorporates the fuzzy logic techniques to describe the syntactic relations of the semantic features extracted from a symbol pattern. The basic idea behind FOHDEL is to develop a simple language with the meaningful features which facilitates interactive rule-base generation. Such a rule-base should cover a wide range of handwriting variabilities with a minimum number of rules. A compact language which is capable of representing complicated patterns with a simple syntactic structure and a reduced set of complex features can implicitly show a good amount of data compression. This can take place in a two-fold manner, primarily by enlarging the flexibility of feature description and thus reducing the number of required features and secondly by creating efficient prototypes which cover a large area of various descriptive deviation and thus minimizing the number of necessary rules. In addition the effect of altering the rules by changing the connective operators can be easily observed. Last but not least through the generative power of such a language its extension to other pattern recognition application can be easily accomplished just by changing the feature space or enlarging the rule-base [6]. Once the main features of the new language are defined, the next step is the definition of the language structure. A language is a collection of sentences of finite length which is constructed from a finite alphabet or in case of syntax description it is limited to a finite vocabulary. A grammar can be regarded as a device that enumerates the sentences of a language. Formal methods of syntax and semantics are often employed to describe symbols. Methods for describing characters in a linguistic form have been presented by various researchers over the last 30 years. The existing linguistic techniques in pattern recognition are based on the structure of underlying relationships between features in a two dimensional pattern.
Published in the Proceedings of the Biennial Conference of North American Information Proecessing Society (NAFIPS’96), Berkeley, IEEE, pp. 147-151, 1996.
Diagnosis
Script analyzer
FOHRES
RECOGNIZED TEXT
Multi-script Dictionary
Figure 2 Multi-script system.
If such a structure is identified then a complex pattern can be described in terms of basic primitives and sub-patterns. But the precision required by formal languages in pattern recognition contradicts with the imprecision or ambiguity of real life patterns. To overcome this mismatch between the imprecise nature of the input and the precise description of the syntax fuzziness was introduced into the structure of formal language. This led to the development of stochastic and fuzzy languages. Based on these facts we have developed a new fuzzy language named “FOHDEL- Fuzzy On-line Handwriting DEscription Language”. What is new about this language? Primarily it incorporates the uncertainty factor at all levels from feature description over rule generation to the classification level. It is designed with the help of fuzzy grammar and supports mainly the description of handwritten symbols [5]. Through its compact form the number of prototypes needed for classification is small (until now five), and the chosen attributes facilitate a hierarchical classification of the handwritten symbols. Due to the chosen operators FOHDEL sentences can be used as input to a grammatical inference engine. The syntax of FOHDEL [5] includes basic primitives, e.g. “vertical straight line” - VSL, fuzzy linguistic terms, e.g. “medium” - M, fuzzy linguistic modifiers, e.g. “more than” - >, and operators, e.g. “or” - |. The set of fuzzy primitives
consists of geometrical, positional and global fuzzy features. FOHDEL production rules are a list of primitives related to attributes through operators. Each rule has a possibility value reflecting the likelihood of the rule to fit a symbol. To demonstrate the rule generation of various scripts we have chosen the Latin symbol for phonetic “[be]”, the Kanji symbol for “[bi]” and the Devanagari symbol for “[wa]” (Fig. 1). After the first process level (segmentation) we have identified the segments with the corresponding criteria (see Sec. 2). For the Latin script the abrupt change in curved-ness, for Kanji the pen-ups and for Devanagari also in this case the pen-ups. In the feature extraction phase (Level 2) we identify for this example some geometric and positional features. While for Devanagari and Latin script rule generation we can apply the same feature classes, for Kanji we introduced some additional geometric features such as “leftward hook” - HKL. The aggregation of the geometric and positional features is accomplished to generate linguistic clauses e.g. “olike curve at bottom-right”. In the last step of hierarchical recognition system FOHDEL rules are generated (Fig. 3). This involves statistical analysis of the acquired data and the most suitable features, linguistic terms and operators are selected to create robust rule-base [8].
Rule L[be]: >VH#VL_L & >VH#O_BR “More than very high vertical line to the left and a more than very high o-like curve at bottom-right “ Rule D[wa]:VH#HL_T & >H#C_ML & VH#VL_R “A very high horizontal line at the top and more than high c-like curve at medium-left and a very high vertical line to the right” Rule K[bi]:>H#HL_T & VH#VL_MR & VH#HKR_R & >M#VL_M “More than high horizontal line at top and very high vertical line at middle-right and a very high Leftward hook at the right and a more than medium vertical line at the middle”
Figure 3 FOHDEL rules for Latin [be], Devanagari [wa] and Kanji [bi].
4
System integration
The handwriting recognition of multiple scripts including Devanagari, Kanji and Latin was conceptually described in the last sections. The integration of this multi-script scheme is accomplished on FOHRES system as shown in Fig. 2. The handwriting recognition system FOHRES consist of modules of various recognition levels. The script specific changes were accomplished in some of the modules. The multi-script dictionary contains the different rule-bases generated in FOHDEL. The rule base is generated with the statistical analysis of the collected handwriting data of multiple scripts in UNIPEN format. The developed recognition system is ported to various platforms like workstations, PCs and pen-tops. An additional feature of the system is the availability of UNIPEN [9] interface, which facilitates comparative benchmark studies. The obtained classification results were satisfactory and were generally above 90%. The presented approach was restricted to the isolated character recognition. Our future work will focus on the recognition of cursive words of multiple scripts. Moreover the rule-base will be enhanced to recognize a larger set of Kanji character symbols.
Acknowledgments We sincerely thank Dr. Shuwei Guo for his help in analyzing Kanji script and providing us the necessary data.
5
References
[1]
T.H. Hildebrandt and W. Liu, “Optical Recognition of Handwritten Chinese Characters: Advances since 1980,” Pattern Recognition, Vol.26, No.2, pp.205-225, 1993.
[2]
R.M.K. Sinha,“Rule based contextual PostProcessing For Devanagari Text Recognition,” Pattern Recognition, Vol.20, pp. 475-485, 1987.
[3]
L.A. Zadeh, “A Fuzzy-algorithmic approach to the definition of complex and imprecise concepts,” Intl. J. on Man-Machine studies, Vol. 8, pp. 249-291, 1976.
[4]
A. Malaviya and L. Peters, “Handwriting recognition with fuzzy linguistic rules,” 3rd EUFIT, Aachen, 1995.
[5]
A. Malaviya et al, “FOHDEL - a fuzzy handwriting description language,” FUZZIEEE, June 1994.
[6]
A. Malaviya and L. Peters, “Extracting meaningful handwriting features with fuzzy aggregation method,”3rd ICDAR, Montreal Canada, 1995.
[7]
A. Malaviya and L. Peters, “On designing a multilayered pattern recognition system”, submitted to Fuzzy Sets and Systems.
[8]
A. Malaviya et al., “Automatic generation of fuzzy rule base for on-line handwriting recognition,” EUFIT’94, Aachen, 1994.
[9]
I. Guyon et al., “UNIPEN project of on-line data exchange and recognition benchmarks,” 13th IEEE-ICPR, pp. 29-33, Israel, 1994.
