Robust Stamps Detection and Classification by. Means of .... Beside these features it is always profitable to use stamp templates (as simplified images) to.
Robust Stamps Detection and Classification by Means of General Shape Analysis Pawel Forczma´ nski and Dariusz Frejlichowski West Pomeranian University of Technology, Szczecin, Faculty of Computer Science ˙ lnierska Str. 49, 71–210 Szczecin, Poland and Information Technology, Zo {pforczmanski, dfrejlichowski}@wi.zut.edu.pl
Abstract. The article presents current challenges in stamp detection problem. It is a crucial topic these days since more and more traditional paper documents are being scanned in order to be archived, sent through the net or just printed. Moreover, an electronic version of paper document stored on a hard drive can be taken as forensic evidence of possible crime. The main purpose of the method presented in the paper is to detect, localize and segment stamps (imprints) from the scanned document. The problem is not trivial since there is no such thing like stamp standard. There are many variations in size, shape, complexity and ink color. It should be remembered that the scanned document may be degraded in quality and the stamp can be placed on relatively complicated background. The algorithm consists of several steps: color segmentation and pixel classification, regular shapes detection, candidates segmentation and verification. The paper includes also the initial results of selected experiments on real documents having different types of stamps.
1
Introduction
From a strictly technical point of view, rubber stamping, also called stamping, is a craft in which certain type of ink made of dye or pigment is applied to an image or pattern that has been carved, molded, laser engraved or vulcanized, onto a sheet of rubber. The ink coated rubberstamp is then pressed onto some type of medium such that the colored image has now been transferred to the medium. The medium is generally some type of fabric or paper. This kind of stamping has not changed for centuries (in fact it is as old as writing itself) and it is supposed that it will not change in the close future. Nowadays, when computer technology is present in various areas of life, the problem of computer crime is becoming more and more important. It covers both strictly electronic and traditional types of law-breakings. On the other hand, there are still many areas of life, where computers and digital media are employed only as tools and play just a supporting role. The most evident example of such domain is an area associated with official documents, identity cards, formal letters, certificates, etc. All these documents are being issued by formal authorities and are often in a form of a paper letter consisting of several typical elements: heading, body text, signatures and stamps which, from this historical point of view confirm
2
its official character. In business environments, they are often used to provide supplemental information (date received/approved, etc). In other words, its main purpose is to authenticate a document which in many cases is a subject to forgery or tampering with help of modern computer means. In general, the process of forgery consists of the following steps: obtaining the original document, high resolution scanning, digital image manipulation and final printing. It is rather easy to recognize fake stamps, even if they are printed using ink-jet printers. This article addresses the problem, which is definitely not new, since the task of seal imprint identification on bank checks, envelopes, and transaction receipts have emerged from mid-1980s. On the other hand reliable recognition of stamps in the documents is not trivial and has not been solved till today [1–3]. The most advanced method found in the scientific literature is described in [2], where the authors present a stamp detection approach, which treats stamps as regions with analytically shaped contours, however this regions are limited to oval shapes only. The general motivation of the research presented in this paper is a need of semi-automatic computer software that is able to analyze an image and detect and localize different types of stamps in it. The application area of this kind of a system is broad, ranging form law-enforcement forces, law offices, official archives and any other institutions that utilize stamp.
2 2.1
Stamp detection and classification Stamps characteristics
All stamps placed on paper documents have specific characteristics which are derived from the process of stamping, These characteristics (shape, complexity, typical patterns) evolved into de-facto standards. The analysis of the problem shows that there are two main groups of stamps having its distinguishable properties: official stamps met mostly on official documents, unofficial stamps used as decoration. The first group (see Fig. 1) consists of regularly-shaped objects (ovals, squares, rectangles) with clearly visible text and mere ornaments. They are often colored red or blue and do not cover large areas. On the other hand, stamps from the second group (see Fig. 2) are more fancy, irregularly-shaped, with decorative fonts and complex patterns. It is a fundamental issue to define the features that can be employed to distinguish stamps from not-stamps and further between official and unofficial stamps. In this paper we focus on official stamps as it plays a meaningful role in practical tasks. The features which are used to describe stamps can be divided into two classes: spatial characteristics [4– 8], including dimensions (proportions of dimensions), edge distributions, mean and variance of gradients, moment representation; color characteristics [9, 10], which include color distribution is HSV and Y Cb Cr color spaces. Beside these features it is always profitable to use stamp templates (as simplified images) to verify the detection and recognition stage. It is worth noticing that we do not employ Hough transform to detect circles, since we deal also with rectangular, triangular and other stamps. Hence, the approach is much more flexible than one presented in [2].
3
Fig. 1. Sample official stamps, often regular and without decorations.
Fig. 2. Sample unofficial stamps, more complex, with many decorative motives.
2.2
Algorithm Overview
The algorithm of stamps processing is divided into several stages: detection, verification and coarse classification, which is depicted in Fig. 3. Detailed descriptions of each stage are presented in the following sections. The stamp detection uses color space transformation and projection method, stamp verification uses simple geometrical characteristics like object size and proportions of its dimensions while classification utilizes a novel approach to general shape analysis.
Document
Detection
/ Candidates
V erif ication
/ Stamps
Classif ication
/ Answer
Fig. 3. Process of stamps retrieval from scanned document.
2.3
Stamp Detection and Verification
An input image of a document has to be stored in a file with possibly lossless compression, high spatial resolution and full color range (24-bit RGB). First it is down-scaled to obtain low resolution representation (256 × 256 pixels) used for preliminary detection. The RGB image is then converted into Y Cb Cr color space (ITU-R BT.709 standard). It is worth noticing that most popular stamps are often blue or red colored, thus we select Cb and Cr components only for further analysis (see Fig. 4 and Fig. 5). In each case presented below (Chinese taxation registration and Czech trade license), several potential areas are detected and passed to the verification/classification stage. For each matrix which represents Cb an Cr channel we perform projections in horizontal and vertical directions in order to find areas of high intensity. Sample projections for one of the test images are presented below, in Fig. 6. As it can be seen, areas occupied by possible stamps are represented by values higher than mean value (assumed to be a background value). Next, the candidates for stamps are segmented and
4
Fig. 4. Sample document containing a stamp (left), Cb and Cr components of its Y Cb Cr representation.
Fig. 5. Sample document containing a stamp (left), Cb and Cr component of its Y Cb Cr representation (right).
passed to the stage where the quasi-geometrical features are calculated. There are two general features used: object size and width to height proportion. The analysis of stamps collected for the experimental purposes, showed that object size should occupy not less than 5% and not more than 15% of the total image area. On the other hand, the proportion of width to height should be not less than 1/3 and not more that 3. This prevents the situation where relatively narrow objects are accepted. The algorithm is capable of detecting more than one stamp in a document as long as they occupy disjoint areas in the image. In case when the centers of two or more stamps are positioned close to each other, they are recognized as one stamp. In case when no stamp is detected (no areas of adequate color can be found), the algorithm ends. Finally, each extracted stamp is passed to the shape analysis stage in order to find its characteristic features.
5 4
4
x 10
3.6
x 10
2.6 3.4 2.4
3.2
2.2
3 2.8
2
2.6 1.8 2.4 1.6 2.2 row projection
1.4
1.2
2
0
50
100
150
200
250
1.8
column projection 0
50
100
150
200
250
Fig. 6. Row projection (left) and column projection (right) and mean values (dashed line) of Cr component of the document image shown in Fig. 4.
2.4
Coarse Classification by Means of the General Shape Analysis
The general shape analysis (GSA) is a problem similar to shape classification or retrieval. There are two important differences. First of all, the number of model classes in the GSA is very small. This limited database covers only the most general objects — triangle, square, ellipse, etc. Secondly, the process of identification is different than in the classical recognition. In GSA the explored object does not have to belong to one of the base classes. In fact usually it does not. Thanks to this and the specifically selected instances in the database we can indicate the general information about a shape, e.g. how triangular, square, elliptical, etc. it is. The general shape analysis approach can be applied in all problems, where the initial coarse classification is desirable. This can be useful for example in retrieval of shapes, where firstly we are trying to identify a general class, and later work within this class only. Moreover, this process can be reiterated and at the consecutive stages the general shapes used may be more precise. However, not only retrieval is an example of the possible usage of GSA approach. It can be helpful also in problems, where we work using less precise description for processed objects. An example in this group is the usage of imprecise voice commands (e.g. ’find round blue objects’) in any system of human-computer interaction. The mentioned properties of GSA allow us to use this approach in the second stage of the developed algorithm. Here we are trying to identify the general shape of an extracted stamp. The result of this step can help in indication of the character of a processed stamp. Basing on information about typical kinds of stamps we can pre-classify a particular one in order to decide if it has to be recognized more precisely. For example the much attention has to be put on round objects than for triangular ones, because usually the governmental, official stamps are round. Various shape description methods can be used in the GSA approach. However, depending on the particular application their different properties can be desirable. Usually, the property of generalization is useful, yet in the problem described in the paper, the ability of distinguishing
6
the round, triangular and rectangular objects is especially helpful. That is why, amongst several algorithms explored in the problem of GSA (e.g. [11]), the most convenient ones are the polar methods. Amongst them the best results in the GSA problem was so far achieved by Point Distance Histogram (PDH). This conclusion was confirmed thanks to the comparison with human benchmark, obtained after analysis of inquiry forms, filled in by almost two hundred persons of different age and gender [11]. The PDH combines the advantages of histogram with the transformation of contour points into polar coordinates. Firstly the mentioned coordinates are derived (with O = (Ox , Oy ) as the origin of the transform) and put into two vectors Θi for angles and P i for radii [11]: q yi − Oy 2 2 . (1) ρi = (xi − Ox ) + (yi − Oy ) , θi = atan x i − Ox The resultant values are converted into nearest integers [11]: ⌊θi ⌋ , if θi − ⌊θi ⌋ < 0.5 θi = . ⌈θi ⌉ , if θi − ⌊θi ⌋ ≥ 0.5
(2)
The next step is the rearrangement of the elements in Θi and P i according to increasing values in Θi . This way we achieve the vectors Θj , P j . For equal elements in Θj only the one with the highest corresponding value P j is selected. That gives a vector with at most 360 elements, one for each integer angle. For further work only the vector of radii is taken — P k , where k = 1, 2, ..., m and m is the number of elements in P k (m ≤ 360). Now, the normalization of elements in vector P k is performed [11]: M = max {ρk } , k
ρk =
ρk , M
The elements in P k are assigned to r bins in histogram (ρk to lk ,[11]): r, if ρk = 1 . lk = ⌊rρk ⌋ , if ρk = 6 1
(3)
(4)
In case of the above algorithm the L2 norm can be used in order to derive the dissimilarity measure between two received shape descriptions h1 and h2 [12]: sX ((h1 (i) − h2 (i))2 . (5) L2 (h1 , h2 ) = i
The parameter r, determining the number of bins in the histogram representing a shape, was experimentally established as 25. There were only five models in the database, including the most popular stamp shapes, i.e. circle, ellipse, rectangle, triangle and hexagon. Forty stamps of various size and rotation, extracted at the previous stages, were tested. The results are depicted in Fig. 7. As usually in the case of general shape analysis [11] three base elements that are closest to the processed one are provided. As we can clearly see the PDH algorithm
7
Fig. 7. The three general shapes the most similar to explored stamps, selected using PDH algorithm. The values of dissimilarity measure are also provided.
gives almost perfect answer. Only in two cases the results are wrong. The test object no. 6 is very small and indistinct and that could influence the results. In the second case only the first indication is incorrect. For test object no. 11 firstly the ellipse was pointed out. The proper answer — triangle — appeared in a second place. However, the difference between the similarity measures for the two first indicated general shapes were significantly small this time. During our experiments we discovered that the results of classification using PDH is actually not influenced by inaccurate segmentation, which shows as irregularities in shape contour. However, in very rare cases it might be corrected by blurring of input image after color separation.
3
Summary
In the paper we presented an approach which leads to the detection and coarse classification of stamps in the scanned documents. This problem has not been broadly discussed and analyzed so far. In fact, only one method, according to our best knowledge, can be found in the scientific literature [2]. Our algorithm is more complex and is composed of several steps, which include color conversion and separation, vertical and horizontal projection of pixel intensities and shape analysis and is more flexible, since detects not only oval stamps. The last stage of our approach was supported by the general shape analysis performed using Point Distance Histogram. That was helpful in general classification of stamps extracted from scanned documents. The selection of the method for this stage was intentional. First of all, we were looking for a method that will work better in discrimination between round, rectangular and triangular shapes versus another ones. It is important in our main problem of determining the potential criminal activity. In order to achieve this goal, the polar shape descriptors were suggested.
8
They are by definition robust to many problems appearing when working with shapes. Basing on the experiments described in [11] PDH was pointed as the most effective algorithm within this groups in the problem of coarse classification. Its performance in this task was significantly better than in case of other polar shape descriptors. That was plainly confirmed by results presented and discussed in this paper. Our experiments conducted on 170 different scanned color documents shown that in more than 82% cases the stamp was successfully segmented and then in 93% successfully identified by means of the general shape analysis. Most of the errors were caused by poor quality of scanned documents as well as by a color different from the fixed standard. It might be easily improved by using more detailed color separation involving custom color models involving GramSchmidt orthonormalization procedure. The future research will be also focused on much more detailed stamp recognition, namely detection and recognition of texts present on stamps, which can have a very wide area of application.
References 1. K. Ueda and Y. Nakamura. Automatic verification of seal impression patterns. In Proc. 7th. Int. Conf. on Pattern Recognition, pages 1019–1021, 1984. 2. G. Zhu, S. Jaeger, and D. Doermann. A robust stamp detection framework on degraded documents. In Proceedings - SPIE The International Society For Optical Engineering, volume 6067, january 2006. 3. G. Zhu and D. Doermann. Automatic document logo detection. In The 9th International Conference on Document Analysis and Recognition (ICDAR 2007), pages 864–868, 2007. 4. T. D. Pham. Unconstrained logo detection in document images. Pattern Recognition, 36:3023–3025, december 2003. 5. D. Zhang and G. Lu. Review of shape representation and description techniques. Pattern Recognition, 37:1–19, 2004. 6. S. Loncaric. A survey on shape analysis techniques. Pattern Recognition, 31:983– 1001, 1998. 7. M. Mehtre, B., M. S. Kankanhalli, and W. F. Lee. Shape measures for content based image retrieval: a comparison. Information Proc. & Management, 33:319– 337, 1997. 8. J. Wood. Invariant pattern recognition: a review. Pattern Recognition, 29:1–17, 1996. 9. Y. Deng, B. S. Manjunath, C. Kenney, M. S. Moore, and H. Shin. An efficient color representation for image retrieval. IEEE Transactions on Image Processing, 10(1):140–147, 2001. 10. B. S. Manjunath, J.-R. Ohm, V. V. Vasudevan, and A. Yamada. Color and texture descriptors. IEEE Transactions on Circuits and Systems for Video Technology, 11:703–715, 2001. 11. D. Frejlichowski. An experimental comparison of seven shape descriptors in the general shape analysis problem. ICIAR 2010, Part I, Lecture Notes in Computer Science, 6111:294–305, June 21–23 2010. 12. M. Miklasz, P. Aleksiun, T. Rytwinski, and P. Sinkiewicz. Image recognition using the histogram analyser. Multimedia and Intelligent Techniques, 1:74–86, 2005.
