Document forgery detection using distortion mutation

Document forgery detection using distortion mutation of geometric parameters in characters Shize Shang Xiangwei Kong Xingang You

Downloaded From: http://electronicimaging.spiedigitallibrary.org/ on 08/23/2015 Terms of Use: http://spiedigitallibrary.org/ss/TermsOfUse.aspx

Journal of Electronic Imaging 24(2), 023008 (Mar∕Apr 2015)

Document forgery detection using distortion mutation of geometric parameters in characters Shize Shang,a Xiangwei Kong,a,* and Xingang Youa,b

a Dalian University of Technology, School of Information and Communication Engineering, No. 2 Linggong Road, Ganjingzi District, Dalian 116024, China b Beijing Institute of Electronic Technology and Application, No. 15 Xinjiangongmen Road, Haidian District, Beijing 100091, China

Abstract. Tampering related to document forgeries is often accomplished by copy-pasting or add-printing. These tampering methods introduce character distortion mutation in documents. We present a method of exposing document forgeries using distortion mutation of geometric parameters. We estimate distortion parameters, which consist of translation and rotation distortions, through image matching for each character. Detection of tampered characters with distortion mutation occurs based on a distortion probability, which is calculated from character distortion parameters. The introduction of a visualized probability map describes the degree of distortion mutation for a full page. The proposed method exposes the forgeries based on individual characters and applies to English and Chinese document examinations. Experimental results demonstrate the effectiveness of our method on low JPEG compression quality and low resolution. © 2015 SPIE and IS&T [DOI: 10.1117/1.JEI.24.2 .023008] Keywords: document forgery detection; distortion mutation of geometric parameters; image matching; translation distortion; rotation distortion. Paper 14587 received Oct. 4, 2014; accepted for publication Jan. 27, 2015; published online Mar. 10, 2015.

1 Introduction Government departments, companies, and universities commonly use printers and photocopiers. A simple, low-cost operation enables counterfeit document creation. A definitive technological need exists for document source verification and validation of authenticity. Well-trained professionals frequently examine documents using specific devices such as high-resolution microscopes, spectrometers, and costly composition analyzers. However, various chemical methods can damage documents during examination. It is necessary to develop lossless, convenient, low-cost forensic methods to provide reliable evidence to judge whether suspicious or questionable documents are actually forged. With pattern recognition advances in recent years, automatic and semiautomatic document examinations are a reality. Forensic analysis utilizing a computer and flatbed scanner is a viable option. This method consists of printer identification,1–8 scanner identification,9–12 and document forgery detection.13–23 During document examination, the first step is to determine the printer source, including the make and model of the printer. The next step is to detect whether document tampering occurred and the location of the tampered regions. Occasionally, only the scanned image is supplied, and not the paper document. In this case, identification of the source scanner should take place. A document is usually tampered with by copy-pasting or add-printing, where copy-pasting means copying one part of a document to another and making a reproduction, and add-printing means printing some contents on the whitespace of the printed document. In document forgery detection, some methods focus on device type identification.19–23 Such devices might have one laser printer, inkjet printer, or

photocopier. In practice, few forgery judicial cases relate to distinct device types; the identification of character morphology in different devices is simpler. Other methods work on statistical similarity detection when examining documents forged via copy-pasting or add-printing.17,18 These methods examine the similarity of related characters; however, they are inappropriate for Chinese because Chinese has a larger set of characters, thus it is inconvenient to examine each character individually; in addition, some characters have low repetition rates in a document. Copy-pasting and add-printing introduce distortion mutations of geometric parameters (DMGP) to documents. Some methods examine the tampered characters based on text-line alignment;13,14 these methods are suitable for the examination of tampered lines in the document, but not for individual characters. As such, it is necessary to develop a new forgery detection method based on individual characters and one that is compatible with Chinese and English. In this paper, we propose a method with which to expose tampered characters based on DMGP. In a printed document, the geometric translation parameters change gradually on the page. Only tampered characters have DMGP when compared with the original characters. Without using text-line attributes to align the characters, we apply an image matching method to calculate the distortion parameters. Optical character recognition (OCR) rebuilds the reference document image, and the image matching is between the scanned and reference character images. We calculate a distortion probability value based on character distortion parameters to identify the tampered characters. A visualized probability map (P-map) displays the degree of distortion mutation on the page in question. We introduce receiver operating characteristic (ROC) curves between true positive rates and false

*Address all correspondence to: Xiangwei Kong, E-mail: [email protected]

1017-9909/2015/$25.00 © 2015 SPIE and IS&T

Journal of Electronic Imaging

023008-1

Mar∕Apr 2015


•

Vol. 24(2)

Shang, Kong, and You: Document forgery detection using distortion mutation of geometric parameters in characters

positive rates to illustrate the performance on detecting tampered characters and lines. This proposed method examines forged documents based on individual characters and applies to Chinese and English. The experimental results demonstrate the effectiveness of the method on practical documents written in both Chinese and English. The extraction of geometric distortion parameters does not depend on statistical characteristics, making it ideal for JPEG compression and lower resolutions (300 dpi). To the best of our knowledge, this is the first work to expose tampered characters based on DMGP in Chinese and English documents. The organization of the paper is as follows: Sec. 2 introduces the related works in document forgery detection. Section 3 contrasts the geometric distortion between authentic and forged documents with regard to the feasibility of the proposed method. Section 4 describes the proposed method with parameter estimation for translation and rotation distortions (including P-map acquisition from distortion parameters). Section 5 presents the performance comparison with other published works, experimental results on different scanning resolutions, and JPEG compression qualities. Section 6 presents the conclusion and future work. 2 Related Work Presentation of document forgery detection techniques is summarized in three categories: detection of geometric distortion in characters,13–16 statistical similarity in characters,17,18 and source of device types.19–23 A summary of their works is as follows. 2.1 Geometric Distortion in Characters Beusekom et al.13,14 presented a technique to detect the document forgery utilizing copy-pasting or add-printing. This technique extracts text-line attributes from each English word and calculates the translation and rotation distortion parameters based on the text-line alignment. This technique can examine tampered lines, but not for individual characters. In addition, the authors proposed another technique15 to detect counterfeit documents based on full-page image matching. Laser printed documents have a distinct geometric distortion related to hardware defects.8 A counterfeit document is identified by matching it against the authentic document. Ahmed and Shafait16 applied image matching to identify tampering using several documents with similar content. Automatic matching and identification of the static portions related to each document assist in identification of the variable portions of the document that exhibit distortion related to tampering. 2.2 Statistical Similarity in Characters Both tampered and original characters in a given document may come from distinct printers. Statistical similarity is applied to detect character tampering related to different printers. Bertrand et al.17 applied a Hu invariant moment similarity to detect a copy-paste operation. Kee and Farid.18 reconstructed the printer models for each character and compared the estimation error value to expose tampered characters. They reconstructed each character using multiple printer models, calculating the error tolerance between the reconstructed character and the original. The model exhibiting the smallest error was determined as the source printer. Journal of Electronic Imaging

This technique has a good performance on printers of different makes and models, but is sensitive to toner density. 2.3 Source of Device Types Some documents are tampered by different device types such as laser printers, inkjet printers, and photocopiers. Device type identification can be used to expose tampered characters. Lampert and Breuel19 extracted a set of 15 features from each character. These features were edge roughness, correlation coefficient, texture, and so on. These features are distinct between laser and inkjet printed characters. support vector machine is used to classify the two device types. Gebhardt et al.20 also used edge roughness to distinguish printing types. Only characters having a vertical edge were chosen for feature extraction. The Grubbs and K-nearest neighbor methods were used to perform unsupervised anomaly detection. Umadevi et al.21 divided the text word image into three regions by expectation maximization including foreground text, noise, and background. A parameter called print index was generated using an iterative method for distinguishing printing techniques. Schulze et al.22 described a frequency-domain method to distinguish laser printing, inkjet printing, and photocopying. They divided a full-page document image into several blocks, featuring the mean and standard deviation of discrete cosine transform coefficients extracted from these blocks. Shang et al.23 proposed a new method to distinguish laser printing, inkjet printing, and photocopying techniques. Features extracted from individual characters include noise energy, contour roughness, and average character-edge gradient. 3 Analysis of Geometric Distortion in a Document All printed documents have geometric distortion, but only the tampered characters have DMGP when compared with the original characters. The geometric distortion is a result of hardware defects in the printer such as the spinning velocity fluctuations of the polygon mirrors and imperfections of the paper feeding mechanism.6,8 Characters in the document have translation distortion on both horizontal and vertical directions in all laser printers. The translation distortion changes gradually, with small differences noted among adjacent characters. As some characters are tampered by copy-pasting or add-printing, these tampered characters have DMGP. Their distortion characteristics exhibit differences when compared against the original characters. Tampered characters are located by comparing DMGP with adjacent characters. In this section, we describe how to generate the reference document image and segment the character images, how to align the scanned document image against its reference document image, and the geometric distortion differences between forged documents and authentic documents. 3.1 Image Preprocessing and Document Image Alignment Before document image alignment, a scanned document image should undergo some preprocessing operations. First, as the text lines may have tilt caused from printing and scanning, the scanned document needs tilt correction based on Hough transform.24 Then, we generate the reference document from the scanned document image based on OCR8 and save the reference document to the TIFF image, which has the same resolution as scanned document

023008-2

Mar∕Apr 2015


•

Vol. 24(2)


Fig. 1 Alignment between the scanned and reference document images.

image, and define it as reference document image. Finally, the characters are segmented from the document image25 and the coordinates of the characters are recorded. Document image alignment is applied to align the scanned and reference document images. A benchmark character in each document image is chosen to align the two document images, and the size of the scanned document image should be adjusted based on the coordinate offsets between benchmark characters. Suppose the sizes of the scanned and reference document images are ðMs ; N s Þ and ðM r ; N r Þ, respectively. The coordinates of the character central point in scanned document image are ½xs ði; jÞ; ys ði; jÞ, and ½xr ði; jÞ; yr ði; jÞ is the coordinates of the character central point for reference document image, where ði; jÞ means the position of the character is in the i’th row and the j’th column. The coordinate offsets for each character are calculated as dx ði; jÞ ¼ xr ði; jÞ − xs ði; jÞ : (1) dy ði; jÞ ¼ yr ði; jÞ − ys ði; jÞ

We set the first character on the upper left corner of the document as the benchmark character and make sure the benchmark character has no geometric distortion, so ds ð1;1Þ ¼ 0 and dr ð1;1Þ ¼ 0. Based on the coordinate offsets, we adjust the size of the scanned document image to ðMs0 ; N s0 Þ by increasing or decreasing some blank rows or columns, and the image size adjustment should satisfy the following conditions: 8 0 M ¼ Mr > > < 0s Ns ¼ Nr : (2) d ð1;1Þ ¼ 0 > > : x dy ð1;1Þ ¼ 0 Based on the size adjustment in Eq. (2), the image alignment between the scanned and reference document images is shown in Fig. 1. The blue characters are on the reference document image and the green characters are on the scanned document image. As we set the first character on the upper left corner of the document as the benchmark character, the two benchmark characters completely overlap, and other characters have different levels of geometric distortion. The distortion parameters, which consist of two translation parameters and one rotation parameter, can be calculated by the proposed image matching method. 3.2 Geometric Distortion Between Authentic and Forged Documents When the scanned and reference document images are aligned, we calculate the coordinate offset ðdx ; dy Þ again for each character and get the translation distortion map for the whole document as

2

ð0;0Þ ðdx ð1;2Þ; dy ð1;2ÞÞ 6 ðdx ð2;1Þ; dy ð2;1ÞÞ ðdx ð2;2Þ; dy ð2;2ÞÞ 6 .. .. Distortion map ¼ 6 6 . . 4 .. . ···

··· ··· ðdx ði; jÞ; dy ði; jÞÞ ···

Each element in the translation distortion map is a twodimensional vector, which consists of horizontal and vertical distortion parameters. Figure 2 illustrates the translation distortion map for a Chinese document printed by a Canon LBP3500. Figure 2(a) illustrates the distortion map for an authentic document, while Fig. 2(b) demonstrates a tampered document containing two tampered lines in the 5th and the 20th rows. In both figures, the horizontal and vertical coordinates denote the row and column positions of the characters in the document. The blue vectors indicate the translation distortion for each character and the start and end points of the blue vector represent the character central point in the reference and scanned character images, respectively. All the translation distortion vectors of the characters in a document will constitute a vector map as shown in Figs. 2(a) and 2(b). The vectors with a large magnitude mean that these characters have a large translation distortion and the blue dots mean the distortion is zero. As the translation vectors are tiny and it is difficult to intuitively render the distribution of translation distortion, we apply color isograms to express the distribution of the Journal of Electronic Imaging

3 ··· ··· 7 7 7: ··· 7 5 .. .

(3)

distortion magnitude. We illustrate the color bars on the right of each figure to show the correspondence between the color and distortion magnitude. Utilizing the first character on the upper left corner of the document as the benchmark character, the distortion magnitude of the benchmark character is zero, and the distortion magnitude of other characters increases as the distance increases from the benchmark point. The color bars in Fig. 2 show that the translation distortion fluctuates from 0 to 30 pixels (600 dpi) and the distribution of the isograms in Fig. 2(a) is caused by hardware defects in the laser printer.6,8 The translation distortion of the two tampered lines in Fig. 2(b) exhibits a significant disturbance on the distortion map. The color isograms in Fig. 2 demonstrate the gradual translation distortion of characters in an authentic document, whereas the translation distortion of tampered characters exhibits mutations. It is feasible to determine character tampering utilizing DMGP. As the translation distortion magnitude has a large range in a full-page document, it is difficult to directly locate tampered characters using a threshold. Each character in the document has at most eight adjoining

023008-3

Mar∕Apr 2015


•

Vol. 24(2)

Shang, Kong, and You: Document forgery detection using distortion mutation of geometric parameters in characters 30

30

5

5 25

10

25

10 15

15 20 15

25 30

20

Rows

Rows

20

20 15

25 30

10

10

35

35 5

40 5

10

15

20

25

5

40

30

5

10

Columns (a)

15

20

25

30

Columns (b)

Fig. 2 The translation distortion map: (a) the translation distortion map in an authentic document, and (b) the translation distortion map in a forged document.

5 5

5.5

5

4.5

5

10

4

10

15

3.5

15

4

20

3

20

3.5

25

2.5 2

30

Rows

Rows

4.5

3

25

2.5 30

2

1.5 35

35

1.5

1

1

40

40 0.5 5

10

15

20

25

0.5 5

30

10

15

20

Columns

Columns

(a)

(b)

25

30

Fig. 3 The magnitude differential map of translation distortion: (a) the magnitude differential map in an authentic document, and (b) the magnitude differential map in a forged document.

characters. We calculate the magnitude differential by averaging the differential with respect to adjacent characters. Figure 3 illustrates the magnitude differential of translation distortion contrasting authentic and forged documents, with a differential value of translation distortion of less than 6 pixels. As shown in Fig. 3(b), the magnitude differential of the tampered lines exhibits significant differences with respect to other lines. As a result, we can differentiate the tampered characters from authentic characters using DMGP. 4 Proposed Method When a document is tampered by copy-pasting or add-printing, it is difficult to accurately paste or print characters in an ideal position. Translation and rotation distortions may be present on the tampered characters. Figure 4 illustrates the geometric distortion of the Chinese word “Yong” after tampering. Figure 4(b) is a character image which is segmented from the reference document image. Figure 4(a) is the character image from the scanned document image which is Journal of Electronic Imaging

obtained based on document image alignment and has the same position and size as Fig. 4(b). Figure 4(c) exhibits positional relationship of Figs. 4(a) and 4(b) after image matching, applying a translation and rotation angle. The red dots denote the center of the character. The blue vectors denote the translation and rotation distortions. We calculate

Fig. 4 The Chinese word “Yong” and its geometric distortion: (a) scanned character image, (b) reference character image, and (c) an illustration of geometric distortion between (a) and (b).

023008-4

Mar∕Apr 2015


•

Vol. 24(2)


Algorithm 1 The pseudo-code for proposed method.

Reference document image restoration by OCR; Tilt correction and character segmentation; Align the scanned and reference document images by the benchmark character and resize the scanned document image; for i ¼ 1∶DocRow for j ¼ 1∶DocColi Estimate rotation distortion parameter θ0 for Charði; jÞ; Estimate translation distortion parameters ðd x ; d y Þ for Charði; jÞ; Calculate distortion probability P char and save it to p-mapði; jÞ;

denoted by f 1 ðx; yÞ and f 2 ðx; yÞ, respectively, where x and y are the coordinates in the character images, M and N are the height and width of the character images, x ∈ ½1; M and y ∈ ½1; N. The reference character image f 2 ðx; yÞ is segmented from the reference document image in the image preprocessing. The scanned character image f 1 ðx; yÞ is from the scanned document image and is obtained by segmenting the same position and size with f 2 ðx; yÞ after the document image alignment. We match two character images based on FMT and consider three parameters: translation, rotation, and scaling. Given an identical resolution between a scanned document image and its reference document image, there is no consideration given to scaling; translation and rotation are the only properties considered. We denote the relationship between f 1 ðx; yÞ and f 2 ðx; yÞ as f 2 ðx; yÞ ¼ f 1 ðx cos θ0 þ y sin θ0 − dx ; −x sin θ0

end

þ y cos θ0 − dy Þ;

Calculate distortion probability P line based on P char ; end Make a decision for tampered characters and lines based on P char and P line by threshold.

the distortion parameters by image matching based on the Fourier–Mellin transform (FMT). The proposed method works as given in Algorithm 1. Where DocRow and DocColi denote the row number in the document and character number in the i’th row, respectively; Charði; jÞ is a character in the position ði; jÞ; Pchar is the distortion probability for the characters; and Pline is that for lines. The first three steps in the above pseudocode are preprocessing steps. We regenerate the reference document by OCR and save it to TIFF image with an identical resolution as the scanned document image. We segment the characters in the reference and scanned document images based on horizontal and vertical projections. A benchmark character is applied to align the document images, and the size of the scanned document image is adjusted to be the same as the reference document image. The preprocessing is executed on the full document image, and the next several steps in the pseudocode are performed on the character image level. We estimate the distortion parameters for each character which consists of two translation distortion parameters and one rotation distortion parameter. We calculate the distortion probability for characters and lines, and the tampered characters and lines can be identified by probability thresholds. We show the degree of distortion relative to each character in the document through visualization of the P-map using Pchar . 4.1 Parameter Estimation for Rotation Distortion When a document undergoes copy-pasting or add-printing, the tampered characters may exhibit different rotation angles relative to other characters. Printer hardware or manual operation may introduce slight errors. Rotation distortion can be a defining characteristic of tampered characters. Given a scanned character image and a reference character image Journal of Electronic Imaging

(4)

where θ0 is the rotation angle between the scanned image and its reference character images, and dx and dy are the translation amplitude on the horizontal and vertical directions, respectively. After Fourier transform, Eq. (4) is transformed to F2 ðu; vÞ ¼ F1 ðu cos θ0 þ v sin θ0 ; −u sin θ0 þ v cos θ0 Þ · exp½−jðudx þ vdy Þ:

(5)

In the Fourier domain, the rotation parameter and translation parameters are isolated as two factors. This isolation allows for separate parameter estimations. As exp½−jðudx þ vdy Þ is the phase information and its absolute value is 1, we calculate the absolute value on the two sides of Eq. (5) and obtain the following equation: jF2 ðu; vÞj ¼ jF1 ðu cos θ0 þ v sin θ0 ; −u sin θ0 þ v cos θ0 Þj:

(6)

This demonstrates the relationship of an amplitude spectrum between character images with rotation angle θ0 as the only parameter. That is to say, the Fourier amplitude spectrum has translational invariance. We estimate the rotation distortion parameter θ0 in a polar coordinate. The aim is a rotation conversion in Cartesian coordinates with respect to translation of the polar coordinates; this improves the computational efficiency. We denote r; θ as the translation and angle in polar coordinates. We write the coordinate transformation as follows:

r cosðθ − θ0 Þ ¼ u cos θ0 þ v sin θ0 ; r sinðθ − θ0 Þ ¼ −u sin θ0 þ v cos θ0

(7)

where u ¼ r cos θ and v ¼ r sin θ. In the Fourier amplitude spectrum, we project the variables to ðu; vÞ in the polar coordinates system, P2 ðθ; rÞ ¼ F2 ðu; vÞ and P1 ðθ; rÞ ¼ F1 ðu; vÞ, and simplify the expression. Combining Eqs. (6) and (7) results in the following equation:

023008-5

Mar∕Apr 2015


•

Vol. 24(2)


jP1 ðθ; rÞj ¼ jP2 ðθ − θ0 ; rÞj:

(8)

Thus, we convert rotation to translation in polar coordinates. Given polar coordinates, we apply image matching to estimate the rotation angle between the scanned and its reference character image. We translate the scanned character image with a specified angle range and calculate the correlation coefficient between the translated character and reference character images. The rotation angle θ0 is determined as θ0 ¼ arg maxfρ½P1 ðθ − θ 0 ; rÞ; P2 ðθ; rÞg; (9) hpffiffiffiffiffiffiffiffiffiffiffiffiffi pffiffiffiffiffiffiffiffiffiffiffiffiffii where ρðP1 ; P2 Þ ¼ CovðP1 ; P2 Þ∕ DðP1 Þ · DðP2 Þ is the correlation coefficient between P1 and P2 . From a practical point of view, we search the angle in the range of θ 0 ¼ ½−30 deg; 30 deg. 4.2 Parameter Estimation for Translation Distortion In addition to the rotation distortion parameter, there is translation distortion in the horizontal and vertical directions, denoted as ðdx ; dy Þ. As we described in Sec. 3.2, the geometric distortion results when the authentic document gradually changes on the page. However, translation distortion parameters for tampered characters have mutations when compared with adjacent characters. Thus, translation distortion is an additional parameter for identifying tampered characters. After estimating the rotation angle θ0 , rotation correction is performed on the scanned character image and makes the two character images only have translation. Given the equation: Gðu; v; θÞ ¼

F2 ðu; vÞ : F1 ðu cos θ þ v sin θ; −u sin θ þ v cos θÞ (10)

Let θ ¼ θ0 , combined with Eq. 5, it yields the following equation: Gðu; v; θ0 Þ ¼ exp½−jðudx þ vdy Þ:

(11)

Gðu; v; θ0 Þ is a virtual function. Its inverse Fourier transform is as follows: gðx; yÞ ¼ F−1 ½Gðu; v; θÞ ¼

1 ; 2πðx − dx Þðy − dy Þ

(12)

where gðx; yÞ is an impulse function with an infinite peak value and the location of the peak is ðdx ; dy Þ. In practice, the peak value on ðdx ; dy Þ is limited, but the value is noticeably higher than that of other regions. The peak coordinates are the values of the translation distortion. Calculation of the distortion parameters is as follows: ½dx ; dy ¼ argmax½gðx; yÞ:

(13)

Calculation of parameters ðdx ; dy Þ by this method results in minor pixel variation. We examine the character image whose center is near ðdx ; dy Þ and calculate the correlation. The peak of the correlation coefficients is chosen and the parameters ðdx ; dy Þ are corrected. Journal of Electronic Imaging

4.3 Acquisition of P-Map We denote three parameters of the rotation and translation distortions as ðθ0 ; dx ; dy Þ. These parameters exhibit minor differences with respect to adjacent characters in the authentic document. Conversely, the tampered characters exhibit major differences. One character may have several adjacent characters. The mean value of the distortion parameters for all adjacent characters constitutes the estimated distortion parameter for the central character. The estimation error is the difference between the estimated distortion parameters and the actual distortion parameters, calculated as (E ¼ θ − θ0 θ 0 Ex ¼ dx − d 0 x ; Ey ¼ dy − d 0 y

(14)

where θ 00 , d 0 x , and d 0 y are the mean values of θ0 , dx , and dy for adjacent characters, respectively. We introduce the character distortion probability as −ðΔEΦ − μΦ Þ2 ; (15) PΦ ¼ 1 − exp 2σ 2Φ where Φ denotes the set consisting of the rotation angle, horizontal translation, and vertical translation (three parameters and Φ ¼ fθ; x; yg). For a tampered character, only one or two parameters may have a mutation. Therefore, we determine character tampering with the assumption that at least one parameter is mutated. We calculate the probability of a character tampering as Y PΦ : (16) Pchar ¼ Φ¼fθ;x;yg

The increase in probability value exhibits a large parameter estimation error, indicating a high likelihood of character tampering. The mean and standard deviation in Eq. (15) are set as μΦ ¼ 0 and σ Φ ¼ 1. If a character has DMGP, then the value of its estimation error EΦ has an apparent difference with its adjacent characters and the probability Pchar is nearly 1. Calculating probability Pchar for each character, we render a visualized P-map composed of Pchar to show the detection result in the document. We denote the probabilities using different colors and the tampered characters will exhibit greater color variance. With respect to the line tampering detection, we calculate the probability by averaging Pchar on the line as follows: Pline ¼

N 1X P ðiÞ; N i¼1 char

(17)

where Pchar ðiÞ denotes the distortion probability of the i’th character on the line, and N represents the number of characters on the line. 5 Experiments and Analysis In this experiment, we focus on forgery detection effectiveness given multiple scenarios. We base our experiment scenarios on document tampering of individual characters and entire lines. We compare the performance of our experiment against Ref. 14. We consider utilizing different document

023008-6

Mar∕Apr 2015


•

Vol. 24(2)


0.8

0.2 0.1 0 0

0.2

0.4

0.6

0.8

0.4

0.6

0.8

1

the two classifiers does so. The statistical model combines the two features in one Bayesian formulation and makes one decision. Figure 5 shows the detection results of the four datasets for the proposed method and the method in Ref. 14. Figure 5(a) is the result for the voting scheme in Ref. 14 and the proposed method, and Fig. 5(b) is for the statistical model in Ref. 14 and the proposed method. From the ROC curves in Fig. 5, the proposed method has a better performance than Ref. 14 on both decision models and remains stable on the four datasets. 5.2 Detection for Tampered Characters With regard to low-scanning resolution and JPEG compression, we anticipate a robust performance. Figure 6 shows the ROC curves for character tampering detection with resolutions ranging from 200 to 800 dpi. The true positive rate increases rapidly with the false positive rate, and the detection accuracies have a small growth at resolutions of 300 dpi and above. The ROC curve at the resolution of 200 dpi is very different compared with that of 300 dpi; this is due to the weakening of tamper traces in decreased resolution. In addition, the size of a scanned character is approximately

0.8

0.3

0.2

Fig. 6 Detection results for character tampering documents with different scanning resolutions.

0.8

0.4

0.3

False positive rate

1

TP300 PPC300 TPLJ TPCLJ TP300 in Ref. 14 PPC300 in Ref. 14 TPLJ in Ref. 14 TPCLJ in Ref. 14

200 dpi 300 dpi 400 dpi 500 dpi 600 dpi 700 dpi 800 dpi

0.4

0

0.9

0.5

0.5

0

1

0.6

0.6

0.1

0.9

0.7

0.7

0.2

True positive rate

True positive rate

5.1 Performance Comparison We compare the performance with the method presented in Ref. 14 published in 2013. Beusekom et al.14 proposed two features denoted as skew angle and alignment feature to expose document forgery. If a text-line is forged, it has a difference in skew angles with other text-lines and the forged text-line cannot be aligned by the left and right alignment lines. The Bayesian rule is used to estimate the forging probability. Two decision models, named the voting scheme and statistical model, are presented to demonstrate the results. The voting scheme is calculating the forging probability separately and a text-line is classified as forged if at least one of

1 0.9

True positive rate

languages, tampering methods, and JPEG compressions with respect to the scanning resolution. Finally, we give the time complexity of proposed method. As there is no public dataset available as a performance reference, we generate two groups of datasets as follows. Group 1: We tampered with 160 document pages, half as character tampering and half as full-line tampering. There are 1360 tampered characters and 480 tampered lines on the documents, and the tampering ratios are 1.14% and 13.6%, respectively. The tampering method focuses on copy-pasting and add-printing, comprising 80 pages per tampering method. We selected two languages for the forged document: Chinese and English. These two languages have vastly different character structures. The scanner is an EPSON V33 and the resolution ranges from 200 to 800 dpi. We saved the scanned document images in TIFF and JPEG formats. We tampered the forged documents with the following printer models: Canon LBP3500, HP 5200Lx, Samsung ML3471ND, and Lenovo LJ6000. We define a distance threshold to limit the search region of adjacent characters as two times a character’s height, which is 180 pixels in 600 dpi. Group 2: As the datasets in Ref. 14 are not public, we regenerate the datasets with the following: Two-pass Print 300 dpi (TP300), Print, Paste, and Copy 300 dpi (PPC300), Two-pass Print LaserJet (TPLJ), and Two-pass Print Color LaserJet (TPCLJ). All datasets are line-tampering documents with different tampering methods.

0.7 0.6 TP300 PPC300 TPLJ TPCLJ TP300 in Ref. 14 PPC300 in Ref. 14 TPLJ in Ref. 14 TPCLJ in Ref. 14

0.5 0.4 0.3 0.2 0.1

1

0 0

0.2

0.4

0.6

0.8

1

False positive rate (b)

False positive rate (a)

Fig. 5 Performance comparison with Ref. 14: (a) the detection results for the proposed method and the method in Ref. 14 using the voting scheme, and (b) the detection results for the proposed method and the method in Ref. 14 using the statistical model.


023008-7

Mar∕Apr 2015


•

Vol. 24(2)

1

1

0.9

0.9

0.8

0.8

True positive rate

True positive rate


0.7 0.6 0.5 0.4 0.3

JPEG JPEG JPEG JPEG TIFF

0.2 0.1 0 0

0.2

0.4

0.6

0.8

Q=30 Q=50 Q=70 Q=90

0.7 0.6 0.5 0.4 0.3

JPEG JPEG JPEG JPEG TIFF

0.2 0.1 0 1

0

False positive rate

0.2

0.4

0.6

Q=30 Q=50 Q=70 Q=90

0.8

1

False positive rate

Fig. 7 Detection results for character tampering documents with different image qualities.

Fig. 9 Detection results for line tampering documents with different image qualities.

90 × 90 pixels at 600 dpi, compared with a pixel size of 30 × 30 at 200 dpi. If a character exhibits translation distortion with fewer pixels, then the distortion magnitude is much smaller at 200 dpi. Figure 7 depicts the detection results in the TIFF format and with different JPEG compression qualities with the resolution of 600 dpi. Of note, the ROC curves remain stable even with low compression qualities. As a result, the proposed character tampering detection method remains viable with JPEG compression.

and tampered by the same printer; we underline the tampered characters in red. Figure 11 depicts the detection results (P-maps) of Fig. 10. The positions of the color blocks correspond to the characters in the document, and the colors represent the distortion probabilities whose values are shown in the color bar on the right of the figure. From the color blocks in Fig. 11, we see that the tampered characters have greater probabilities than the other characters. In addition, characters adjacent to the tampered characters also have a great probability because the adjacent characters also have DMGP with respect to the tampered characters. Thus, if tampering is evident with one character, we simply determine that the central character is tampered with in the probability region. This is also suitable method for line tampering detection. In practice, we need to consider the distance between two lines, as we calculate DMGP with respect to adjacent characters. As shown in Fig. 10(b), the title “CONTRACT” and the last line for the signatures have large distances with their adjacent characters. The estimation error may increase with the actual value in this case, so we use a threshold to limit the search region of adjacent characters to two times character’s height.

5.3 Detection for Tampered Lines Line tampering detection is also viable with the proposed method. As with character tampering detection, line tampering detection considers the quality of the scanning resolution and JPEG compression. Figure 8 depicts the detection results of the line tampering detection at different resolutions with TIFF format images. The true positive rate rapidly increases to 1 then maintains stable growth as the resolution increases. Figure 9 depicts the results of the line tampering detection in TIFF format and with different JPEG compression qualities, and shows that the performance is robust to JPEG compression. 5.4 Detection for Practical Documents Figure 10 shows two forged documents written in Chinese and English; the documents are printed by Canon LBP3500 1 0.9

True positive rate

0.8 0.7 0.6 0.5 200 dpi 300 dpi 400 dpi 500 dpi 600 dpi 700 dpi 800 dpi

0.4 0.3 0.2 0.1 0 0

0.2

0.4

0.6

0.8

1

False positive rate

Fig. 8 Detection results for line tampering documents with different scanning resolutions.


5.5 Time Complexity Although tampered document detection does not require real-time execution, we examine the time complexity and computational efficiency to show the performance of our proposed method. Suppose the number of characters in a document is N and the resolution of character image is X × Y. Taking into account the tilt correction, OCR, and character segmentation in the preprocessing steps, the time complexity is Oð1Þ þ OðNÞ þ OðNÞ ¼ OðNÞ. In the parameter estimation step, the time complexity for FFT (fast Fourier transform) and polar coordinate conversion in Sec. 4.1 are O½XY · log2 ðXYÞ and OðXYÞ, respectively. The time complexity for inverse FFT and calculation of parameters ðdx ; dy Þ in Sec. 4.2 are O½XY · log2 ðXYÞ and O½XY · log2 ðXYÞ, respectively. So the time complexity of parameter estimation for one character is O½XY · log2 ðXYÞ þ OðXYÞ þ O½XY · log2 ðXYÞ þ O½XY · log2 ðXYÞ ¼ O½XY · log2 ðXYÞ. In the P-map acquisition step, the time complexity is OðNÞ. In sum, the total time complexity for proposed method is OðNÞþOðNÞ·O½XY·log2 ðXYÞþOðNÞ¼O½NXY·log2 ðXYÞ.

023008-8

Mar∕Apr 2015


•

Vol. 24(2)


Fig. 10 Forged documents: (a) a forged Chinese document and the tampered characters are labeled with red lines, and (b) an English forged document and the tampered characters are labeled with red lines.

1.000

1.000

0.875

0.875

0.750

0.750

0.625

0.625

0.500

0.500

0.375

0.375

0.250

0.250

0.125

0.125

(a)

(b)

Fig. 11 The detection results for forged document: (a) the detection result for Fig. 10(a), and (b) the detection result for Fig. 10(b).

The computer specifications used for the experiment are as follows: Intel(R) Core(TM) i7-3770 CPU @ 3.40 GHz (main processor), and 16 GB RAM. With a resolution of 600 dpi and 1500 characters per single document page, the detection time is approximately 17 min. 6 Conclusion We propose a method of document forgery detection based on DMGP with translation and rotation distortion parameters. This method is suitable for document examination in both Chinese and English, can examine documents based on individual characters, is robust to JPEG compression, and is effective in low-resolution documents as well. The experimental results demonstrate that this method performs well for practical document examination. Journal of Electronic Imaging

The printed documents exhibit gradual, full-page geometric distortion, and only the tampered characters have DMGP compared with the original characters. We applied image matching between the scanned and reference character images to estimate the distortion parameters for each character, and the reference document image was restored by OCR. As the decisive step, we introduced the concept of probability to describe the distortion degree related to the geometric parameters. We also introduce a visualized P-map to render detection results for the document. In future work, we will consider improvements in accuracy on practical documents. When a character and line are distant with respect to adjacent characters, the distortion estimate may have significant errors, leading to a reduced accuracy in tampering detection. We will also consider estimating the page distortion model and calculation of

023008-9

Mar∕Apr 2015


•

Vol. 24(2)


distortion parameters in any position in the document. This will improve the accuracy of the distortion estimation error and enhance the robustness of practical document detection. Acknowledgments The work is supported by the National Natural Science Foundation of China (Grant No. 61172109) and the Research Fund for the Doctoral Program of Liaoning Province (Grant No. 20131014). References 1. A. K. Mikkilineni, N. Khanna, and E. J. Delp, “Forensic printer detection using intrinsic signatures,” Proc. SPIE 7880, 78800R1 (2011). 2. A. K. Mikkilineni et al., “Printer identification based on graylevel cooccurrence feature for security and forensics application,” Proc. SPIE 5681, 430–440 (2005). 3. A. K. Mikkilineni, N. Khanna, and E. J. Delp, “Texture based attacks on intrinsic signature based printer identification,” Proc. SPIE 7541, 75410T (2010). 4. P. J. Chiang et al., “Printer and scanner forensics,” IEEE Signal Process. Mag. 26, 72–83 (2009). 5. G. N. Ali et al., “Intrinsic and extrinsic signatures for information hiding and secure printing with electrophotographic devices,” in Final Program and Proc. IS and Ts NIP19: Int. Conf. Digital Printing Technologies, pp. 511–515, Society for Imaging Science and Technology, Springfield, Virginia (2003). 6. O. Bulan, J. Mao, and G. Sharma, “Geometric distortion signatures for printer identification,” in IEEE Int. Conf. Acoustics, Speech and Signal Processing, 2009. ICASSP 2009, pp. 1401–1404, IEEE, Piscataway, New Jersey (2009). 7. M. J. Tsai, J. Liu, and J. S. Yin, “Digital forensics of printed source identification for Chinese characters,” Multimedia Tools Appl. 73, 2129–2155 (2014). 8. Y. Wu, X. Kong, and X. You, “Printer forensics based on document’s geometric distortion,” in Proc. 2009 16th IEEE Int. Conf. Image Processing (ICIP 2009), pp. 2909–2912, IEEE, Piscataway, New Jersey (2009). 9. J. S. Aronoff and S. J. Simske, “Effect of scanner resolution and character selection on source printer identification,” J. Inf. Sci. Technol. 55, 050602 (2011). 10. H. Gou, A. Swaminathan, and M. Wu, “Robust scanner identification based on noise features,” Proc. SPIE 6505, 65050S (2007). 11. N. Khanna, A. K. Mikkilineni, and E. J. Delp, “Scanner identification using feature based processing and analysis,” IEEE Trans. Inf. Forensics Secur. 4, 123–139 (2009). 12. A. E. Dirik, H. T. Sencar, and N. Memon, “Flatbed scanner identification based on dust and scratches over scanner platen,” in IEEE Int. Conf. Acoustics, Speech, and Signal Processing—Proc., ICASSP 2009, Vol. 6505, pp. 1385–1388, IEEE, Piscataway, New Jersey (2009). 13. J. V. Beusekom, F. Shafait, and T. M. Breuel, “Document inspection using text-line alignment,” in Proc. 9th IAPR Int. Workshop on Document Analysis Systems, DAS ‘10, ACM Int. Conf. Proc. Ser., pp. 263–270, ACM, New York (2010). 14. J. V. Beusekom, F. Shafait, and T. M. Breuel, “Text-line examination for document forgery detection,” Int. J. Doc. Anal. Recognit. 16, 189–207 (2013).


15. J. V. Beusekom and F. Shafait, “Distortion measurement for automatic document verification,” in Proc.—11th Int. Conf. Document Analysis and Recognition, ICDAR 2011, pp. 289–293, IEEE, Piscataway, New Jersey (2011). 16. A. Ahmed and F. Shafait, “Forgery detection based on intrinsic document contents,” in Proc.—11th IAPR Int. Workshop on Document Analysis Systems, DAS 2014, pp. 252–256, IEEE, Piscataway, New Jersey (2014). 17. R. Bertrand et al., “A system based on intrinsic features for fraudulent document detection,” in Proc. Int. Conf. Document Analysis and Recognition, ICDAR, pp. 106–110, IEEE, Piscataway, New Jersey (2013). 18. E. Kee and H. Farid, “Printer profiling for forensics and ballistics,” in MM and Sec’08: Proc. 10th ACM Workshop on Multimedia and Security, pp. 3–9, ACM, New York (2008). 19. C. H. Lampert and T. M. Breuel, “Printer technique classification for document counterfeit detection,” in Int. Conf. Comput. Intell. Secur. ICCIAS, pp. 639–644, IEEE, Piscataway, New Jersey (2006). 20. J. Gebhardt, M. Goldstein, and F. Shafait, “Document authentication using printing technique features and unsupervised anomaly detection,” in Proc. Int. Conf. Doc. Anal. Recognit., pp. 479–483, IEEE, Piscataway, New Jersey (2013). 21. M. Umadevi, A. Agarwal, and R. Rao, “Printed text characterization for identifying print technology using expectation maximization algorithm,” Proc. Int. Conf. Doc. Anal. Recognit., pp. 201–212, IEEE, Piscataway, New Jersey (2011). 22. C. Schulze et al., “Using DCT features for printing technique and copy detection,” Adv. Digital Forensics 306, 95–106 (2009). 23. S. Shang, N. Memon, and X. Kong, “Detecting documents forged by printing and copying,” EURASIP J. Adv. Signal Process. 2014, 140 (2014). 24. A. Amin and S. Fischer, “A document skew detection method using the Hough transform,” Pattern Anal. Appl. 3, 243–253 (2000). 25. D. Lee and S. Lee, “A new methodology for gray-scale character segmentation and recognition,” in Proc. Int. Conf. Doc. Anal. Recognit. Vol. 1, pp. 524–527, IEEE, Piscataway, New Jersey (1995). Shize Shang is a PhD student at Dalian University of Technology. He received his BS degree in electronic and information engineering from Dalian University of Technology in 2008, and he began to pursue his PhD degree in March 2010. He was a visiting research scholar at NYU-Poly from September 2012 to December 2013, supervised by Professor Nasir Memon. His current research interests include digital image forensics and document forensics. Xiangwei Kong is a professor at Dalian University of Technology. She received her PhD degree at Dalian University of Technology in 2003. She was a visiting research scholar at Purdue University from September 2006 to September 2007 and at NYU from December 2014 to June 2015. Her research interests include multimedia forensics, pattern recognition, and information retrieval. Xingang You is a researcher at Beijing Institute of Electronic Technology and Application and is also a professor at Dalian University of Technology. His research interest is multimedia security.

023008-10

Mar∕Apr 2015


•

Vol. 24(2)