Multimedia Tools and Applications https://doi.org/10.1007/s11042-018-6891-7
Two stages object recognition based copy-move forgery detection algorithm Mohamed A. Elaskily 1,2 & Heba A. Elnemr 3 Osama S. Faragallah 2,4
2
& Mohamed M. Dessouky &
Received: 28 March 2018 / Revised: 24 September 2018 / Accepted: 13 November 2018 # Springer Science+Business Media, LLC, part of Springer Nature 2018
Abstract Copy-Move Forgery Detection (CMFD) is a key issue of image forensics. A copy-move forgery is a type of image tampering that is created by copying a part of the image and pasting it on another part of the same image to perniciously hide or clone certain regions. This paper presents a new methodology for CMFD in digital images. The proposed algorithm is performed in two successive stages; matching stage and refinement stage. In the matching stage, close morphological operation and Connected Component Labeling (CCL) are used to segment the target image into different objects. The Speeded Up Robust Features (SURF) are extracted from each object and used to build an object catalog. The objects in the catalog are compared to each other, and matched objects are determined. If matched objects exist, the image is categorized as forged image. Otherwise, it is categorized as original image. The refinement stage, on the other hand, is implemented to ensure the originality of the target image. Thus, the candidate image that is classified as original is fed into the refinement stage to certify its originality. In this stage, close and open morphological operations as well as CCL are utilized to obtain the various objects in the image. Afterward, the SURF features are extracted from each object and used to build a new object catalog. The match between the objects in this catalog is obtained. If similar objects are found, the candidate image is classified as forged. Otherwise, the image is categorized as original. The proposed technique is assessed on four popular datasets. The results demonstrate the capability and robustness of the proposed technique in detecting the copy-move forgery under different geometrical attacks. Furthermore, the outcomes show that the suggested technique outperforms the previous CMFD methods in terms of Accuracy and execution time. Keywords Copy-move forgery detection . Morphological operation . Object detection . Connected component labeling . Speeded up robust features
* Heba A. Elnemr
[email protected] Extended author information available on the last page of the article
Multimedia Tools and Applications
1 Introduction Images are one of the important means of communication that is used for interaction among people around the world in the current digital era. Images are used in different vital areas such as crime evidence, medical diagnostics, environment and weather research, military information, and many other applications. Owing to the need to transfer images among different unguaranteed communication ways, image authentication has been demanded. Image authentication is used to ensure that there is no changes occurred on the image. Two approaches are used for authenticating an image, active and passive authentication [10]. Active authentication requires applying operations such as embedding a watermark or using a digital signature firstly on the original images. In the active approach, the original content of the image must be available for comparing it with the suspected image. Unlike active authentication, passive authentication is used for examining suspected images without any previous knowledge about original images [13]. Forgery methods have been advanced tremendously, in ways that not to leave any visual evidence to affect the image. Image forgeries appear in many cases such as judges in courts, social media, cybercrimes, military and intelligence deception, newspapers, or defamation of important characters. Any image has some statistical, geometric, or physical properties. These statistics or properties are used for detecting image forgery by searching for any unusual features or any perturbations [12]. Image forgeries have many different types such as copy-move forgery, image splicing, image morphing, image resampling, and image retouching [2]. Copy-move forgery is applied by copying a patch region from an image and pasting it into the same image to hide or duplicate an object and gives the image a fake meaning or fake scenery [36]. Image splicing is applied by composing two or more different scenes from two or more different images to create a new fake image [7]. Image morphing is using two different shapes from two different images to gradually merging them into a new one [34]. Image resampling is applied by increasing/decreasing height/width of a specific object in an image or in all content of the image to give deceptive scene [1]. Image retouching is enhancing an object or image to hide or highlights some features such as lighting, coloring or background changing to attract attention or to divert attention about an object in an image [16]. Figure 1 illustrates different types of digital image forgeries. Copy-Move forgery are applied to give the digital images other meanings or to deceive the viewers. Copy-move forgery is the widest forgery type because it is easy to apply and its effects in the tampered image are limited. However, copy-moved tampered image is difficult to detect because the copied patch has the same features of the whole image. Hence, the final tampered image is homogeneous as one context image. In this paper, a novel, fast and robust Copy-Move Forgery Detection (CMFD) algorithm has been developed and tested. The proposed algorithm is based on detecting the objects within the contributed image by building a contour among its borders. After that, an object catalog is built. The Speeded Up Robust Features (SURF) are extracted from each object and employed to build an object’s catalog. The objects included in the catalog are matched, where the matched objects reflect the copy-moved attack. Owing to SURF features, the proposed algorithm proved to be reliable against scaling, rotation, and translation operations. The contribution of the proposed approach may be summarized as follows: 1) Constructing a high-performance algorithm that contributes on CMFD by increasing its power to detect the forged images.
Multimedia Tools and Applications
A) Copy- Move Forgery: Begins from left, the original image first followed by its tampered image
B) Image splicing: Begins from left, different images are composed and followed by the tampered images
C) Image morphing: The left and right images are original and the middle is the morphed
D) Image resampling: left Image is the original and right image is resampled
E) Image retouching: Left is the original image followed by two tampered images Fig. 1 Digital image forgery types
2) Decreasing of the false detection rate of the copy-move forgery caused by forgers deceiving. 3) Building an objects’ catalog that contains features of all objects in the tested image to facilitate the matching process. 4) Speeding up the forgery detection process by decreasing the computational time taken by the proposed algorithm to detect the copy-move forgery. 5) Using SURF algorithm makes deceived operations such as scaling, rotation, and translation have no effect on the extracted features, since SURF features are scaling, rotation and translation invariant. The structure of the paper is organized as follows. The next section shows the related work. In Section 3, the proposed approach is discussed in details. In Section 4, experimental results on different datasets are presented with different types of attacks. Finally, Section 5 concludes the paper.
Multimedia Tools and Applications
2 Related work Forensic methods are considered as blind methods, owing to the fact that the authenticity test is performed without any previous knowledge about the original image. Currently, various researches have presented many algorithms to detect copy-move forgery. There are two main approaches for CMFD algorithms; block-based methods, and non-block based methods. Block-based techniques are generally based on dividing the image into overlapping, circular or rectangular blocks. Afterward, an extraction process is performed to these blocks to detect their features. A matching process is then applied to find the matched features, which reflects the repeated blocks [20]. Discrete Cosine Transform (DCT) technique is used for CFMD by [29, 33]. Both techniques are based on dividing the image into fixed size overlapped rectangle or circular blocks, and then extract the DCT coefficients from each block. These DCT coefficients are sorted lexicographically. Neighboring blocks in the sorted order are matched to detect identical regions in the image. Similar technique that extracts DCT from each block is presented in [26]. Furthermore, Gaussian RBF kernel Principal Component Analysis (PCA) is applied to reduce the dimension of the feature vector representation in order to improve the efficiency of the feature matching process. Another approach that hybridized DCT with PCA is presented in [37]. DCT algorithm is used to extract the features after overlapping block division is performed, while PCA is used to compress the feature vector of these overlapping blocks. Furthermore, the work in [17] utilized both DCT and Singular Value Decomposition (SVD) for detecting and locating copymove forgery. DCT is applied to each block to represent its features, whereas SVD is used for improving the ability to resist noise and dimensionality reduction. Block matching strategy becomes very complicated when the forgery operation blurs the edges of the forged patches. An algorithm for detection of copy-move forgery, using Stationary Wavelet Transforms (SWT) and SVD, is proposed in [11] to solve this problem. SWT is a shift invariant technique that is used to find the similarities according to noise caused due to blurring. SVD is used to represent features extracted from each block. In addition, image moments are used for CMFD to benefit from its ability to analyze the shapes and recognize the image parts. Various types of image moments are used for CMFD, such as blur Moment, Hu Moment, Krawtchouk Moment, Histogram Moment, and Zernike Moment [28]. Image moments techniques overcome blurring, noise addition, and other operations such as rotation, JPEG compression, and partially scaling. However, they sustained from high complexity due to performing several matching procedures. Latterly, Zernike Moments have been implemented for CMFD [21, 27, 35]. On the other hand, non-block based CMFD techniques are applied by extracting features from the entire image [20]. Keypoints-based techniques are a major form of that method. Keypoints-based techniques are based mainly on utilizing Scale Invariant Feature Transform (SIFT) and SURF for detecting copy-move forgery [18]. SIFT algorithm is proposed by David G. Lowe [25] which presents a method for extracting distinctive invariant features from images. These features are invariant to translation, scale and rotation, and gives robust matching against affine transformation, noise addition, and illumination changes. The SIFT features can be used to execute reliable matching between different views of a scene or object. SURF algorithm is a development of SIFT algorithm to enhance the algorithm speed [6]. SURF features are fast and robust comparing with SIFT features due to their 64 bytes dimensional descriptor comparing with 128 bytes dimensional descriptor of SIFT features.
Multimedia Tools and Applications
By focusing on the recent keypoints based algorithms, Amerini et al. [3] proposed an algorithm that extracts SIFT features from the whole image. A clustering process for the extracted SIFT features is performed with dynamic thresholding step. Finally, a matching process is applied to detect similar matched features, which reflects the copy-move forgery regions in that image. A two stage algorithm that applies both SIFT and SURF techniques for CMFD is presented in [32]. To speed up the examination of CMFD, the best ten matched keypoints in both SURF and SIFT keypoints are selected, separately. The matching strategy is performed according to g2NN test. The tampered region is detected by using SURF features matching and confirmed by using SIFT features matching. Hashmi et al. [14] applied Dyadic Wavelet Transform (DyWT) as a preprocessing step to decompose the image into four sub-images; LL, LH, HL, and HH. SIFT keypoints are extracted from the LL sub-image as it contains all the details of the image. By finding the descriptor vector for each keypoints a matching process is performed to detect the similarities between various descriptor vectors. These similarities are referred to the copy-moved patched regions. Recently, image attackers are innovated to hide their image forgeries by removing the original SIFT keypoints from the image and re-introducing forged keypoints instead of the original ones [4]. This process leads to extract fake SIFT features which hide the similarities among the copymove forgery regions. Costanzo et al. [9] proposed three forensic detectors that are capable to resist SIFT keypoints removal and injection attack, including Keypoint-to-Corner Ratio Detector, CHI-Square Distance Detector, and Support Vector Machine Detector. These detectors search for inconsistencies within the image texture, such as the heterogeneous distribution of the keypoints. In the work of [8], fifteen most notable feature sets are examined, such as SIFT, SURF, Blur moments, Hu moments, and Zernike moments, in addition to block-based feature sets such as DCT, DWT, and Singular Values Decomposition (SVD), PCA, Kernel-PCA, and Fourier– Mellin Transform (FMT), as well as other features that are characterized by intensity features as Luo features, Bravo features, Lin features, and Circle features. The comparison between these features exhibits that keypoints-based features give good results in CMFD comparing with other block-based algorithms. Algorithms which are using SIFT and SURF features are very fast especially algorithms that are using SURF features.
3 Proposed algorithm The proposed CMFD algorithm is based on segmenting the tested image into several objects and building an objects’ catalog that comprises the attributes of each object within the input image. The objects’ catalog is created by, first; segmenting the input image into consistent regions. Then, SURF descriptors are extracted from each labeled region. These descriptors are finally used to build the objects’ catalog. A matching procedure is carried out among the objects within the objects’ catalog to measure the similarity of these objects, and thus identifies the possible tampered objects. We observed that small tampered regions are hardly ever discovered because they have very limited contributions to the descriptors. Additionally, some forged regions exposed a high level of similarity with their neighbor regions. Thus, they may be considered part of the surrounding area, and therefore they are hard to be detected. For that reason, we adopted a multi-task learning paradigm in order to leverage the valuable information contained in various related tasks to assist improving the generalization performance of all tasks [22–24]. Therefore, we developed a two stages CMFD approach. The first stage is responsible for detecting the
Multimedia Tools and Applications
copy-move forged images and the images that candidate to be original. The second stage is applied on the candidate categorized to be original image, either to ensure their integrity or to detect a copy-move forgery within this candidate. Figure 2 illustrates the first stage of the proposed CMFD algorithm, while the second stage is presented in Fig. 3. The two main stages are detailed in the following subsections.
3.1 Objects’ catalog Objects’ catalog is deemed the main step for the proposed CMFD technique. It is a profile that consists of the different objects in the image associated with their attributes. The object’s catalog is created through two steps. In the first step, the various objects within the image are detected. While in the second step, SURF features are extracted from each detected object.
3.1.1 Object detection In this work, Connected Components Labeling (CCL) is applied to segment the image and detect the location of various objects in the image. CCL is a distinctive process that is used for multipurpose applications, such as image understanding, image analysis, pattern recognition, and computer vision [15]. It is used as a type of image segmentation or image regions labeling, which utilizes blob discovery and blob extraction. Blob represents regions that have constant or approximately constant properties. These regions indicate that there is a strong sign for the presence of objects or part of objects, which are used in turn for object detection or object tracking. The object detection process is fulfilled through three steps; close morphological operation application, edge detection and image segmentation by contour tracking. These steps are discussed as follows: A) Close Morphological Operation Application: Morphological operations are set of nonlinear image processing procedures that aim to analyze the images based on shapes. Morphological operations are used for edge detection, skeletonization, noise removal, etc. Closing morphological operation is an important morphological operator that removes small holes and connects small cracks. Closing operation is performed using a binary image and a fixed size structuring element. Closing operation exhibits object outlines by filling small and thin holes in its boundaries and removing small projections. It tends for growing the foreground pixels of an object to smooth the boundaries or contours of that object. It also shrinks the background holes or points belonging to these regions for the distinctness of region’s borders. B) Edge Detection: Edge detection is carried out through three stages: noise reduction, edge enhancement, and edge localization. In this work, Sobel operator is used to detect the edges by calculating the gradient of intensity at each pixel within the image. It detects the direction of the largest increase from light to dark and the amount of change in that direction. Sobel detector refines the image changes at each pixel, then decides if that pixel represents an edge or not. This step and the previous one aim to prepare the image for contour tracking and then image segmentation using CCL.
Multimedia Tools and Applications Tested Image
Tested Image
Preprocessing Operations
Object's Catalog Object Detection
Binary Image
Apply close morphological operation
Sobel edges detector
Image segmentation using CCL
Closed morphological image
Extract SURF features for each region
Construct the objects catalog Edge detected image
Refining stage
No
Objects matching (Two or more features match) Objects localization image
Yes Build the candidate copy-move forgery objects table
Check intersection between candidate forgery objects
Intersection exists
Copy-move forgery exists between the matched objects
No
Original image
Yes Delete the intersected objects and rebuild the final copy-move forgery objects table
Yes
The table empty
No
Fig. 2 Matching stage flowchart in addition to example with images
C) Image Segmentation: In this work, CCL is adopted to perform the image segmentation task using contour tracking. This task is considered the key task for object detection process. The CCL method proceeds by
Multimedia Tools and Applications
Image candidate to be original
Object’s Catalog Object Detected Apply close morphological op.
Apply open morphological op.
Sobel edges detector
Sobel edges detector
Image segmentation using CCL
Image segmentation using CCL
Intersected and neighbor regions merging
Extracts SURF features for each region
Construct the objects catalog
Original image
No
Objects matching (Two or more features) Yes
Build the candidate copy-move forgery objects table
Check intersection between candidate forgery objects
Intersection exists
No
Yes Delete the intersected objects and rebuild the final copy-move forgery objects table
Yes
The table empty
Fig. 3 Refine matching stage flowchart to ensure image originality
No
Copy-move forgery exists between matched objects
Multimedia Tools and Applications
scanning the edge detected binary image, pixel by pixel, from top to bottom and left to right to find the connected pixel regions. Each pixel takes a label, being either foreground or background, according to its intensity value. If a pixel is not the background then it is a part of a specific object and a connectivity check is performed to assign it to that object. After assigning each pixel to a specific foreground object or a background region, objects bounding boxes are portrayed. The obtained objects are localized in the grey level image. These localized objects represent the regions of interest which are fed to the feature extraction step.
3.1.2 Features extraction In this step, SURF algorithm is applied to the grey level image to acquire the objects’ features. SURF descriptors are obtained for each candidate region. SURF method is a fast, robust and reliable local feature detector and descriptor. It, first, detects the points of interest within each region, and then it computes a feature vector of 64 dimensions for each interesting point. SURF descriptors are invariant against rotation, scaling, translation, and illumination changes. Finally, an object’s catalog is created using the extracted objects associated with their SURF descriptors.
3.2 Object matching After building the objects’ catalog, the matching process is accomplished to detect the copymove forgery. The matching process is conducted in two stages. In the first stage, matching stage, all objects in the object’s catalog are compared with each other to detect the similar objects. While the second stage is reserved to refine the results, all images that are categorized as original are undergoing the refine matching test to guarantee their originality.
3.2.1 Matching stage After building the objects’ catalog, the matching stage is carried out, as illustrated in Fig. 2, to detect the copy-move forgery. In the matching stage, all objects are compared with each other. These comparisons are performed according to the Euclidean distance between the corresponding features that extracted from each object, with a similarity threshold of 0.6. When two or more objects are found to be similar, a candidate copy-move forgery object table is created. This table contains all suspected copy-moved objects. These suspected objects are either actually forged, or their similarities are due to the intersection regions between these objects. Therefore, the candidate copy-move forgery table is searched for intersected objects. The position (x,y) of the four corners of each object bounding box is compared against other objects bounding boxes to check if there is an intersection among these objects. Consequently, these intersected objects are removed from the table. After removing the intersected objects, if the table still contains candidate objects, the image is categorized as forged image and the forged regions are the objects remaining in the table. Otherwise, the image is categorized as original and undergoes the refine matching stage.
3.2.2 Refine matching stage In this stage, the images that are nominated to be original are subjected to a second matching procedure. This stage is a refinement stage that is performed to improve the system performance by guaranteeing the image originality.
Multimedia Tools and Applications
Close and open morphological operations are applied separately to the candidate images to be original. Both close and open morphological operations are performing a smoothing process for objects boundaries. Open morphological operation tends to smooth the contours of the objects by eliminating small and thin elements. It also removes the dim pixels in the object boundaries. While, close morphological operation tends to smooth the contours of the objects by filling and enhancing the small and thin holes in the object edges. The CCL method is employed to detect the objects in the closed and opened images separately. Afterward, the two segmented images are joined, and the intersected and adjacent objects are merged. In such a way, the proposed algorithm will cover extra spatial regions in the image by merging regions resulted from the two morphological operations. SURF descriptors are then extracted from the detected objects, and a new objects’ catalog is built. The matching stage is achieved for all objects in the new catalog, and the similar objects are stored in a copy-move object table. If the table is empty, the tested image is categorized as original. Otherwise, the intersections between the detected objects in the table are investigated and the objects are removed if intersections exist. Finally, the copy-forgery objects’ table is explored, if the table is empty, the image is positively original, else it is classified as forgery and the fraud regions are the ones remaining in the table. The refine stage is displayed in Fig. 3. Figure 4 shows the implementation of the object matching strategy on two sample images. One is a forged image (Fig. 4(a)), which is classified in the matching stage as original, yet the refine stage discovered the forgery regions and classified it as a forgery. While the other (Fig. 4(b)) is an original image, which is categorized in the matching and refine stages as an original image. As shown in Fig. 4(a), the forgery regions are not detected in the closed morphologically image. This causes the candidate image to be misclassified as original. While, as illustrated in Fig. 4(a), the copy-move regions are clearly detected in the open morphologically image. It is obvious that close morphological operation is unable to perfectly detect the tampered regions, whereas merging close and open detected regions yields effectively detection of the forged region.
4 Experimental results This section illustrates and evaluates the results of the proposed CMFD system. Furthermore, the proposed method is compared with the previous methods reported in the literature. All experiments are achieved on a machine with Intel core i7, 64 bits processor, 8 GB RAM, and operates by Linux Ubuntu 14.04.4 with software tool MATLAB R2017a.
4.1 Datasets Four traditional datasets are used to evaluate the suggested copy-move forgery algorithms. These datasets are MICC-F220 [3], MICC-F2000 [3], MICC-F600 [5], and SATS-130 [8]. MICC-F220, MICC-F2000, and MICC-F600 datasets are developed by Media Integration and Communication Center (MICC), while SATS-130 dataset is developed by Christlein et al. [8]. The MICC-F220 dataset consists of 220 images with a resolution varies from 722 × 480 to 800 × 600 pixels. The dataset is divided into 110 original images and 110 forged images. The size of the forged patched region compared to the size of the whole image is on the average 1.2%. The MICC-F2000 dataset contains 2000 images with an average resolution 2048 × 1536 pixel. The dataset is divided into 1300 original images and 700 forged images.
Multimedia Tools and Applications Image candidate to be original from matching stage
Binary Image
Closed morphological image Opened morphological image
Image candidate to be original from matching stage
Binary Image
Closed morphological image Opened morphological image
Edge detected image
Edge detected image
Edge detected image
Edge detected image
Regions resulted
Regions resulted
Regions resulted
Regions resulted
Merged regions from both close and open op. Merged regions from both close and open op.
Original image
(a): Candidate original image and categorized as forgery image after refine matching stage
Original image
(b): Candidate original image and categorized as original image again after refine matching stage
Fig. 4 a Candidate original image and categorized as forgery image after refine matching stage b Candidate original image and categorized as original image again after refine matching stage
In the MICC-F2000 dataset, the size of the patched region compared to the size of the whole image is on the average 1.12%. For both datasets MICC-F220 and MICC-F2000, the forged images are created by randomly copying a rectangular or square shape patched area and pasting it over the image. Before pasting the patched regions to the image, they are modified by applying a number of attacks such as rotation with different angles which ranges from 0o to 90o, scaling with
Multimedia Tools and Applications Table 1 Results of applying the proposed matching stage Datasets Metrics
MICC-F220
MICC-F2000
MICC-F600
SATS-130
TPR FPR FNR TNR ACC MCC CT (mm:ss)
89.09% 8.18% 10.91 91.82% 90.45% 80.94% 2:12
87.40% 12.35% 12.6% 87.35% 84.23% 72.34% 30:58
82.34% 29.09% 17.66% 70.91% 70.79 58.18% 14:30
80.47% 21.28% 19.53% 78.72% 69.89% 56.26% 3:20
different scaling factors (sx, sy) applied to x and y-axis, translation modification, and a combination of them. The details of ten attacks applied to MICC-F220 and fourteen attacks applied to MICC-F2000 are listed clearly in [3]. MICC-F600 is a much-challenged dataset because the manipulated patched regions have been post-processed before pasting it to the image to look well in a natural manner through the area in which they have been pasted. MICC-F600 dataset is a high-resolution dataset consisted of 600 images with about 3888 × 2592 pixels. It is divided into 448 original images and 152 forged images. These forged images constructed from four main types listed as follows: a) b) c) d)
38 images with one duplicated region and then apply translation. 38 images with two or three duplicated regions and then apply translation. 38 images with one patched region rotated by 30 degrees. 38 images with one patched region rotated by 30 degrees and scaled by 120%.
Finally, SATS-130 dataset consists of 96 images with variable resolutions varies from 1024 × 683 to 3264 × 2448 pixels, and it is divided into 48 original images and 48 forged images. These datasets achieve a wide variety of copy-move forgeries with different difficulties that greatly helps in testing the efficiency of the proposed algorithm.
4.2 Evaluation metrics To evaluate the proposed algorithm, a number of common evaluation metrics are employed. These metrics are listed as follows: True Positive Rate (TPR), False Positive Rate (FPR), False Negative Rate (FNR), True Negative Rate (TNR), Accuracy (ACC) and Matthews Correlation Coefficient (MCC). The Table 2 Results of applying the proposed two-stage CMFD algorithm Datasets Metrics
MICC-F220
MICC-F2000
MICC-F600
SATS-130
TPR FPR FNR TNR ACC MCC CT (mm:ss)
100% 1.80% 0% 98.20% 99.09% 98.20% 2:48
98.40% 6.35% 1.60% 93.65% 93.55% 83.39% 46:58
94.50% 11.35% 5.50% 88.65% 91.05% 80.79% 17:37
91.67% 20.83% 8.33% 79.17% 85.42% 71.39% 7:24
Multimedia Tools and Applications 100
89.09 100
87.4
98.4
94.5 82.34
80
91.67 80.47
60
TPR for matching stage only.
40
TPR for matching & refining stages.
20 0 MICC-F220
MICC-F2000
MICC-F600
SATS-130
Fig. 5 TPR values for matching stage only and TPR values for matching & refining stages
MCC is used for measuring the efficiency even if a number of original images are not equal to the number of forged images in the tested dataset [30], such as MICC-F2000 and MICC-F600. These metrics are computed by eqs. (1-6), respectively. TPR ¼ T P =ðT P þ F N Þ ¼ ð1−FNRÞ
ð1Þ
FPR ¼ F P =ð F P þ T N Þ ¼ ð1−TNRÞ
ð2Þ
FNR ¼ F N =ð F N þ T P Þ
ð3Þ
TNR ¼ T N =ðT N þ F P Þ
ð4Þ
ðT N þ T P Þ 100 ðT P þ F P þ T N þ F N Þ
ð5Þ
ACC ¼
ðT P T N Þ−ð F P F N Þ MCC ¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi 100 ððT P þ F P Þ ðT P þ F N Þ ðT N þ F P Þ ðT N þ F N ÞÞ
ð6Þ
where; a) True Positive (TP): is the number of forged images that were truly detected as forged images b) False Positive (FP): is the number of original images that were falsely detected as forged images c) False Negative (FN): is the number of forged images that were falsely detected as original images d) True Negative (TN): The number of original images that were truly detected as original images Furthermore, the Computational Time (CT) represents an important assessment factor that participates in evaluating the proposed method and comparing it with the traditional algorithms. 100 80 60 40 20 0
FPR for matching stage only.
8.18 1.8 MICC-F220
12.35 6.35 MICC-F2000
29.09 20.83 11.35 21.28 MICC-F600
FPR for matching & refining stages.
SATS-130
Fig. 6 FPR values for matching stage only and FPR values for matching & refining stages
Multimedia Tools and Applications 100 80 60 40 20 0
90.45 99.09
93.55 84.23
91.05 70.79
85.42 69.89 ACC for matching stage only. ACC for matching & refining stages.
M ICC-F220
M ICC-F2000
M ICC-F600
SA TS-130
Fig. 7 ACC values for matching stage only and ACC values for matching & refining stages
4.3 The results The results of applying the proposed algorithm using only the matching stage are illustrated in Table 1. For MICC-F220, MICC-F2000, MICC-F600, and SATS-130 datasets, the TPR values achieved are 89.09%, 87.40%, 82.34%, and 80.47%, respectively, while the FPR values attained are 8.18%, 12.35, 29.09%, and 21.28%, respectively. Furthermore, the proposed matching stage algorithm achieved ACC values of 90.45%, 84.23, 70.79%, and 69.89% and MCC values of 80.94%, 72.34%, 58.18%, and 56.26, for MICC-F220, MICC-F2000, MICCF600, and SATS-130 datasets, respectively. In addition, the processing time in (min: sec) for the MICC-F220, MICC-F2000, MICC-F600, and SATS-130 datasets are 2:12, 30:58, 14:30, and 3:20, respectively. Table 2, On the other hand, exhibits the results of performing the proposed two-stage algorithm. The TPR values are 100%, 98.40%, 94.5%, and 91.67%, and the FPR values are 1.8%, 6.35%, 11.35%, and 20.83% for MICC-F220, MICC-F2000, MICC-F600, and SATS130 datasets, respectively. Moreover, the ACC values are listed as 99.09%, 93.55%, 91.05%, and 85.42% while the MCC values are recorded as 98.20%, 83.39, 80.79%, and 71.39% for MICC-F220, MICC-F2000, MICC-F600, and SATS-130 datasets, respectively. Additionally, the computational time for MICC-F220, MICC-F2000, MICC-F600, and SATS-130 datasets are 2:48, 46:58, 17:37, and 7:24, respectively.
4.4 Discussion This work developed an automatic two-stage CMFD algorithm that is based on dividing the tested image into several objects, extracting SURF descriptors for each object, building an object’s catalog and computing the similarity among these objects. The algorithm is carried out through two stages, matching stage and refinement stage. The matching stage categorizes the candidate images into forgery or original and localizes the tampered regions for the forgery images. The refinement stage, on the other hand, examines the images that are classified as original images to guarantee their originality. 100 80 60
98.2 80.94
72.34
83.39
80.79 58.18
56.26
71.39
40 20 0
MCC for matching stage only. MCC for matching & refining stages.
M CC-F220
M CC-F2000
M ICC-F600
SA TS-130
Fig. 8 MCC values for matching stage only and MCC values for matching & refining stages
Multimedia Tools and Applications Table 3 Comparison between proposed algorithm and previously reported methods on MICC-F220 dataset
TPR FPR FNR TNR CT (mm:ss)
The Proposed Algorithm
Amerini et al. [3] Amerini et al. [5] Mishra et al. [31] Kaur et al. [19]
100% 1.80% 0% 98.20% 2:48
100% 8% 0% 92% 24:13
100% 6% 0% 94% 17:05
73.64% 3.64% 26.36% 96.36% 0:2.85
97.27% 7.27% 2.73% 92.73% N/A
The results indicate that the matching stage fails to discriminate between forged and original images. Yet, by applying the refinement stage, the performance of the proposed CMFD system improved significantly. As illustrated in Fig. 5, the TPR, using only the matching stage, has maximum and minimum values of 89.09% and 80.47%, respectively, which means that in the worst case nearly 80% of the tampered images were effectively detected. By putting on the refinement stage the TPR values were increased with maximum and minimum values of 100% and 91.67%, respectively, which means that in the worst detection scenario about 92% of the forged images were effectively identified. Another relevant result is the FPR, which has maximum and minimum values of 29.09% and 8.18%, respectively, using only the matching stage, as presented in Fig. 6. This shows that, in the worst case scenario, approximately 29% of the original images are incorrectly classified as forged. On the other hand, applying the refinement stage declined the FPR values to 20.83% and 1.8%, as maximum and minimum values, respectively, pointing out that about 20% of the original images, in the worst case, were misclassified in dataset SATS-130. Furthermore, Fig. 7 shows that a high ACC value is achieved when applying both stages. The matching stage achieved maximum and minimum ACC values of 90.45% and 69.89%, respectively. Whereas, fulfilling both stages extremely raised the maximum and the minimum ACC values to 99.09% and 85.42, respectively. This indicates that the refinement stage remarkably improved the proposed CMFD system performance. Finally, the MCC values obtained from different datasets are displayed in Fig. 8. Applying the matching stage returned an MCC value between 56.26% and 80.94%, while utilizing matching and refinement stages yielded an MCC value between 71.395 and 98.2%. This reveals the importance of the refinement stage as well as the effectiveness of the proposed twostage CMFD in detecting the tampered regions. In accordance with the previous results, the proposed two-stage CMFD showed a superior performance than one-stage CMFD (matching stage). The refinement stage succeeded in decreasing the missing detection rate, by unveiling all copy-move forgeries that have eluded the matching stage, most probably because they are pretty similar to the surrounding regions that may be thought part of it, or they are extremely small to impact on the SURF descriptors. Table 4 Comparison between proposed algorithm and previously reported methods on MICC-F2000 dataset
TPR FPR FNR TNR CT (mm:ss)
The Proposed Algorithm
Amerini et al. [3]
Amerini et al. [5]
98.40% 6.35% 1.60% 93.65% 46:58
93.42% 11.61% 6.58% 88.39% 312:18
94.86% 9.15% 5.14% 90.85% 180:15
Multimedia Tools and Applications Table 5 Comparison between proposed algorithm and previously reported methods on MICC-F600 dataset
TPR FPR FNR TNR CT (mm:ss)
The Proposed Algorithm
Amerini et al. [3]
Amerini et al. [5]
94.50% 11.35% 5.50% 88.65% 17:37
69.20% 12.50% 30.80% 87.50% 115:00
81.60% 7.27% 18.40% 92.73% 76:21
In addition, the proposed system evaluation using diverse datasets demonstrated that our method is reliable and robust in detecting copy-move forgeries comprising different transformations (e.g., resize and rotation) even if these operations are implemented jointly on the same image.
4.5 Comparison with traditional methods In this section, the results of the proposed algorithm are compared with that of the latest and strongest traditional CMFD algorithms. The algorithms Amerini et al. [3] and Amerini et al. [5] that performed on datasets MICCF220, MICC-F2000, and MICC-F600 had been implemented on the same machine which used for implementing the proposed algorithm. For the algorithm Mishra et al. [31], its hardware specifications are Intel core i5, 64 bits processor and operate by windows 8.1 with software MATLAB R2013a. For the algorithms Kaur et al. [19] and Christlein et al. [8] their hardware and software specifications not specified in their papers and they did not compute the computational time. Tables 3, 4, 5 and 6 summarize the comparison results between the proposed CMFD technique and the previously reported methods. The obtained out findings on MICC-F2000 and MICC-F600 indicate that our proposed algorithm outperform significantly the previously reported CMFD methods. For MICC-F220 dataset, although the methods stated by Amerini et al. [3] and Amerini et al. [5] have the highest TPR (100%) as the proposed algorithm, they achieved the FPR (8%) and (6%) while the proposed method attained the lowest FPR (1.8%). This implies that the proposed system is more capable to identify original images. For SATS-130 dataset, the proposed algorithm achieved the highest TPR (91.67%) compared to previously reported methods with a significant difference. This means that our system can efficiently detect copy-move forgeries. However, the suggested method attained the highest FPR (20.83%) which indicate that about 21% of the original images are misclassified. Moreover, in accordance with the computational time, the proposed system has the lowest processing time compared to other reported algorithms. Table 6 Comparison between proposed algorithm and previously reported methods on SATS-130 dataset
TPR FPR FNR TNR CT (mm:ss)
The Proposed Algorithm
Amerini et al. [3]
Christlein et al. [8]
91.67% 20.83% 8.33% 79.17% 7:24
67.13% 11.89% 32.87% 88.11% 47:00
79.17% 11.63% 20.83% 88.37% N/A
Multimedia Tools and Applications
5 Conclusion This paper presents a novel CMFD methodology that is based on segmenting the target image into different objects, and exploring the similarity among these objects. This method proceeds in two consecutive stages; matching stage and refinement stage. In the matching stage, the candidate image is categorized into forged or original, while the refinement stage aims to certify the originality of the image that is classified as original in the matching stage. Our findings point out that the refinement stage enhances significantly the overall system performance. Furthermore, the proposed algorithm shows effectiveness with different datasets under different cloning conditions whether single or multiple cloning. The achieved ACC for the MICC-F220, MICC-F2000, MICC-F600, and SATS-130 datasets are 99.09%, 93.55%, 91.05%, and 85.42%, respectively. In addition, the proposed CMFD algorithm remarkably outperforms the previous reported methods, regarding its performance and processing time. The experimental results confirm that the proposed algorithm offers very low computational time comparing with other existing algorithms. This low computational time resulted from SURF feature descriptor which is characterized by its small dimensional size and speed of extracting. Additionally, building the objects’ catalog, which contains all the objects in the tested image, facilitates the matching process. In the future work, Cloud-based Deep Learning may be applied to build a CMFD system. Additionally, mobile-based CMFD algorithm may be developed. Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References 1. Ali Qureshi M, Deriche M (2014) A review on copy-move image forgery detection techniques. MultiConference on Systems, Signals & Devices (SSD), 11–14 2. Al-Qershi OM, Khoo BE (2013) Passive detection of copy-move forgery in digital images: state-of-the-art. Forensic Sci Int pp 284–295 3. Amerini I, Ballan L, Caldelli R, Del Bimbo A, Serra G (2011) A SIFT-based forensic method for copy– move attack detection and transformation recovery. IEEE Trans Inf Forensics Secur 6(3) 4. Amerini I, Barni M, Caldelli R, Costanzo A (2013) SIFT keypoint removal and injection for countering matching-based image forensics. EURASIP J Inf Secur 8. https://doi.org/10.1186/1687-417X-2013-8 5. Amerini I, Ballan L, Caldelli R, Del Bimbo A, Del Tongo L, Serra G (2013) Copy-move forgery detection and localization by means of robust clustering with j-linkage. Signa Process Image Commun 28(6):659–669 6. Bay H, Ess A, Tuytelaars T, Van Gool L (2008) Speeded-up robust features (SURF). Comput Vis Image Underst 110:346–359 7. Birajdar GK, Mankar VH (2013) Digital image forgery detection using passive techniques: a survey. Digit Investig pp 226–245 8. Christlein V, Riess C, Jordan J, Riess C, Angelopoulou E (2012) An evaluation of popular copy-move forgery detection approaches. IEEE Trans Inf Forensics Secur 7(6):1841–1854 9. Costanzo A, Amerini I, Caldelli R, Barni M (2014) Forensic analysis of SIFT Keypoint removal and injection. IEEE Trans Inf Forensics Secur 9(9) 10. Derroll D, Divya B (2015) Image authentication techniques and advances survey, compusoft. Int J Adv Comput Technol 4(4), Volume-IV, Issue-IV 11. Dixit R, Naskar R, Mishra S (2017) Blur-invariant copy-move forgery detection technique with improved detection accuracy utilising SWT-SVD. IET Image Process 301–309 12. Elaskily MA, Aslan HK, Abd El-Samie FE, Elshakankiry OA, Faragallah OS, Dessouky MM (2017) Comparative study of copy-move forgery detection techniques. Intl Conf on Advanced Control Circuits
Multimedia Tools and Applications
13. 14.
15. 16.
17. 18. 19. 20. 21. 22.
23. 24. 25. 26. 27. 28. 29. 30. 31. 32.
33. 34. 35. 36. 37.
Systems (ACCS) Systems & Intl Conf on New Paradigms in Electronics & Information Technology (PEIT), Alexandria, Egypt Haouzia A, Noumeir R (2008) Methods for image authentication: a survey. Multimed Tools Appl 1–46. https://doi.org/10.1007/s11042-007-0154-3 Hashmi MF, Anand V, Keskar AG (2014) Copy-move image forgery detection using an efficient and robust method combining un-decimated wavelet transform and scale invariant feature transform. AASRI Conf Circ Sig Process (CSP 2014) pp 84–91 He L, Ren X, Gao Q, Zhao X, Yao B, Chao Y (2017) The connected-component labeling problem: a review of state-of-the-art algorithms. Pattern Recogn pp 25–43 Huynh TK, Le-Tien T, Huynh KV, Nguyen SC (2015) A survey on image forgery detection techniques, IEEE RIVF International Conference on Computing & Communication Technologies Research, Innovation, and Vision for Future (RIVF) Kang X, Lin G, Chen Y, Zhang E, Duan G (2012) Detecting tampered regions in digital images using discrete cosine transform and singular value decomposition. Int J Digit Content Technol Appl (JDCTA) 6 Kaur R (2016) Image forgery and detection of copy move forgery in digital images: a survey of recent forgery detection techniques. Int J Comput Appl 139(5) Kaur H, Saxena J, Singh S (2015) Simulative comparison of copy- move forgery detection methods for digital images. Int J Electron Electr Comput Syst IJEECS, ISSN 2348-117X, 4 Kaur R, Zail G, Kaur A (2016) A review of copy-move forgery detection techniques. Int J Comput Sci Inf Technol Secur (IJCSITS) 6(2) Le-Tien T, Huynh-Kha T, Pham-Cong-Hoan L, Tran-Hong A, Dey N, Luong M (2017) Combined Zernike moment and multiscale analysis for tamper detection in digital images. Informatica 41:59–70 Liu Y, Nie L, Han L, Zhang L, Rosenblum DS (2015) Action 2 activity: recognizing complex activities from sensor data. Proceedings of the Twenty-Fourth International Joint Conference on Artificial Intelligence (IJCAI 2015) Liu Y, Zhang L, Nie L, Yan Y, Rosenblum DS (2016) Fortune teller: predicting your career path. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI-2016) Liu Y, Nie L, Liu L, Rosenblum DS (2016) From action to activity: sensor-based activity recognition. Neurocomputing pp 108–115 Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110 Mahmood T, Nawaz T, Irtaza A, Ashraf R, Shah M, Mahmood MT (2016) Copy-move forgery detection technique for forensic analysis in digital images. Math Probl Eng 2016, Article ID 8713202 Mahmoud K, Abu-AlRukab A (2016) Copy-move forgery detection using zernike and pseudo Zernike moments. Int Arab J Inf Technol 13(6A) Mahmoud K, Husien A, Al-Rukab A (2016) Moment based copy move forgery detection methods. Int J Comput Sci Inf Secur (IJCSIS) 14(7) Maind RA, Khade A, Chitre DK (2014) Image copy-move forgery detection using block representing method. Int J Soft Comput Eng (IJSCE) 4(2):2231–2307 Matthews BW (1975) Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochim Biophys Acta Protein Struct 405(2):442–451 Mishra P, Mishra N, Sharma S, Patel R (2013) Region duplication forgery detection technique based on SURF and HAC. Hindawi Publishing Corporation. Sci World J Pandey RC, Singh SK, Shukla KK, Agrawal R (2014) Fast and robust passive copy-move forgery detection using SURF and SIFT image features. 9th International Systems Conference on Industrial and Information (ICIIS), Gwalior Prakash CS, Maheshkar S (2016) Detection algorithm for copy-move forgery based on circle block. Proc Int Conf Comput Vis Image Process pp 355–364 Redi JA, Taktak W, Dugelay J-L (2011) Digital image forensics: a booklet for beginners. Multimedia Tools Appl 51:133–162 Ryu S-J, Kirchner M, Lee M-J, Lee H-K (2013) Rotation invariant localization of duplicated image regions based on Zernike moments. IEEE Trans Inf Forensics Secur 8(8):1355–1370 Sreenivasu T, Sudha Vani G (2017) Copy-move digital image forgery detection techniques: a review. Int J Recent Sci Res 8(6):17389–17393 Sunil K, Jagan D, Shaktidev M (2015) DCT-PCA based method for copy-move forgery detection. ICT and Critical Infrastructure: Proceedings of the 48th Annual Convention of Computer Society of India, Volume II pp 577–583
Multimedia Tools and Applications
Mohamed A. Elaskily received the B.Sc. with honor degree and M.Sc. degree in computer science and engineering from Faculty of Electronic Engineering (FEE), Menuofia University, Egypt, in 2008 and 2013, respectively. Mohamed got the CCNA and CCNP certifications from CISCO. He registered for Ph.D. study in October 2014. He was worked as Demonstrator at 2008. In September 2009, he was worked in the Egyptian Universities Network. He was worked in many Network companies until 2014. Since 2014, he has been a Ph.D. researcher in the Department of Informatics, Electronics Research Institute (ERI), Cairo, Egypt. The major fields of his research interests are Computer Networks, Information Security, Ad-hoc security, Image Processing, and Digital Image Forensics.
Heba A. Elnemr is an Associate Professor at Electronics Research Institute, Cairo-Egypt. She received her B. Sc. degree, M. Sc. degree and Ph.D. degree in Electronics and Communications Engineering from Faculty of Engineering, Cairo University, Egypt. She has supervised several masters and Ph. D. students in the field of image processing. Her research interests include: pattern recognition, signal processing, biometrics, computer vision and image processing.
Multimedia Tools and Applications
Mohamed M. Dessouky received the B.Sc., (Hons.), M.Sc., and Ph.D. degrees from Faculty of Electronic Engineering (FEE), Menoufia University, Egypt, in 2006, 2011, and 2016, respectively. Since 2016, he has been a teaching staff member in the department of Computer Science and Engineering at FEE. He is a CISCO Academy Curriculum Lead for mare than 10 years. He is a CISCO Academy Curriculum Lead for more than 10 years. His research interesnt include artificial intelligence, image processing and computer network security.
Osama S. Faragallah received the B.Sc. (Hons.), M.Sc., and Ph.D. degrees in Computer Science and Engineering from Menoufia University, Menouf, Egypt, in 1997, 2002, and 2007, respectively. He is currently a Professor with the Department of Computer Science and Engineering, Faculty of Electronic Engineering, Menoufia University, where he was a Demonstrator from 1997 to 2002 and has been Assistant Lecturer from 2002 to 2007 and since 2007 he has been a Teaching Staff Member with the Department of Computer Science and Engineering, Faculty of Electronic Engineering, Menoufia University. Since 2015, He joined with the Department of Information Technology, College of Computers and Information Technology, Taif University, Al-Hawiya 21,974, Kingdom of Saudi Arabia. He is a coauthor of about 150 papers in international journals and conference proceedings, and two textbooks. His current research interests include network security, cryptography, internet security, multimedia security, image encryption, watermarking, steganography, data hiding, medical image processing, remote sensing, and chaos theory. Email:
[email protected],
[email protected], Mobile: 00966540923647, 00966548395560.
Multimedia Tools and Applications
Affiliations Mohamed A. Elaskily 1,2 & Heba A. Elnemr 3 & Mohamed M. Dessouky 2 & Osama S. Faragallah 2,4 1
Informatics Department, Electronics Research Institute (ERI), Cairo, Egypt
2
Computer Science and Engineering Department, Faculty of Electronic Engineering, Menoufia University, Menouf 32952, Egypt
3
Computers and Systems Department, Electronics Research Institute (ERI), Cairo, Egypt
4
Information Technology Department, College of Computers and Information Technology, Taif University, Al-Hawiya 21974, Kingdom of Saudi Arabia
