An Efficient Clustering Technique for Cameras Identification using Sensor Pattern Noise ∗ Electrical
O. M. Fahmy∗ , Engineering Department, Future University in Egypt, Cairo, Egypt. omarfarouk
[email protected]
Abstract—In multimedia forensics, source-based image classification problem is a well known problem. It addresses the issue of identifying those images that were captured by a specific source camera from a given set of N data images. This is possible as each camera has a unique Photo Response NonUniformity (PRNU) that is registered in every image captured by this camera. Classification is made by clustering images having the same PRNU. In this paper, an efficient technique is proposed to group images originated by the same camera. It is based on evaluating the similarity functions between the estimated PRNUs as well as their Hu’s moment invariant information. The proposed technique has the ability to group images taken by a specific camera, even if the input images correspond to the same scene that was captured by different cameras. The classification algorithm is performed by means of a hierarchical clustering. Simulation examples are given to verify the superiority of the proposed technique compared to other techniques.
Keywords: Camera Identification; Clustering; Multimedia Forensics
I.
I NTRODUCTION
Nowadays, there is an increasing demand for a series of reliable digital forensic techniques for using in security and privacy applications. This is mainly due to ease of manipulating digital images. One of the most important forensic tasks is investigating the source cameras of a set of given digital images. This task has a number of applications, ranging from fighting against child pornography, to obtaining a PRNU estimate per camera, that can be later used for integrity checking purposes. The primary goal is to verify the image origin and distinguish between images that have been captured by different cameras. This is feasible as each digital camera has its Sensor Pattern noise. This Sensor Pattern noise uniquely identifies images captured by this camera. Pattern noise consists of two main components: the dominant part Photo Response Non Uniformity noise (PRNU) and the fixed pattern noise (D). The PRNU is extracted by means of a digital filtering operation [1]–[6]. Thus, to assign an image to its mother camera, we extract the PRNU of this image and compare it with the average PRNU’s of images taken by this camera. In view of unknown source cameras, many clustering algorithms were proposed to assign images to its mother cameras [7]–[9]. In [7] the clustering algorithm depends on setting a certain threshold to decide whether an image belongs to a specific cluster or not. When the correlation between a new image and the average fingerprint of the cluster exceeds the threshold, it is joined to this cluster. The process continues until no more images exceed the threshold. Then, the process is repeated for the remaining images to create
a new cluster. In [8], a classification system applied among images taken by unknown digital cameras was proposed. Firstly, PRNU is extracted and enhanced for each image. Secondly, an unsupervised classifier is applied to a training set of PRNUs to cluster them into classes (clusters), centroids of previously identified classes are used as the trained classifier to test a new data set. In [9], a clustering algorithm based on hierarchical clustering was proposed. However, in view of no prior knowledge about the images, these clustering techniques suffer from poor performance. In [10], a classification system based on Multi-Class Spectral Clustering MCSC was proposed. Such a technique has two main open issues: firstly, it shows performance variability due to its random initialization and, secondly, it needs a stop criterion to select the best number of K clusters. In [11], an improved version using Normalized Cuts criterion was proposed to overcome the problem of stop criterion. This technique suffers from the random initialization and a pre-defined threshold for the stop criterion problems. In summary, most of these techniques are not guaranteed to accurately classify unknown source images as simulation examples show. In this paper, a clustering algorithm based on combining hierarchical clustering with Hu’s moments [12] is proposed. It is performed in two steps. The first step, named coarse clustering. It is based on hierarchical clustering and silhouette coefficient computations. To improve the classification performance and overcome problems like very small silhouette coefficient values that hamper the ability of the coarse clustering stage in achieving correct classification, the second refinement clustering stage is proposed. The second clustering stage is based on applying Hu’s moment vector to images that fail the coarse clustering stage, in order to achieve correct classification. The paper is organized as follows: Section II, briefly summarizes the MCSC technique used for image clustering. In Section III, the proposed clustering algorithm that is based on Hu’s moments is described. Simulation examples based on natural images drawn from, [13] are given in Section IV. Finally, Section V concludes the paper. In this paper, capital letters will denote matrices, e.g. K while bold small letters denote vectors, e.g. x and small italic letters for variables. II.
M ULTI -C LASS S PECTRAL C LUSTERING T ECHNIQUE
This section briefly describes one of the most popular modern clustering algorithms, the MCSC technique based on Normalized Cuts criterion, [11]. It is summarized as follows: 1)
Given a set of images, set up a weighted graph
G = (V, E), the edge E connecting each pair of images (i, j) is weighted by a similarity matrix W (i, j). The similarity matrix W (i, j) is the normalized correlation between each pair of images (i, j), [9], [11]. The graph G is partitioned into two disjoint graphs A and B. The degree of dissimilarity between these two parts P is called the cut and defined as cut(A, B) = u∈A,v∈B W (u, v) and the normalized cut (Ncut) is N cut(A, B) =
cut(A, B) cut(A, B) + assoc(A, V ) assoc(B, V
normalized correlation values are grouped together in clusters whereas weakly correlated PRNUs are sent to an auxiliary set, that will be considered in the refinement step. The algorithm is described as follows: 1) 2)
(1)
where the assoc(A, V ) represents the total connections from nodes in A to all nodes in the graph (similarly for B). 2)
Coarse step: in this step, images whose PRNUs have high
Let x be a vector of dimension N = |V |, where xi = 1 if the node i belongs to A, and -1 otherwise. Let also the totalPconnection from node i to all other nodes be d(i) = j W (i, j) and D an N × N diagonal matrix with d on its diagonal. The optimal bi-partition of a graph is obtained by minimizing the normalized cut value, defined as:
3)
Estimate the PRNU K, for each image Xj in the Nsized image data-set as in [1]–[6] . At the start of classification, compute an N × N correlation matrix C = [ci,j ], i, j = 1, . . . N . The element ci,j =< Ki , Kj > where means the inner product and Ki , Kj , are zero-mean PRNU and i, j = 1, . . . , N . Evaluate the normalized correlation (similarity) matrix H = [hi,j ]. the element hi,j is given by ci,j hi,j = √ √ (4) ci,i cj,j
y = (1 +P x) − b(1 − x) d(i) P b = xi >0 d(i)
The first cluster L(1) is constructed by picking up the pair of images (m,n) that have largest PRNU correlation coefficient. The center of this cluster contains these two PRNUs of images (m,n), has the average n PRNU KL(1) = Km +K where Km , Kn are the 2 PRNU of images (m,n). Scan the PRNU of other images in the data-set. The normalized correlation matrix H is updated between a new PRNU of image j and cluster L(1) according to: cL(1),j hL(1),j = √ (5) √ cL(1),L(1) cj,j
Equation 2 can be solved using the generalized eigenvalue system
where L(1) is the cluster that contains images m & n n , KL(1) = Km +K 2
N cut(A, B) = miny
yt (D − W )y yt Dy
4)
(2)
where
xi , bj =< Λ(j) and Λc > where Λ(j) , is the Hu’s moments vector of the the image j that lie in the auxiliary array, ΛcL(r) is Hu’s moments vector of the centroid of the cluster L(r) and Λc is
Hu’s moments vector of all other clusters except the L(r). Again; the image j is assigned to the cluster having the most negative value of sj . IV.
E XPERIMENTAL R ESULTS
The following experiments have been performed to compare the proposed clustering technique with one of most recently published techniques, [11] as well as the classical technique [7], as they represent the general aspects of image clustering. Every PRNU K is perturbed by = 0.001 to allow Hu’s moment computations. It is worth mentioning that the proposed scheme doesn’t depend on the initial ordering of input images, as in every cycle it picks up the maximum correlated pair. The following two examples have been considered: Ex 1: In this example, a set of 80 images is randomly chosen from a set of 5 different cameras drawn from Dresden Image data base, [13]. The cameras are Nikon D70 (6 MP), two another different Nikon D70S (6 MP) cameras, Canon Ixus70 (7.1 MP), and Kodak M1063 (10.3 MP). First, the Maximum Likelihood Estimate (MLE) technique of [2], [3] is used for extracting the PRNU of each image. The clustering algorithm starts by assigning images using the clustering approach described in the coarse step. This step ends up with 7 images moved to the auxiliary array. Next, in the refinement step, images moved to the auxiliary array are clustered using the proposed Hu’s moments technique, [12]. Table I, compares the proposed clustering technique with the clustering algorithm in [7] as well as MCSC using Normalized Cuts in [11]. The comparison for each camera j is shown in terms of TPR and FPR (the ideal case TPR = 100% & FPR = 0%) where No. of images correctly assigned to a cluster j No. of images actually belong to this cluster j No. of images are erroneously assigned to a cluster j FPR(j) = No. of images actually belong to other clusters Figure 1 illustrates the TPR (left) & FPR (right) values of this simulation for each camera. The proposed technique takes an execution time 3 seconds against 1.5 seconds of the MCSC technique and 22 seconds for the classical technique. Although the proposed technique has slightly larger computation time than MCSC technique, yet it manages to better classify of images where the MCSC technique fails. The computations have been run on LENOVO G510 laptop PC, equipped with an Intel Core i5-4200M 64 bit CPU @2.50 Ghz, RAM 8 GB. Ex 2: In this example, 5 different sets consist of 500 images. Each set contains 100 images. They are also drawn from Dresden Image data base, [13]. The cameras are, Nikon D60 (10.2 MP), Nikon D200S (10.2 MP) cameras, Canon Ixus70 (7.1 MP), Kodak M1063 (10.3 MP) and Panasonic FZ50 (10.1 MP). Table II, compares the performance of the proposed clustering technique with the clustering algorithm in [7] as well as MCSC using Normalized Cuts in [11] in terms of TPR and FPR. Figure 2, shows TPR (left) & FPR (right) of this comparison. The results have shown that the proposed clustering technique has higher TPR and lower FPR, which indicates the efficiency of this technique over other proposed techniques. TPR(j) =
V.
C ONCLUSION
This paper proposes a robust efficient clustering technique for grouping a set of images belonging to a capturing camera. The technique doesn’t depend on images initial ordering. It only computes the similarity matrix once at the start and does not need any pre-specified threshold for clustering. In addition, it overcomes some of the shortcomings of other clustering algorithms and it yields an improved TPR values (more than 90 %). Besides, the proposed technique has the ability of identifying cameras capturing the same scene that previous clustering techniques fail to do. Further performance enhancement, especially in case of very weakly correlated images, can be achieved through incorporating Hidden Markov model in the clustering decision process which in turn may lead to improvement in the TPR & FPR ratio. ACKNOWLEDGMENT The author acknowledges the support of Prof. M. F. Fahmy, Electrical Engineering Dept., Assiut University, Egypt.
TABLE I.
Camera Camera Camera Camera Camera
1 2 3 4 5
MSCSC Technique in [11] TPR FPR 85% 5% 86.6% 1.5% 80% 3% 90% 8.3% 80% 7%
TABLE II.
Set Set Set Set Set
1 2 3 4 5
TPR & FPR VALUES FOR E XAMPLE 1 Classical Technique in [7] TPR FPR 55% 15% 60% 9.2% 66.6% 12.3% 65% 15% 50% 15.7%
Moments Proposed Technique TPR FPR 100% 1.6% 93.3% 0.00% 100% 0.00% 95% 3.3% 90% 5%
TPR & FPR VALUES FOR E XAMPLE 2
MCSC Technique in [11] TPR FPR 88% 2% 80% 2.5% 90% 1.25% 89% 2.25% 85% 1.5%
Classical Technique in [7] TPR FPR 58% 8% 45% 8.7% 50% 7.5% 48% 8.7% 35% 10%
Moments Proposed Technique TPR FPR 94% 1.25% 95% 1.25% 95% 0.75% 95% 0.75% 94% 0.5%
R EFERENCES [1]
[2]
[3] [4]
[5]
[6]
[7]
[8]
[9]
[10]
[11]
[12] [13]
J. Lukas, J. Fridrich, and M.Goljan, ”Digital camera identification from sensor pattern noise”, IEEE Trans. Inform. Forensics Security, vol. 1, no.2, pp. 205-214, Jun. 2006. M. Chen, J. Fridrich, M. Goljan, and J. Lukas, ”Determining image origin and integrity using sensor noise,” IEEE Trans. Inform. Forensics Security, vol. 3, no. 1, pp. 74-90, Mar. 2008. J. Fridrich, Digital Image Forensics, IEEE Signal Processing Magazine, pp. 26-37, Mar. 2009. M. Goljan, J. Fridrich, and T. Filler, ”Managing a large database of camera fingerprints”, in Proc. SPIE, San Jose, CA, vol. 7541, pp. 0801-08-12, Electronic Imaging, Media Forensics and Security XII, Jan. , 2010. C.-T. Li, ”Source camera identification using enhanced sensor pattern noise”, IEEE Trans. Inform. Forensics Security, vol. 5, no. 2, pp. 280287, Jun. 2010. X. Kang, Y. Li, Z. Qu and J. Huang, ”Enhancing Source Camera Identification Performance with a Camera Reference Phase Sensor Pattern Noise”, IEEE Trans. Inform. Forensics Security, Vol. 7, no. 2, pp. 393 402, Apr. 2012. G.J. Bloy, ”Blind Camera Fingerprinting and Image Clustering”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 532-535, Mar. 2008. C. T. Li ”Unsupervised Classification of Digital Images Using Enhanced Sensor Pattern Noise”, IEEE International Symposium on Circuits and Systems (ISCAS 10), pp. 3429-3432, 30th May - 2nd Jun. 2010. R. Caldelli, I. Amerini, F. Picchioni and M. Innocenti, ”Fast Image Clustering of Unknown Source Images”, IEEE International workshop on Information Forensics and Security (WIFS), pp. 1-5, 12-15 Dec., 2010. B. B. Liu, H. K. Lee, Y. Hu, C. H. Choi, ”On Classification of Source Cameras: a graph Based Approach, IEEE International workshop on Information Forensics and Security (WIFS), pp. 1-5, 12-15 Dec., 2010. I. Amerini, R. Caldelli, P. Crescenzi, A. Del Mastio, A. Marino, ”Blind Image Clustering Based on the Normalized Cuts Criterion for Camera Identification, Image Communication, ELSEVIER, pp. 1 - 13, 2014. M. K. Hu, ”Visual Problem Recognition by Moment Invariant”, IRE Trans. Inform. Theory, Vol. IT-8, pp. 179-187, Feb. 1962. Dresden Image data base ”http://forensics.inf.tu-dresden.de/ddimgdb/ ”.
Figure 1. TPR & FPR Comparison of Ex1 between MCSC technique and classical technique with the moments proposed Technique
Figure 2. TPR & FPR Comparison of Ex2 between MCSC technique and classical technique with the moments proposed Technique
