Multiple Frames Combination Versus Single Frame ...

Journal of Information Assurance and Security. ISSN 1554-1010 Volume 8 (2013) pp. 230-239 © MIR Labs, www.mirlabs.net/jias/index.html

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation Jinjuli Jameson1, Siti Norul Huda Sheikh Abdullah1, Nik Nur Aisyah Nik Ghazali1 and Nazri A. Zamani1, 2 1

Pattern Recognition Research Group, Center for Artificial Intelligence Technology, Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600, Bangi, Selangor D.E., Malaysia [email protected], [email protected], [email protected] 2

Digital Forensics Department, CyberSecurity Malaysia, 43300, Seri Kembangan, Selangor D.E., Malaysia [email protected]

Abstract: Generating high-resolution images or videos have become an essential need for digital image processing and analysis especially in the forensic field. Compressed and at low resolution video frames of common security surveillance videos are found to be very low in clarity and degraded with many noises, distortions, blurs, bad illumination and video compression artifact. This could interfere during image interpretation and analysis process. Using super resolution methods, high resolution image is obtained from a set of low resolution images, after it had undergone two main processes; image registration process based on Keren algorithm and image reconstruction process based on Projection onto Convex Set (POCS) on frequency domain. The validation process of output is done by calculating the Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR) value to show the comparison of image quality. The proposed combination methods were evaluated against other typical resampling methods and sparse representation method. The experimental results had shown that the sparse representation method has the highest PSNR mean value but our proposed combinatorial method is comparable to it as the difference of the mean is too small. Keywords: Super resolution, Projection onto Convex Set, Video Forensics, Peak Signal to Noise Ratio.

I. Introduction The aim of the enhancement process in video forensics analysis is to produce a good clarity quality with a good sharpness from a video exhibit. A sharp image could be defined as an image that looks like a real scene, a natural one. On the other hand, blurred image is defined as an image that has lost the high-frequency information, which often prevent from its interpretation or analysis. Blurred image is due to factors such as weather, motion and other interruptions during image acquisition. Also, when we zoom into an image or video frame, the necessary interpolation causes blur while compression causes artifacts. Image resampling causes the size of image to increase. The amount of pixel increases as the size of image increases. However, by referencing to the original image, the details of the resampled image could not be created. Thus, every

resampled image is prone to produce blurry issue [1]. Many typical resampling methods such as nearest-neighbor, bicubic, bilinear and lanczos fail to create clearer resampled image [2]. Capturing high resolution image from single or multiple low resolution images is a classic problem. The high resolution image with more pixels per area offers more detail scene than a low resolution image, thus, provides more valuable information in fields like medical imaging and pattern recognition in computer vision [3]. Super resolution (SR) is a term for a set of methods of resampling video or images. It works effectively with a single low resolution image or by combining several low resolution images of the desired scene There are two methods in processing super resolution image; a) using a single frame super resolution [4], [5] and b) based on multiple frames super resolution [2], [6]. Both methods have its own advantage in different environment. Single frame super resolution is based on the redundancy patches that occur in an image, while multiple frames super resolution works by aligning the sub pixels of all the input images to form a high resolution image. A single high resolution image can only be constructed if the low resolution frames are shifted with respect to the high resolution grid differently from each other and with sub-pixel increment. Each of the low resolution frame contains unique information which cannot be obtained from other frames, thus, this information will be exploited in order to obtain a high resolution image. The goal of the study is to develop a solution to the one of the image interpretation problems which is to generate a single, high resolution image from a series of low resolution images. This paper is an extension from our previous proposed work in [6]. In our previous work, it is proven that the combinatorial SR performs better than SR and the other typical single frame resampling methods. In this paper we further our research by evaluating our method performance with the state-of-the-art single frame super resolution method, the sparse representation that had been proposed in [5]. MIR Labs, USA

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation In this research, multiple frames SR method was applied to create high resolution image. We referred to Keren et al. [7] for motion estimation algorithm and we used Projection onto Convex Sets (POCS) for image reconstruction. Apart from that, we combined the super resolution method with other typical resampling methods in order to improve our findings. The rest of the sections are arranged as follows. Section II introduces related works. Section III mentions the description of single and multiple frames resampling methods, while Section IV describes the solution that we had used to solve the problem. Section V presents the experimental result and Section VI concludes the paper

II. Related Works Originally, the idea of super resolution was founded in 1984 by Tsai and Huang [8]. Recently, this approach has been applied in many research areas to tackle the issue of image resolution enhancement [2], [4], [5], [6]. In addition, detailed comparisons and evaluation have been also reported in [1], [3]. In general, SR algorithms can be categorized in two types which is reconstruction-based SR algorithms and learning-based SR algorithms. Reconstruction-based SR algorithm basically combine information from a sequence of low resolution (LR) images of the same scene in order to generate a high resolution (HR) image. Freeman et al. [9] had found a solution using one-pass algorithm that has the same capabilities as iterative Markov network. It is able to generate the missing high frequency content of a zoomed image based on training set using Nearest Neighborhood algorithm. He and Siu [10] combined single image super resolution with Gaussian process regression (GPR) to obtain a sharp image. Protter et al. had come up with a simple super resolution technique by applying denoising method called Nonlocal-Means algorithm (NLM) [11]. Other approaches to the SR problems include Maximum a posteriori (MAP) [12], iterative back-projection [13], POCS [2], [14], [15] and wavelet-based algorithm [16]. Meanwhile, learning-based SR algorithms work by extracting redundant high-frequency image information from training samples which contained known HR components. This approach has been actively studied in the past few years [4], [5], [17], [18], [19].

231

POCS is probably the most promising algorithm in SR image fusion. The initiative of implementing POCS for super resolution was first suggested by Stark and Oskoui [20]. The authors proposed a solution to SR problem based on POCS in the spatial domain. It works by considering the solution as an element on a convex set defined by the input LR images. For computational efficiency, Wheeler et al. applied POCS in the frequency domain [21]. Aguena and Mascarenhas [22] applied POCS technique and super resolution for multispectral image data fusion in order to obtain more flexible result. Super resolution has been implemented in several applications including to obtain more detail image of the earth and the galaxy such as used by NASA, to enhance video traffic surveillance, to render image and to convert standard TV recordings to a high definition TV format. In [23], SR technique has been applied in video analysis for forensic investigation.

III. Single Frame vs. Interpolation Methods

Multiple

Frames

In this paper we divided the resampling into two main methods, single frame and multiple frames. As shown in Figure 1, we then further classify them into typical resampling method and sparse representation. Meanwhile, for multiple frames method, it is divided into super resolution based resampling method and super resolution combination based resampling methods. Focusing on the single frame methods, we found five common resampling methods that we then used in our experiment. They are bilinear, bicubic, nearest neighbor, lanczos and bicubic-spline (or bspline). We also consider the sparse representation as a single frame method. Sparse representation is basically a technique that uses learning dictionaries for discriminative tasks such as image segmentation and classification. The dictionaries continue to update with each new training input [6]. It is part of super resolution but using single image as the input instead of using multiple input images. Single frame method tries to magnify the image without introducing blur. It manipulates the pixel of the LR image to create the HR image. On the other hand, multiple frames super resolution method uses the information from all the LR images in the dataset to create the HR image. The LR images are of the same scene.

Figure 1. The single frame and multiple frames resampling methods

232

Jameson et al. The unknown signal

IV. Multi Frames Super Resolution There exist many super resolution methods in the literature. The general idea behind this relationship of high and low resolution images is simple. Imagine O is the original high resolution image and I is the low resolution input image that we obtained. Then we can observe that:

A. Registration using Keren algorithm Keren et al. algorithm works by looking at the frames as two different functions; f and g .Function f is the reference image and function g is the low resolution images. The following relation holds for the horizontal shift a , the vertical shift b , and the rotational angle around the origin  :

(2)

cos 

to the first terms in the

g ( x, y)  f ( x  a  y  x 2 / 2, y  b  x  y 2 / 2) (3) By expanding f to the first term of its own Taylor series, it gives the first order equation:

g ( x, y )  f ( x, y )  ( x  a  y  x / 2)f / y

f k 1  TCm , TCm1..., TCifk

(6)

i  PCi  I  is the projection operator projecting signal to convex set Ci and i is the

where TCi  I 

PCi that

multiplier factor with value of 0 < λi < 2. The low resolution image g ( x, y ) can be modeled as a high resolution image

f ( x, y) that experienced shift

S , S  x

y

and

had

undergone degradation process by the point spread function

(PSF) h  x, y  and addition of blur N  x, y  . It can be defined as:

g ( x, y)  h( x, y) f ( x  S x , y  S y )  N ( x, y)

g ( x, y)  f ( x cos   y sin   a, y cos   x sin   b)

and

and their respective projection operators Pi for every

(1)

where the matrices B and D represent blurring and downsampling respectively, and n is the noise in the generation of I from O. In our experiment, to obtain a high resolution image from multiple images, we had implement two previous algorithms; Keren [7] and POCS [11] and add resampling processes to increase the performance.

sin 

Ci  H , i  1, 2,...m and f  C0  im1 Ci , given that the intersection C0 is nonempty. Provided the constraint sets C

f ( x, y) ,will generates:

I  BDO  n

By expanding the Taylor series:

f ( x, y) and convex sets Ci is assumed

to be an element of an appropriate Hilbert space. Thus,

(7)

From (7), convex set Ci is defined as:

Ci  { f :| g ( x, y)  h( x, y) f ( x, y) | N ( x, y)}

(8)

Then, the solution of (8) is an iteration process of the orthogonal projection to convex set that has been determined by the constraints of the noise from that particular low resolution image. Thus, the projection operator is replaced into (6) until the equation (9) has finally obtained:

Fk 1  f k  i  gi – hi ' f k / || hi || hi 2

(9)

2

(b  x  y 2 / 2)f / y

(4)

where g i is the ith element of vector g ( x, y ) and hi ’ is the ith row of matrix

While the error function between g and by a and

f after translation

b and rotation at  can be approximated by

E (a, b, )  

[ f ( x, y )  (a  y  x 2 / 2)f / y  (b  x  y 2 / 2)f / y  g ( x, y )]2

h( x, y) .The remaining error from

gi – hi ' f k is used to modify the solution after the iteration to

(5)

where the summation is over the overlapping of f and g . This whole process is repeated until an image of one pixel is reached. B. Reconstruction using Projection onto Convex Sets (POCS) Next, we employ POCS for image reconstruction process. POCS algorithm is simple yet could give detailed image information. This algorithm is an approach by doing repeated iterations using information from low resolution images [6].

and satisfy the cross section of the convex set. Figure 2 shows our proposed super resolution combination method framework. After preprocessing the input images, the LR images were registered using Keren algorithm and then the images were reconstructed using POCS. Consequently, we also run resampling process on the LR images. We choose five resampling methods which are bilinear, nearest neighbor, lanczos, bspline and bicubic. The outputs of the resampling processes are used as reference image in the next step of SR process. Then, each image were evaluated and compared. Finally, in order to improve the performance of our super resolution method, we then iteratively processed the resampled output images into the super resolution process. The outputs of this process are HR images exceed the quality of normal super resolution and resampled images.

MIR Labs, USA

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation

233

Figure 2. The proposed super resolution combination framework

(a)

(b)

(c)

(d)

(e)

(f)

Figure 3. Original image; (a) Person 1, (b) Person 2, (c) Building , (d) Building 2, (e) Vehicle 1 and (f) Vehicle 2

MIR Labs, USA

234

Jameson et al.

V. Experimental Result The super resolution combination methods described above were tested with three different images in terms of type, size, and background as shown in Figure 3. The first row of image in Figure 3(a) was obtained from a closed circuit camera (CCTV) of a person. This image dataset were obtained from the Digital Forensic Department of CyberSecurity Malaysia. Figure 3(b) is a face of a person we acquired from our research group face database. Meanwhile, Figure 3(c) and 3(d) are of a building and the third row of image, Figure 3(e) and 3(f) is of vehicles where the main focus is on the license plate. The image was captured using IP camera. Figure 3(c), 3(d) and 3(e) were obtained from an open source. These images are significant in forensic cases, where these images help as benchmarked for suspect’s face recognition cases, robbery and arson in buildings and also stolen vehicle and kidnapping cases. In the first step, we generate our dataset of low resolution (LR) images from the dataset given. In order to do this, we apply downsampling method on the image until it reached to 50 percent or in other words, we decrease the image resolution substantially. Then, we apply interpolation by a factor of two to the image using the typical resampling methods, sparse representation, super resolution and combinatorial super resolution. Table I shows the size of original image, the downsampled input image and the output image. Image Size (pixel) Image

Original Image

Input Image

Output Image

Person 1

704x576

352x288

704x576

Person 2

96x148

48x74

96x148

Building 1

480x640

240x320

480x640

Building 2

384x288

192x144

384x288

Vehicle 1

720x576

360x288

720x576

Vehicle 2

640x480

320x240

640x480

Figure 5 to Figure 10 shows the comparison between state of the art resampling methods and our proposed super resolution combination methods on image Person 1, Person 2, Building 1, Building 2, Vehicle 1 and Vehicle 2 respectively. In order to measure and assess the quality of the images, we used a well known evaluation algorithm called the Mean Square Error (MSE) and Peak Signal to Noise Ratio (PSNR). MSE is the simplest and widely used quality measurement where it computes the error signal by subtracting the test signal from the reference, and then computing the average energy of the error signal [24]. The equation of MSE is defined as such: M

N

PSNR  20 log10 255 /  MSE

(11)

The higher the value of PSNR, the higher the quality of an image could be seen. We conduct several experiments by comparing the original image with the high resolution output image. The result of the experiment can be seen in Table II. Table II shows the result of PSNR value of each method from the lowest up to the highest. From Table II, we can observe that the mean of PSNR value reported by sparse representation method outperformed in comparison to other methods. As illustrated in Figure 4, sparse representation has the highest mean. SR Nearest Neighbor has the second highest mean with different mean of 0.01 to sparse representation. However, the performance of every resampling method varies depending on the type of the input image. As we can see in the experiment result in Figure 5 and Figure 6, sparse representation method produce a better quality for image Person 2 but give a lower quality for image Person 1 when compare to the combination of super resolution and nearest neighbor method. Some of the typical resampling methods generate a higher quality image than the combinatorial super resolution methods. Referring to Table II and Figure 6, bicubic resampling method has the highest PSNR value and better image quality among all the resampling methods. From the result obtained, it is also concluded that the multiple frames super resolution and the combinatorial of super resolution method performs better for the high contrast image such as image of Person 1, Building 1 and Vehicle 1. Meanwhile, the typical resampling method and the sparse representation method generate a better quality image for the low contrast image such as image of Person 2, Building 2 and Vehicle 2.

VI. Conclusion and Future Direction

Table 1. Specification of dataset

MSE  1/ MN ( y 1) ( x 1) [ I ( x, y)  I ' ( x, y)]2

decibels and is inversely proportional the Mean Squared Error. The PSNR equation can be defined as:

(10)

where I ( x, y ) is the value of pixel of the original image,

I ' ( x, y ) is the value of pixel of upsampled image and M and N is the image dimension. The PSNR is evaluated in

This paper is an extension of our previous research on super resolution combination method. As a conclusion, a combination of super resolution with state-of–the-art resampling methods was proposed in this paper. Keren registration algorithm and Projection onto Convex Sets (POCS) were used to generate the high resolution images. Based on our experiment result and evaluation, even though sparse representation outperformed the super resolution combination method but the mean is too small to be meaningful or significant. Thus, we can conclude that both the combinatorial of super resolution and sparse representation are comparable and the result is actually depend on the type of input image and its contrast ratio. The experiment has also proven that the combinatorial of super resolution and nearest neighbor method achieved better results than other typical resampling techniques. Our future work is to extend this framework into the complete CCTV forensic identification process based on enhanced Bayesian algorithm. This method also will be further absorbed for license plate recognition system (LPR), optical recognition system (OCR) and face recognition system.

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation

Acknowledgment This research uses some code of Patrick Vandewalle and has been funded by the Ministry of Science, Technology and Innovation (MOSTI) through ERGS/1/2011/STG/UKM/2/48 (TK) under the title of 2D-3D Hybrid Face Matching via Fuzzy Bees Algorithm for Forensic Identification. The research also would like to thank CyberSecurity Malaysia and Royal Police of Malaysia's Forensics Lab for their support of the research.

References [1] C. Papathanassiou and M. Petrou. “Super resolution: An Overview”. IEEE International Geoscience and Remote Sensing Symposium (IGARSS05), pp. 5655-5658, 2005. [2] N.A. Zamani, M.Z.A. Darus, S.N.H.S. Abdullah and M.J. Nordin. “Multiple-Frames Super-Resolution for Closed Circuit Television Forensics”. International Conference on Pattern Analysis and Intelligent Robotics (ICPAIR), pp.36-40, 2011. [3] S.C. Park, M.K. Park, and M.G. Kan, “Super Resolution Image Reconstruction: A Technical Overview”. IEEE Signal Processing Magazine 20(3), pp. 21–36, 2003. [4] D. Glasner, S. Bagon and M. Irani. “Super-Resolution from a Single Image”. IEEE 12th International Conference on Computer Vision, pp. 349-356, 2009. [5] N.A. Zamani, A.D.M. Zahamdin, S.N.H.S. Abdullah and M.J. Nordin. “Sparse Representation Super-Resolution Method for Enhancement Analysis in Video Forensics”. International Conference on Intelligent Systems Design and Applications (ISDA), pp. 921-926, 2012. [6] N.N.A N. Ghazali, N.A. Zamani, S.N.H.S. Abdullah and J. Jameson. “Super Resolution Combination Methods for CCTV Forensic Interpretation”. International Conference on Intelligent Systems Design and Applications (ISDA), pp. 853-858, 2012. [7] D. Keren, S. Peleg and R. Brada. “Image Sequence Enhancement Using Sub-pixel Displacements”. Computer Society Conference on Computer Vision and Pattern Recognition, pp. 742-746, 1988. [8] R. Y. Tsai and T. S. Huang. “Multipleframe Image Restoration and Registration”. Advances in Computer Vision and Image Processing, pp. 317-339, 1984. [9] W.T. Freeman, T.R. Jones and E.C. Pasztor. “Example-Based Super-Resolution,” IEEE Computer Graphics and Applications, pp. 56-65, 2002. [10] H. He and W.C. Siu. “Single Image Super-Resolution Using Gaussian Process Regression”. Computer Society Conference on Computer Vision and Pattern Recognition, pp. 449-456, 2011. [11] M. Protter, M. Elad, H. Takeda and P. Milanfar. “Generalizing the Nonlocal-Means to Super-Resolution Reconstruction”. IEEE Transactions on Image Processing 18(1), pp. 36-51, 2009. [12] S. Belekos, N.P. Galatsanos and A.K. Katsaggelos. “Maximum A Posteriori Video Super-Resolution Using A New Multichannel Image Prior”. IEEE Transaction Image Processing 19(6), pp. 1451-1464, 2010. [13] M. Irani and S. Peleg. “Motion Analysis For Image Enhancement: Resolution, Occlusion And Transparency”. Journal of Visual Communication and Image Representation 4(4), pp. 324-335, 1993.

235

[14] H. Huang, X. Fan, C. Qi and S. H. Zhu. “A Learning-Based POCS Algorithm for Face Image Super-Resolution Reconstruction”. Proceedings of the Fourth International on Machine Learning and Cybernetics, pp. 5071-5076, 2005. [15] C. Fan, J. Zhu, J. Gong and C. Kuang. “POCS Super-Resolution Sequence Image Reconstruction based on Improvement Approach of Keren Registration Method”. International Conference on Intelligent Systems Design and Applications (ISDA), pp. 333-33, 2006. [16] H. Ji and C. Fermuller. “Robust Wavelet-based Super-resolution Reconstruction: Theory and Algorithm”. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(4), pp. 649-660, 2009. [17] M. Elad and D. Datsenko. “Example-based Regularization Deployed to Super-resolution Reconstruction of a Single Image”, The Computer Journal 52(1), pp. 15-30, 2009. [18] J. Yang, Z. Wang, Z. Lin, S. Cohen and T. Huang. “Coupled Dictionary Training For Image Super-resolution”. IEEE Transactions on Image Processing 21(8), pp. 3467-3478, 2012. [19] X. Ma, J. Zhang and C. Qi. “Hallucinating Face by Position-patch”. Pattern Recognition 43(6), 2224-2236, 2010. [20] H. Stark and P.Oskoui. "High-resolution Image Recovery From Image Plan Arrays Using Convex Projections”, Journal of the Optic Society of America A 6(11), pp 1715-1726, 1989. [21] F.W. Wheeler, R.T. Hoctor and E.B. Barret, “Super-resolution Image Synthesis using Projection onto Convex Sets in the Frequency Domain”. International Society for Optics and Photonics in Electronic Imaging, pp. 479-480, 2005. [22] M.L.S. Aguena and N.D.A. Mascarenhas. “Multispectral Image Data Fusion Using POCS and Super-resolution”. Computer Vision and Image Understanding, pp. 178–187, 2006. [23] A. Gehani and J. Reif. “Super-Resolution Video Analysis for Forensic Investigations”. International Federation for Information Processing in Advances in Digital Forensics 242(3), pp. 281-299, 2007. [24] C.S. Varnan, A. Jagan, J.Kaur, D.Jyoti and D.S.Rao, “Image Quality Assesment Techniques on Spatial Doman”. International Journal on Science and Technology 2(3), pp. 177-184, 2011.

Author Biographies Jinjuli Jameson received her bachelor degree in information technology (intelligent system) from Universiti Kebangsaan Malaysia in 2012. She is currently pursuing her Master's degree in computer science at Universiti Kebangsaan Malaysia. Her research interests include pattern recognition, computer vision and robotics.

Siti Norul Huda Sheikh Abdullah obtained her computing degree from University of Manchester Institute of Science and Technology (UMIST), UK and her master degree specializing in artificial intelligence fron Universiti Kebangsaan Malaysia. She completed her PhD degree in image processing at Universiti Teknologi Malaysia. Holding an Associate Professor post in Faculty of Information Science and Technology, she also serves

236

Jameson et al. as a research fellow at Center for Artificial Intelligence Technology, UKM. Her research interests are pattern recognition, computer vision and robotics, and document analysis and recognition.

Nazri Ahmad Zamani is currently an active senior digital forensics analyst and researcher in Digital Forensics Department of CyberSecurity Malaysia. His expertise is in audio, photo, video and biometrics forensics. He is an AccessData Certified Examiner (ACE) and is currently pursuing his MSc. at Universiti Kebangsaan Malaysia in 2D+3D face recognition system in forensics. He has been in the field of R&D for more than 10 years now in the field of audio, image and video processing, industrial machine vision, robotic vision, biometrics, pattern recognition and artificial intelligent.

Nik Nur Aisyah Nik Ghazali received her bachelor degree in information and communication technology from Universiti Teknologi Petronas, Malaysia in 2009. She is currently pursuing the Master's degree in computer science at Universiti Kebangsaan Malaysia. Her research interests include pattern recognition and computer vision.

Image Resampling Method

BSpline Bilinear SR-BSpline Lanczos NN SR-Lanczos SR-Bilinear SR-Bicubic Bicubic SR SR- NN Sparse Representation

PSNR Mean Person 1

Person 2

Building 1

Building 2

Vehicle 1

Vehicle 2

PSNR

MSE

PSNR

MSE

PSNR

MSE

PSNR

MSE

PSNR

MSE

PSNR

MSE

23.26 23.35 25 23.42 23.41 25.1 25.14 25.76 23.44 24.35 25.93

306.959 300.663 205.627 295.856 296.538 200.946 199.104 172.616 294.497 238.825 165.989

23.04 23.67 18.84 24.7 22.94 19.82 19.63 20.12 25.44 21.8 21.8

322.91 279.31 849.34 220.33 330.43 677.77 708.08 623.53 185.82 429.62 429.62

24.47 24.54 25.3 24.66 24.56 25.4 25.8 25.17 24.67 25.8 25.81

232.32 228.6 191.9 222.37 227.55 187.53 171.03 197.73 221.86 170.64 170.64

22.5 22.54 22.62 23 22.93 23.43 23.12 23.51 23.12 24.29 24.29

365.66 362.31 355.7 325.9 331.19 295.18 317.02 289.79 317.02 242.15 242.15

23.37 23.48 25.3 23.53 23.5 25.37 26.71 25.41 24.66 26.71 26.71

299.28 291.8 191.9 288.46 290.46 188.83 138.7 187.1 222.37 138.7 138.7

23.8 23.95 25.12 24.09 27 25.26 25.12 26.62 27.29 26.4 25.1

271.07 261.87 200.02 253.56 121.36 193.68 200.02 141.61 113.82 148.96 200.95

23.41 23.59 23.70 23.9 24.06 24.06 24.25 24.43 24.77 24.89 24.94

23.56

286.471

25.23

195.02

24.68

221.35

23.7

277.38

24.8

215.32

27.74

109.42

24.95

Table II. Result of PSNR

Figure 4. Mean of PSNR

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation

237

Figure 5. Comparison between state-of- the-art methods methods on image Person 1; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

Figure 6. Comparison between state-of- the-art methods methods on image Person 2; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

238

Jameson et al.

Figure 7. Comparison between state-of- the-art methods methods on image Building 1; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

Figure 8. Comparison between state-of- the-art methods methods on image Building 2; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

Figure 9. Comparison between state-of- the-art methods methods on image Vehicle 1; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

Multiple Frames Combination Versus Single Frame Super Resolution Methods for CCTV Forensic Interpretation

Figure 10. Comparison between state-of- the-art methods methods on image Vehicle 2; (a) Bspline, (b) Bilinear, (c) SR-Bspline, (d) Lanczos (e) Nearest Neighbor, (f) SR-Lanczos, (g) SR-Bilinear, (h) SR-Bicubic, (i) Bicubic, (j) SR, (k) SR-Nearest Neighbor and (l) sparse representation.

239