A New SVD-Based Image Quality Assessment Mohammad Esmaeilpour
Azadeh Mansouri
Ahmad Mahmoudi-Aznaveh
Department of engineering, Faculty of computer engineering Kharazmi University Tehran, Iran
[email protected]
Department of engineering, Faculty of computer engineering Kharazmi University Tehran, Iran
[email protected]
Department of electrical and computer engineering Shahid Beheshti University Tehran, Iran
[email protected]
Abstract—In recent years, many efforts have been performed in order to design an algorithm assessing perceptual image quality based on human visual system. Although some impressive metrics have been presented, full reference image quality assessment (IQA) is still a challenging issue. In this paper, we present a new SVD-based IQA method in which the structural similarity between the reference and distorted image is utilized as a key factor for measuring the imposed distortions. The experimental results show that the proposed algorithm can effectively evaluated the image quality in a consistent manner with human visual perception. Keywords—image quality assesmnet, Human visaul system, Singular value decomposition, M-SVD, R-SVD
I.
INTRODUCTION
Recently designing a good image/video quality metric has become one of the most important issues in the field of image/video processing. An image or video signal can be affected by a wide range of modifications/distortions such as transmission errors or compression. In many cases, the best judges for evaluating the quality of the modified/distorted signal are humans. In this case, the subjective quality assessment methods are reliable and accurate. However, this type of evaluation is very time consuming, cumbersome and thus cannot be applicable for real-time applications. As a result, our goal is to design a method evaluating the distortions which is correlated with human visual systems (HVS). Objective quality assessment methods can be divided into three categories: if an original reference signal is supplied against the test image, the model is considered as full reference (FR). Conversely, No-reference image quality assumes that just distorted image is available for evaluating. A third group, which is entitled as reduced reference, supposes that some information of the original image can be accessible. There are so many applications utilizing image quality assessment such as visual processing, image coding and information hiding. For instance, recently SSIM [1], as one of the well known image quality assessment method, has been utilized in performance optimization of h.264 video compression standard [2-3]. The most popular and widely used image quality measurements are MSE and PSNR. However, these two quality metrics cannot be reliable enough since the results achieved by
them are not well correlated with subjective evaluations. The evaluations of these two metrics are poor especially in case of structural distortion. In this paper, a new full reference image quality assessment is introduced in which the structural similarly is employed in order to evaluate the quality. In the next section, an overview of the most important relevant methods is described. In section 3, the proposed method is explained. The experimental results are given in section 4 and the conclusion is illustrated in section 5. II.
BACKGROUND
It is well-known that the traditional image quality metrics such as PSNR do not take the human visual system (HVS) into account. As a result, numerous methods have been proposed in order to quantify the perceived perceptual quality based on human visual system. In spite of presenting impressive metrics, full reference image quality assessment is still a challenging issue. UQI [4](Universal Quality Index) can be regarded as a pivotal scheme which affects subsequent methods such as SSIM [5], MS-SSIM [5], VIF [6] and IW-SSIM [7]. SSIM =
(2 μx μ y +c1 )(2σ xy +c 2 ) 2 2 2 2 ( μx μ y +c1 )(σ x σ y +c 2 )
(1)
where x and y are reference and distorted images. In addition, μ and σ represent mean and variance of the image intensity. Moreover, σxy stands for covariance of x and y. Finally, the constants c1 and c2 are employed to avoid instability. In [8] a theoretical study has been performed in which concluded that the SSIM value can be predicted form PSNR and vice versa. Another approach for image quality assessment is based on singular value decomposition. Shnayderman et. al. exploit the difference between the singular values of the reference and the distorted images [9]. This method employs only the singular values for quality evolution. Consequently, the structural information is less effective in this algorithm. To overcome this drawback, some efforts have been conducted to utilize the structural information for quality assessment [10-11]. By applying SVD, an image (A) can be decomposed into three matrices, a diagonal eigen value (S) and two orthogonal eigen vectors (U and V).
A
m ×n
=U
m ×m
×S
m ×n
T ×V n ×n
(2)
In other words, A can be deemed as the following summation: A
m ×n
k = ∑ Ui ×S i ×ViT , i=1
k =min ( m ,n )
(3)
where Ui and Vi illustrate the ith column of left and right singular vector matrices (U and V) respectively. Moreover; the ith singular value is represented by Si. To put it more simply, Ui×ViT can be regarded as the ith basis image of the original input and Si is its cooperation to construct the whole image. As a result, the matrix UVT can be considered as a collection of basis images of matrix A. In brief, the singular vector matrices mostly contain the structural information. To show the pivotal role of basis images in demonstrating the structure, the product of UVT is illustrated in Fig 1.
in which Dmid shows the mid point of the sorted Di. Regarding aforementioned points, mostly structural information can be achieved through singular vectors. In this case, singular values show the brightness or weight of the basis images in constructing the whole image. However, this method mainly concentrated on singular values and this fact can be regarded as neglecting the role of structural information. On the other hand, R-SVD[11] is utilized just structural information obtained from singular vector matrices. Since the original and the distorted images are not structurally very different, right singular vector of the original image is utilized in order to obtain matrix Uˆ and Sˆ . In other words, to extract the structural information, the distorted image is decomposed into the basis images utilizing the reference image structures. ˆ .V &, i = 1, 2, 3, ..., m Sˆ = & A i i
(6)
⎧ , Sˆi =0 ˆ = ⎪⎨ 0 if U i ˆ ⎪⎩ A.Vi / S i , otherwise
(7)
Then a referee matrix can be constructed as: ˆ R
m ×n
ˆ =U
m ×m
Λ
T V m ×n n ×n
( 8)
where Λ is a matrix with ones on the main diagonal and zeros elsewhere. In fact, the disparity of singular values of Rˆ (Referee matrix) from one shows the difference between the original and the distorted images. The elements of these singular values are called as di as it shows the deviation from one. Finally the R-SVD metric is illustrated as follows: a b Fig 1) a-Original image b) structural information The most important parts of the structural information lies in the first basis images. By increasing the index, more detailed information is added until complete image is constructed After illustrating the characteristics of the SVD, two SVD based image quality metrics are analyzed in order to investigate their pros and cons. As it is mentioned, M-SVD [9] utilizes the difference of singular values between the original and the distorted images for evaluating the rate of distortion. In this case, the original and the distorted images are decomposed into 8×8 non-overlapping blocks; then, SVD is applied on each block. The comparison between the singular values of the original and the distorted blocks is performed as follows: 8
D
j
=
∑ (s
i
− sˆi )
2
,
j = 1 : #blocks
(4)
i =1
in which si and sˆi are diagonal elements of the singular values matrix (matrix S ) of the original and the distorted image respectively. In addition, #blocks stands for number of blocks. Finally, M-SVD is defined as: #blocks D i − D mid ∑ i=1 M − SV D = #blocks2
(5)
R − SV D =
m 2 ∑ (d i −1) i =1 m 2 ∑ (d i +1) i =1
(9)
In this paper, we combine the singular values differences together with the singular vectors dissimilarities in order to introduce a more effective image quality metric. III.
PROPOSED METHOD
In the proposed scheme, the structural information extracted from singular vector matrices and the singular values are both exploited. In doing so, both the original and the distorted images will be decomposed into 8×8 non-overlapping blocks; then, SVD is applied on each block. Likewise [9] the distance between singular values of the original and the distorted block is calculated as shown in equation 10. Hence; for each block a quality factor relating to singular values is obtained which is referred to as F1 1 F = j
8
∑ (s
i
− sˆi ) 2 ,
j = 1 : #blocks
(10)
i =1
Similar to M-SVD, F1 employs the singular values differences to evaluate perceptual quality. As mentioned before, the SV’s represent the contribution weights of the basis images. Despite the fact that these values can somehow express the structural information, the M-SVD metric neglect the
singular vectors which more effectively represent image structure. Since human visual system mostly extracts structural information from a scene[1], in this paper the singular vector matrices are utilized to better consider the structural information. Owing to the fact that R-SVD exploits singular vectors and shows that this information has a better effect on perceptual quality, the proposed method makes use of this factor. Thus, a similar approach to R-SVD algorithm is adapted to enhance the results. In doing so, for each 8×8 block, the referee matrix is calculated. As a result, F2 is obtained for each block through equation 11. 8
2 j
F =
∑d
i
( j ) −1
j =1 8
∑ d i ( j ) +1
,
j = 1: #blocks
(11) (a)
j =1
Likewise equation 9, di shows the singular values of the referee matrix. Here, j represents the block number. It is clear that, the dynamic range of F2 is between zero and one. The best value, zero, is achieved for two identical images for both factors (F1 and F2). Finally, the proposed quality measure as the combination of these two factors can be written as: #blocks
Proposed Metric =
∑
Fi1 × Fi 2
(12 )
i=1
# blocks
In which zero illustrates the best value when the original and distorted images are the same. IV.
EXPRIMENTAL RESULTS
To evaluate the performance of the proposed method MATLAB R2011a is used. All experiments have been conducted using LIVE database (consist of 982 images) [6].
(b)
Two performance metrics have been adapted for this paper. The first is the linear correlation coefficient (CC) between DMOS (Difference Mean Opinion Score) and algorithm result after nonlinear fitting. The CC between x and y is as follows: ∑ ( x − x )( y − y ) 2 2 ∑ ( x −x ) ∑ ( y − y )
(13)
where x and y illustrate means of x and y. The higher the CC between an image quality metric and DMOS illustrates the better performance of the presented metric. The second performance evaluator is root mean square error (RMSE) between DMOS and algorithm score. RMSE ( x , y ) =
1 ∑ ( x − y )2 n
(14)
The smaller value of RMSE indicates better and more effective algorithm.
(c)
Scatter plots of the proposed score for five distortion types versus DMOS along with the best fitting curves are depicted in Fig. 2. As it is illustrated in Table 1 and Table 2, for three types of distortions; Jpeg, Jpeg2k and white noise, the proposed method in terms of both CC and RMSE outperforms the other schemes. In addition, by considering the overall RMSE, the superiority of the proposed method can be concluded. Comparing the proposed method with both R-SVD and MSVD indicates its better performance Table 1. RMSE after nonlinear regression
(d)
Proposed R-SVD M-SVD PSNR MS-SSIM
JP2K
JPEG
5.411 6.422 5.611 9.125 5.915
5.153 7.594 6.512 8.684 6.942
White Noise 3.721 12.281 4.564 8.360 4.412
Gaussian Blur 6.631 4.914 8.480 8.191 5.695
Fast Fading 6.424 7.214 6.452 11.364 6.065
Overall 7.741 10.882 10.302 12.365 9.260
Table 2. Linear correlation coefficient Proposed R-SVD M-SVD PSNR MS-SSIM
JP2K
JPEG
0.951 0.931 0.944 0.778 0.945
0.955 0.902 0.945 0.780 0.950
White Noise 0.981 0.689 0.929 0.769 0.892
V.
(e)
Gaussian Blur 0.913 0.949 0.837 0.756 0.902
Fast Fading 0.915 0.921 0.910 0.763 0.924
Overall 0.887 0.778 0.827 0.751 0.917
CONCLUSION
In this paper, we proposed a full reference image quality assessment based on singular value decomposition. To extract the structural information, the distorted image is decomposed to the basis images using the reference image structure. Consequently, the structural deviation of the distorted image can be evaluated. In addition, the singular values differences between the original and the distorted images are employed as a measure representing mostly the luminance distortion. Combining structural and luminance distortion leads to an effective quality metric conforms to human visual perception. The experimental results show appropriate performance of the presented algorithm. REFERENCES [1]
[2]
[3] (f) Fig. 2). Scatter plots of DMOS versus proposed method score by five types of distortions. (a) Overall data (b) jp2k, (c) jpeg (d) White Noise, (e) Gaussian Blur, (f) Fast fading
H. R. Sheikh, M. F. Sabir, and A. C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," Image Processing, IEEE Transactions on, vol. 15, pp. 3440-3451, Nov 2006. S. S. Channappayya, A. C. Bovik, and J. R. W. Heath, "Rate Bounds on SSIM Index of Quantized Images," Image Processing, IEEE Transactions on, vol. 17, pp. 1624-1639, 2008. H. Yi-Hsin, O. Tao-Sheng, S. Po-Yen, and H. H. Chen, "Perceptual Rate-Distortion Optimization Using Structural Similarity Index as Quality Metric," Circuits and Systems for Video Technology, IEEE Transactions on, vol. 20, pp. 1614-1624, 2010.
[4]
[5]
[6]
[7]
[8]
Z. Wang and A. C. Bovik, "A universal image quality index," Signal Processing Letters, IEEE, vol. 9, pp. 81-84, Mar 2002. Z. Wang, E. P. Simoncelli, and A. C. Bovik, "Multiscale structural similarity for image quality assessment," in Signals, Systems and Computers, 2003. Conference Record of the Thirty-Seventh Asilomar Conference on, 2003, pp. 1398-1402 Vol.2. H. R. Sheikh and A. C. Bovik, "Image information and visual quality," Image Processing, IEEE Transactions on, vol. 15, pp. 430-444, 2006. W. Zhou and L. Qiang, "Information Content Weighting for Perceptual Image Quality Assessment," Image Processing, IEEE Transactions on, vol. 20, pp. 1185-1198, 2011. A. Hore and D. Ziou, "Image Quality Metrics: PSNR vs. SSIM," in Pattern Recognition (ICPR), 2010 20th International Conference on, 2010, pp. 2366-2369.
[9]
[10]
[11]
A. Shnayderman, A. Gusev, and A. M. Eskicioglu, "An SVD-based grayscale image quality measure for local and global assessment," Image Processing, IEEE Transactions on, vol. 15, pp. 422-429, Feb 2006. A. Mahmoudi-Aznaveh, A. Mansouri, F. TorkamaniAzar, and M. Eslami, "Image quality measurement besides distortion type classifying," Optical Review, vol. 16, pp. 30-34, 2009. A. Mansouri, A. Mahmoudi-Aznaveh, F. TorkamaniAzar, and J. Jahanshahi, "Image quality assessment using the singular value decomposition theorem," Optical Review, vol. 16, pp. 49-53, 2009.
