each block we compute sharpness using Kurtosis of the. DCT. The final metric is the average sharpness of the blocks in the edge profile. The advantages of this ...
NO-REFERENCE SHARPNESS METRIC BASED ON LOCAL EDGE KURTOSIS Jorge Caviedes and 'Sabri Gurbuz (Intern) Philips Research USA, Briarcliff Manor, NY 10510; *Clemson University, Clemson, South Carolina 29634 jorge.caviedes@,philips.com, sabriek3,ces.clemson.edu
ABSTRACT Sharpness metrics that use the frequency spectrum of the image cannot separate the sharpness information from the scene content. Sharpness metrics that use spatial gradients of the edges of objects work only for comparisons among versions of the same image. We have developed a content independent, no-reference metric. In this approach, we create an edge profile by detecting edge pixels and enclosing them with 8x8 pixel blocks. For each block we compute sharpness using Kurtosis of the DCT. The final metric is the average sharpness of the blocks in the edge profile. The advantages of this method include robust combination of spatial and frequency domain information, flexibility in the edge detection component, and a normalized sharpness scale. The results show high correlation with subjective quality for sharpness-enhanced video.
1.
INTRODUCTION
The perception of sharpness is related to the clarity of detail and edge definition of an image. The relative sharpness of an image can be measured, in the spatial domain, by the definition of edges in comparison to a reference image; or, in the transformed domain, by the difference in high frequency energy associated with edges and fine details, also with respect to the reference. Those are the main two approaches to sharpness measurement, i.e. using either frequency domain information, or spatial domain information. From the frequency domain camp, Zhang er al. [7] published a procedure for periodic sharpness self-test of SEM. In that case, a drop in image sharpness indicates need for maintenance and calibration of the SEM. Their sharpness metric is based on a statistical measure known as multivariate kurtosis', applied to the distribution of the two-dimensional Fourier transform coefficients over the entire image. Such a measure is global rather than local, the information about the spatial coherency is not taken
'
Kurtosis is a measure ofpeakedness relative to the normal distribution.
0-7803-7622-6/02/$17.00 02002 IEEE
into account, and, most importantly, the measure is relative to a reference or target value. Using the spatial domain approach, Rangayyan et al. [6] have proposed a method based on edge profile acutance, which uses a root mean square gradient normal to the edges of objects. This algorithm depends on accurate region (or object) detection and it is aimed at sharpness measurements of multiple versions of the same image. In this paper we present a new approach to measuring sharpness, which does not require a reference image and yet uses the same scale for any image. This method was inspired by initial observations on statistical measures of the frequency distribution, namely kurtosis and skewness (i.e. asymmetric distortion relative to the normal distribution), which showed that although very precise to measure the relative sharpness of multiple versions of the same scene, skewness and kurtosis show a baseline associated with the specific scene content, i.e. the average value jumped when the scene changed (and the value range was also not uniform). The new method is a combination of the spatial and frequency domain approaches that overcomes the disadvantages of the frequency domain approach. This paper is organized as follows. Section 2 deals with the computation of the 2D Kurtosis in the DCT domain, Section 3 presents the approach to segmenting the edge profile, Section 4 describes the full sharpness metric algorithm, Section 5 presents results, and Section 6 provides conclusions and future research directions. 2.
COMPUTING 2D KURTOSIS OF THE DCT
Let D(u,v) be an 8x8 block DCT coefficient matrix. The distribution of the coefficients, excluding D(0,O). can be normalized to form a legitimate bivariate probability density function (PDF), p ( x ,y ) , where
CCp(x,y)=l
(1)
In the theory of probability, the classical measure of the non-gaussianity of a random variable, e.g. A,' is kurtosis. It is a measure of the departure of a probability distribution from the gaussian (normal) shape. The kurtosis measure is zero for a Gaussian variable, and greater than zero for most non-gaussian random variables. There are non-gaussian random variables that
111 - 53
IEEE ICIP 2002
have zero kurtosis, but they can be considered as very rare. Using the notation of Zhang et al. [7],for a given random variable x with meanpl,, and existing finite moments up to the fourth, the kurtosis is defined by:
/L;
kurt (x) = p2 = m 4
'
where m4 and m 2 are the fourth and second central moments, respectively; and, the k" central moment is defined by [SI:
(4)
E denotes the probability expectation or the mean value of a random variable. Furthermore, kurtosis has the following linearity property: If x, and x2 are two independent random variables, then it holds: P 2 ( X I
+ x2) =
PZ(a.xl) =
a:
P 2 ( X l ) +
4
(6)
P 2 ( X I ) ,
r
= EE(W -PIT
r-'(W-
(7)
where T denotes the transpose of a vector. Zhang [7] used this definition with p=2 and a random vector W=[X Y]' , applied to the Fourier transform of the entire image, as the sharpness measure of the still images of SEM. The actual equations are the following: = [ Y w +YO.< +ZY,,,+4P(PY,,, -Yu -Y,,,)l/(~-P*Y (8) where P2,l
Figure 1. Edge detection and creation of edge profile. The success of the sharpness metric does not depend strongly on the precision of the edge detector. A tunable definition of edge detection parameters is sufficient, e.g. the standard deviation parameter value for the Canny edge detector, should be fine enough to exclude weak edges often associated with MPEG blocking artifacts, low level noise, and to focus on strong and medium strength edges. The frequency spectra of the blocks in the edge profile is then used to calculate the sharpness measure, thus combining the spatial and the frequency domain approaches. 4. THE SHARPNESS METRIC ALGORITHM
3
c "
Yk.,
To focus on the sharpness of the edges, first we detect all edges stronger than a certain threshold, and then assign each edge pixel to the center of an 8x8 pixel block. Figure 1 shows the original and edge images, and examples of possible block assignments. The collection of all those blocks helps define the edge profile region. Edges may he detected using one of many options available; in this work we use a Canny edge detection onerator 121.
(5)
P2(x2),
wherea is a scalar. Let W be a p-dimensional random vector (i.e. W=[w,,y,..w,l) with finite moments up to the fourth, a n d p and be mean vector and covariance matrix of W, respectively. Mardia [3] proposed the p-dimensional kurtosis:
P,,,
The 2-D kurtosis is a sharpness indicator for the entire mxn image, or any region within. Thus, for a given 2-D image region&X,Y), e.g. an 8x8 pixel block, where m, n=0,1,.. .7, the corresponding 2-D DCT coefficients F(u,v) can be obtained, where U and Y are the spatial frequencies. F(u,v) is then normalized, after excluding F(O,O),to form the bivariate probability density function P(U.V). 3. EDGE PROFILE SEGMENTATION
m
c c x i - P,)"Y, - P J ' P ( X , > Y ,)
(9)
i = o ,=a
= ,=o
- P x ) ' P ( * , ) ) * " ( 2 (Yi - f l y ) * P(Yi))"* ,=o
4y= m X - P J y - P y ) I ,
Figure 2 shows the flowchart of the no-reference sharpness metric algorithm. It starts by creating an edge image for each input video frame. After the edge image has been obtained, the algorithm assigns each edge pixel to a position at the center of an 8x8 block. Then, the 8x8 DCT, F(u,v), is computed and normalized to obtain the PFD p(u.v).
(10)
I11 - 54
Next, the 2-D kurtosis calculation described in Section 2 is performed on each block of the edge profile, according to Equation (8). That value is the local kuttosis of the edge frequency profile and therefore it is a local measure of sharpness.
includes four original sequences (i.e. doll, dolphins, lawnmowerman, and meteo) and 5 sharpness enhanced versions of each. The highest sharpness values caused degradation. Subjective quality scores were obtained by using a panel of video experts in our lab. 3 . Original video plus noise and sharpness enhancement. The set includes three original video sources (same as case 1) plus 20 processed versions of them. Subjective scores were obtained in the same formal subjective test as case I
Get next frame
Create edge
t e 52
Sequence Doll
Assign block to edge
pixel
Get next edge pixel
Calculate DCT
Dolphins
Done ?
Compute PDF
Average kurtosis oyer image
Lawnmowerman
&
Calculate 2-D kurtosis of PDF
Meteo
Done?
c .
72nu7hi
Average kurtosis
Table 1. Description of test sequences.
oyer sequence
8 Figure 2. Block diagram of sharpness metric. Finally, when all edge pixels have been processed, the kurtoses are averaged over all 8x8 blocks. That average is the frame sharpness measure; it can be output per frame or image, or as a running average over a certain time window depending on the application. S.
Description Head of a blond doll with a coast city skyline, 50 frames. Original format 72Ox576i. Dolphins swimming in the ocean, water ripples, motion, sun reflection; 100 frames. Original format 72Ox576i. Garden scene, two men, lawnmower, flowers, grass; 200 frames. Original format 60Ox4OOp. Weatherman giving report in front of map; 36 frames. Original format
6.1 Original and sharpness enhanced video Figures 3, 4, and 5 show scatter plots of subjective score vs. sharpness for sequences doll, lawnmowerman, and dolphins. The Pearson’s correlation coefficient for the doll sequence is ,966, for lawnmowerman is ,751, and for dolphins is ,959.
Doll
RESULTS
We have tested the 2-D kurtosis of the edge profile as a sharpness metric for digital video sequences. Four sets of sequences have been used 1. Original and coded video that has undergone sharpness enhancement. The set includes 3 original sequences (doll, dolphins, lownmowerman; see descriptions in Table I), and 34 processed versions of them. The sharpness enhancement method is a peaking algorithm. Subjective scores were obtained in a ITU formal test, which used the recommendations [4]. 2. Original video plus broad sharpness enhancement including extreme levels. The set
I11 - 55
(
Y
e y
D
l
,
‘Z
8 ’
s
v 1 .
Sharpnrrr
Figure 3. Subjective score vs. sharpness for doll.
For the lawnmoweman sequence shown in Figure 4, large errors are observed at the endpoints of the range. When sharpness enhancement is too high, quality drops as a result of overshooting artifacts. And when quality is too low, it is mainly driven by coding artifacts.
Lawnmowerman
also in the high end of the range (in Figure 4) for the lawnmowerman sequence. 6.3 Sharpness enhanced noisy video In this set we observed that sharpness follows subjective quality as long as the noise level is not above a noticeable level.~Oncethe noise is too high, it becomes the dominant factor for overall quality, and the correlation with sharpness is not significant.
6. CONCLUSIONS AND FUTURE RESEARCH
P
s
v1
, 0 1
Sharpness
Figure 4. Subjective score vs. sharpness for lawnmowerman. For dolphins, although there is increased motion and water in the scene, the sharpness metric shows excellent correlation with the subjective scores.
Dolphins
We have proposed a new, no-reference objective sharpness metric algorithm based on averaged edge profile kurtosis. The proposed algorithm is a combination of the spatial domain edge profile acutance [6], and the kurtosis of the frequency spectrum [7] algorithms. We have shown very encouraging results for sharpnessenhanced video, using the peaking method, up to the point where excess sharpness renders images non-natural. When noise, blockiness, or other artifacts become dominant, the metric alone, or without the ability to compensate for those impairments, cannot be used to predict quality. However, it is possible to use our sharpness metric as a sub-metric of an overall quality metric, like the one we proposed in [I]. 7.
REFERENCES
[ I ] J.E. Caviedes, and J. Jung, No-Reference Metric for a Video Quality Control Loop, Proc. SCI 2001, Vol. 13, Part 2, Orlando, FL, July 2001, pp.290-295. [2] J. Canny, A Computational Approach to Edge Detection, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-8, No. 6, November 1986.
131 K.V. Mardia, Measures of Multivariate skewness and Kurtosis with Applications, Biometrika 57, pp. 519-530, 1970. [4] Methodology for the subjective assessment of the quality of television pictures, Recommendation IT(/-R BT.500-IO, 2000. [5] A. Papoulis, Probability. Random Variables and Stochastic Processes, McGraw-Hill, Inc., 1991. Sharpness
[6] R.M. Rangayyan, and S.G. Elkadiki, Algorithm for the Computation of Region-Based Image Edge Profile Jan. 1995 pp62-70. Acutance. J. of Elect. Imaging, 4( I),
Figure 5. Subjective score vs. sharpness for dolphins. 6.2 Broad range sharpness enhanced video The results for broad range sharpness (not included due to limited space) indicate that the sharpness metric follows quality up to the point where degradation occurs caused by excessive sharpness. This was observed before
[7] N.F. Zhang, M.T. Postek, R.D. Larrabee, A.E. Vladar, W.J. Keery, and S.N. Jones, Image Sharpness Measurement in Scanning Electron Microscope -Part 111, Scanning, 21, 1999, pp.246-252.
III - 56
