PYRAMIDAL PERCEPTUAL FILTERING USING MOON AND SPENCER CONTRAST R. Iordache
A. Beghdadi, P. Viaris de Lesegno
Tampere University of Technology Signal Processing Laboratory FIN-33101 Tampere, Finland email:
[email protected]
Universit´e Paris 13 L2TI-Institute Galil´ee FR-93430 Villetaneuse, France email: beghdadi,
[email protected]
ABSTRACT A perceptual multiresolution filtering method based on human perception models is proposed. The main idea is to detect and remove the irrelevant structures at different scales using the just-noticeable contrast notion and luminance adaptation. The processing is done in the Laplacian and Gaussian pyramid decompositions of the image. The consistency of the method is demonstrated on a gray-level image. 1. INTRODUCTION Modeling the human perception mechanisms has opened new perspectives in the analysis and processing of visual and audio signals [1]. The understanding and modeling of the human visual and auditory systems limitations in perceiving certain distortions have resulted into lower bit-rate coding and more efficient processing. These limitations lead to the notion of perceptual irrelevancy, which is intimately related to masking and detection [1]. It is well known that an image contains structures (information) that are irrelevant to the human observer. In this paper we use the just-noticeable contrast (JNC) as irrelevancy criterion [2]. We exploit a model of multiresolution decomposition to mimic the early processing stages of the human visual system (HVS) [3]. The interest of the perceptual filtering is to eliminate the irrelevant components of an image and to keep a simplified and, of course, relevant image representation. The detection effect is quantified using a contrast measure. A consistent contrast definition for complex images was proposed in [4], where the concept of local bandlimited contrast was introduced. The local bandlimited contrast was successfully used in image quality assessment [5]. In this paper it is used a local contrast definition based on the work of Moon and Spencer [6]. This approach is attractive because their model is justified by psycho-visual experiments on complex images. To account for the multichannel decomposition in HVS, a multiresolution visibility evaluation is done, by computing the contrast at each level
0-7803-6725-1/01/$10.00 ©2001 IEEE
of the Gaussian pyramid image decomposition. The remaining of the paper is organized in 3 sections. Section 2 describes the pyramid perceptual filter, section 3 discuss experimental results, while conclusions and future work are presented in the last section. 2. PYRAMIDAL PERCEPTUAL FILTERING The perceptual filtering is a nonlinear operation, which preserves the visible details and smoothes the non-visible structures. Ideally, a human observer should not make any difference between the original and the perceptual filtered images. The processing is done on luminance values. The image is filtered using a normalized low-pass contrast sensitivity function (CSF) [7], to account for the frequency contrast selectivity of the HVS. Then, a Gaussian and the corresponding Laplacian pyramids are constructed, the first to approximate the image at different resolution, and the second to ensure the reconstruction of the image at the original scale [8]. At every stage of the Gaussian pyramid, the visibility of the pixels is assessed, resulting into a visibility map, which is a bit map where 1’s represent visible pixels and 0’s represent non-visible pixels. The visibility map is used to modulate the information in the Laplacian pyramid, so that the reconstructed image from the Laplacian pyramid takes into account the visibility information at every resolution. 2.1. Gaussian and Laplacian pyramids The Gaussian pyramid is a set of low-pass filtered images , where is the original image and is a reduced version of , in that both resolution and sample density are decreased [8]:
REDUCE
!
(1)
The REDUCE operator is a low-pass filtering followed by down-sampling by a factor of two in both vertical and horizontal directions. As low-pass filter, we use a separable [8]. filter having the 1-D mask
146
$
&
$
(
&
$
$
The Laplacian pyramid is a sequence of error images , where is the difference between two levels of the Gaussian pyramid [8]:
*
*
for a ratio of between the pixel sizes of the window and the object, and with being the global luminance, equal to the mean luminance level of the image. We point out that in our approach the ratio between the sizes in pixels of the window and of the object is , but, for simplicity, we still use (7) to approximate the adaptation luminance. The visibility map of the image is defined as: rs
*
*
6
-
.
!
EXPAND /
q
t
!
*
(2)
u
where EXPAND is the reverse of REDUCE (1), i.e. upsampling by a factor of two in both vertical and horizontal directions, and then interpolating using the same low-pass filter as in REDUCE. The Laplacian pyramid is a perfect reconstruction multiresolution decomposition [8]:
6
VM u
(8) 6
where the Boolean operator VM compares the local contrasts (4) against the local JNC’s (6): @
@
=
>
f
R
=
>
:
u
:
=
>
S
w
;
:
(9) ;
otherwise ;
3
4
*
EXPAND
(3)
2.3. Perceptual Laplacian pyramid
*
y
*
y
*
y
The perceptual Laplacian pyramid is a sequence of images derived from the Laplacian pyramid (2), taking into account the visibility information obtained by applying the VM operator (8) to every level in the Gaussian pyramid (1):
2.2. Visibility map As previously mentioned, at each level of the Gaussian pyramid the visibility of each is assessed. As measure of visibility, we use a contrast measure inspired from Moon and Spencer contrast model [6]. By comparing the contrast at each pixel against a local visibility contrast threshold, one obtains the visibility map. Let be an arbitrary luminance levels image. The image is analyzed with a window. The object is the middle pixel, and the remaining 8 pixels constitute the immediate surround of the object. The contrast at pixel is defined as:
y *
8
=
VM
(10)
{
y
7
:
{
with the operator denoting the Hadamard (element-wise) product. The perceptual filtered image is reconstructed from the perceptual Laplacian pyramid in the same way the original image is reconstructed from the Laplacian pyramid (3):
6
7
*
>
;
@
=
>
=
>
6
:
;
:
=
(4) >
;
3
y y
6
C
:
;
=
C
:
=
:
4
(11)
>
where is the immediate surround luminance, calculated as the average luminance of the 8 neighbors of the , i.e. pixel 6
*
EXPAND
;
>
3. EXPERIMENTAL RESULTS
;
3
6
C
:
=
>
;
6 E
F
G
4
I
F
F
L
4
M
:
;
=
>
N
N
(5) P
The proposed perceptual filtering method is tested on the gray levels man image, of size (Fig. 2(a)). In our experiments we consider six levels of decomposition, . i.e. In Fig. 1 there are shown the visibility maps at 3 levels of the Gaussian pyramid ( ), for , when the perceptual reconstruction (Fig. 2(b)) is visually identical to the original image. The visibility maps are not evident to interpret, especially for the lower pyramid levels, but it can be seen that edges are always visible (marked by white in the visibility map), and majority of the pixels in smooth regions are not-visible (marked by black in the visibility map). We are interested to observe what happens to the percepis increased. We expect that tual reconstruction when the reconstructed image is visually identical to the original for small values of , and, as grows, first low contrast $
&
$
&
8
As visibility contrast threshold, we use the just-noticeable contrast (JNC) is [9]:
$
@
7
-T
T
U
V
T
F
\
]
^
=
>
b
d
_
T
T
X
Y
Z
[
6
N
`
a
:
;
@
=
>
f
=
>
.
R
=
>
S 6
a
:
;
6
C
:
(6) ;
T F
:
\
]
;
T
X
U
i
V
Z
[
Y
/ d T
b
F
\
] ^
F _
\
]
T
T X
Y
Z
[
X
N
6
@
a
j
Z
[
h
:
=
>
k
;
6
C
:
=
>
;
l
is the Weber-Fechner JNC, where imentally measured constant and luminance, which is given by [10]: 6
a
:
E
is an experis the adaptation
_
=
>
;
@
@
6
a
:
;
=
>
o
&
7
6
C
:
;
=
>
N
p
p
6
q
(7)
147
l
l
@
l
$
l
&
(a)
(b)
(c) s
t
(d) VM
(e) VM
(f) VM
s
t
Fig. 1. Gaussian pyramid images and the corresponding visibility maps for @
$
7
.
l
details are lost, and then more and more contrasted details disappear. This is indeed the case, and in Fig. 2 the perceptual reconstructions for three values of (0.02, 0.08, and 0.15) are shown, which correspond to visually different reconstructions. One cannot make the difference between the (Fig. 2(b)), original and the filtered image for since the calculated JNC’s (6) are equal to or lower than the (Fig. 2(c)) low contrast regions real JNC’s. At of the image are blurred, but the high contrast parts remain (Fig. unchanged. This effect is stronger for 2(d)), the low contrast regions being filtered out.
for which they still make no differthe largest value of ence between the original and the perceptual reconstruction, for a set of test images and a fixed number of pyramid levels. Another important issue is to construct an image quality assessor based on the proposed perceptual decomposition. In this respect, we are considering the possibility of adapting a masking model inspired from the existing frequencyselective masking models. Another potential application of the perceptual filtering technique is as a pretreatment for image analysis and compression.
4. CONCLUSIONS AND PERSPECTIVES
5. REFERENCES
The proposed multiresolution perceptual filter has a consistent dependence on , as it is experimentally demonstrated. Before using the filtering procedure in any application, and the depth of the pyramidal decomposition have to be adjusted. At a first approximation, the optimal value , i.e. the largest value for which an average for observer cannot differentiate the original from perceptual reconstructed image, is independent of . Therefore, the value of can by chosen by performing perceptual assessment tests, in which the subjects are asked to indicate
[1] N. Jayant, J. Johnston, and R. Safranek, “Signal compression based on models of human perception,” Proceedings of the IEEE, vol. 81, pp. 1385–1422, 1993.
@
@
l
l
@
&
l
E
@
l
@
@
$
l
l
@
l
@
@
[2] T. N. Cornsweet, Visual Perception, Academic Press, New York, NJ, 1970.
l
[3] D. Hubel and T. N. Wiesel, “Brain mechanism of vision,” Scientific Amer., vol. 241, pp. 150–162, 1979.
l
[4] E. Peli, “Contrast in complex images,” J. Opt. Soc. Amer. A, vol. 7, pp. 2032–2040, Oct. 1990.
148
@
(a) Original
&
(b) Perceptual filtered ,
@
l
@
(c) Perceptual filtered ,
l
l
E
$
(d) Perceptual filtered , @
l
Fig. 2. Perceptual reconstructions for three values of
[5] N. Damera-Venkata, T. D. Kite, W. S. Geisler, B. L. Evans, and A. C. Bovik, “Image quality assessment based on a degradation model,” IEEE Transactions on Image Processing, vol. 9, pp. 636–650, April 2000.
(
$
).
mid as a compact image code,” IEEE Transactions on Communications, vol. 31, pp. 532–540, April 1983. [9] F. Kretz, “Subjectively optimal quantification of pictures,” IEEE Transactions on Communications, vol. 25, pp. 1288–1292, 1975.
[6] P. Moon and D. E. Spencer, “The visual effect of nonuniform surrounds,” J. Opt. Soc. Amer. A, vol. 35, pp. 233–248, 1945.
[10] K. Belkacem-Boussaid and A. Beghdadi, “A new image smoothing method based on a simple model of spatial processing in the early stages of human vision,” IEEE Transactions on Image Processing, vol. 9, pp. 220–226, Feb. 2000.
[7] T. Mitsa, K. L. Varkur, and J. R. Alford, “Frequency channel based visual models as quantitative quality measures in halftoning,” in Proc. SPIE, Feb. 1992, vol. 3, pp. 390–401. [8] P. J. Burt and E. H. Adelson, “The Laplacian pyra-
149
