The D-Dimensional Inverse VectorGradient Operator and its Application for Scale-Free Image Enhancement Piet W. Verbeek and Judith Dijk Delft University of Technology, Faculty of Applied Sciences Pattern Recognition Group, Lorentzweg 1 2628CJ Delft The Netherlands
[email protected]
Abstract. A class of enhancement techniques is proposed for images in arbitrary dimension D. They are free from either space/frequency (scale) or grey references. There are two subclasses. One subclass is the chain: {Negative Laplace operator, Multiplication by Power (γ−1) of modulus-Laplace value, Inverse Negative Laplace operator} together with generalized versions. The generalization of the Negative Laplace operator consists of replacing its isotropic frequency square transfer function by an (equally isotropic) modulusfrequency to-the-power-p transfer-function. The inverse is defined accordingly. The second subclass is the chain: {Vector-Gradient operator, Multiplication by Power (γ−1) of modulus-gradient value, Inverse Vector-Gradient operator} together with generalized versions. We believe the Inverse Vector-Gradient operator (and its generalized version) to be a novel operation in image processing. The generalization of the Vector-Gradient operator consists of multiplying its transfer functions by an isotropic modulus-frequency to-thepower-(p−1) transfer-function. The inverse is defined accordingly. Although the generalized (Inverse) Negative Laplace and Vector-Gradient operators are best implemented via the frequency domain, their point spread functions are checked for too large footprints in order to avoid spatial aliasing in practical (periodic image) implementations. The limitations of frequency-power p for given dimension D are studied.
1
Introduction
Scale-free operators in image processing are operators that contain no reference length, reference frequency or reference grey value. A scale-free operation is based on a function of the form out = inpower . Examples are spatial differentiation, where the Fourier transform is multiplied by frequencypower and grey-value gamma-correction where
(
out grey = in grey
)γ
. In practice regularization may introduce a reference, but we
shall assume it to be far outside the range of interest. We shall apply scale-free operators to image enhancement in a three step way. First we use a scale-free derivative operator, next we process the derivative i.e. we apply a gamma correction (e.g. to reduce small derivatives probably due to noise) and then we
apply the (again scale-free) inverse of the derivative operator. The method can be similarly applied when the second step is replaced by a (noisy) transmission channel. Then the third step is reconstruction or de-emphasis. A similar approach has been proposed by Fattal et al. [3] for contrast compression of the logarithm of luminance in 2D images. They perform a numerical reconstruction, as they cannot guarantee integrability after the non-linear operation (cf. our gamma correction). We shall show this to be a minor problem in 2D images. A series of D-dimensional derivative operators and corresponding inverse operators is studied, for arbitrary dimension. The simplest are the Negative-Laplace operator (frequency power =2) and the first derivative along a line, which is anisotropic. The direction average of reconstructions from the latter is isotropic. The (vector-)gradient is also isotropic. So far no reconstruction from the vector-gradient was available. We shall prove that the direction average of reconstructions from the first derivative along a line is equivalent to a reconstruction from the vector-gradient. Thus we have found the inverse vector-gradient operator. Like the Negative-Laplace operator and its inverse also the vector-gradient and its inverse can be generalized to arbitrary frequency power and be used for enhancement or pre-emphasis / de-emphasis. For frequency to the power zero the generalized vector gradient is a multidimensional Vector Hilbert Transform as discussed by Felsberg and Sommer [4].
scene
lens
ori
der
a
proc
screen
scene
lens
ori
lens
ori
b
COMPUTER
proc
+
screen
scene
lens
int
d/dY
pre-emphasis
channel / recording
TRANSMITTER
int
d/dX scene
int
de-emphasis
screen
+
screen
RECEIVER
der1
proc
der2
proc
der3
proc
ori
low
c
COMPUTER
d
COMPUTER
Fig.1. Related methods (ori=image of the scene, der=derivative, proc=(nonlinear) point operation, int=integrating reconstruction). a) Derivative enhancement. b) Pre-emphasized transmission. c) Vector Gradient enhancement. d) Granlund enhancement [1].
2
Related methods
The derivative methods (fig.1a) are high emphasis methods like the ones used in the (one dimensional) Dolby system for audio recording and in coding systems for signal transmission (fig.1b). Such systems lack the vector approach of the vector-gradient method (fig.1c). Indeed, the vector approach is present in the enhancement methods of the Granlund school [1]. The Transfer Functions of Granlund’s directed derivative filters are carefully made to add to an isotropic sum. The essential difference with the
vector-gradient method is that no integration step is applied. Instead the low frequencies are removed before processing and reinstalled in the last step (fig.1d). 3
Reconstruction from the Directed Line Derivative
A simple model for derivative and integration is the single line 1D model (fig.2a). The derivative along an X’-direction (direction vector n ) can be integrated along that direction starting from a zero reference rim around the image, which may be extended to infinity. Differentiation followed by integration yields the identity operator, such that the output image equals the input image, O = I 0
On ( X ',, Y ') =
dx ' [ dI ( x '+ X ',, Y ') / dx ']
∫
X 'ref − X '
Expressed in the usual image coordinates X ≡ ( X ,, Y ) 0
On ( X ) =
4
∫ dx ' n i ∇I ( x ' n + X )
−∞
Reconstruction from the Vector Gradient
There is no natural preferred X’-direction. Hence an average of directed line reconstructions is a better (more transmission noise robust) method (fig.2b)
O( X ) =
1
∑1
∑ On ( X ) n
n
x = x ' n,
with
=
1 S ( D)
∞
x
∫ d Ω ∫0 dr −r i ∇I ( x + X )
r ≡ x = − x ' and full solid angle S ( D) = ∫ d Ω
where S ( D = 2m) = Dπ m / m!, S ( D = 2m + 1) = Dπ m 2 D m!/ D ! e.g. S (1) = 2, S (2) = 2π , S (3) = 4π . Y’
Y X’ (X,Y) grad
gra inte
tion
th pa
reference grey
n
X
reference grey
reference grey
Fig.2. 1D derivatives and integration. a. Single oriented line. b. Average of many lines in different orientations. c. Average of many lines with limited range integration footprint.
Moreover, as
∫
image
d x f ( x) ≡
∫∫
image
dx..dy f ( x) = ∫ d Ω ∫ dr r D −1 f ( x) ⇒
∫ d Ω∫ dr g ( x) = ∫
image
dx
g ( x) r D −1
the average of directed line reconstructions can be written as a sum of convolutions D −D Xk 1 ∂I ( X ) O( X ) = − dx x x i ∇I ( x + X ) = ∑ ∗ ∫ D S ( D ) image ∂X k k =1 S ( D ) X In the frequency domain this can only correspond to the identity D u −2π iuk O (u ) = ∑ 2π iuk I (u ) ≡ I (u ) based on i u ≡ 1. 2 2 k =1 (2π u ) u
{
}
Thus we have found that the reconstruction corresponding to the vector gradient operator, the inverse vector-gradient operator, is a sum of axis-wise convolutions with the (Integration) Point Spread Functions (IPSF’s) Xk with R ≡ X . S ( D) R D The (Integration) Transfer Functions (ITF’s) corresponding to these IPSF’s are −2π iuk with ρ ≡ u . (2πρ ) 2 Note that only D convolutions are needed to get the average over all directions. The vector gradient model is seen to have a Derivative Transfer Function with characteristic power of frequency p=1. We generalize the vector gradient to arbitrary p inserting an isotropic factor (2πρ)p−1.
5
Derivative operators and their inverse operators (integrations)
According to the chain (cf. fig. 1c)
{Derivative operator, Non-linear Point Processing, Inverse Derivative operator} a variety of enhancement methods can be constructed. Table 1 gives a survey of the derivatives to be chosen. The integrations are up to zero-derivative terms. The derivation of the Fourier transforms is given in [2]. The functions α and β allow noninteger powers of frequency. They are based on the Γ-function and defined in Table 2. Integrations can be of limited range (IPSF truncated by Gaussian, fig. 1c). In implementations we avoid the infinities of IPSF’s or ITF’s using Cauchy functions (R2+T2)−1 instead of R−2 and (ρ2+τ2)−1 instead of ρ−2. Cauchy functions are related to electronic-integration where τ = resistance × capacity.
Table 1. Derivatives and their inverses: reconstructions by integration. D is the dimension. p is the characteristic (also non-integer) power of frequency in the Derivative Transfer Function. Note that for p=0 the Generalized Vector Gradient and its Inverse are equal to the Vector Hilbert Transform and its Inverse as discussed in [4].
Derivative Type
Derivative DPSF
Derivative DTF
R≡ X
ρ≡ u
Directed Line Der. Negative Laplace
n
d / dX ' −∇
(Negative Laplace)n
{−∇ } R
Vector Gradient
∇
α (− p, D)
−p, D+p ≠ 0,−2,−4,
VG (NL)n Generalized VG
− p−D
n
{ }
∇ −∇ R
n
(2π iu ')
− ( D + p ) −1
β ( − p, D ) X
−p+1,D+p+1 ≠ 0,−2,−4,
Modulus of Grad
n
( 2πρ )−2
R
2− D
( 2πρ )2 n
( 2πρ )−2 n
R
2n−D
( 2πρ )
( 2πρ )
R
p−D
α ( p, D)
R
−D
β (1, D ) X
R
−D
R
− ( D − p ) −1
( 2πρ )
n
( ∫ dX ' )
−n
( 2πρ )2
p
2π iu 2
Integration IPSF R≡ X
(2π iu ')
2
2
Generalized NL
n
Integration ITF
2n
2π iu
( 2πρ ) p −1 2π iu
∇I
α (2, D )
D−2 ≠ 0 α (2 n, D )
D−2n ≠ 0,−2,−4,
−p
p, D−p ≠ 0,−2,−4
− ( 2πρ )
−2
− ( 2πρ )
−2 n − 2
− ( 2πρ )
− p −1
2π iu
β(1,D)=1 / S(D) 2π iu
β (2 n + 1, D ) X
D−2n ≠ 0,−2,−4, 2π iu
β ( p, D ) X
p+1,D−p+1 ≠ 0,−2,−4, Grey-weighted Distance Transform
Table 2. The functions α and β that allow non-integer powers of frequency.
α ( p, D ) ≡ 2− p π − D 2 Γ ( ( D − p ) 2 ) Γ ( p 2 ) , −p
β ( p, D) ≡ 2 π
−D / 2
Γ(( D − ( p − 1)) / 2) / Γ (( p + 1) / 2),
p, D − p ≠ 0, −2, −4, p + 1, D − ( p − 1) ≠ 0, −2, −4,
( D − ( p + 1))α ( p + 1, D ) = β ( p , D ) = α ( p − 1, D ) ( p − 1)
6
Limited range integration
Although the generalized (Inverse) Negative-Laplace and Vector-Gradient operators are best implemented via the frequency domain, their point spread functions are checked for too large footprints in order to avoid spatial aliasing in practical implementations (with periodic images, like in discrete and fast Fourier transforms). We emulate limited range by truncation of the IPSF by a Gaussian window W ( X ) , with standard deviation σGauss , e.g. for the Vector Gradient model (fig.1c) O( X ) =
D
∑ {W ( X ) X k S −1 ( D) R − D } ∗ {∂I ( X ) k =1
∂X k
}
In the frequency domain this reads as applying a smoothed ITF W (u ) ∗ [(2πρ ) −2 2π iuk ] . Also in the other models truncated integration is tantamount to a smoothed ITF, i.e. W (u ) ∗ (2πρ )−2 for the Negative-Laplacemodel, W (u ) ∗ (2πρ ) − p for the Generalized NL-model and W (u ) ∗ (2πρ ) − p −1 2π iuk for the Generalized Vector Gradient model. How do the different derivatives react to truncated integration? In our twodimensional experiment (D=2) the GVG integration allows IPSF truncation without problems whereas the GNL integration is very sensitive to truncation of its IPSF. The reason might be the following. In the frequency domain the GNL-ITF (ρ2)−p/2 (or in practice (ρ2+τ2)−p/2 ) has a (very) sharp peak at zero frequency (0,0) which is blunted by the smoothing. The GVG-ITF components (ρ2)(−p−1)/2 (iuk) have zero crossings at u1=0 and u2=0 respectively, the zero value of which is not affected by the smoothing.
7
Derivative enhancement
The different types of derivatives constitute as many image representations. In each representation one can apply a non-linear operation. In particular a non-linear function can be applied (point operation). Reconstruction by integration then yields a non-linearly processed image. By an appropriate choice of the non-linear function the processing can be made an enhancement. Noise suppression can be achieved by removing or reducing small modulusderivatives (this encompasses the vector gradient). As a reference for what is small one can take the maximum modulus encountered in the image. This emulates the automatic contrast stretching we are used to in our luminance perception. For the reduction one may think of several functions of relative modulus-derivative: thresholding, smooth thresholding, and symmetric gammacorrection. We shall use the latter in our tests. The non-linear point operation
(
out grey = sign ingrey
)
in grey
γ
= ingrey
γ −1
in grey
is applied to the derivatives: the (G)NL or the components of the (G)VG. For γ>1 this produces noise suppression, as demonstrated in fig. 3, for γ
