Jul 3, 1991 - State University of New York at Stony Brook. Stony Brook ... a function, the new transform corresponds to convolution of the image with a point.
Stony Brook
Department of Electrical Engineering State University of New York at Stony Brook Stony Brook, New York 11794-2350
Computer Vision Laboratory
Tech. Report No. 91.07.03
July 3, 1991
Spatial-Domain Convolution/Deconvolution Transform Muralidhara Subbarao
Abstract A new linear transform is de ned for real valued functions which can be expanded in Taylor series. For an image which can be expressed by the Taylor expansion of a function, the new transform corresponds to convolution of the image with a point spread function. The point spread function must satisfy the condition that it's zero-th moment is non-zero and positive integer moments are nite. The forward part of the new transform is based on a spatial-domain convolution formula rst derived by Papoulis in 1962. The inverse of the new transform corresponds to deconvolution. This transform suggests a direct spatial-domain approach to convolution and deconvolution operations on images. In practice, the transform can be implemented by local operations on images using global characteristics of point spread functions speci ed by their rst few moments. Therefore, this transform is better suited than the Fourier transform for many image ltering operations such as restoration of defocused images and image enhancement. The new transform is rst derived for one-dimensional signals. It is then extended to two-dimensional images and multi-dimensional signals. A modi ed version of the transform is de ned for discrete signals.
1
1 Introduction Convolution in the spatial domain becomes multiplication in the frequency domain. This is the ultimate justi cation for the introduction of the complex machinery of the frequency domain. { B. K. P. Horn (1986) [1].
During the last few years, we have been doing research on determining the distance of an object using it's defocused image in a camera system [5, 6, 7]. Our early work described in [5, 6] is based on a Fourier domain approach, whereas the later work [7] is based on a spatial-domain approach. The latter work dealt only with simple objects such as points, lines, blobs, and stripes. However, this work provided us some new insight into the way the convolution operation a ects an image. This motivated us to nd a spatial-domain approach to determine the distance of objects with arbitrary appearance. We were particularly interested in a spatial-domain method for determining distance because our earlier work based on the Fourier domain approach had certain practical disadvantages. We succeeded in nding a spatial-domain approach [9] after we derived a new deconvolution formula in the spatial-domain [8]. Based on this new spatial-domain deconvolution formula, we have derived a new transform which is the subject of this report. The new transform is a linear transform. The forward part of the transform corresponds to convolution, and it is based on a convolution formula rst derived by Papoulis [3, 4] in 1962. (This formula was however derived independently by us in 1989.) The inverse part of the transform corresponds to deconvolution, and it is based on a new deconvolution formula rst derived by us and reported in [8]. For images, the transform is expressed in the spatial-domain. Therefore we call it the Spatial-domain Convolution/Deconvolution Transform, or simply S Transform. The domain and range of S transform are the space of real valued functions which can be expanded in Taylor series. (In this report, we use the words `function' and `signal' interchangeably, and `image' and `two-dimensional signal' interchangeably.) First we de ne the tranform for one-dimensional continuous signals. Then we extend it to two-dimensional images and multi-dimensional (n-dimensional) signals. Later we de ne a modi ed version of the transform for discrete signals obtained by sampling continuous signals which can be expressed in Taylor series. Veri cation of the derivation of S transform and its inverse transform are easier for polynomial functions. Therefore, rst we present our results for real valued polynomials. These results can be extended to arbitrary functions which can be expanded in Taylor series by considering the Taylor series of the function to be an in nite-order polynomial. This is a preliminary report on the new transform. Some related topics under investigation are mentioned near the end of this report. 2
2 Forward S Transform In this section, we develop two possible de nitions of the forward S transform. Let P N be the space of all real valued polynomial functions of degree less than or equal to N for N = 0; 1; 2; . Let f 2 P N . Then we can write +1
+1
f (t) =
N X k
ak tk
(1)
=0
where ak are real valued coecients and t is a real variable. Let the nth moment of a real valued function h(t) be denoted by hn for n = 0; 1; 2; . We have Z1 hn tnh(t) dt: (2) ?1
Let be the space of all real valued functions h(t) such that h 6= 0, and all positive integer moments of h upto order N are nite, i.e.,
MN
+1
0
jhnj < 1 for n = 0; 1; 2; N:
(3)
De nition 1: The S transform of a polynomial f (t) where f 2 P N with respect to a kernel h(t) where h 2 M N is denoted by SH ff g( ) or FH ( ) and is +1
+1
de ned by the integral
SH ff g( ) FH ( )
Z1 ?1
h(t) f ( ? t) dt;
(4)
where is the transform variable and 2 R (i.e., is real) The integral in Eq. (4) above can be recognized as the convolution integral. In order to facilitate de ning the inverse transform corresponding to the S transform, an alternate de nition of the S transform is derived below. The rst de nition of the S transform given earlier is in integral form whereas the second de nition of the S transform to be derived next is in di erential form. Since f (t) in Eq. (4) is a polynomial of order less than or equal to N , we can write f ( ? t) in a Taylor series expansion around the point as
f ( ? t) =
N (?t)k X k ( ) f k! k
(5)
=0
where
k f k ( ) d d k f ( ): Substituting Eq. (5) in Eq. (4) we obtain Z1 N k X FH ( ) = h(t) (?kt!) f k ( ) dt: ?1 k =0
3
(6) (7)
Interchanging the order of summation and integration above, we have
FH ( ) =
Z N (?1)k X k ( ) 1 tk h(t) dt f k! ?1 k
(8)
=0
From the de nition of the kth moment hk of h(t) as de ned in Eq. (2), Eq. (8) becomes
FH ( ) =
N (?1)k X hk f k ( ): k ! k
(9)
=0
Eq. (9) is the alternate de nition of the S transform. This will be referred to as the di erential form of the de nition of S transform due to the presence of the derivative term f k ( ); we shall also refer to it as the convolution formula. Note that, this formula holds even if h(t) is not a continuous function. Papoulis [3, 4] was the rst to derive the convolution formula (9) in 1962 for N = 1. (It was, however, derived independently by us in 1989.) He has suggested that, when the series on the right hand side of Eq. (9) converges rapidly, the rst few terms of the series can be used to numerically compute the result of the convolution operation. He has shown that the series converges fast when h(t) is \time-limited", i.e. h(t) = 0 for jtj > T; (10) for some nite T , and f (t) is smooth in the interval (t ? T; t + T ). De nition 2: The S transform of f (t) where f 2 P N with respect to a kernel function h(t) where h 2 M N is denoted by SH ff g( ) or FH ( ) and is de ned by +1
+1
SH ff g( ) FH ( )
N (?1)k X hk f k ( ) k ! k
(11)
=0
One may think of H as representing the moment vector de ned by
H (h ; h ; h ; ; hN ) 0
1
2
(12)
and SH as an operator de ned as
SH
N (?1)k dk X k! hk d k : k
(13)
=0
Note that SH is a Linear Constant Coecient Di erential Operator. The coecients (?1)k hk =k! will be called S transform coecients, or S-coecients for short. Clearly, S transform is a linear transform and has properties of a general linear transform. In the next section, we derive a deconvolution formula based on the convolution formula (9). Our initial restriction that f (t) is a polynomial is very helpful in deriving the inverse transform corresponding to the S transform.
4
3 Inverse S Transform The S transform de ned in Eq. (11) can be written as N (?1)k X hk f k ( ) k ! k ! N (?1)k X 1 k ) f ( ) = h FH ( ) ? k! hk f ( ) : k
FH ( ) = h f ( ) +
(14)
0
=1
0
(15)
=1
Taking the mth derivative with respect to on either side of Eq. (15),
f m ( ) =
!
?m (?1)k 1 F m( ) ? NX hk f k m( ) ; h H k ! k
(16)
+
0
=1
where
m FHm( ) d d m FH ( ): (17) Equations (15) and (16) are key to deriving the inverse S transform (actually, Eq. (15) is a special case of Eq. (16) when m=0). Equation (16) expresses the mth derivative of f ( ) in terms of its own higher order derivatives (m + 1 to N ). Therefore we can substitute for f k ( ) in Eq. (15) using Eq. (16). In the resulting equations, we can again substitute for all derivatives of f ( ) using Eq. (16). This procedure of using Eq. (16) to eliminate the derivatives of f ( ) from Eq. (15) is carried out recursively through repeated substitution. This procedure terminates in a nite number of steps because derivatives of order higher than N are zero. The resulting expression can be summarized as below.
up
( ); jI j 0 (i ; i ; i ; ; ip) for p = 1; 2; ; N i + i + i + + ip N ? jIp? j (?1)p jIpj hi1 hi2 hi3 hip i ! i ! i ! ip ! hp r3 r2 X r1 X r2 r1 X r X X X1 1 j I j j I j 2 1 u F jI3j + uF + f = h F+ uF + i1 i2 i3 i1 i2 i1
I Ip jIpj rp 0
0
1
2
1
3
2
3
1
+
+1
1
0
2
3
+
=1
r3 r2 X r1 X X
i1
=1
i2
=1
i3
=1
rN X
iN
=1
(22)
3
2
1
0
(18) (19) (20) (21)
=1
=1
=1
=1
uN F jIN j:
(23)
=1
In the above expression, we reexpress each summation term in a more convenient form using the relation r3 r2 X r1 X X i1
=1
i2
=1
i3
=1
rp X ip
=1
upF jIpj =
X pi1 i2 i3 ip N +
5
+
+
+
upF i1
i2 i3 ip
+
+
+
+
(24)
where the summation on the right hand side is done over all possible values of i ; i ; i ; ; ip subject to the additional condition that 1
2
3
i 1; i 1; i 1; ; ip 1: In the resulting expression, we regroup the terms to obtain N X f ( ) = wk FHk ( ) k where w = 1 and h p h k X (?1)p k Y X iq wk = p q iq ! p jIpj k h 1
2
3
(25) (26)
=0
0
0
+
+1
=1
=
0
(27) (28)
=1
iq 2 f1; 2; 3; ; kg for q = 1; 2; 3; ; k: (29) The right hand side of Eq. (28) should be interpreted as summing over all possible permutations of i ; i ; ; ip (for p = 1; 2; 3; ; k) such that they sum to k (this condition is speci ed by jIpj = k). This corresponds to the di erent ways in which the integer k can be expressed as the summation of positive integers 1; 2; ; k. One may also think of this as the di erent ways of putting k balls into p di erent boxes with iq balls in the q-th box, q = 1; 2; p. This analogy can be used to construct a recursive algorithm to generate all possible Ip for a given value of k. One such algorithm written in the C programming language is given here. 1
2
/* A program which prints all possible Ip for a given k */ #include #define KMAX 20 /* maximum value of k */ int box[KMAX+1]; /* buffer which stores Ip during its' generation */ main(argc,argv) int argc; char *argv[]; { int k,cur_box,atoi(); void Ip(); if(argc != 2) { fprintf(stderr,"usage: program k\n"); exit(1); } k = atoi(argv[1]); /* read k as a command line argument */ if((k>KMAX) || (k
