transformation and Tikhonov regularization with trial-and-error selection of the regularization parameter . Software is developed. Numerical results are presented ...
Reconstruction of smeared and out-of-focus images by regularization V. S. Sizikov and I. A. Belov St. Petersburg State Institute of Precision Mechanics and Optics (Technical University), St. Petersburg, Russia
共Submitted September 27, 1999兲 Opticheski Zhurnal 67, 60–63 共April 2000兲
The reconstruction of smeared and out-of-focus images is considered. The problem is described by Fredholm-type integral equations of the first kind. They are solved using Fourier transformation and Tikhonov regularization with trial-and-error selection of the regularization parameter ␣. Software is developed. Numerical results are presented. © 2000 The Optical Society of America. 关S1070-9762共00兲01304-X兴
INTRODUCTION
One of the inverse problems of optics is the reconstruction of distorted images.1 The term image can refer to a photograph of a person, text, or an object in nature 共including a photograph taken from space兲, a television or motionpicture image, a telescopic photograph, an optoelectronic reproduction of a celestial body, etc. However, to fix ideas, we shall henceforth use the term image to refer to a photograph. We assume that preliminarily treatment of the image has been carried out and that, specifically, the scratches on the photograph have been removed, its contrast has been adjusted, and other operations 共not requiring mathematical treatment兲 have been performed. We shall dwell on the most difficult problem, i.e., the mathematical reconstruction of images distorted as a result of smearing 共displacement, movement兲 or defocusing. RECONSTRUCTION OF SMEARED IMAGES
Let us examine this problem in the case of a smeared 共displaced, moved兲 photograph. Let the photographed object 共which is assumed to flat because of the large distance to it兲 and the photographic film in the camera be oriented parallel to the camera lens aperture at the distances f 1 and f 2 , respectively, from the lens. In this case 1/f 1 ⫹1/f 2 ⫽1/f and f 1 ⭓ f 2 , where f is the focal distance of the lens 共Fig. 1兲. We assume that the film underwent a linear and uniform displacement 共movement兲 by ⌬ or that the object 共for example, a fast-moving target兲 underwent a displacement by ⫺⌬ f 1 / f 2 during the exposure time. As a result, the image on the photographic film is smeared. The problem is described mathematically by the relation2–4 1 ⌬
冕
x⫹⌬
x
w 共 ,y 兲 d ⫽g 共 x,y 兲 ,
J. Opt. Technol. 67 (4), April 2000
冕
⬁
⫺⬁
k 共 x⫺ 兲 w 共 ,y 兲 d ⫽g 共 x,y 兲 ,
where k共 x 兲⫽
再
1/⌬
for x苸 关 ⫺⌬,0兴 ,
0
for x苸 关 ⫺⌬,0兴 .
⫺⬁⬍x,y⬍⬁,
共2兲
共3兲
If the x axis is antiparallel to the displacement, then k共 x 兲⫽
再
1/⌬
for x苸 关 0,⌬ 兴 ,
0
for x苸 关 0,⌬兴 .
共4兲
We note that this formulation of the problem can be extended to the case of nonparallel object and film planes, as well as the case of nonuniform and/or nonlinear displacement of the film 共or object兲, as was done, for example, in Ref. 5. Finding the solution of Eq. 共2兲 is an ill-posed problem. Fourier transformation and Tikhonov regularization2,6,7 are used to solve it in this paper. The regularized solution has the form w ␣ 共 ,y 兲 ⫽
1 2
冕
⬁
⫺⬁
W ␣ 共 ,y 兲 e ⫺i d .
共5兲
Here the regularized Fourier-transform spectrum of the solution is
共1兲
where g(x,y) is the intensity on the film 共the smeared image兲 as a function of the rectangular coordinates x and y, the x axis being directed along the displacement 共smear兲, and w( ,y) is the true image 共the image which would have been recorded on the film in the absence of the shift, i.e., if ⌬ ⫽0兲. The relation 共1兲 is a one-dimensional integral equation relative to x( ,y) at each fixed value of y, which plays the 351
role of a parameter. It can be written in the form of a Fredholm-type integral equation of the first kind in convolution form3,4
FIG. 1. Reconstruction of a smeared image.
1070-9762/2000/040351-04$18.00
© 2000 The Optical Society of America
351
W ␣ 共 ,y 兲 ⫽
K 共 ⫺ 兲 G 共 ,y 兲 , L共 兲⫹␣ M 共 兲
where L 共 兲 ⫽ 兩 K 共 兲 兩 2 ⫽K 共 兲 K 共 ⫺ 兲 , K共 兲⫽
冕
⬁
⫺⬁
G 共 ,y 兲 ⫽
冕
k 共 x 兲 e i x dx, ⬁
⫺⬁
g 共 x,y 兲 e i x dx, M 共 兲 ⫽ 2 ,
冕冕
w共 , 兲
冑共 x⫺ 兲 2 ⫹ 共 y⫺ 兲 2 ⭐ p
2
共6兲
共7兲
and ␣ ⬎0 is the regularization parameter. There are several methods, for example, the discrepancy method, for choosing ␣.2,6,7 However, as tests have shown, the trial-and-error method is most effective for image reconstruction.3,4 According to this method, w ␣ ( ,y) is calculated for a series of values of ␣ using formula 共5兲 关along with 共6兲 and 共7兲兴, the solution w ␣ ( ,y) is displayed in graphical form, and the value of ␣ which gives the best reconstructed image from the standpoint of physiological 共rather than mathematical兲 perception criteria is chosen. This method is similar to tuning the contrast of a television image 共in that case ␣ is inversely proportional to the contrast兲. We note that the value ⌬ is usually not known in practice, and it should also be determined by trial-and-error. As for the smear direction 共along which the x must be oriented兲, it is determined from marks on the photograph 共see Fig. 3b below兲. Thus, after correctly selecting the direction of the x axis 共along the smear兲 and the value of ⌬, solving Eq. 共2兲 共more precisely, set of equations兲, using formulas 共5兲–共7兲, and selecting ␣ by trial and error, we can reconstruct the undistorted photograph w( ,y) from the distorted 共smeared兲 photograph g(x,y). RECONSTRUCTION OF OUT-OF-FOCUS IMAGES
Let the object photographed 共which is assumed to be flat兲 and the photographic film be oriented parallel to the lens at the distances f 1 and f 2 ⫹ ␦ from it, respectively, where ␦ is the image focusing error 共Fig. 2兲. In this case the following relation holds:2,8
d d ⫽g 共 x,y 兲 .
共8兲
Here g(x,y) is the intensity on the out-of-focus photograph, w( ,y) is the intensity sought on the undistorted photograph 共on the photograph which would have been obtained if ␦ ⫽0兲, ⫽a ␦ / f 2 is the radius of the diffraction circles on the film 共circles into which each point A ⬘ or A ⬙ is transformed; see Fig. 2兲, and a is the lens aperture radius. The relation 共8兲 can be brought into a standard form, or, more specifically, transformed into a two-dimensional Fredholm-type equation of the first type in convolution form:2,8
冕 冕 ⬁
⬁
⫺⬁
⫺⬁
k 共 x⫺ ,y⫺ 兲 w 共 , 兲 d d ⫽g 共 x,y 兲 ,
⫺⬁⬍x,y⬍⬁, where k 共 x,y 兲 ⫽
再
共9兲
1/ 2
for
0
for
冑x 2 ⫹y 2 ⭐ , 冑x 2 ⫹y 2 ⬎ .
共10兲
In Eq. 共9兲 g(x,y) is the measured right-hand side, w( ,y) is the function sought, and the kernel k(x,y) is called the point scattering function. We note that the problem of defocusing was also considered in Ref. 5 for the case of nonparallelism between the plane of the object and the plane of the film. Finding the solution of Eq. 共9兲, as in the case of Eq. 共2兲, is an ill-posed problem. The solution obtained by Tikhonov regularization and two-dimensional Fourier transformation has the form2,6,7 w ␣共 , 兲 ⫽
冕 冕 ⬁
1 4
2
⫺⬁
⫺⬁
W ␣共 1 , 2 兲
⫻e ⫺i 共 1 ⫹ 2 兲 d 1 d 2 .
共11兲
Here W ␣共 1 , 2 兲 ⫽
K 共 ⫺ 1 ,⫺ 2 兲 G 共 1 , 2 兲 , L共 1 ,2兲⫹␣ M 共 1 ,2兲
共12兲
where L 共 1 , 2 兲 ⫽ 兩 K 共 1 , 2 兲 兩 2 ⫽K 共 1 , 2 兲 K 共 ⫺ 1 ,⫺ 2 兲 , K共 1 ,2兲⫽ G共 1 ,2兲⫽
冕 冕 冕 冕 ⬁
⬁
⫺⬁
⫺⬁
⬁
⬁
⫺⬁
⫺⬁
k 共 x,y 兲 e i 共 1 x⫹ 2 y 兲 dxdy, g 共 x,y 兲 e i 共 1 x⫹ 2 y 兲 dxdy,
M 共 1 , 2 兲 ⫽ 共 21 ⫹ 22 兲 2 ,
FIG. 2. Reconstruction of an out-of-focus image. 352
J. Opt. Technol. 67 (4), April 2000
共13兲
and ␣ ⬎0 is the regularization parameter, which is chosen, as in the reconstruction of smeared images, by trial and error. We note that the value of ␦ 共or 兲 is not known a priori in practice and that it is also usually determined by trial and error.2,8 V. S. Sizikov and I. A. Belov
352
Thus, after choosing the values of ␦ 共or 兲, the stable relations 共11兲–共13兲 can be used to reconstruct the undistorted photograph w( ,y) from the out-of-focus photograph g(x,y).
NUMERICAL REALIZATION
The IMAGE software 共for Windows 95兲 was written in Visual C⫹⫹ for solving the problem of reconstructing smeared and/or out-of-focus images by Tikhonov regularization and Fourier transformation according to Eqs. 共1兲–共7兲 and 共8兲–共13兲 with trial-and-error selection of ␣ 共as well as ⌬ and ␦兲 and display of the processing results on a monitor. Both the ordinary problem of simulating the distorted image g(x,y) according to 共1兲 or 共8兲 and the inverse problem of reconstructed the undistorted image w( ,y) according to 共5兲 or w( , ) according to 共11兲 are solved. The ordinary and inverse one-dimensional and two-dimensional Fourier transformations 关see Eqs. 共5兲, 共7兲, 共11兲, and 共13兲兴 are performed in the form of discrete Fourier transformation, which, in turn, is realized in the form of fast Fourier transformation according to programs that are modifications of the Fortran programs FTFIC 共see pp. 183–184 and 190–192 in Ref. 7兲 and FTFTC 共see p. 190 in Ref. 7兲. Black-and-white image are processed using gray 共a mixture of red, green, and blue in equal proportions兲 to obtain a large number of brightness gradations, and color images are processed using separate processing in the three primary colors followed by superposition of the three images. Representation of the colors in the Windows operating system according to the RGB 共red, green, and blue兲 scheme is used more extensively in processing black-and-white images. Each component is varied from 0 to 255 共there are thus 256 brightness gradations兲. Therefore, by mixing the three components in equal proportions we can obtain: 0,0,0 for black; n,n,n, where n⫽1,...,254, for gray; and 255, 255, 255 for white. The ordinary problem is solved in the following manner. If it is a 共distorted兲 photography, it is translated into a digital representation by scanning. For simplicity, the common BMP 共bitmap兲 format for a graphic file is used in the present work. If it is a model example, an undistorted image 共Fig. 3a兲 is first formed using a graphics editor 共Word, Paintbrush, etc.兲, and then a smeared or out-of-focus image is formed in the computer according to 共1兲 or 共8兲 共Figs. 3b and 4a兲. Figure 3a shows an original image of text, Fig. 3b shows the imaged smeared at a 30° angle relative to the lines of text, and Figs. 3c–3e show the regularized solution w ␣ ( ,y) for the optimal value of the regularization parameter ␣ ⫽2.5⫻10⫺3 , for ␣ ⫽0 共i.e., without regularization兲, and for the overestimated value ␣ ⫽0.1, respectively. A total of 256 discrete values of y were used, and 256 discrete readings of g were taken at each y along x. If there were less than 256 readings, zeros were added. Figure 4 shows the analogous results for the reconstruction of an out-of-focus image 共256⫻256 discrete reaches were taken兲. 353
J. Opt. Technol. 67 (4), April 2000
FIG. 3. a—Original 共exact兲 image. b—Smeared image (⌬⫽02). Reconstructed images for ␣ ⫽2.5⫻10⫺3 共c兲, ␣ ⫽0 共d兲, and ␣ ⫽0.1 共e兲.
CONCLUSION
1. The problems of the smearing and defocusing of images are described by Fredholm-type integral equations of the first kind 共for the one-dimensional and two-dimensional cases兲. 2. These equations are effectively solved by Tikhonov regularization with trial-and-error selection of the regularization parameter ␣ 共as well as ⌬, ␦, and 兲.
FIG. 4. a—Out-of-focus image ( ⫽10). Reconstructed images for ␣ ⫽5 ⫻10⫺4 共b兲, ␣ ⫽0 共c兲, and ␣ ⫽5⫻10⫺2 共d兲. V. S. Sizikov and I. A. Belov
353
3. For its practical realization, the method is supplemented by the use of gray in processing black-and-white images and by separate processing of color images in the three primary colors. 4. Everything taken together permits the efficient reconstruction of smeared and out-of-focus images. 5. The method described can be used to process old lowquality photographs, for reconstructing images of fastmoving targets, telescopic images of celestial bodies, or photographs of terrestrial objects taken from space, for improving the quality of tomography, etc. 1
R. H. T. Bates and M. J. McDonnell, Image Restoration and Reconstruction 关Clarendon Press, Oxford 共1986兲; Mir, Moscow 共1989兲, 336 pp.兴. 2 A. B. Bakushinski and A. V. Goncharski, Ill-Posed Problems. Numerical Methods and Applications 关in Russian兴, Izd. MGU, Moscow 共1989兲, 199 pp. 3 V. S. Sizikov, M. V. Rossiskaya, and A. V. Kozachenko, ‘‘Processing a smeared image by differentiation, Hartley transformation, and Tikhonov
354
J. Opt. Technol. 67 (4), April 2000
regularization,’’ Izv. Vyssh. Uchebn. Zaved., Priborostroen. 42共7兲, 11 共1999兲. 4 V. S. Sizikov and I. A. Belov, ‘‘Modelling problem of distorted image reconstruction by regularization method,’’ in Abstracts of 2nd International Conference ‘‘Tools for Mathematical Modelling,’’ St. Petersburg 共1999兲, pp. 127–128. 5 A. N. Tikhonov, A. V. Goncharski, V. V. Stepanov, and A. G. Yagola, ‘‘Inverse problems for processings photographic images,’’ Ill-posed Problems in the Natural Sciences, A. N. Tikhonov and A. V. Goncharsky 共Eds.兲 关Mir Publishers, Moscow; Imported Publications, Chicago 共1987兲; Izd. MGU, Moscow 共1987兲, pp. 185–195兴. 6 A. F. Verlan’ and V. S. Sizikov, Integral Equations: Methods, Algorithms, and Programs 关in Russian兴, Naukova Dumka, Kiev 共1986兲, 544 pp. 7 A. N. Tikhonov, A. V. Goncharski, V. V. Stepanov, and A. G. Yagola, Numerical Methods for the Solution of Ill-Posed Problems 关Kluwer Academic Publishers, Dordrecht–Boston 共1995兲; Nauka, Moscow 共1990兲, 232 pp.兴. 8 V. S. Sizikov, A. V. Kuz’min, and A. V. Kozachenko, ‘‘Treatment of out-of-focused images by two-dimensional Hartley transformation and Tikhonov regularization,’’ Izv. Vyssh. Uchebn. Zaved., Priborostroen. 42共8兲 共1999兲.
V. S. Sizikov and I. A. Belov
354
