Filtering and Luminance Correction for Aged Photographs Alfredo Restrepo (Palacios) and Giovanni Ramponi IPL - DEEI - University of Trieste, Via Valerio 10, 34127 Trieste, Italy ABSTRACT We virtually restore faded black and white photographic prints by the method of decomposing the image into a smooth component that contains edges and smoothed homogeneous regions, and a rough component that may include grain noise but also fine detail. The decomposition into smooth and rough components is achieved using a rational filter. Two approaches are considered; in one, the smooth component is histogram-stretched and then gamma corrected before being added back to a homomorphically filtered version of the rough component; in the other the image is initially gamma corrected and shifted towards white. Each approach presents improvements with respect to the previously separately explored techniques of gamma correction alone, and the stretching of the smooth component together with the homomorphical filtering of the rough component, alone. After characterizing the image with the help of the scatter plot of a 2D local statistic of the type (local intensity, local contrast), namely (local average, local standard deviation), the effects of gamma correction are studied as the effects on the scatter plot, on the assumption that the quality of the image is related to the distribution of data on the scatter plot. Also, the correlation coefficient between the local average and the local deviation on the one hand, and the global average of the image play important descriptor roles. Keywords: Objective quality measure, faded print restoration
1. Introduction The acquisition and the virtual restoration of photo prints, via the processing of digital copies of the original art work, is an approach of recognized cultural importance at least for two reasons: it allows the public at large to appreciate collections which would otherwise be practically unaccessible for reasons of costs and of safety of the artistic product; additionally, it makes available an image that effectively represents the original piece, freezing the deterioration process which, even under a proper conservation environment, often can only be slowed but not stopped. This is particularly true of the restoration of faded antique photographic prints such as the collection in the historical archive of the Fratelli Alinari Museum, in Florence. For this application, in this paper we combine the tools of separate processing of the smooth and rough image components, histogram stretching, nonlinear filtering, and gamma correction in an adaptive scheme; appropriate parameter values are automatically chosen, also based on the computation of objective quality indicators. The separate role of power-law correction has been previously explored in [1], where the intensity component of the pixels was gamma corrected, the appropriate value of γ being found automatically for each image. As reported, the correlation coefficient ρ between the sample mean and the sample deviation for windowed data shows in general negative values when γ = 1 and increases for values of γ > 1. Likewise, the effects of linear histogram stretching of a certain additive smooth component of the image, together with proper nonlinear filtering of the corresponding rough image component, have been explored in [10]; a successful filtering technique for the virtual restoration of faded prints was developed there. We review in some detail these two techniques in Section 2. Section 4 describes the main contribution of the paper, consisting in the combination of the two restoration techniques previously mentioned; before that, in Section 3, we explore in some detail the characteristics of scatter plots of the two-dimensional statistic given by the pair (local average, local standard deviation), that a E-mail:
[email protected], Telephone: +39.040.5587140 (A. Restrepo is currently on leave from Dpt. Ing. Electr. y Electr., Universidad de los Andes, Bogota, Colombia;
[email protected]) c E-mail:
[email protected], Telephone: +39.040.5587853
was found useful in the characterization of the effects of gamma-correction in [10]. In Section 4 we combine the techniques of histogram stretching, filtering and gamma correction; it is found that it is a very successful technique; several examples of the effects of the technique are given. In Section 5, we elaborate further on the subject of image quality assessment, an important issue in the implementation of automatic and/or adaptive restoration techniques. The paper is concluded in Section 6.
2. PREVIOUS RESULTS We briefly describe the separate techniques of the processing of the smooth and the rough image components, and of the power-law correction which have been previously explored for the virtual correction of faded photo prints.
2.a. Separate Correction of Smooth and Rough Components The restoration technique proposed in10 is mainly a technique for enhancing the luminance component of faded BW or sepia photo images. Using a rational filter ,11 the image is additively decomposed into a smooth component that contains edges as well as smoothed homogeneous regions, and a rough component that includes fine detail and may include grain noise as well. Due to the properties of the rational filter, the smooth component contains edges and filtered homogeneous regions. The smooth component is histogram-stretched before being added back to the rational homomorphically filtered rough component. The main reason for filtering the rough component is to get rid of possible grain noise; however, this step may be skipped if it is desired that the grain characteristics of the photograph remain. Values for the maximum MX and the minimum MN of the intensity of the image to be used for the stretch of the histogram are obtained from a median filtered version of the original image; the reason for using a median filtered version is that certain outliers such as small stains and thin scratches, which may be nearly black and nearly white respectively, should not be taken into account at this point. The dynamic range of the smooth component is linearly expanded as follows; to each pixel intensity i there corresponds a parameter α such that i = αMN + (1-α)MX. Using the parameter α, the intensity of the pixel is changed to i = αBk + (1-α)Wt where Bk and Wt respectively correspond to black and white. This corresponds to a conventional stretching of the histogram of the (intensity) image. The resulting image is the addition of this range-corrected smooth component and a scaled and processed version of the rough component. The rough component of the image is either homomorphically filtered or scaled with scale factor 0.5Wt/(MX-MN). The filtering is done in order to attenuate the effects of grain noise is usually modeled as multiplicative noise. The logarithm of the rough component is filtered with a rational filter; after taking antilog the processed rough version is added back to the processed smooth component.
2.b. Automatic Gamma Correction for Contrast Enhancement Locally everywhere, faded prints have a large sample average; the sample average being a measure of local intensity. They also have a low sample variance; the sample variance being a measure of local contrast. As a related fact, faded photo prints have a large global average as well. The application of gamma correction with γ > 1 alleviates this extreme situation: it reduces local averages µ allowing for larger local variances σ 2 . Two-dimensional statistics of the type (location, variation) are found to be useful in the characterization of the effects of gamma correction and in the automatic choice of a nearly optimal value of γ. The automatic gamma-correction method for the virtual restoration of faded black and white prints reported in1 looks for a value of γ that makes the global average intensity nearly 0.5 and nearly uncorrelates the local standard deviation from local average, for the trimmed subset of sampled windows on the lower 75% percentile, along the average component. For any sample, there are inherent restrictions between a value of σ and the corresponding possible values of µ, and viceversa. For faded black and white prints, the correlation coefficient ρσ,µ between the sample mean and the sample deviation for windowed samples increases with γ, starting at a negative value for γ = 1. For the trimmed, low-luminance subset of the points in the µ − σ plane, the value of γ for which ρσ,µ = 0 was observed
to be also a value that makes the average of the intensity of the image equal or near to 0.5. For this optimal value of γ, the image looks visually correct.
3. ON GAMMA CORRECTION AND σ − µ SCATTER PLOTS We use scatter plots of (local average, local standard deviation) for the double purpose of giving an indicator of the quality of the image and choosing an appropriate value of γ, for the automated enhancement of the contrast in images affected by uniform fading or poor, grayish luminance distribution. It is therefore convenient to explore in some detail this relation between σ − µ scatter plots and gamma correction.
3.a. A Characteristic of the (σ − µ) - Plane Given a windowed sample of intensity data, coded in the interval [0, 1], 0 corresponding to black and 1 to white, consider the resulting ordered pair: (sample average, sample deviation) which can be seen as an instance of a local measure of location.vs.dispersion of the image. Such pairs can live only in a certain permitted region of the plane; in fact, the possible positions for the points (σ, µ) are restricted horizontally to abscissae in the interval [0, 1] and are vertically bounded by the horizontal axis below, and above by a certain curve that we derive next. The relative position in the permitted region of the points (σ, µ) gives valuable local information regarding the image. Note that if µ=0 or µ=1 then necessarily σ= 0 since the sample must be constant in those cases; related to this is the fact derived below that the maximum possible value of σ occurs for µ=0.5. For faded images, the distribution of the points in the permitted region is highly concentrated near the point (1, 0), indicating an image with an overall high intensity and low contrast. The effect of the application of gamma correction is to cause a migration of the points, according to a certain flow, making the scatter plot more uniformly distributed on the one hand and also making the overall average near 0.5, on the other; such images look better in general. To get the upper bound curve for the scatter plot (σ, µ), we first derive an alternate expression for the sample variance. Let n be the window (sample) size and let [x1 , x2 , ...xn ] be the corresponding sample data. Pn Pn σ 2 = n1 i=1 x2i - ( n1 i=1 xi )2 Pn Pn Pn = n1 i=1 x2i - n12 i=1 j=1 xi xj Pn Pn−1 Pn Pn Pi−1 Pn = nn2 i=1 x2i - n12 ( i=1 j=i+1 xi xj + i=2 j=1 xi xj + i=1 x2i ) Pn Pn−1 Pn 2 2 = n−1 i=1 xi - n2 i=1 j=i+1 xi xj n2 Pn P Pn n−1 1 = n2 [ i=1 (n − 1)x2i − i=1 j=i+1 2xi xj ] Pn−1 Pn Pn−1 Pn = n12 [ i=1 j=i+1 (x2i + x2j ) − i=1 j=i+1 2xi xj ] Pn−1 Pn = n12 i=1 j=i+1 (x2i + x2j − 2xi xj ) Pn−1 Pn = n12 i=1 j=i+1 (xi − xj )2 P = n12 1≤i
