PLENARY PAPERS
Correction of Distortions in Color Images Based on Parametric Identification V. A. Fursova, A. V. Nikonorova, S. A. Bibikova, P. Yu. Yakimovb, and E. Yu. Minaevb a
Image Processing Systems Institute, Russian Academy of Sciences, ul. Molodogvardeiskaya 151, Samara, 443110 Russia b Samara State Agronomy University, Moskovskoe shosse 34, Samara, 443086 Russia email:
[email protected],
[email protected] Abstract— The paper under consideration deals with a certain information technology for correction of color digital images distortions. These distortions result from image registration process and interaction with light ing environment. In this study, we consider the basic scheme of the correction technology for various types of distortions. The scheme consists of three main stages: detection of distorted areas, identifying of color dis tortion model, and constructing the correction color mapping. Efficient algorithms are proposed for each stage. The results obtained for some kinds of distortions are given in the paper. DOI: 10.1134/S1054661811020349
1. FORMULATION OF THE PROBLEM In recent years, we have seen increasing demand for new information technology development of digi tal image color correction in various applications, such as digital image color correction in prepress, dig ital photo correction, video color correction in Inter net television and video surveillance, etc. Artifacts and distortions associated with the technology of record ing and interaction with the ambient lighting environ ment arise in the process of digital image recording. Sometimes a particular lighting environment supple ments the image with grace and originality and is deliberately formed for this purpose. However, fre quently it distorts the image of the object of interest or introduces into it unnecessary artifacts. The correction task of the lighting environment influence on the image can be called the problem of correction of technological distortion. The nature and intensity of the lighting environment impact can vary greatly in different image areas, leading to different types of image distortion. The following types of dis tortion are common: shading or color distortion on the part of the image (matt speck), due to significant differences in light intensity of different parts of the recorded image; occurrence of multiple point artifacts on the image, as a consequence of multiple reflections from the small object elements or finely dispersed contamina tion of detecting apparatus (the dust on the scanner glass); emergence of extended artifacts (specks) that arise due to the higher reflectance of the local object areas. A case study is the correction problem of shadows and specks on digital photocopies of works of painting. Received January 22, 2011
They are characterized by the following types of dis tortion: glare, shadow stripe at the edges of the lateral frames, and matte specks. Shadow stripes at the edges of the lateral frames appear due to different distances from lighting devices. The cause of multiple artifacts (specks) is a light reflection from the oil paint strokes in the paintings. Matte specks, appearing in a smooth decrease in color strength due to more intense lighting of areas that are closer to the lightbox, occur due to the secondary reflection effect. The correction problem of technological distortions occur in many other areas related to image processing; for example, the task of speck elimination is important for video surveillance systems [1], and the task of illuminance leveling is important in “panoramic” image building from frag ments [2]. Digital photocopy correction is usually realized through local image transformations performed by an operator manually using the system Adobe Photoshop or similar ones. Unfortunately, the technological and computational capabilities of these systems are lim ited, and the correction execution may take a long period of time, and, moreover, requires an operator of high qualifications and wide experience. A number of efforts in many works have been made to develop procedures for automating the color cor rection processes. In particular, for this purpose work [3] used an autoregressive model and built the proce dure of parameter estimation. Currently, color correc tion methods are being developed on the basis of the idea of color division of the lighting environment and the color of the observed object. It is the method of color constancy [4] and color invariants [5]. Work [6] proposes a method for color correction retinex based on the use of color images the eye can perceive. This paper considers a range of methods and algo rithms for color correction based on the use of para metric identification of formal models. Along with a
ISSN 10546618, Pattern Recognition and Image Analysis, 2011, Vol. 21, No. 2, pp. 125–128. © Pleiades Publishing, Ltd., 2011.
126
FURSOV et al.
significant acceleration of the color correction pro cess, an important advantage of this approach is that the user does not require high skills and knowledge of color theory. In this case it is enough to have good color perception to specify the desired colors on small image fragments, whose natural colors are a priori known. Of course, the desired color assignment is of a subjective character, but the goal of this work is limited to the creation of instruments that provide a high degree of process automation of color correction in the network with subjective color representations of the user. 2. THE GENERAL SCHEME OF THE COLOR CORRECTION STAGES The general scheme of color correction usually involves two steps: the localization of distortion and color correction. The localization problem of distor tions consists of building a circuit that limits one or more areas of distortion. The second problem is the actual color correction of the distorted areas. Often a situation arises where the model construction of color correction of distorted fragments is not required or impossible, for example, due to the lack of necessary information. In this case, the computer retouch is exe cuted, i.e., substitution of the image fragments with more suitable distribution of the brightness function. Various methods of contour allocation, particularly the method of active contour [7], can be used for iden tification of the distortion area in the automatic mode. However, the method proposed by the authors, based on the calculation of the socalled contrast function [8], has appeared more effective in most problems considered: f k ( s, a ) = 1 s
2s + a
∑
i = s+a
L k ( x i ),
– 1 s
s
∑ L ( x ), k
i
(1)
i=1
where Lk(xi) is a function count of the brightness of the kline of the image. Variation of the s parameter in the vicinity of the expected boundaries is realized to deter mine the boundary point. As a boundary point of dis tortion, the area receives an s value, for which the con trast function of contrast (1) is maximum. Improve ment of the reliability of the method in comparison with work [9] is achieved by subsequent tracing of a path with regard to its allowable variation. The problem of model identification of color trans formation is solved for the implementation of color correction phase. For that, for example, N small parts (fragments) on the original image are extracted, whose colors, in the expert’s opinion, should be changed. A corresponding test piece, on which a desired color is specified, is created for each of these selected frag ments. At the same time colors of analogous subjects or a priori knowledge of the socalled recognized nat ural colors (parts of face, sky, green spaces, etc.) can be used.
On a set of fragment pairs (source and desired), the problem of model identification in a specified para metric family of conversion functions is solved, which, in general, can be represented as z ( x i, y i ) = F ( s ( x i, y i ), c, x ( i ) ),
i = 1, N ,
(2)
where z(xi, yi), s(xi, yi), i = 1, N , are the desired and actual values of color coordinates (the brightness val ues) accordingly at the point (xi, yi), c is the vector of the required model parameters, and x(i) is the error (observation and specifying the model order) at the i fragment. Then, elementwise the image transforma tion is realized in accordance with the obtained model: ˆz ( x i, y i ) = F ( s ( x i, y i ), cˆ ), i = 1, 2, …, (3) ˆ where z (xi, yi) are the values of color coordinates (the brightness values) at the point (xi, yi), obtained as a consequence of processing, as well cˆ is the estimation vector of model parameters. When distortions are so large that information about the colors on some part of the image is com pletely absent, model (2) fails to specify the desired and true colors. This case solves the problem of the model identifying computer retouch. Usually, it leads to parameter determination of some assigned “trans fer” model of colors from the immediate vicinity of the distorted plot. One of the main problems in the general scheme of the color correction stages on the basis of identifica tion is identification of a parametric model family that describe the transformation type of color coordinates and brightness values. The parametric model classes depend on the type of distortion. Linear or linear parameters of the model are the most simple and con venient in terms of solving the identification problem. Using the notation in (2), the linear model of color correction in the general case can be represented as M
z ( x i, y i ) =
∑ c s ( x , y ) + ξ ( i ), j j
i
(4)
i
j=1
where sj(xi, yi), j = 1, M , are the defined function of color coordinates and brightness values at the point (xi, yi), and cj is the parameter of the linear model to be determined. When the desired color coordinates z(xi, yi), i = 1, M , are defined for N of the different image frag ments (N > M) equally with known values of the color coordinate functions sj (xi, yi), j = 1, M , by model (2) N row vectors can be written as s i = [ s 1 ( x i, y i ), s 2 ( x i, y i ), …, s M ( x i, y i ) ], i = 1, N . (5) With the use of these N ratios and the original model (4), it is clear how to present a model in matrix form: (6) Z = Sc + Ξ.
PATTERN RECOGNITION AND IMAGE ANALYSIS
Vol. 21
No. 2
2011
CORRECTION OF DISTORTIONS IN COLOR IMAGES (a)
(b)
127 (c)
Fig. 1. Examples of correction. (a) Matte specks correction; (b) specks separation in the feame of video surveillance system; (c) shadow correction.
The identification problem is to construct the Ξ esti mation of the parameter vector cˆ for an unknown N × 1 vector of errors c on the N × M matrix S and N × 1 vec tor Z (N > M). Within the described linear parameter model, the quality of color correction to a consider able degree is determined by the successful choice of the function type of color coordinate transformation (5). Further, we consider the models of color correc tion applied to the task of image color changing and the shadow removing distortion and glare. 3. PARAMETRIC CLASSES OF DISTORTION MODELS We construct parametric models of different classes of technological distortions. The simplest model is the componentwise color correction with independent transformation of three color coordinates, R, G, B. This model is constructed in the segment form of the Taylor series of low uniformity for each component. The parameter estimation problem of these transfor mation functions is solved according to the specified color on the test fragments. The problem reduces to solving the identification task (6) that is linear in the model parameters of the form (4). The problem of speck elimination considers a general approach to the model construction, which consists in the additional distortion presentation introduced by the specks by means of two simpler functions (4) formed in different spaces: I ( x, y ) = I* ( x, y ) + P ( x, y )Ψ ( x, y ). (7) Function P(x, y) is a normalized function of the speck brightness Δf(x, y) on the assumption that the minimum value of speck brightness is equal to zero: Δf ( x, y ) , P ( x, y ) = (8) max Δf ( x, y ) x, y
where Δf(x, y) = f*(x, y) – f(x, y), where f(x, y) is the brightness of the image points, f*(x, y) is the bright PATTERN RECOGNITION AND IMAGE ANALYSIS
ness of the reference image, Ψ(I(x, y)) is a function that ensures the reference value transformations of color coordinates to their real values in the speck areas with the maximum brightness. The known fact that the information about the luminous exitance is contained in the brightness com ponents of some color spaces, for example, CIE Lab [3], is a prerequisite for the specified representation. In so far as the largest color distortions occur in the speck field at maximum brightness, the errors of color correction in areas close to the speck borders with such representation would be small. The problems of parameter identification of the polynomials in different orders are solved to determine the functions P(x, y) and Ψ(I(x, y)) using the specified desired color on the test fragments. Removing the shadow distortion, the transforma tion function in the class of models, which are linear on parameters, can be identified, when the shadow only changes the color, but does not mask them com pletely. In this case, we consider the region of propor tional shadiness and the transition area. The middle line of the transition area is received as a shadow boundary. Contrast indicator (1) is used for determi nation. The paper shows that this indicator in this case “works” and is more stable than other known methods for contour determination [9]. RESULTS The developed methods and algorithms are imple mented on a desktop supercomputer system and tested in preparation for printing a large number of digital photocopies, reproductions, and paintings in the Agni publishing house (Samara). A number of algorithms are used in video surveillance systems. Several correc tion examples are shown in Fig. 1. Vol. 21
No. 2
2011
128
FURSOV et al.
ACKNOWLEDGEMENTS We are grateful to experts from the Agni publishing house for their proficient help in color correction and testing of the proposed methods on real images. This work was supported by the Russian Founda tion for Basic Research, project no. 090700269a. REFERENCES 1. “Computer Image Processing,” Part II: Methods and Algorithms, Ed. by V. A. Soifer (VDM Verlag, 2009). 2. H. Dersch, “Panorama Tools Open Software for Immersive Imaging,” in Proc. VR Photography Conf. (Berkeley, 2007). 3. G. M. Emelyanov and S. A. Popov, “Color Correction of Digital Images,” Pattern Recogn. Image Anal. 14, No. 4, 45–49 (2003).
4. D. A. Forsyth, “A Novel Approach to Color Con stancy,” in Proc. Int. Conf. on Computer Vision (Tarpon Springs, FL, 1988), pp. 9–18. 5. B. V. Funt and G. D. Finlanson, “Color Constant Color Indexing,” IEEE PAMI 17, 522–529 (1995). 6. E. Land, “An Alternative Technique for Computation of the Designator in the Retinex Theory of Color Vision,” Proc. Nat. Acad. Sci. 83, 3078–3080 (1986). 7. M. Kass, A. Witkin, and D. Terezopoulos, “Snakes: Active Contour Models,” Int. J. Comp. Vision, No. 1, 321–331 (1987). 8. S. A. Bibikov, V. A. Fursov, and A. V. Nikonorov, “Shadow Artifacts Correction of Fine Art Reproduc tions,” in IMTA in Conjunction with VISIGRAPP 2010 (Angers, May 2010), pp. 3–12. 9. A. Rosenfeld and M. Thurston, “Edge and Curve Detection for Visual Scene Analysis,” IEEE Trans. Comp. 20 (5), 562–569 (1971).
PATTERN RECOGNITION AND IMAGE ANALYSIS
Vol. 21
No. 2
2011
