An automatic algorithm of degraded color document image enhancement is proposed. It is based on previous work on Natural Rendering of Color Image using ...
DEGRADED COLOR DOCUMENT IMAGE ENHANCEMENT BASED ON NRCIR Shaohua Chen1, Azeddine Beghdadi1 and Mohamed Cheriet2 1
L2TI, University Paris 13, France, 2Synchromedia ETS, University of Quebec, Canada ABSTRACT
An automatic algorithm of degraded color document image enhancement is proposed. It is based on previous work on Natural Rendering of Color Image using Retinex (NRCIR) with respect to document image characteristics. In the proposed work, an adaptive workflow is designed to enhance document image in luminance and chrominance contrast while keeping degradations within tolerance and hue-shift minimized. Tests with degraded document image databases are effectuated, the results of which prove an encouraging performance of the proposed method. Index Terms — NRCIR, image enhancement, degraded document image, Retinex, CIELch color space 1. INTRODUCTION Document image processing is of great importance in document imaging which can produce more suitable result for further image analysis and understanding. An enhanced version of image often enables more successful segmentations and less recognition failure. Many approaches have been proposed in literature [1]. However, most of them are natural-scene image oriented. Adaptations should be made before applying them to document images for an optimized performance. This paper is mainly focused on some recent methods based on Retinex theory [2] and its diversities. Since the pioneer work of Edwin H. Land [3], wide applications of Retinex have been realized in industrial scenario, medical diagnostics or aerospace photography [4] [5]. Many algorithms have been proposed such as path version [6], iterative version [7] [8] and center/surround version [9]. As discuss in our previous work of natural rendering of color image using Retinex (NRCIR) [10], these techniques are not supposed to be applied directly to degraded images although they work efficiently indeed in extracting as more details as possible from images. Many of them disregard light conditions of scene which usually result in unnaturally sharpened appearance, and the document image degradations are often over-emphasized as well. We propose in this paper an algorithm of degraded document image enhancement serving as a preprocessing of image analysis and understanding such as text recognition for instance with consideration of document image characteristics. The structure of this paper is organized into 5 sections including the current section 1 of introduction. A problem statement will be given first in next section to demonstrate the insufficiency of some enhancement techniques in document image case. Section 3 describes
the flowchart of the proposed method. Details of analysis and practical implementations will be presented with some extensive tests results in section 4 followed by conclusion and perspective works in section 5. 2. PROBLEM STATEMENT As stated in previous work [10], natural enhancement algorithm shall avoid dramatic alternation of lighting conditions to image scene, and shall not introduce additional artifacts or amplifying hidden distortions of images. Many old document images suffer from physical degradations resulted from aging process, low-quality materials, and even human errors [11]. Although many document images are scanned and stored in high definition version, over-enhancement of them will still result in either unnatural appearance or dramatic white balance change which is not friendly to viewers, or overemphasized degradations which might be misleading in future image content analysis. In awareness of these constraints, NRCIR realizes a natural rendering of color image while preventing from over-enhancement. However, some modifications need to be made under document image scenario for more evident and robust performance. NRCIR was originally designed to be conservative especially for enhancing dark zone of image to avoid emphasizing blocking or ringing effects inside degraded image. However, this restriction can be relaxed thanks to high definition of document scanning and storage, and hence more force of enhancement is permitted to realize a better detail extraction. Meanwhile, moderate saturation and hue-constancy properties are appreciated in order to render the enhanced image version with more friendly appearance to historical scholars and ordinary viewers. Targeting to these requirements, the proposed method adopts a more optimized flowchart which includes an iterative implementation of NRCIR to luminance channel for contrast enhancement, and applies color space CIELch to directly manipulate chrominance and hue component of document images. Figure 1 illustrates the problems discussed above. As can be seen from figure 1(b), a simple histogram equalization to 3 channels not only destroys the channel balance but also highlights both target (image content) and noise (image degradation) which makes the recognition more difficult than ever. Indeed, some color constancy algorithm (figure 1(c)) enhances the contrast but at the expense of white balance misadjustment which would produce unrealistic color casts in the image. The proposed method, however, demonstrates encouraging performance with sufficiently improved luminance and chrominance contrast and preserved white balance for a friendly rendering of document images.
Figure 1: (a) 1st column: original images (b) 2nd column: histogram equalization (c) result of Retinex [7] (d) the proposed work
3. GENERAL FLOWCHART The NRCIR algorithm includes a global tone mapping serving as pre-processing, a modified Retinex with histogram rescaling of luminance channel (we called it here the NRCIR kernel), and a chrominance enhancement using a reference map, with another histogram rescaling serving as post treatment [10]. For document image enhancement, we designed another flowchart to cope with document image characteristics as shown in figure 2. The proposed method carries out image enhancement in three steps. The global tone mapping in step 1 is the same as first step of NRCIR serving as pre-processing. Thereafter, the NRCIR kernel (modified Retinex and histogram rescaling as described in [10]) is applied iteratively in order to strengthen luminance contrast since most documents images are now in high definition and do not suffer from blocking or ringing effect which relaxed the enhancement force restriction as in NRCIR. And with application of CIELch color space, the chrominance component can be proportionally enhanced with respect to luminance channel, and hue-constancy preservation can be realized by maintaining pre-processed hue component as illustrated in the flowchart. A simple gamma correction is applied at last serving as post treatment because the histogram rescaling generally renders image back to canonical illumination condition which makes such gamma correction a reasonable operation for a pleasant global appearance of image. 4. IMPLEMENTATION AND DISCUSSIONS Some essential points of NRCIR are recalled for a self complete description with special focus on the adaptations to document images. Albeit with modern techniques of scanners, it is still not rare to find some poorly controlled scanned documents with either too dark in appearance like
Figure 2: Global Flowchart of the proposed work
under-exposed scene or over bright appearance like overexposure which makes a pre-processing of global tone correction fairly necessary.
4.1 Global Tone Mapping In the proposed work, the same mapping curve as in NRCIR is adopted. It is circular arc whose radius is an empirical function of image key which is an index of dominating tone of image. (1)
!
"
'& (
!
"
#
#
$ %&
+ ) *&
(2)
where r is the radius of the circle, L(x,y) is the luminance of pixel (x,y) of image I. For over-bright scanned document, the global mapping step compress image tone level to make image dimmer, whereas for a darkly scanned version, it will increase tone level to brighter appearance. After pre-processing with global tone mapping, the document image is supposed to exhibit middle tone appearance and ready for enhancement of step 2. 4.2 Luminance and Chrominance Enhancement The mapped luminance is first enhanced by NRCIR kernel, that is, a modified one-filter version Retinex followed by a histogram rescaling step. The enhanced version of luminance is then fed back to Retinex step for an iterative implementation. This operation aims to strengthen the enhancement force to luminance channel since the enhancement restrictions to dark zone is largely relaxed for high-definition scanned document images. As discussed in [10], the modified Retinex luminance is defined as: ,-
. /
01234 5/
(3)
where Lr and Lgm refers to Retinex luminance and mapped luminance respectively. Im is an image mask obtained by a convolution between the mapped luminance and Retinex filter referring to formula (4) to (6) as defined in [12]. Note that the logarithm function applied to Im is essential in the proposed method in preventing halo effect. 67
9-
C
,27 8 9-
=012> ? @ : AB
7DE FAG 5 H
(4) ;< ; ;;
(5) (6)
where (x,y) is the coordinate of the image pixel varying from 1 to K. The profile of Retinex filter demonstrates a pointed shape which helps to enhance regional detail and a large base which helps to reduce artifact introduced by enhancement. The enhancement of luminance may sometimes result in unpredictable extreme bright points due to the division operators in NRCIR kernel such as equation (3). A white point correction is necessary before
linear normalization since the maximum value of the obtained result does not represent significantly the image content. The same principle of histogram rescaling as in NRCIR is adopted. It is applied as well to the chrominance channel for chrominance normalization. The number of iterations is empirically set to 3 according to our test which seems to be a wise tradeoff between enhancement performance and computational cost. Hue constancy property is usually preferred for image enhancement to render the image into more friendly appearance compared to its original version. Thanks to CIELch color space, the manipulation of chrominance and hue component is direct and intuitive. The hue component after pre-processing will be maintained unaltered in the proposed work for hue-constant reconstruction of image as shown in the global flowchart. 4.3 Gamma Correction as post-treatment The normalization using histogram rescaling renders image appearance back to canonical illumination. However, in document image scenario, a relatively bright appearance can to some extent facilitate reading and therefore be preferred. Hence a post-treatment using gamma correction is followed to render image with global pleasant appearance, and gamma value is empirically set to 1/1.8 according to our tests. 4.4 Eperiments and Discussion Extensive tests have been performed in order to evaluate the efficiency and robustness of the proposed method. Many historical document images suffering various degradations have been tested, and the results obtained confirm an encouraging performance of the proposed method. A typical test result is shown in figure 3. Figures 4(a) and 4(b) illustrate another example where selected zones are zoomed for comparison (text region and graphic region). 5. CONCLUSIONS AND PERSPECTIVE WORKS An automatic and adaptive degraded document image enhancement based on NRCIR is proposed in order to improve the luminance and chrominance contrast of document image while avoiding dramatic white balance changes and artifacts. The proposed method applies an iterative workflow using NRCIR kernel to enhance contrast information in CIELch color space after an adaptive global tone mapping. This iterative operation increases enhancement force without introducing artifacts for high definition scanned document images. The chrominance component is moderated enhanced and hue component is maintained unchanged for a visually coherent reconstruction of image. The proposed method outperforms other color constancy algorithms which usually perform independent channel operation in RGB or LAB color spaces and result consequently in a dramatic white balance changes. The gamma correction of post-treatment also helps to render
Figure 3: Left image is original, and right image is enhanced.
The image with more pleasant appearance. Albeit with some empirical parameters, the proposed method needs no parameter modification in practice, and its adaptability and robustness are proved by extensive test using document image database. However, the proposed flowchart cannot be generalized as an enhancement tool for natural scene image because for the latter more chrominance and saturation enhancement are usually desired. The iterative number, global mapping curve and gamma value for posttreatment are also to be optimized in future works. 6. REFERENCES [1] R. C. Gonzalez, R. E. Woods, S. L. Eddins, “Digital Image Processing”, Prentice Hall, 2004 [2] J. J.McCann, “Capturing a black cat in shade: past and present of Retinex color appearance models,” Journal of Electronic Imaging, Vol. 13, No. 1, pp. 36-47, 2004 [3] E. H. Land, “Recent advances in retinex theory," Vision Research 26(1), pp. 7-21, 1986. [4] http://dragon.larc.nasa.gov/ [5] R. Sobol, “Improving the retinex algorithm for rendering wide dynamic range photographs," in IS&T/SPIE Electronic Imaging 2002. The Human Vision and Electronic Imaging VII Conference., 4662, pp. 341-348, (San Jose), 2002. [6] A. Rizzi, D. Marini, L. Rovati, and F. Docchio, Unsupervised corrections of unknown chromatic dominants” Journal of Electronic Imaging , Vol 12(3),2003,
Figure 4(a):Upper image is original, lower image is enhanced.
[7] B. Funt, F. Ciurea, and J. McCann "Retinex in Matlab," Proceedings of the IS&T/SID Eighth Color Imaging Conference: Color Science, Systems and Applications, pp 112-121, 2000. [8] J. McCann, “Lessons learned from mondrians applied to real images and color gamuts,”, in Proc. IS&T/SID Seventh Color Imaging Conference: Color Science, Systems, and Applications, Scottsdale, AZ, pp. 1-8, 1999 [9] Z. Rahman, D. J. Jobsonz, G. A. Woodellz, “Retinex Processing for Automatic Image Enhancement,” J. Electron. Imaging, Vol. 13, 2004 [10] S. Chen, A. Beghdadi, “Natural Rendering of Color Image based on Retinex”, ICIP09, Cairo Egypt, Nov. 2009 [11] R. F. Moghaddam, M. Cheriet, “Low Quality Document Image Modeling and Enhancement”, International Journal on Document Analysis and Recognition, Vol. 11, Issue 4, March 2009
Figure 4(b): 1st row: zoomed text region (left image is original, right image is enhanced by the proposed work); 2nd row: zoomed graphic region comparison.
[12] L. Meylan and S. Süsstrunk, “Bio-inspired color image enhancement,” Human vision and electronic imaging. Conference No9, San Jose CA, USA, vol. 5292, pp. 46-56, 2004
