Estimation of Color for Gray-level Image by Probabilistic Relaxation Takahiko HORIUCHI Fac. Soft. & Info. Sci., Iwate Prefectural University Sugo 152-52, Takizawa, Iwate, 020-0193, Japan
[email protected] Abstract A color estimation method for a gray-level image is proposed by giving a few color pixels. It is known that a density value in the gray-level image will be calculated by linear combination of an RGB vector of the color image. The problem dealt with in this study can be formulated as an ill-posed problem which searches for an RGB vector from a density value as a solution. By assuming a restricted condition to minimize the total of the color difference defined among adjacent pixels, the color will be optimized by the probabilistic relaxation method. The performance of the proposed method is verified b y experiments. The proposed algorithm works very well when the solution is known with confidence in a few percents of the image.
1. Introduction Development of multimedia technology in recent years has enabled it to treat a vivid image easily on a computer. Then, the demand of restoring original color has been increasing to an image recorded as gray-level information, such as a BW movie and a BW photograph (they shall be called a gray-level image in this paper). For example, in amusement fields, such as a movie and a video clip it is colored by human’s labor and many gray-level images are distributed as vivid images. In other fields such as archaeology dealing with historical gray-level data and security dealing with gray-level images of a crime prevention camera, it can imagine easily that it is useful if color can be restored. There is no systematic study on colorization of BW imagery. Although there was a paper which restores the full-color image from a reduced color image as a similar study[1], it was a specific method using a color index, and cannot apply to gray-level images in this study. In this paper, the color image is expressed by RGB colorimetric system. The conversion technique from RGB to other colorimetric system is known. Though a density value of the gray-level image can be calculated uniquely
by linear combination of the RGB vector, searching for the RGB vector from a density value poses conversely an ill-posed problem in which two or more solutions exist. That is, in an image without any restrictions, it is impossible to restore a color image from a gray-level image completely. In order to solve this ill-posed problem, in this paper, a restricted condition is set up so that total of color difference among adjacent pixels may serve as the minimum, and color is estimated by solving this using the probabilistic relaxation method[2]. The solution of the probabilistic relaxation depends on the initial probabilities. In this paper, a few percent solutions are given as initial probabilities.
2. Problem Setting Let consider MxN image. Let Ni,j, (1< i < M, 1 < j < N) be an image coordinate (i,j). Let Ii,j be a RGB color vector (Ri,j, Gi,j, Bi,j) and Yi,j be a corresponding density value. The element of Ii,j and Yi,j are quantized by L1 and L2 bits, respectively. It is known that a color vector Ii,j and the density value Yi,j have the following relation: Yi,j = (0.299,0.587,0.114)Ii,j
T
(1)
Here, symbol T expresses transposition of a matrix. By Eq. (1), Yi,j can be given uniquely from Ii,j. Meanwhile, Ii,j cannot be given uniquely from Yi,j. It becomes the theme of this study how suitable color is estimated out of many candidate solutions.
3. Color Estimation Method 3.1 Setting of restricted condition The problem defined in Sec.2 cannot derive Ii,j from Yi,j uniquely. Therefore we try to derive an optimum solution by setting a restricted condition.
1051-4651/02 $17.00 (c) 2002 IEEE
Generally, as a natural image becomes highly resolution, change of a color with adjacent pixels becomes smaller. Of course, although color difference becomes large around the edge, it must be an appropriate assumption by considering the whole image. Then, in this paper, color difference with adjacent pixels gives restrictions so that it may become small by total of the whole image, and it tries estimation of the color vector for each pixel. Let di,j be a value of color difference for a pixel Ni,j among adjacent pixels. Then the restricted condition becomes (2) and the estimation problem is equivalently represented by the problem to derive a color vector I*i,j=(R*i,j, G*i,j,B*i,j) which satisfies Eq.(2). Note that the solution I*i,j also satisfies Eq.(1). In this paper, adjacent pixels are defined as 4-connected pixels in vertical and horizontal directions and the definition of color difference is a square norm ||.|| of difference between RGB vectors “.” so that conditions might become simple as much as possible. That is, the value of color difference di,j is defined as di,j=||Ii,j-Ii-1,j||+||Ii,j-Ii,j-1||+||Ii,j-Ii+1,j||+||Ii,j-Ii,j+1||. (3)
3.2 Estimation by probabilistic relaxation The problem to derive the optimum color vector I*i,j can be solved by round-robin method as combinatorial optimization problem. However, if an image is quantized by 8 bits (L1=L2=8), the number of candidate solution is about 17 millions as the maximum. Since the maximum number of candidate solutions exceeds more than 400 by the case of L1=L2=4, the combination of the (MxN)-th power of about 400 has to be investigated. Therefore, it is impossible to solve the problem in real-time even if L1=L2=4. Then, it is necessary to use a certain optimization technique. In this paper, a solution will be solved by using the probabilistic relaxation method[2]. The method is a technique mainly used in the various fields of image processing as the optimization technique for solving labeling problems. By using the relaxation method, it is possible to evaluate color difference by the "compatibility coefficient." Furthermore, since that is the technique of calculating the local optimu m solution, it is expectable to make it converge on a suitable solution, even if the image does not fulfill restricted conditions. On the other hand, it is the problem that the relaxation needs much calculation time. Since, it has been studied for reducing calculation time by parallel architecture[3],[4], it must be possible by using parallel processing.
Let Q be a set of candidate solutions Ii,j (it abbreviates to I for convenience henceforth) for density Yi,j of a pixel Ni,j (it abbreviates to N). The probabilistic relaxation gives candidates a probability pN(I),(sum.of pN(I)=1) for each pixel and a candidate I* with the maximum probability is selected as the solution for pixel N. The probability is updated by the following iterative equations:
. (4) In this paper, the following compatibility function (CF) q N(I) are used: (CF) (5) Here, N’ means an adjacent pixels of a pixel N , and I’ means a candidate solutions of N’. Symbol rNN’(II’) means the compatibility coefficients. The CF evaluates only a candidate with maximum probability for an adjacent pixel. The compatibility coefficients are formulated for evaluating color difference among adjacent pixels as follows:
(6) Equation(6) becomes the minimum value 2, when color difference is zero. When the color difference is the maximum, it becomes the minimum value 0, and it is formulated so that it may change linearly. Since the probabilistic relaxation gives semi -optimal solution, the result depends on the initial probabilities. In this paper, by setting partial probabilities as 1, partial solutions are given.
4. Experiments 4.1 Specification of data In order to evaluate the proposed method, the estimating experiments using SIDBA standard image database were performed. In the experiments, Milkdrop which has little change of a color among adjacent pixels, Parrots which includes various colors, and Lenna, which is standard are used from the database. Those original images consist of 256x256 pixels and quantized by 8bits for each RGB. Since there were restrictions of the memory of a computer, the image reduced to 80x80 and 4bits quantization (L1=L2=4) was used for this experiments. Figure 1 shows the color images (see in CD proceeding). Figure 2
1051-4651/02 $17.00 (c) 2002 IEEE
shows the gray-level image converted from Fig.1 by Eq.(1).
(a)Milkdrop (b)Parrots (c)Lenna Fig.1 Original images using for experiments.
(7) By giving a few percent correct pixels, the estimation with 26-28dB[pp/rms] SNR is possible.
Fig.2 Gray-level images for Fig.1.
1%
2%
3%
4%
5%
4.2 Color estimation by giving solutions partially When estimating color from a gray-level image such as the crime prevention camera, partial colors, such as a criminal's dress, may have become clear before estimation from the witness's information. Then, it was experimented what color estimation is performed when correct solutions were given partially. A correct candidate's initial probability is set as 1, and the candidate solutions of other pixels gave it equally. The pixels of correct solution were selected by random number. Figure 3 shows the result at the time of giving 3% correct solutions (see in CD proceeding). The number of iteration is 10. Although the purple solution could see partially, almost the same color as the original image is estimated subjectively.
6% 7% 8% 9% 10% Fig.4 Estimated images by giving solutions partially.
Fig.5 SNR between original image and Fig.4.
Fig.3 Estimated images by giving a correct solutions partially(3%). Figure 4 shows estimated images by giving correct solutions from 1% to 10% (see in CD proceeding). The number of iteration is 10. In order to verify the results objectively, SNR between an original image and the estimated one is shown in Fig.5. The definition of SNR is as follows:
When 3% solutions are given, it is shown in Fig.6 how the color is estimated (See in CD proceeding). Since something needs to display one color when a candidate's maximum probability is equal, the value around blue has been displayed for convenience. When there is little nu mber of iteration, therefore, the whole is displayed blue. In order to verify the results objectively, SNR defined in Eq.(7) is shown in Fig.7. The SNR increases in monotone and converges gradually.
4.3 Estimating possibility of color In this paper, the algorithm works when the solution is known with confidence in a few percent of the image. In this subsection, two kinds of experiments were performed
1051-4651/02 $17.00 (c) 2002 IEEE
for investigating the estimating possibility without giving solutions.
t=1
t=2
t=3
t=4
t=5
t=6 t=7 t=8 t=9 t=10 Fig.6 Estimated images by giving 3% solutions.
4.3.2 Experiment 2 In Experiment 2, a behavior is investigated by giving initial probabilities around the correct solution. The initial probability for correct candidate is twice others’ candidates . The initial probability of all other candidates is given equally. That is, for every pixel, 2/(“number of elements for Q”+1) is given to only correct candidate as initial probability, and 1/(“number of elements for Q”+1) is given to the remaining candidate solutions. The estimated result is shown in Fig.9 (see in CD proceeding). The number of iteration is 10. It can be confirmed that it remains the local optimum solution that is the original image, if the initial probability can be set up near a correct solution.
Fig.9 Estimated images by giving initial twice probabilities for solutions. Fig.7 SNR between original image and Fig.6. 4.3.1 Experiment 1 In Experiment 1, the equivalent initial probability was given to candidate solutions in each image. That is, 1/(number of elements for Q) was given as initial probability to all candidate solutions. Figure 8 shows the estimated results (see in CD proceeding). The number of iteration is 10 and a candidate color with maximum probability is displayed as the estimated result for every pixel.
5. Conclusion In this paper, a color estimating method from a graylevel image by the probabilistic relaxation method was proposed. The proposed method works well when the solution is known with confidence in a few percent of the image. This can be useful in cases where the colors of the objects in the picture is known, and instead of painting them manually, the user specifies isolated points in the picture. Application includes movie coloration and criminal identification. Since this study is just starting stage, many problems remain as follows: how to select initial probabilities, how to narrow down candidates and how to give the correct solution, etc.
References Fig.8 Estimated images by giving equivalent initial probability. The result shows that the estimated image is covered with purple and co mpletely differs from the color of original images. By analyzing the estimated color it turns out that many solutions have taken the value around I*=(7,7,7). It is considered that the result stays around the global optimum solution to minimize the color difference among adjacent pixels and converges into the middle of the quantization range.
[1] Barbala E. Schmitz and Robert L/ Stevenson, “Color Palette Restoration,” Graphical Models and Image Processing, vol.57, no.5, pp.409-419, 1995. [2] A.Rosenfeld, R.A.Hummel and S.W.Zucker, “Scene Labeling by Relaxation Operations,” IEEE Trans. System, Man, and Cybernetics, vol.SMC-6, no.6, pp.420-433, 1976. [3] K.Kamada, K.Toraichi, R.Mori, K.Yamamoto and H.Yamada, “A parallel architecture for relaxation operation,” Pattern Recognition, vol.21, no.2, pp.175-181, 1988. [4] K.Mori, T.Horiuchi, K.Wada and K.Toraichi, “A parallel relaxation architecture for handwritten character recogn ition,” Proc. IEEE PACRIM ’95, pp.74-77, 1995.
1051-4651/02 $17.00 (c) 2002 IEEE
