conducted to confirm that color conversion within the cm color space generates more realistic ..... and the term corresponding to lightness correction in Eq. (3.1).
A New Color Conversion Method for Realistic Light Simulation Toshiya Naka, Kenji Nishimura, Fumihiko Taguchi, and Yoshimori Nakase
ABSTRACT In this paper a new color conversion method to replace the RGB approximation based on the perceived change in color when a colored light source illuminates a surface is proposed. In our study 1200 uniformly distributed samples were taken from Commission Internationale de l'Eclairage 1976 L*a*b* uniform color space and used to construct a uniform subset. Photographic patches of the colors in this subset were then subjected to color light sources and the color changes were measured. From these measurements rules which govern the corresponding change within the color space were determined. Changes within the color space were then defined by simple linear equations for any light source illuminating any surface. Lastly, experiments were conducted to confirm that color conversion within the cm color space generates more realistic images than RGB approximations for a variety of light sources.
Key Words
Photorealistic, Radiosity, Color Conversion, Uniform Color Space,
lllumination
1 INTRODUCTION Currently, in computer graphics, methods for faithfully expressing physical phenomena are being widely researched. Optical simulation is one of the most impomlnt elements necessary for realistically generating computer graphics. The present paper will discuss a new color conversion method which provides faithful reproduction of changes in hue and saturation of an object under various illuminating conditions. In computer graphics based optical simulation, two aspects of light must be carefully considered: energy and color. Viewing light as energy is the most common way of considering light. In the 1980's two scientists (Whitted 1980; Hall 1986) proposed the ray-tracing method which considers only the specular reflection component of light energy on an object's surface. In
N. M. Patrikalakis (ed.), Scientific Visualization of Physical Phenomena 345 © Springer-Verlag Tokyo 1991
346 order to reproduce more realistic lighting conditions, two other scientists (Kajiya 1986; Cohen 1986; Cohen 1988) developed the radiosity method which takes into consideration diffuse inter-object reflection. These methods fonnulate a light energy relationship between objects. For photorealistic image generation the color of light must also be taken into consideration. All types of light have a relative spectral energy distribution which causes the human eye to perceive color. Accurate simulation of color changes of an illuminated object requires the calculation of the, energy transfer for all wavelengths in the visible range ( ). = 380 to 780 nm). Such simulation, however, requires a great deal of computation, and therefore is not practical in computer graphics. Thus, in conventional computer graphics, the consideration of the color of light, such as is needed for reproducing color changes of objects under color light sources, has been used only in a few cases (Hall 1987; Hall 1988; Meyer 1988). Our present study proposes a new color conversion model for expressing the change in color of an object under a light source with a smoothly varying spectral distribution. The features of this method can be summarized as follows: (1) The CIE ( Commission Internationale de l'Eclarirage ) color space is used instead of
conventional spectral band approximation such as ROB, because it is highly unifonn with respect to human visual perception. (2) Rules governing color changes, based on empirical findings, are defined by simple linear equations. (3) This method can be incorporated into conventional illumination calculation algorithms by converting ROB values to the CIELAB color space. Repeated experiments perfonned using this technique with a number of color light sources confmn that color conversion within the CIELAB color space produced less color difference than the conventional ROB approximation. Section 2 discusses problems with color calculation methods and then proposes a new color conversion method to be used for illumination simulation under a color light source. Section 3 describes simulation experiments with a number of color light sources. Section 4 provides an evaluation and Section 5 a summary.
2 COLOR EXPRESSION METHOD 2.1 Problems In the ROB system the color of an object is quantified by the three values, R, 0, and B, which can be calculated by Eq. (2.1). In this equation, light with spectral distribution L()' ) is incident on an object surface with spectral reflectance distribution p (). ). The ROB values are also weighted by visual cells R()' ), O()' ), and B( ). ) of the human eye which have three different spectral distributions.
347 R G B
= }; P ( A )L( A )R(). ) = }; p (). )L( A )G(). ) = }; P ( A )L( A )B( A )
~ ~ ~
A A A
where A is wavelength in the visible range from 380 run to 780 nm, ~
A is the change in wavelength. (2.1)
According to this mechanism, accurate simulation of the color of an object requires calculation for all possible A 's in the visible range. This calculation is computationally expensive and therefore not very practical. Hence, in conventional illumination algorithms an energy relation equation has been established with three spectral bands to express light colors: RGB. Achieving white balance is fundamental to satisfactory reproduction of colors. Experiments were conducted to determine the number of wavelength samples necessary for acceptable color expression. In these experiments two lights, magenta and cyan are projected onto a standard white surface as shown in Fig. 2.1(a). The relative spectral energy distributions of the light sources are shown in Fig. 2.1 (b). The results of these experiments are shown in Fig. 2.2. The vertical axis of Fig. 2.2 represents ratios RIG and BIG. The closer to 1.0 both of these are, the better the white balance. Using Eq. (2.1) for calculation and varying the number of wavelength samples taken it was determined that at least 20 wavelength samples must be taken in the visual range to achieve white balance. Summarized below are problems with conventional illumination simulation. (1) In conventional methods, such as ray tracing and radiosity, the energy equivalence at the
surface of an object can be obtained to a relative degree of accuracy. However, in converting the colors of surface texture based on this illumination data, shades are produced using the RGB approximation, thereby introducing a degree of inaccuracy. (2) Furthermore, in conventional methods, all light sources are colorless so the change in color on the surface of an object due to a colored light source cannot be simulated. Light A (Cyan)
LightB (Magenta)
(a)
(White)
Energy
~----r-----r----'
400
500 600 700 Wavelength (run) (b)
Fig.-2.1 Experimental (a) to obtain the white balance and (b) the relative spectral energy distribution of the lights.
348 RIG ,BIG 1.3 ""--"""T""---,r---.,....---,
Light source
Imax--...
RIG
1.2 t-+--+----1f---*B~/G~-I
l.lr.-\---+----1f----+----I
te b ance 0.9 L..-_.....L..._----lL.....-_....I......_---l o 10 20 30 40 Sampling number Fig. 2.2 The number of samples with wavelength ). and ratios RIG and BIG.
Patch
Fig. 2.3 Relationship between illuminance and luminance on an object surface
In order to find a solution to these problems a new color conversion method which could replace the RGB approximation method was investigated.
2.2 Uniform Color Space In response to the fact that the RGB color system corresponds poorly to the peculiarities of human vision the Commission Internationale de I'Eclairage (CIE) in 1976 proposed the CIELAB In any region within this space, human perception of the
uniform color space (CIE 1976).
difference in any two colors will always correspond to the actual distance between the two colors in the space. Therefore, with the introduction of this space it became possible to formulate linear approximations of color conversion rules.
Hence, in the following discussion,. the CIELAB
uniform color space is used instead of the RGB color system. In order to convert from the conventional RGB system to the CIELAB color space, RGB values in Eq. (2.1) must first be converted to CIEXYZ color system using Eq. (2.2). A point worth noting about the XYZ system is that the primaries X and Z, representing hue and saturation, are chosen on the non-luminance plane. Because of this, lightness information is represented in Y only. (X, Y, Z)
t
=
A (R, G, B) t
where A is the 3x3 color space conversion matrix. Refer to Appendix, Color Transfonnation.
(2.2)
349 Final conversion into the CIELAB unifonn color space is accomplished by using the resulting X,Y,and Z values from Eq. (2.2) in Eq. (2.3) where X o ' Yo and ~ are the values for standard white light and YIY0 expresses the reflectance of the object L"
116(YIY0>1f.l - 16
(YIYo > 0.(08856) (YIYo ~ 0.(08856)
L"
903.25(YIY0>
a
500[ (xlXif.l - (YIYif.l ]
b"
200[(YIYif.l - (ZIZo)If.l]
XO
= 0.9804,
Yo
= 1.0000,
~
= 1.1812 (2.3)
Color in the uniform color space is expressed in three components, (L", a", b") where L" represents lightness, and ( a", b") contain saturation and hue information.
2.3 New Color Conversion Method Conventional illuminance algorithms calculate the surface illuminance, I, of an object as shown in Fig. 2.3.
This value represents the energy that illuminates a surface under given lighting
conditions. For realistic images, it is necessary to convert texture color on a surface based on this illuminance value. Formerly, as an approximation, each spectral band of ROB was weighted with illuminance I. As an alternative to this method, the following discussion proposes using the uniform color space and converting illuminance data, I, into texture lightness, L". We propose to use the uniform color space instead of approximation to convert illuminance I to lightness L" of the texture. The relation equations for this are given in (2.4) - (2.6). Mter illuminance calculations have been done for all patches, illuminance of a given patch i is represented as Ii' the maximum value of all Ii is I max , and reflectance of patch i as P i' Assuming Yo to be the value of luminance Y for standard
white, the luminance value of light entering patch i, represented as Yin' is given by Eq. (2.4):
(2.4) Thus, luminance Yi of light reflected from patch i is equal
~o
the Y value of the incident light
multiplied by the reflectance, as given by Eq. (2.5), where constant k defaults to a value of 1.0 but may change depending on dynamic range and other characteristics of the display device.
Yi
k PiYin k P i (~JIma,.) Yo (2.5)
Further, by converting the relationship between luminance Y and lightness L" using the fIrst
350 expression in Eq. (2.3), the relationship given in Eq. (2.6) can be obtained (for simplicity, the coefficient Yo on the right side of Eq. (2.5) is assumed to be a.)
L·i in Eq. (2.6) is the lightness
of the texture illuminated by illuminance~. L·i
=
a
III
(L·iO + 16) - 16
where Cio is the value of texture lightness under standard C light source. (2.6) Thus, conversion to texture lightness values is possible by calculating the illuminance of individual objects under given illuminating conditions using conventional illuminance calculation algorithms and then introducing this illuminance ~ into the lightness calculation in accordance with Eqs. (2.4) to (2.6). As demonstrated above, when light energy is converted to the lightness of the surface texture of an object, color changes in the texture when subjected to color light can be expressed via color conversion within the uniform color space. Next, experiments were conducted to determine color changes of an object under color light
3 EXPERIMENT 3.1 Lig htness Correction In order to determine the color change when a texture, mapped on an object surface, is
illuminated with a non-colorimetry source, a number of color samples were prepared, and these changes were measured. Surfaces were assumed to be completly diffuse planes in this study.
L*
a*
Saturation
Fig. 3.1 Distribution of color samples in the uniform color space.
Fig. 3.2 Color patches used in color conversion experiment. (L * =75 to 90 intervals, hue angle = 0 to 360 degree and saturation = 70)
351
40
CCD image en or
b* 01---+-++--+-l"--+-~-H-1H----l
Light sources
Fig. 3.3
-20 '-----'-_ _---''--_ _. . . L - - . I 20 -20 a*
o
Configuration of the measuring system
Fig. 3.4 Example of measurments of changes in the color space under light sources differing in illuminance
First, photographic samples of 1200 colors distributed within the ClELAB color space were prepared. These color samples were uniformly distributed within the uniform color space as shown in Fig. 3.1. Each lattice intersection in Fig. 3.1 represents the chromaticity coordinate of a color sample in the color space. Fig 3.2 shows color samples with lightness L' ranging from 75 to 90 in increments of 5, with hue ranging from 0 to 360 degrees, and with a saturation value of 70. The illuminance of the light on these 1200 color samples was varied in four steps (with L' set to 45, 55, 60 and 70), and the amount of color change within the color space was measured. Table 3.1 provides the illuminating conditions. For measurement, a high performance CCD image scanner was used (PIC-2350, Ikegami Tsushinki Co.) Color sample data were converted from RGB values read from the scanner to the ClELAB uniform color space via Eqs. (2.2) and (2.3). Fig. 3.3 shows the configuration of the measuring system . In this measuring system it was possible to vary light source illuminance by varying power and to vary hue by using color filters. Fig. 3.4 provides examples of measurements taken of changes in color samples within the space (a' - b' plane; L' = 70). In the uniform color space lightness and luminance change virtually linearly with changes in light source illuminance. Based on these measured results and taking into consideration the color continuity within the space, changes within the color space as illuminance varies were approximated by the linear equation shown in (3.1). Chromaticity coordinates (L' i' a'i' b',) in the color space changes to (L'j, a'j, b') as the light source illuminance changes. Table 3.1 Dluminating condition
Light source
Table 3.2 Chromaticity coordinates
FL205W-EDL-50K (Toshiba Co., Ltd.)
Color temperature
5000K
Maximum illuminance
4000 Ix
Resolution
250 dpi
L1 L2 L3 L4 L5
Color
L*
a*
b*
Yellow Green Red Yellow Red
80 -4 70 -20 80 10 90 10 90 20
40 20 40 40 20
352
(j-:J L"w) a"i
a"j b", J
(l:JL"w) b"i
where L"w is the lightness of standard white under standard light source, and L"c the lightness of standard white as illuminance varies. (3:1)
Eq. (3.1) allows the conversion of illuminance data obtained under specific illuminating conditions to scenes where illuminance is arbitrarily changed. In this conversion the illuminance of the light source is converted to lightness value L"c using Eqs. (2.4) to (2.6). The second and third expressions in Eq. (3.1) are the formulation of relationship between the hue and saturation of the texture as light source illuminance is changed.
3.2 Color Conversion of Color Light Source This section describes a color conversion method for applying the method described in Section 3.1 to color light sources. In section 3.1 we considered cases where the light source was colorless. Conventional illumination algorithms handle only non-colorimetry sources so no accurate color conversion model has been presented for color light sources. The principal purpose of using the uniform color space in the present study is to obtain faithful visual reproduction of a scene with a color light source. To this end the illuminating condition under color light is reproduced by converting texture color. Hence, as in Section 3.1, existing color samples were illuminated with a color source, and changes within the color space measured. Table 3.2 shows the chromaticity coordinates (values converted into the CIE space) of three representative light sources selected from those used for the experiment. Fig. 3.5 shows sample measurements of color changes within the color space of 30 color samples when illuminated with light source L1 ( a" - b" plane; L" = 70, in two types).
40D~~§;:=IJ 20
-20
o
a*
20
40
Fig. 3.5 Example measurements of changes in color samples under color light source Ll.
353 L* p b*
a*
./-- - -- .....~ b*
o
a*
Fig. 3.6 Changes of color samples in the color space under a color light source. The color of the region marked ~ under standard white Light P. changes to the one in the region marked
under a color light Q
These experimental results indicate that changes in existing colors under color light follow certain rules, and thus, by using the method of least squares, the 1200 sample colors in the space can be approximated as given by Eq. (3.2).
(L·w / L·max ) L·i (L·w / L·... ) a·i + (1 +m) (L.J L·max ) a·w (L·w/L·max) b·i + (1 +m) (L.J L·max ) b·w (
m =0
)
)
(3.2) The rules of changes of color samples measured under color light is given below. In Fig. 3.6, point P represents the chromaticity coordinate value (L·max ,0,0) of standard white light in the uniform color space, and point Q the chromaticity coordinate value (L·w' a ·w' b·w) of the color light source. When texture at chromaticity point I (L·i , a·i , b·i ) is illuminated with color light, the point changes to chromaticity point J (L.j, a .j, b·j ) within the color space. First, the color space shrinks proportional to the amount of change in light source lightness from point P to Q ( L·w/ L·max ). This is indicated by the fIrst term of each expression in Eq. (3.2), and the term corresponding to lightness correction in Eq. (3.1). In Fig. 3.6, the color existing on the shaded plane changes to the one on the hatched plane, within the color space. Moreover, since light source Q has color components, the color space is distorted such that the amount of distortion is proportional to the amount of inclination of colorless axis OP to axis OQ. This is indicated by the second expression in Eq. (3.2). The coeffIcient m in this expression is an approximation of the saturation reduction of the color corresponding to the complementary color of the light source. In
354 Eq. (3.2), all hues within 0
0
of 0 are unifonnly distorted by IIlo only. These coefficients
correspond to the color purity of the light source. 0 /2 is equal to the angle of expansion from plane OPQ shown in Fig. 3.6. These coefficients are determined based on measurements, using a least square fitting method.
4 EVALUATION 4.1 Comparison with Conventional Method As a means of evaluating the color conversion model proposed in Section 3, approximation by the conventional ROB color system was compared with conversion using Eqs. (3.1) and (3.2). In the ROB method each ROB value of a texture is merely weighted with the illuminance of the object obtained from an illuminance calculation. The color difference within the color space,
t:. Eab"
between the respective approximations and the measured values was obtained with respect to the three light sources used for measurement in Section 3.
This color difference,
t:. E.b', is the
CIELAB color difference expressed by Eq. (4.1), and is the difference between two colors in a color space expressed quantitatively (Robertson 1977).
(4.1) Table 4 Optimum parameter and color difference Parameter 20 b*
mO
o +--+---+---+---0+--1
-20
o
a * 20
Ll L2 L3 L4 L5
lb
0.2 0.2 0.1 0.3 0.4
Color difference t:.Eab*
[deg.]
CIELAB
39 32 48 20 23
2.6 4.1 2.2 2.3 3.1
ROB 12.8 18.9 14.8 13.6 15.5
40
Fig. 4.1 Relationship between Changes of the color space Table 4 provides color differences between optimized values obtained by the method of least squares of coefficients IIlo and 0
0
and measured values with respect to each light source used for
measurement (Light source parameters are shown in Table 3.2. Light source Ll is yellow, L2 green, and L3 red). Fig. 4.1 reveals a close correspondence between the measured values of color distortion under light source Ll, and those obtained through approximation using Eq. 3.2 ( a' - b' plane; L' = 70). When m is 0.2 and 0 is 39 degrees in Eq. (3.2), the optimum approximation is
355
achieved, reaching values marked with broken line. For all three light sources used for the experiment, using our method it is possible to reduce
f).
Eab" to 1/3 of that obtained using the ROB
approximation. Although these color differences are clear in blue color where large measuring errors occur, the differences are within a permissible tolerance in other hues. Thus this color conversion method can provide sufficiently acceptable image quality for illumination simulation with computer graphics.
4.2 Simulation
with
Radiosity
lllumination scenes in which the present color conversion is applied to texture were constructed using illuminance data calculated by the radiosity method (the values of coefficients 1llo and 8
0
taken to be as shown in Table 4). Fig. 4.2 shows the results of our simulation. The scene in Fig. 4.2 uses seven light sources and some 1300 polygons, and some 110,000 patches employed for radiosity illuminance calculation. Fig. 4.2 (a) is an illumination scene under standard C light source, and (b) and (c) rooms generated by the color conversion method under color light sources L2 and L3, respectively. Fig. 4.3 (b) shows the scene of a room converted under another light source L6. Light L6 has the chromaticity coordinates (L" = 80, a" = -20, b" =-15) and coefficients are 8 0 = 35 degrees and 1llo = 0.7.
Repeated experiments with different light sources further confirmed that more faithful
simulation was possible with the proposed method than with the ROB approximation.
5 SUMMARY In the conventional illumination algorithm, conversion equations to correct texture color on an
object surface based on illuminance data are not fully established, and, accordingly, conversion is carried out by multiplying texture ROB values by illuminance. Moreover, since no consideration is given to color light sources, it is difficult to achieve faithful reproduction of subtle color changes caused by color light sources. In the present color conversion method, these problems have been solved by introducing the
uniform color space to process image data. The features of this method can be summarized as follows: (1) The method succeeds in formulating the relationship between illuminance and
lightness; (2) The method succeeds in approximating changes in texture hue and saturation linearly based on measured values. Models determined by Eq. (3.2) permit the generation of illumination scenes under given color light sources with as small an amount of computation as the conventional method. There is a
356
growing need for computer graphics to efficiently model color components of light sources in order to obtain photorealistic images. The presented method is widely applicable to such a need.
(a) Conventional radiosity and texture mapping
(b) Dluminated under light source L2
(c) Dluminated under light source L3 Fig. 4 .2 A test scene for color illumination
(a) Conventional radiosity and texture mapping
(b) Dluminated under light source L6
Fig. 4.3 Another test scene for color illumination
357
ACKNOWLEDGEMENTS The authors would like to express their thanks to Mr. Nishizawa, Mr. Nishimura, Mr. Hirai and other members of the SIG group of Matsushita Electric Industrial Co., Ltd. for their encouragement and assistance. The autors also thanks Richard Doerksen and Dabney Israel for their help in improving the manuscript
REFERENCES Cohen, M.P., D.P. Greenberg, D.S. Immel, and P.I.Brock, " An Efficient Radiosity Approach for Realistic Image Synthesis," Computer Graphics and Applications, 6(3), March 1986, pp 26-35. Cohen, M.F., S.E. Chen, J.R. Wallace, and D.P.Greenberg," An Progressive Refinement Approach to Fast Radiosity Image Generation," ACM SIGGRAPH 88, New York, August 1988, pp 75-84. Hall, R.A. and D,P. Greenberg, "An Testbed for Realistic Image Synthesis," Computer
Graphics and Applications, 3(8), November 1983, pp 10-20. Hall, R.A., " Hybrid Techniques for Rapid Image Synthesis," in Whitted, T., and Cook, eds.,
Image Rendering Tricks, Course Notes 16 for ACM SIGGRAPH 86 Dallas, TX, August 1986. Hall, R.A., " Color Reproduction and Illumination Models," from techniques for Computer
Graphics, edited by D.F. Rogers and R.A Earnshaw, PA, 1987, pp 194-238. Hall, R.A, " Illumination and Color in Computer Generated Imagery," Springer- Verlag, New York, 1989. Kajiya, J.T., "Rendering Equation," ACM SIGGRAPH 86, Dallas, TX, August 1986. pp 143-150. Meyer, Gray W., " Wavelength Selection for Synthesis Image Generation," Computer
Vision, Graphics, and Image Processing, vol. 41, 1988, pp 55-79. Robertson, AR., "The CIE 1976 Color Difference Formulae," Color Research vo1.2, 1977, pp 7-11. Supplement No.2 to CIE Publication No.l5 Colorimetry, "Official Recommendation on Uniform Color Spaces, Color-difference Equations, and Metric Color Terms," 1976. Whitted, T., "An Improved Illumination Model for Shaded Display," Communications of the A~ 23(6), June 1980, pp 343-349.
