Visual Neuroscience (2004), 21, 341–345. Printed in the USA. Copyright © 2004 Cambridge University Press 0952-5238004 $16.00 DOI: 10.10170S0952523804213074
Color constancy by asymmetric color matching with real objects in three-dimensional scenes
VASCO M.N. de ALMEIDA,1 PAULO T. FIADEIRO,1 and SÉRGIO M.C. NASCIMENTO 2 1 2
Remote Sensing Unit0Department of Physics, University of Beira Interior, 6201-001 Covilhã, Portugal Department of Physics, Gualtar Campus, University of Minho, 4710-057 Braga, Portugal
(Received September 7, 2003; Accepted January 20, 2004)
Abstract Color matching experiments use, in general, stimuli that are poor representations of the natural world. The aim of this work was to compare the degree of color constancy for a range of illuminant pairs using a new matching technique that uses both real objects and three-dimensional (3-D) real scenes. In the experiment, observers viewed a 3-D real scene through a large beamsplitter that projects on the right-hand side of the scene (match scene), the virtual image of a 3-D object (match object) such it appeared part of the scene. On the left-hand side of the scene (test scene), observers viewed a symmetrical scene containing a test object identical to the match object. Test and match objects were both surrounded by the same reflectances with identical spatial arrangement. The illuminant on the test scene had always a correlated color temperature of 25,000 K. The illuminant on the match scene could be any of seven different illuminants with correlated color temperatures in the range 25,000 K– 4000 K. In each trial, the observers, who were instructed to perform surface color matches, adjusted the illuminant on the match object. Constancy indices were very high (0.81–0.93), varied with the color of the match object, and increased with the extent of the illuminant change. Observer’s mismatches, however, were independent of the extent of the illuminant change. Keywords: Color constancy, Color matching, Real scenes, Color vision
image of a three-dimensional (3-D) real object on a real scene and therefore relaxed some of the main constraints on stimuli.
Introduction Human color constancy has been tested with a variety of experimental approaches: color naming (Troost, 1992), simultaneous asymmetric color matching (Arend & Reeves, 1986; Tiplitz Blackwell & Buchsbaum, 1988; Lucassen & Walraven, 1996; Brainard et al., 1997; Bäuml, 1999; Foster et al., 2001), successive asymmetric color matching (Brainard & Wandell, 1992; Bäuml, 1995), and achromatic settings (Fairchild & Lennie, 1992; Bäuml, 1994; Brainard, 1998; Kraft & Brainard, 1999). The main constraints on the stimuli used in these experiments are its abstract nature, the reduced dimensionality of the scene and0or test object, a limited set of illuminants tested, low luminance levels, and a limited color gamut. Constancy levels obtained are, in general, lower than expected from our everyday life but, typically, when more realistic environments are used as stimuli (Brainard, 1998; Kraft & Brainard, 1999) higher degrees of constancy are reported. The purpose of the present work was to test how color constancy varied with a large range of illuminant pairs and for test objects of different colors and with full three-dimensionality. The technique (de Almeida et al., 2002) optically created a virtual
Materials and methods Apparatus Fig. 1 shows a simplified diagram of the experimental apparatus. Observers viewed a real scene through a large beamsplitter that projected on the right-hand side of the scene the virtual image of a 3-D object (O). The virtual image was projected onto a dark carbon 3-D mask (K) with the same dimensions of the object O. The object O was a colored cube that was seen as part of the scene and indistinguishable from the other real objects on the scene. The scene was symmetric and in the left-hand side contained a real test object (T) with the same shape and size of the object O; both objects were covered with identical test papers, cut from the same sheet of paper. Fig. 2(a) shows the 3-D scene seen from the observer’s viewing position but without the beamsplitter; Fig. 2(b) shows the scene seen through the beamsplitter. Test object and match object (M), the virtual image of the object O, were both symmetrically surrounded by the same reflectances with an identical spatial arrangement. The scene was illuminated by a computer-driven liquid-crystal display (LCD) data projector (Epson EMP-5600)
Address correspondence and reprint requests to: Vasco M.N. de Almeida, Remote Sensing Unit0Department of Physics, University of Beira Interior, 6201-001 Covilhã, Portugal. E-mail:
[email protected]
341
342
V.M.N. de Almeida, P.T. Fiadeiro, and S.M.C. Nascimento scene (test scene) was always illuminated by an illuminant with correlated color temperature 25,000 K (test illuminant) and the illumination on the right-hand-side (match scene) could be any from seven different illuminants with correlated color temperatures of 25,000 K, 10,000 K, 7500 K, 6700 K, 6000 K, 4700 K, and 4000 K (match illuminants). All these illuminants had chromaticities drawn from the daylight locus (Wyszecki & Stiles, 1982). The middle region of the scene had a short and smooth transition between the illuminants.
Scene
Fig. 1. Diagram of the optical setup used in the experiment. Observers viewed the test object (T) through the large beamsplitter that projected a virtual image of the three-dimensional object (O) onto a dark threedimensional mask (K). This object and the entire scene were illuminated independently by high-precision computer-driven LCD projectors. The object O was seen as if it was part of the scene, and could not be distinguished from the other real objects on the scene.
and the object O was independently illuminated by another LCD data projector (3M MP8640) driven by a Visual Stimulus Generator VSG205 (Cambridge Research Systems, UK) with 8-bit resolution in each of the R, G, and B signals. The beamsplitter was a 400 mm 3 300 mm borosilicate glass, HEBBAR 50050 coating with 1 mm of thickness (Melles Griot Inc.). The illumination of both LCD projectors was defocused in the object plane to avoid unrealistic sharper shadows on the scene. The object O and its surroundings were carefully adjusted to reproduce the mutual reflections in the frontal and lateral faces as if the object was in the scene. The accuracy of the procedure was verified visually by comparing the appearance of the virtual object with that of the real test object (T) under the same illuminant. The left-hand side of the
The 3-D scene was viewed binocularly. The luminance level of test and match scenes was always 229 6 3 cd{m22, measured on a white sample of BaSO 4 placed in front of the test or match object. Because of the chromatic nonuniformity of the LCD arrays, both test and match scenes presented a chromatic variation of the illuminant from top to bottom of around 10%, which is within the limits indicated by the manufacturer. This situation, very common in nature, was not visually notorious. The distance from the objects to the observer ranged from 2.0 m to 2.5 m; the grey wall in the background was at a distance of 2.5 m. Test and match objects were placed 8.3 deg apart, center-to-center, and both subtended a visual angle of 1.5 deg.
Objects Objects were located on steps and were of two types: solids covered with colored papers and pieces of flat rectangular colored papers. The background wall was painted grey. In particular, objects T and O were covered with colored matte papers that could be red, green, yellow, blue, and grey with CIE 1931 ~ x, y! chromaticities of (0.436, 0.355), (0.322, 0.429), (0.411, 0.412), (0.243, 0.275), and (0.323, 0.338), respectively, when illuminated by CIE standard illuminant D65 . The chromaticities of the papers in the test scene were selected in order to be color name distinctive from any of the tested objects when illuminated by the test illuminant. The minimum distance between their chromaticities was 0.55 times the distance between
Fig. 2. (a) Three-dimensional scene viewed from the observer’s point of view but without the beamsplitter. The dark cube was placed exactly where the virtual image was projected. (b) The same scene viewed through the beamsplitter. Test (T) and match object (M) are indicated by arrows. Test and match scenes are also indicated and in this condition were illuminated by 25,000 K and 4000 K, respectively.
Color constancy in 3-D real scenes the positions of the 25,000 K and 6700 K illuminants in the CIE 1976 ~u ', v ' ! chromaticity diagram (Foster et al., 2001). The only conflicting point was between one of the papers tested (grey) and the wall painting (also grey) that were at a distance of 0.011 in the CIE 1976 ~u ', v ' ! chromaticity diagram. Observers The experiment was carried out by four observers with normal color vision verified by Rayleigh and Moreland anomaloscopy and normal acuity. Two were naïve, one was one of the authors of this paper, and the other was aware of the purpose of the experiment. Three observers were male, aged 23–33 years old, and one was female, aged 41 years. Procedure The total number of different conditions tested was 35: five colored papers under seven different match illuminants. Each of the four observers made six matches in each of these conditions. Observers were instructed to perform surface-color matching with control over the chromaticity and luminance of the match object, accomplished by adjusting the projector that illuminated O. In each experimental session, observers performed 14 matches with a 20-s pause between each match. Each observer only performed one experimental session per day that lasted about 50 min. The order of papers and illuminants was balanced over sessions. To accurately define the task, the observer read the following text with a picture of the scene: “The objects on the left-hand side of the scene are
343 made from exactly the same sheet of papers as the corresponding objects on the right-hand side of the scene. Eventually, they appear different because the color of the light may not be the same all over the scene. Your task is to adjust the light on only one object, the central cube on the right-hand side of the scene, in order to make the paper that covers it to look as if it was cut from the same sheet of paper as the corresponding cube in the left-hand side. When you have finished, the two sides of the scene should look as they really are made up of exactly the same pieces of papers, but eventually illuminated by light of different colors”. Calibration In a climatizated room with constant temperature, the LCD data projectors were warmed up 40 min before the calibration session. Illuminant calibration was then checked for test and match by pointing a telespectroradiometer (SpectraColorimeter PR-650; Photo Research, Inc., CA) to a BaSO4 sample placed in front of the test and mask object and avoiding the light that comes from O. The maximum error allowed in illuminant chromaticities was 0.002 in the CIE 1931 ~ x, y! space and 1.3% in luminance. The stability of the color of the illuminants was checked several times per day. After each match produced by the observer, the spectral result was acquired by pointing the PR-650 at the match object. Results Fig. 3 shows the results for two of the five colored papers tested: (a) red and (b) blue. Solid circles represent the chromaticity of the
Fig. 3. Results for two of the five colored papers tested: (a) red, and (b) blue. Solid circles represent the chromaticity of the test paper under the test illuminant, always 25,000 K, and open circles represent its chromaticity under each of the seven match illuminants. Triangles represent the mean match for four observers for each illuminant condition. Vertical and horizontal bars represent 61 S.E.M. across observers. The insets show the corresponding a and b values (see Fig. 4).
344
V.M.N. de Almeida, P.T. Fiadeiro, and S.M.C. Nascimento matches were averaged in CIE 1976 ~u ', v ' ! space and the corresponding color-constancy indices, represented in Fig. 5, ranged from 0.81–0.93. Discussion
Fig. 4. Representation of the Euclidian distances between test and matchilluminant conditions for the tested object (b value) and between exact- and adjusted-illuminant condition for the tested object under match illuminant (a value) used to compute the constancy index as (1 2 a0b!.
test paper under the test illuminant, always 25,000 K, and open circles represent its chromaticity under each of the seven match illuminants. Triangles represent the mean match for four observers for each illuminant condition. Vertical and horizontal bars represent 61 SEM across observers. For the red paper [Fig. 3(a)], observers’s matches were very close to and below the exact match. Similar regularity was found for the green, yellow, and grey papers, not shown. For the blue paper [Fig. 3(b)], the matches were, except for 4000 K, located above the exact match. SEM was similar within and across observers. A standard way of computing the observer’s color constancy is to consider how close the adjusted match is to the exact match (Arend & Reeves, 1986) and quantify it by a constancy index (Arend et al., 1991). In Fig. 4, consider b as the Euclidian distance between the chromaticity of the object under test and match illuminants, and a the extent of chromaticity change between the exact and the adjusted match. Then, the constancy index is computed as 1 2 a0b. The insets in Figs. 3(a) and 3(b) represent the variation of a and b with the correlated color temperature of the match illuminant. The mismatch a showed no significant effect of the correlated color temperature of the illuminant ~P . 0.3) although it depended on the color of the test paper ~P , 0.05). Fig. 5 shows the mean color-constancy indices for four observers and for all tested papers and illuminants. The indices were high and an analysis of variance showed that they increased with the correlated color temperature of the illuminant ~P , 0.01) and depended on the color of the test paper ~P 5 0.04). Observer’s
Fig. 5. Mean color-constancy indices for four observers and for all tested papers and illuminants. The insets represent observer’s matches averaged in CIE 1976 ~u ', v ' ! space and the corresponding color-constancy indices.
This work tested color constancy with real 3-D objects for a large collection of illuminant changes and for objects of different colors. The novelty of the approach consisted in using 3-D test objects whose color and luminance could be controlled by the observer. Color-constancy indices obtained with these experimental conditions were very high (0.81–0.93) and increased with the extent of the illuminant change. The high indices obtained may result from the fact that the scene and test and match objects were real. With such a setup, the observers understood the task easily and expressed little uncertainty about the procedure which is not always the case when using stimuli of more abstract nature. The increase of constancy indices with the extent of the illuminant change is consistent with matching predictions based on minimizing spatial cone-excitation ratios (Nascimento et al., 2004). Surprisingly, the amount of mismatch (a value) was independent of the extent of the illuminant change. If constancy is expressed simply as the extent of observer’s mismatch, it may be considered illuminant independent. These illuminant-independent results are consistent with those obtained in an achromatic locus experiment (Brainard, 1998) for illuminant changes out of the daylight locus.
Acknowledgments This work was supported by the Unidade de Detecção Remota da Universidade da Beira Interior, Covilhã, Portugal, and by the Centro de Física da Universidade do Minho, Braga, Portugal. We thank P.M.L. Monteiro and F.M.B. Ferreira for critical reading of the manuscript.
References Arend, L. & Reeves, A. (1986). Simultaneous color constancy. Journal of the Optical Society of America A 3, 1743–1751. Arend, L.E., Jr, Reeves, A., Schirillo, J. & Goldstein, R. (1991). Simultaneous color constancy: Papers with diverse Munsell values. Journal of the Optical Society of America A 8, 661– 672. Bäuml, K.H. (1994). Color appearance: Effects of illuminant changes under different surface collections. Journal of the Optical Society of America A 11, 531–542. Bäuml, K.H. (1995). Illuminant changes under different surface collections: Examining some principles of color appearance. Journal of the Optical Society of America A 12, 261–271. Bäuml, K.H. (1999). Simultaneous color constancy: How surface color perception varies with the illuminant. Vision Research 39, 1531–1550. Brainard, D.H. (1998). Color constancy in the nearly natural image. 2. Achromatic loci. Journal of the Optical Society of America A 15, 307–325. Brainard, D.H. & Wandell, B.A. (1992). Asymmetric color matching: How color appearance depends on the illuminant. Journal of the Optical Society of America A 9, 1433–1448. Brainard, D.H., Brunt, W.A. & Speigle, J.M. (1997). Color constancy in the nearly natural image. 1. Asymmetric matches. Journal of the Optical Society of America A 14, 2091–2110. de Almeida, V.M.N., Fiadeiro, P.T., Nascimento, S.M.C. & Foster, D.H. (2002). Colour constancy under illuminant changes with 3-D and 2-D views of real scenes. Perception 31, 135–135, Suppl. Fairchild, M.D. & Lennie, P. (1992). Chromatic adaptation to natural and incandescent illuminants. Vision Research 32, 2077–2085. Foster, D.H., Amano, K. & Nascimento, S.M.C. (2001). Colour constancy from temporal cues: Better matches with less variability under fast illuminant changes. Vision Research 41, 285–293.
Color constancy in 3-D real scenes Kraft, J.M. & Brainard, D.H. (1999). Mechanisms of color constancy under nearly natural viewing. Proceedings of the National Academy of Sciences of the U.S.A. 96, 307–312. Lucassen, M.P. & Walraven, J. (1996). Color constancy under natural and artificial illumination. Vision Research 36, 2699–2711. Nascimento, S.M.C., de Almeida, V.M.N., Fiadeiro, P.T. & Foster, D.H. (2004). Minimum-variance cone-excitation ratios and the limits of relational color constancy. Visual Neuroscience 21, 337–340.
345 Tiplitz Blackwell, K. & Buchsbaum, G. (1988). Quantitative studies of color constancy. Journal of the Optical Society of America A 5, 1772–1780. Troost, J.M. (1992). The invariance of color-perception. Irish Journal of Psychology 13, 440– 454. Wyszecki, G. & Stiles, W.S. (1982). Color Science: Concepts and Methods, Quantitative Data and Formulae. New York: John Wiley & Sons.
