J. Vis. Commun. Image R. 18 (2007) 429–438 www.elsevier.com/locate/jvci
Displaying colourimetrically calibrated images on a high dynamic range display Alexa I. Ruppertsberg a
a,*
, Marina Bloj a, Francesco Banterle b, Alan Chalmers
b
Bradford Optometry Colour and Lighting Lab, Division of Optometry, School of Life Sciences, University of Bradford, Bradford, UK b Warwick Digital Laboratory, Warwick Manufacturing Group, University of Warwick, Warwick, UK Received 15 November 2006; accepted 12 June 2007 Available online 25 July 2007
Abstract When colourimetrically characterising a high dynamic range display (HDR) built from an LCD panel and an LED backlight one is faced with several problems: the channels may not be constant; they may not be independent and there may be a significant radiant output at the black level. But crucially, colour transforms are underdetermined, which means that the number of colourimetric dimensions is smaller than the number of device channels. While the first three problems are associated with the LCD, the fourth problem stems from the additional channel in the HDR, the backlight. A 3700 flat-panel Brightside DR37-P HDR display was characterised. Using a spectroradiometer we recorded spectral radiance, chromaticities and luminance and estimated the true increase in gamut of the display due to the additional LED layer. We present a basic characterisation, propose a method for accurately presenting a desired luminance and chromaticity output despite the underdetermined problem and give an estimate of the available gamut. 2007 Elsevier Inc. All rights reserved. Keywords: Colour calibration; Display colourimetry; High dynamic range display
1. Introduction Since its introduction in 2003 [1] high dynamic range displays (HDRs) are becoming more and more popular, because of their increased luminance output and contrast ratio in comparison for example to cathode-ray-tube (CRTs) and flat-panel liquid-crystal displays (LCDs). HDRs can have a luminance output of up to 3000 cd/m2. They create a very vivid viewing experience for the observer, similar to watching a slide show. The design of a HDR varies, but in principle it consists of a LCD display with a strong backlight, which can be in the shape of a DLP or an LED layer [2]. Here, we consider a HDR with an LED layer. The LED layer, or the fourth channel, however is not independent, unlike in [3]. If the LED channel is set to 0, there is no output from the LCD channels. *
Corresponding author. Fax: +44 1274 235570. E-mail address:
[email protected] (A.I. Ruppertsberg).
1047-3203/$ - see front matter 2007 Elsevier Inc. All rights reserved. doi:10.1016/j.jvcir.2007.06.007
In this paper, we describe measurements to characterise a 3700 flat-panel Brightside DR37-P display. The device is capable of displaying so-called low dynamic range images with 8 bits per channel (e.g. .jpeg, .gif, .tiff, .png), high dynamic range images with 10, 12, 16 or more bits per channel (e.g. .hdr, .exr) and images that encode the LCD and LED information (e.g. 8bit+ and JPEGHDR). All these image formats code for RGB values, which are relative values in the sense that they do not represent absolute luminance values from a picture. While two same RGB values within a scene will correspond to the same luminance value, two same RGB values from different scenes will not. Conversely, taking two pictures of the same scene with different exposure times using a digital camera will lead to different RGB values in the two pictures from the same luminance value in the real scene. On the other hand, if we had an image in absolute luminance and chromaticity values, for example an CIEXYZ-tristimulus image, we could calculate the
430
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
corresponding RGB values for a given display device such that the luminance and chromaticity values measured from the display are the same as in the original XYZ-image (provided the colour gamut and luminance range of the display allow for this). XYZ-images can be calculated from hyperspectral images obtained from rendering programs such as RADIANCE [4], or from multiple images taken by a calibrated camera (e.g. [5]). To calculate the corresponding values for the LCD and LED layers we need to first colourimetrically assess the display device. Once we know the characteristics of the device the forward colour transform allows us to calculate for each RGB and LED setting the corresponding XYZ values. Because the number of colourimetric dimensions (n = 3) is smaller than the number of device channels (R+G+B+LED = 4) the inverse colour transform is underdetermined and a method needs to be developed. An additional problem is that the number of pixels in the LCD layer (2,073,600) far exceeds the number of LEDs (1380), making it difficult to display for example a black-and-white pattern with very thin stripes. To counteract this problem the manufacturer of the particular display we investigated developed proprietary software, which corrects this blur originating in the LED layer by manipulating the pixel values in the LCD layer. Note that if the local image contrast exceeds the dynamic range of the LCD layer the blur will not be removed entirely. Because we wanted to measure the output of the device by changing LCD and LED values systematically we used the 8bit+ image format. Images in floating point format such as .hdr images cannot be used for precise control over LCD and LED values because the device automatically scales the output [6]. A further consideration when displaying images with large bright areas on the display is to protect the device against overheating and to keep the power consumption to a reasonable level. The display will limit the power supplied to the LEDs when large areas would natively require a high LED level. While the blur correction is a desired feature, the automatic reduction in power to the LED layer when the image would cause the device to become too hot can cause problems. However, using the display control panel of the display it is possible to assess whether a given image will lead to a clamped power supply. 1.1. Outline In this paper, we describe the hard- and software set up for the HDR display and measurements (Section 2) to characterise the DR37-P display by establishing the spectral power distributions of the channels, the temporal stability, the black-level emissions and the optoelectronic transfer function (Section 3). We describe a method for obtaining the forward and inverse transforms to display a desired colourimetric output on a HDR (Section 4) and finally, we give an estimate of how many different colours can be displayed on this HDR (Section 5).
2. Hard- and software for display and measurement instrumentation 2.1. Display hardware The display hardware consisted of a 3700 flat-panel Brightside DR37-P HDR display (82 · 46 cm) controlled by a PC equipped with an Intel Pentium D at 3.2 GHz, 4GB of RAM and two NVidia GeForce 7900GTX graphics cards. The overall pixel resolution of the LCD panel was 1920 · 1080 pixels and the input values had a resolution of 8-bit per channel. The LED layer consisted of four LED panels in a 2-by-2 arrangement with altogether 1380 LEDs distributed in a honey-comb lattice over the entire size of the display (46 · 30 LEDs). Thus, on average one LED ‘illuminated’ about 42 · 36 pixels of the LCD. The LED channel had a resolution of 8-bit. 2.2. Light measurement instrument The light measurement instrument was a PhotoResearch Spectrascan 650 spectroradiometer. This instrument allows measurements of the emitted light in narrow-band spectral regions. Measurements are based on the light entering the spectroradiometer averaging across a small region of the monitor (1 deg of visual angle). For our measuring set up this corresponded to a 1.05 cm diameter patch of the monitor. We recorded the spectral power from 380 to 780 nm in 5 nm-steps (81 data samples), from which we calculated XYZ tristimulus values (CIE 1931; [7]), chromaticity (x, y) and luminance values. 2.3. Lighting All measurements were taken under identical lighting conditions, in a completely darkened room, whose walls and ceiling were painted dark grey minimising any ambient flare. The measurement laptop was in the same room but its screen was shielded to minimise light pollution. 2.4. Display control software To control the HDR we used the Control Display Panel Software of the HDR (HDRCtl (DR37-P) SW Ver. 48 FPGS Ver. 4.8) with single-link DVI, 8bit+ mode and hardware blur correction enabled. 2.5. Measurement software We used Matlab (Mathworks) to control the measurements by interfacing the spectroradiometer via the serial port to the measurement laptop [8]. To display test images on the HDR we used XnView, a free image converter program that also allows setting up slideshows (http:// www.xnview.com). However, any bitmap viewer, which allows full screen mode, would also be usable.
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
431
independent. Channel independence is a fundamental assumption in the characterisation of a display. In our case, it means the colour signal is a linear combination of the three primary lights from the LCD at a specific LED intensity. We can express the colour signal C(k) in the following form: CðkÞ ¼ LLED ðr C r ðkÞ þ g C g ðkÞ þ b C b ðkÞÞ
Fig. 1. Test image for the green channel (size 1920 · 1080 pixels). Measurements are reported from the left bottom patch. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
2.6. Test images Test images were generated with Matlab in bmp-format. A test image was 1920 · 1080 pixels and contained five square stimuli (each 250 by 250 pixels), one centred in each of the four LED panels and the fifth in the centre of the HDR occupying a small part of all four LED panels (see Fig. 1). Within a single test image all five squares were set to the same colour, the surround was set to black, i.e. the lowest digital input count (=0). For each channel (R, G and B) and all channels (W) we generated nine test images with the following digital counts as input: 0, 32, 64, 96, 128, 160, 192, 224 and 255. These bmp-images were then loaded into HDR Viewer (the proprietary software of the display manufacturer) with blur correction enabled, shown with HDR Viewer and saved as bmp-images, which converted them into 8bit+ images. With a script we modified the LED values in the header of the image file for the five squares. The display control panel shows the actual and the desired power consumption for a given image. Prior to measuring we confirmed that none of our test images lead to a clamped power supply.
ð1Þ
Cr(k), Cg(k) and Cb(k) are the relative spectral power distributions of the light emitted by the red, green and blue channel. The scalars r, g and b give the fraction of maximum output for each channel and LLED is the sum of the maximum luminance output of all three channels at a given LED intensity. Once the spectral power distributions Cr(k), Cg(k) and Cb(k) are known and the relationship between LED intensity and maximum channel output are known, only the remaining three scalars r, g and b are required to determine C(k). Fig. 2 shows the spectral power distribution of the R, G and B channel at maximum channel intensity for LED intensities of 32, 64, 128, 192 and 255. The description of the emitted light as a linear combination of three constant spectral power distributions (Eq. (1)) assumes that the three channels are independent, which means that the output of a given channel depends only on its input setting and not on the input that the other channels receive. We tested this assumption by comparing the tristimulus values for W with the sum of the pure colours as in Eq. (1) for LED intensities 64 and 192 for nine digital counts. Table 1 lists the absolute mean tristimulus value differences and maximum for each LED intensity. The percentage after the mean indicates the relative deviation with respect to the mean X, Y or Z tristimulus value. For both LED levels the deviation is 2% or less and close to the measurement error of the spectroradiometer (+/2%).
0.35
0.3
3. Measurement procedures and results spectral power [W/sr.m2]
This section describes the measurement procedures in detail and presents the results of the measurements. We report measurements from the patch in the left bottom LED panel, as the variation across the five test patches was small (+/4%) and comparable to that of a standard CRT.
0.25
B G
0.2
0.15
R
0.1
0.05
3.1. Spectral power distributions and channel independence 0 350
The light emitted by a monitor at a given location is described by its spectral power distribution and is called the colour signal. If the colour signal is a linear combination of three primary lights with fixed spectral power distributions, then it can be said that the device is channel
400
450
500
550
600
650
700
750
800
wavelength [nm]
Fig. 2. Spectral power distributions for the R, G and B channel (digital count = 255) at different LED intensities (32, 64, 128, 192 and 255). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
432
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
count = 128) for two LED levels (64 and 192). In Fig. 3a each point represents the two measurements at a single wavelength (Fig. 3 shows the data for LED level 192, the data for LED level 64 shows similar behaviour). If the channels were completely constant, the data points would fall onto a perfect line. A linear regression analysis reveals that all regression coefficients R2 are very high (LED = 64; R: 0.9962; G: 0.9932; B: 0.9974. LED = 192; R: 0.9969; G: 0.9929; B: 0.9977.). For LED level 64 the rms (root mean square) error between the linear regression and the data points is highest for the B channel in absolute terms (followed by the G and R channel). But in relation to the measured spectral power distribution peak, the rms error for the G channel is 2.8%, followed by the B and R channel with 1.7%. For LED level 192 the rms error is highest for the G channel in absolute terms (followed by the B and R channel). But in relation to the measured spectral power distribution peak, the rms error for the R channel is 3.1%, followed by the G (2.9%) and R channel (1.6%). Another test of the channel constancy assumption compares the measured spectral power distribution when all channels were set to maximum (i.e. a white patch) with the sum of the spectral power distributions of the individual R, G and B channels. Fig. 3b shows summed power over the measured power for all wavelengths for LED level 192. Again, if channels were completely constant, the data points would fall onto a perfect line. The regression coefficients R2 for both LED levels are 0.9999 and the rms errors in relation to the measured spectral power distribution peak are 1.9% and 2% for LED level 64 and 192, respectively. Note the different scales of the y-axes in Fig. 3a and b.
Table 1 Tristimulus value differences between mixed colours and sum of pure colours for LED intensities 64 and 192 |DXj
jDYj
jDZj
LED = 64 Mean Maximum
3.25 (2.1%) 6.32
3.51 (2.1%) 6.75
5.06 (2.4%) 9.75
LED = 192 Mean Maximum
8.01 (1.9%) 16.09
8.92 (1.9%) 17.81
11.57 (1.9%) 24.35
The percentage in brackets indicates the relative deviation with respect to the mean X, Y or Z tristimulus value.
We do not compute delta E values as this would involve transforming the CIEXYZ values into either CIELAB or CIELUV values, which both require the definition of a white point, whose intended luminance value is 100 cd/ m2 [7, p. 165] The white points measured here are clearly higher. Since neither colour space was developed for high luminance values, it is not clear whether the obtained delta E values would be meaningful. 3.2. Channel constancy
power for low input value [W/sr m2]
A further requirement for displaying calibrated images on a monitor is that the channels are constant, which means that the spectral power distribution for a given channel changes only by a scalar factor as a function of channel input (see also Fig. 2). To test for channel constancy [9] we compared the full power measurements (digital count = 255) to a lower power measurement (digital
0.012 0.018
R
0.01
0.045
G
0.014
0.035
0.006
0.01
0.025
0.004
0.006
0.015
0.002 0
0.005 0
. 0.008
0.002 0 0
0.02 0.04 0.06
0.08
0
0.04
0.08
0.12
B
0
0.05 0.1
0.15 0.2 0.25
power for maximum input value [W/sr.m2]
power for R+G+B [W/sr m2]
0.25
.
0.2 0.15 0.1 0.05
LED=192 0 0
0.05 0.1 0.15 0.2 0.25
power for W [W/sr.m2]
Fig. 3. Test for channel independence (LED = 192). Channel power for low digital count against channel power at corresponding wavelengths for maximum digital count. (a) For R, G and B. (b) Channel power for summed R, G and B at maximum digital count against measured channel power for W.
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
433
To assess whether the peak wavelength shifts towards shorter wavelengths with decreasing digital counts [10] we computed the peak wavelength from our spectral measurements. For the R channel the peak wavelength was 605 nm, for the G channel 545 or 550 nm and for the B channel always 455 nm. However, unlike [10] we could not observe a systematic shift of the peak wavelength towards shorter wavelengths with lower digital counts.
for a given LED intensity we measured and averaged the spectral power distribution of all channels (R, G, B, W) at channel intensity 1. For LED = 64 this yielded: x = 0.2930, y = 0.2850, Y = 0.59; and for LED = 192: x = 0.2923, y = 0.2862, Y = 1.76. Subtracting the black level emission from the measured data results in an improvement of the chromaticity values for low digital counts (see Fig. 4, filled symbols).
3.3. Black level emission
3.4. Temporal stability
Part of the channel constancy assumption is that the colourimetric values of a channel at various digital counts result in the same chromaticity but different luminance values. In Fig. 4 (open symbols) we show the chromaticity values for the R, G and B channel as a function of channel and two LED levels (64 and 192). It is clear that for low digital input counts (1, 32, 64) the chromaticity value is different from higher digital input counts (96, 128, 160, 192, 224, 256). As black level emission
To yield a valid calibration it is important to know how the monitor output behaves over time. There are two aspects to this; one is the immediate future and the other the more distant future, i.e. months. To assess how the monitor output behaves over a short time span (hours) we measured the spectral power distribution for a grey test image (digital count = 128) for one LED intensity over 3 h at various intervals. In Fig. 5 we show the luminance output for the grey test image for an LED intensity of 16. The
0.8 0.7
R
chromaticity
0.6 0.5 0.4 0.3 0.2 0.1 0 0.8 0.7
G
chromaticity
0.6 0.5 0.4 0.3 0.2 0.1 0 0.8 0.7
B
chromaticity
0.6 0.5 0.4 0.3 0.2 0.1 0
LED=64
LED=192 Channel and LED intensity
x y
Before black level correction
x y
After black level correction
Fig. 4. Black level emission. Chromaticity values for R, G and B channels for different digital counts and LED intensities before (open symbols) and after (filled symbols) the black level estimate was subtracted. Apparently missing chromaticity values after black level correction are outside the y-scale. This is due to the fact that the corresponding values before black level correction were close to their intended chromaticity already.
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438 60 50
1
40 30 20
0.8
R Relative luminance
Luminance [cd/m2 ]
434
LED=16
10 0 0
50
100
150
200
Time [min]
G B
0.6 0.4 0.2
Fig. 5. Temporal stability of luminance output for grey test image (digital count = 128) for LED intensity 16.
50
0 1
100
150
200
250
R G
0.8 Relative luminance
mean luminance output was 37.8 cd/m2 (SD = 0.08). The short-term temporal stability is extremely good. It is also evident that the display does not need a long warm-up phase, as the luminance output is stable from the very first minute. Because we have obtained the display only recently we cannot make any comments about the long-term temporal stability. We will make measurements in the future to assess the display’s long-term temporal stability.
LED = 64 0
B
0.6 0.4 0.2
3.5. Optoelectronic transfer function
LED= 192 0
The optoelectronic transfer function (OETF) describes the relationship between the input voltage to the display and the radiant output, and is also called gamma-function. Typically, the measured values can be described by an analytical function of the form: LðDACÞ ¼ a DACc þ b
ð2Þ
where a is a gain factor, b the offset, c indicates the power and DAC the digital input count. Although, LCD displays are designed to match the OETF of typical CRTs, their OETF can deviate from this model substantially and their functions are better described by lookup tables (LUTs) based on measurements and interpolation [11]. In Fig. 6 we have plotted the normalised OETF of our display system for all three channels at LED intensity 64 and 192 over nine digital input counts. The shape of the OETF is very similar to that of a CRT. However, the values for the R, G and B channel differ from each other. We fitted a model according to Eq. (2) to all three channels and obtained for both LED levels a gain factor a = 1.00, an offset b = 0.01, for LED level 64 a power c = 2.98 and for LED level 192 a power c = 2.92. The power (gamma) values are high. Normally, one expects gamma to be around 2.2. The rms error is highest for the B channel (0.12), followed by the R channel (0.05) and G channel (0.01) for both LED levels. Potentially, measuring in smaller intervals would reveal a further departure from the model [10] in which case a one-dimensional LUT for each channel could define the OETF more accurately [12]. 4. Colour transforms 4.1. Forward and backward transforms To accurately represent a given image on any display it is necessary to derive a mathematical description of the
0
50
100
150
200
250
Digital count
Fig. 6. Relative OETF of the display system for all three channels at LED intensities 64 and 192.
display response to a known input, also known as the forward characterisation transform [13]. To generate a mapping that determines the display input required for a desired response, the forward transform must be inverted. The transform maps between a device-dependent and a device-independent colour representation. The device-independent colour space is usually based on a three-dimensional (3D) colourimetric representation such as CIEXYZ or CIELAB, therefore explicitly incorporating the human visual system into the colour imaging path. The device-dependent colour space is also based on a 3D colour space, such as RGB. For a standard three channel device, a unique combination of device signals results in a unique 3D colourimetric representation of the displayed colour. These transforms are simple linear 3 · 3 matrix transforms. Conversion from RGB to XYZ: 2 3 2 3 0 1 2 3 Xr Xg Xb X R R 4Y 5 ¼ M 4G5 ¼ @Yr Yg Yb A 4G5 ð3Þ Zr Zg Zb Z B B The matrix M contains the tristimulus values of the red, green and blue channel at maximum intensity (Xr, Yr and Zr are the tristimulus values of the red channel at maximum intensity, similarly for the green and blue channel). Because of the unique mapping, the inverse transform is straight forward and involves computing the inverse of matrix M. From XYZ to RGB:
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
2
R
3
2
X
3
6 7 6 7 4 G 5 ¼ M 1 4 Y 5 B Z
ð4Þ
However, HDRs have not only three colour channels (from the LCD) but also a fourth channel (the backlight), which means that the number of device channels is greater than the number of colourimetric dimensions. Therefore, different combinations of device signals can result in the same colourimetric output, because the forward transform is a projection from a four-dimensional device space to a three-dimensional colourimetric space. Equivalently, all RGB-LED combinations that produce the same colour form a 1D-manifold in the 4D-device space. The inverse transform, i.e. the mapping from a desired colourimetric output c to the required device settings (RGB-LED) is therefore underdetermined. There are several settings that will yield the same desired colourimetric output c. For each LED intensity LED we can find a conversion matrix MLED that allows a mapping from RGB to XYZ (as in Eq. (3)). The effect of the LED intensity however, does not change the chromaticity of the colour channels but only the luminance output. Thus, all conversion matrices M can be written as a relative conversion matrix Mrel multiplied by a scalar lLED, where lLED corresponds to the summed maximum luminance output of all three colour channels at LED intensity LED. First, we compute matrix Mrel 2 3 1 2 3 2 3 0 X rel r X rel g X rel b R X rel R 6 7 C 6 7 6 7 B 4 Y rel 5 ¼ M rel 4 G 5 ¼ @ Y rel r Y rel g Y rel b A 4 G 5 Z rel r Z rel g Z rel b B B Z rel ð5Þ based on Xr, Yr and Zr from Eq. (3) (accordingly for the G and B channel): Yr Yr þ Yg þ Yb xr X rel r ¼ Y rel r yr Xr xr ¼ X r þ Y r þ Zr Yr yr ¼ X r þ Y r þ Zr Xr X r þ Y r þ Zr Xr X rel r ¼ Y rel r ¼ Y rel r X r þ Y r þ Zr Yr Yr Zr Z rel r ¼ Y rel r Yr
Y rel r ¼
ð5aÞ To yield the absolute tristimulus values X, Y and Z we multiply by the scalar lLED: 0 1 2 3 2 3 X rel r X rel g X rel b R X B C 6 7 6 7 ð6Þ 4 Y 5 ¼ lLED @ Y rel r Y rel g Y rel b A 4 G 5 Z rel r Z rel g Z rel b B Z
435
The scalar lLED is the sum of the maximum luminance output for the chosen LED intensity of all three channels (YLED_r is the luminance output for the R channel at maximum intensity for a given LED level, YLED_g and YLED_b accordingly). lLED ¼ Y LED r þ Y LED g þ Y LED
ð7Þ
b
To determine the required RGB and LED settings for a desired colourimetric output we use the inverse of matrix Mrel and divide by the scalar lLED: 2 3 2 3 R X 1 6 7 7 1 6 M rel 4 Y 5 ð8Þ 4G5 ¼ lLED B Z To account for the black level emission, we can rewrite Eq. (3): 2 3 2 3 X R 6 7 6 7 4Y 5 ¼ M 4G5 Z B 0 1 2 3 2 3 Xr Xk Xg Xk Xb Xk R Xk B C 6 7 6 7 ¼ @ Yr Yk Yg Yk Yb Yk A 4G5 þ 4Yk 5 Zr Zk
Zg Zk
Zb Zk
B
Zk ð9Þ
For convenience of writing the formulae we have assumed that the RGB values are gamma-corrected, i.e. the inverse of the OETF function (Section 3.5) has been applied [13], p. 152]. 4.2. Computing matrix Mrel from measurements From our measurements of the spectral power distribution of the R, G and B channel at maximum digital count for various LED intensities we derived the X, Y and Z tristimulus values. Table 2 lists all the conversion matrices and the relative conversion matrices that were calculated according to Eqs. (5–10), the mean relative conversion matrix and its inverse. As an example, we wish to know what the X, Y and Z tristimulus values of RGB-triplet Æ1, 1, 0æ at LED intensity 64 are: 2 3 0 1 2 3 421:5 0:3394 0:3859 0:1775 1 6 7 B C 6 7 4 523:8 5 ¼ 581:1 @ 0:1826 0:7188 0:0986 A 4 1 5 67:5 0:0058 0:1103 1:0030 0 Compare that to the measured tristimulus values of 427.3, 529.0 and 65.2. For the inverse transform the problem is in determining which scalar lLED to use. We need to find an LED intensity that will generate the required luminance output (=Y value). Fig. 7 shows the luminance output for each channel set to maximum (=255) for different LED intensities. We are looking for the LED intensity that is
436
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
Table 2 Conversion matrices M (based on measured tristimulus values), relative conversion matrices Mrel and luminance scalar lLED for six LED intensities (1, 32, 64, 128, 192 and 255) LED intensity
M
1
4.4 2.4 0.1
4.9 9.2 1.4
2.3 1.3 13.1
0.3409 0.1832 0.0048
Mrel 0.3856 0.7166 0.1096
0.1804 0.1001 1.0189
12.895
32
101 54.4 1.8
114.1 211.6 32
52.0 28.6 293.6
0.3429 0.1846 0.0060
0.3874 0.7185 0.1085
0.1767 0.0969 0.9969
294.51
64
198.8 107 3.6
225.2 417.8 63.6
102.3 56.3 578.7
0.3421 0.1841 0.0061
0.3875 0.7189 0.1094
0.1760 0.0969 0.9958
581.12
128
387.3 208.4 6.7
440.1 819.8 125.4
201.2 111.7 1138
0.3398 0.1828 0.0059
0.3861 0.7192 0.1100
0.1765 0.0979 0.9983
1139.9
192
564.4 303.6 10.1
644 1204 186.4
296.9 165.9 1678
0.3372 0.1814 0.0060
0.3848 0.7195 0.1114
0.1774 0.0991 1.002
1673.5
255
711.9 383.1 12.9
818.6 1536 240.2
379.8 214.3 2145
0.3337 0.1796 0.0061
0.3837 0.7199 0.1126
0.1780 0.1004 1.005
2133.4
Mean Mrel 0.3394 0.1826 0.0058
0.3859 0.7188 0.1103
0.1775 0.0986 1.0030
lLED
Mrel 1 4.0955 1.053 0.0921
2.1195 1.9575 0.203
0.5164 0.0061 1.0007
Averaging across the relative conversion matrices yielded mean Mrel and inversion of that the inverse conversion matrix Mrel 1 .
0 1 2 3 3 4:0955 2:1195 0:5164 325:4 0:0013 1 B C 6 7 6 7 @ 1:053 1:9575 0:0061 A 4 477:2 5 4 0:3512 5 ¼ 1673:5 0:0921 0:203 1:0007 617:5 0:3293 2
G
1600
Luminance [cd/m 2 ]
required to produce the desired luminance output. Because the slope for the G channel is steepest a lower LED intensity would be necessary to produce the desired luminance output than for channel R or B. However, this luminance output is only obtained by setting the G channel to its maximum setting. Potentially, the actual chromaticity we want to display has no green component at all and then the combined luminance output of the R and B channels will not be enough to reach the desired luminance output. If we chose the LED intensity based on the B channel we might always be using the highest LED intensity because the B channel has the lowest luminance output. Therefore, we suggest using the R channel to determine the LED intensity. Let Æ325.4, 477.2, 617.5æ be our desired XYZ tristimulus value; the luminance value is 477.2 cd/m2. By creating a lookup table from our measured data (Fig. 7), we can see that the luminance output of the R channel for 128 was 208.4 cd/m2 and 303.6 cd/m2 for 192. Thus, we set the LED intensity to 192 as this would make sure we obtain the necessary luminance output. lLED is then the sum of the maximum luminance outputs of all three channels at an LED intensity of 192, i.e. 303.6 + 1204 + 165.9 = 1673.5 cd/m2. The RGB triplet is then:
1200
800
R
400
B 0
0
50
100
150
200
250
LED level
Fig. 7. Luminance output against LED intensity for maximum digital count for the R, G and B channel.
So we have Æ0.0013, 0.3512, 0.3293æ at LED intensity 192. Our originally desired XYZ tristimulus value Æ322.2, 472.5, 611.4æ was actually measured from the RGB triplet Æ0, 1, 1æ at LED intensity 64, showing that different RGB and LED settings yield the same colourimetric output. To summarise the inverse transform a lookup table of the luminance outputs of all three channels for different LED intensities needs to be generated. Then, determine the LED intensity for which the R channel yields the desired luminance value. This will be the required LED intensity. lLED is then the sum of the luminance outputs for all three channels at that LED intensity.
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
437
Table 3 Estimate of the number of new colours added to gamut for each different LED intensity and their sum LED
1
32
64
128
192
255
Sum
#
16,777,216
14,586,320
1,968,624
1,937,250
531,360
160,272
35,961,042
5. Size of gamut The complete colour space of a display is called ‘gamut’ and is determined by the primaries of the device, i.e. the colour channels. If one plots the chromaticities of all three primaries in a chromaticity-luminance space (CIE xyY) for different digital counts (0–255 for the red, green and blue channels individually), the data points for each channel form a column in this space because they share the same chromaticity (x, y) but differ in their luminance values (Y). All data points of all three channel lie on the border of the gamut, and all colours that can be displayed lie within the irregular 3D-shape that these points form. The shape has about nine sides and can be described as an irregular elongated triangular dipyramid [14] (a polyhedron). These are two triangular pyramids that are joined by a common triangular prism. The bottom pyramid end corresponds to black (all channels set to 0, therefore no luminance output), the top to white (all channels set to 255, maximum luminance output, the sum of the maximum luminance output of the red, green and blue channel). If channel independence holds, 2563 = 16,777,216 colours can be displayed in an 8-bit per channel device. There are a plethora of colours that cannot be displayed because they lie outside the gamut (there are also colours within the gamut that cannot be displayed, because the resolution of the graphics card is finite). The colour space of a HDR has similar chromaticities to any other 8-bit per channel device, so there is no increased chromaticity resolution, but it is in the luminance dimension where the colour space is hugely increased. To visualise the gamut of a HDR another irregular elongated triangular dipyramid is added for each level of the backlight. However, they are not stacked on top of each other, but overlap, as they all share a common low luminance black level. So, while each of these dipyramids contains more than 16 million colours, they are not 16 million new colours. Therefore to calculate the number of different colours as 2564 = 4.295 · 109, i.e. more than 4 billion colours, is misleading. To obtain an estimate of how many different colours there are, we suggest the following approach. We know that for the lowest backlight intensity (LED = 1) we have 256 settings for each channel, producing 2563 = 16,777,216 colours. For the next backlight level we ask which of the 256 settings for a given channel produce higher luminance outputs than for the previous backlight level. For example for the red channel the digital settings (0–255,0,0,1) (R,G,B,LED) will produce luminance outputs between 0 and 40 cd/m2. The digital settings (0– 255,0,0,2) will produce, let us say, luminance outputs between 0 and 50 cd/m2. Which of the digital settings
produced a luminance output of more than 40 cd/m2? Assume only the settings (250–255,0,0,2) did achieve this, so 6 digital counts out of 256. If we do the same for the two other channels (e.g. 10 digital counts for the green channel and 4 for the blue channel), then only 6 · 10 · 4 = 240 colours were added by the different backlight level. In theory, one would do this for all 255 backlight levels. In practice, we approximated the overall number of colours in the gamut of the HDR by first evaluating the number of new colours added for six different LED levels (1, 32, 64, 128, 192, 255) and then summing them (see Table 3). Thus, the overall number of different colours for this particular display using 8bit+ images is approximately 36 million. Notice that the covered luminance range for each LED intensity increases as the LED level increases. This means that the luminance resolution for higher LED levels is much reduced. From a discrimination threshold point of view that does not pose a problem, because of Weber’s law [15], which states that a discriminable intensity increase is proportional to the current intensity. In plain English, you need to turn up intensity more to see a difference when the background intensity is already high. 6. Summary We have presented a characterisation of a HDR display and a forward and inverse colour transform for such a device. Even though the HDR has a 4D device space it is possible to derive a general solution to display a calibrated image from a 3D colour space. To be able to display such a calibrated image we need an image file format that allows us to manipulate the R, G, B and the LED channel. Some examples are 8bit+, JPG-HDR and MPEG-HDR [2]. The 8bit+ image format is typically a bmp-file format and the top raster line of the image codes the LED intensities for the backlight. Since there are far fewer LEDs than pixels it is crucial to apply the manufacturer’s blur correction. We have also estimated the number of different colours that the display is able to present as approximately 36 million, which is more than twice as large as what a standard 8-bit display device is capable of. The increase in luminance output of the display is profound and therefore the display comes closer to representing the lighting levels found in reality. Acknowledgments We would like to thank Timo Kunkel for help and support during the pilot measurements. This research was funded by a joint EPSRC/dstl Grant (Grant No. EP/ D032008/1).
438
A.I. Ruppertsberg et al. / J. Vis. Commun. Image R. 18 (2007) 429–438
References [1] H. Seetzen, L. Whitehead, G. Ward, A high dynamic range display using low and high resolution modulators, in: Society for Information Display Symposium Digest of Technical Papers, 2003. [2] E. Reinhard et al., High Dynamic Range Imaging. The Morgan Kaufmann Series in Computer Graphics, Elsevier, Amsterdam, 2006. [3] D.R. Wyble, M.R. Rosen, Color management of four-primary digital light processing projectors, Journal of Imaging Science and Technology 50 (2006) 17–24. [4] G.J. Ward, The RADIANCE lighting simulation and rendering system, in: SIGGRAPH 94, ACM Press, New York, Orlando, Florida, 1994. [5] F. Xiao et al., High dynamic range imaging of natural scenes, in: IS& T/SID Tenth Color Imaging Conference, Springfield, VA, 2002. [6] M. Trentacoste, Photometric image processing for high dynamic range displays (MSc thesis), in: The Faculty of Graduate Studies (Computer Science), The University Of British Columbia, Vancouver, 2006. [7] G. Wyszecki, W.S. Stiles, Color Science, second ed., (Wiley Classics Library), John Wiley & Sons, New York, 2000. [8] D.H. Brainard, The psychophysics toolbox, Spatial Vision 10 (1997) 433–436. [9] D.H. Brainard, Calibration of a computer controlled colour monitor, Color Research and Application 14 (1) (1989) 23–34. [10] R.S. Berns, S.R. Fernandez, L.A. Taplin, Estimating black-level emissions of computer-controlled displays, Color Research and Application 28 (2003) 379–383. [11] D.H. Brainard, D.G. Pelli, T. Robson, Display characterization, in: J.P. Hornak (Ed.), The Encyclopedia of Imaging Science and Technology, John Wiley & Sons Inc., 2002, pp. 172–188. [12] E.A. Day, L.A. Taplin, R.S. Berns, Colorimetric characterization of a computer-controlled liquid crystal display, Color Research and Application 29 (2004) 365–373. [13] D. Travis, Effective color displays, in: B.R. Gaines, A. Monk (Eds.), Computers and People Series, Academic Press, London, 1991. [14] E.W. Weisstein, Elongated Triangular Dipyramid. Available from: http://mathworld.wolfram.com/ElongatedTriangularDipyramid.html. [15] H.B. Barlow, J.D. Mollon (Eds.), The senses. Cambridge Texts in the Physiological Sciences 3, Cambridge University Press, Cambridge, 1982.
Alexa I. Ruppertsberg is a postdoctoral research fellow in the Vision Science Research Group at the University of Bradford. She obtained her first degree in Biology from the University of Tu¨bingen (1996) and her doctorate in Human Visual Perception from the Max-Planck Institute for Biological Cybernetics in 1999. She is a member of the Applied Vision Association Executive Committee and her research interest is in understanding human visual perception in natural settings and conditions.
Marina Bloj is a Senior Lecturer in Optometry in the School of Life Sciences at the University of Bradford. She has a BSc in Physics from Argentina, an MPhil (1996) and PhD (1999) from the Medical School at the University of Newcastle (UK). Her main research interest lies in exploring how humans perceive object and material appearance with particular emphasis on colour perception. She conducts research with both real and computer simulated stimuli and has become more interested in trying to establish what constitutes a perceptually realistic rendered image and what are the technical aspects that limit the realism of simulations.
Francesco Banterle is a PhD Student at Warwick Digital Laboratory at the University of Warwick. He has BSc in Computer Science (2004), magna cum laude, and a MSc in Computer Science (2006), magna cum laude, from the Universita` degli Studi di Verona. His research interests are High Dynamic Range Imaging, Graphics Processing Unit Programming, and Real-Time Rendering.
Alan Chalmers is a Professor of Visualisation at the new Warwick Digital Laboratory, WMG at the University of Warwick. He has an MSc with distinction from Rhodes University, 1985 and a PhD from University of Bristol, 1991. He has published over 140 papers in journals and international conferences on high-fidelity graphics and parallel rendering. He is Honorary President of Afrigraph, a member of the Eurographics Executive Committee and a former Vice President of ACM SIGGRAPH. His research goal is ‘‘Realism in Real-Time’’, obtaining physically-based realistic images at interactive rates through a combination of parallel processing and visual perception techniques.
