Linear Scale-Space of Spectral and Color Images - Semantic Scholar

FORTH-ICS / TR-263

December 1999

Linear Scale-Space of Spectral and Color Images Jon Sporring and Panos E. Trahanias

Abstract This paper addresses the problem of mixing spectral bands and space in a smooth, linear, and casual manner. Such properties are of fundamental importance for many basic multi-spectral-image processing algorithms that probe inter-spectral structure. The example of color images is used in a simple spectral perspective. An example is given comparing the deep structure of the color and the gray gradient magnitude.

Keywords: Linear Scale-Space, Spectral and Color Images, Deep Structure, Color and Gray Gradient Edges.

Linear Scale-Space of Spectral and Color Images Jon Sporring and Panos E. Trahanias Institute of Computer Science, Foundation for Research and Technology { Hellas, Vassilika Vouton, P.O. Box 1385, GR-71110 Heraklion, Crete, Greece

Technical Report FORTH-ICS / TR-263 | December 1999

c Copyright 1999 by FORTH

Abstract This paper addresses the problem of mixing spectral bands and space in a smooth, linear, and casual manner. Such properties are of fundamental importance for many basic multispectral-image processing algorithms that probe inter-spectral structure. The example of color images is used in a simple spectral perspective. An example is given comparing the deep structure of the color and the gray gradient magnitude.

Keywords: Linear Scale-Space, Spectral and Color Images, Deep Structure, Color and Gray Gradient Edges.

This work was supported by EC Contract No. ERBFMRX-CT96-0049 (VIRGO http://www.ics.forth.gr/virgo) under the TMR Programme. 

1 What's the Use of Color? Dierential analysis of color images is an aspect of image processing that has gained only little attention in the literature in comparison to gray images. One reason might be that much of the scene structure can be accessed through the dierential structure of the gray image. Another view is that there is no need to actually consider the full detail of color structure, since most humans perceive surfaces as having a single color despite quite large variations. However, the color image in Figure 2(RGB) has a constant gray image but clearly regions of dierent colors. A color gradient easily nds these areas, and tends to be much simpler than color constancy algorithms. Unfortunately, color gradients are rather unstable w.r.t. noise. The goal of this article is therefore to suggest a tool, from which color and spectral images can be analyzed for the spatial and color structure in a unifying, linear, smooth, and causal manner.

2 Spectral and Color Images In the following, the sampling nature of spectral and color images will be discussed. It will be shown that the simplest image analysis of spectral images is performed through Gaussian smoothing, and an example will be given of, how color images can be processed as a spectral image. Spectral images consist of a number of gray images, one for each selected wavelength (henceforth called channels). For example, typical color images use channels centered around wavelengths 435.8nm, 546.1nm, and 700nmy corresponding to blue, green, and red. A gray image is typically a set of pixels sampled on a regular grid independent of the scene. Objects captured in an image are larger than a pixel and smaller than the image size (inner and outer scale [1]). For analysis it is convenient to augment images with a scale-parameter to study images at all scales in a uniform manner, and possibly the simplest scale-space is the Linear Scale-Space (see [2] for a recent review), in which the original image is smoothed with a Gaussian of increasing width. To avoid confusion the Linear Scale-Space of gray images will be called the gray scale-space. A spectral image may be conceived as a stack of images taken with the same gray camera using dierent optical lters tuned for the increasing wavelengths. In this perspective, each channel clearly share the same spatial grid, and a spectral scale-space must therefore incorporate equal spatially smoothing of each channel. However, where a gray image represents the incoming light by a single scalar, a spectral image samples the light spectrum on a regular grid. In analog to, but independently of the spatial sampling, the spectral sampling encapsulates all spectral structure between the inner and outer spectral scales, and Gaussian smoothing of the spectral function is therefore the unique linear scale-space of spectra [2]. Speci cally, for a spectral image sampled on a 3-dimensional grid I (x; y; ), where x, and y are spatial coordinates and  denotes wavelength, the spectral scale-space is then obtained The values correspond to the most blue, green, and red wavelength according to Commission Internationale de L'Eclairage (CIE). y

2

by smoothing I with an axis-aligned 3-dimensional Gaussian kernel, L(x; y; ; ; ) = G(x; y; ; ; )  I (x; y; ); h

i

(1)

using G as the Gaussian kernel, G(x; y; ; ; ) = (22 )?3=2 exp ? x22+2y2 ? 2 22 . The structure of L(x; y; ; ; ) is the main study of this article with emphasis on processing color images as spectral images. It may be observed that CIE's primary color wavelengths are not equally spaced in wavelength, implying that a spectral smoothing of color images must be slightly unsymmetrical. The model used in the coming examples converts color to spectrum as follows: I (x; y; ) = B (x; y ) ( ? 435:8nm) +G(x; y)( ? 546:1nm) + R(x; y)( ? 700nm)

(2)

with B (x; y), G(x; y), and R(x; y) as the blue, green, and red channel, and () as the Dirac Delta function. Under Gaussian smoothing I thus becomes the Parzen estimator. The inverse operation is the simple process of reading the values of I at wavelengths f 435.8nm, 546.1nm, 700nmg. The model suggested in (2) is the mathematically simplest in spectral scale-space. Other models have been tested (see e.g. [3] and references therein), but lead to visually similar results.

3 Structure of Spectral Images The structure of spectral images is an essential tool for writing algorithms to correctly and suciently process spectral images. For example, for edge (scale-)focusing algorithms the spectral image structure entails all necessary information for implementation. The structure is given by the diusion equations, @ @2 @2 L = L + L; @ ( 2 =2) @x2 @y 2

@ @2 L = L @ ( 2 =2) @2

(3)

with L as short-hand for L(x; y; ; ; ). This is a special case of the Ane Scale-Space for which the catastrophe structure is given in [4]. In Figure 1 snapshots of a color image are given in spectral scale-space using (2). In this example, the scale parameters  and have been synchronized for practical reasons. While both space and wavelength can be related to meters, the wavelength range of visible light is far below pixel sizes in typical cameras. A small, common step in scale would make the color image completely gray but not smooth the channels noticeably. A pragmatic approach is therefore to de ne () = a setting a such that  and reach in nity at the same time, in which case the (3) simpli es to a single, linear diusion equation. For an N  N color image this amounts to sample the color spectrum in N points and use an isotropic Gaussian. The present implementation uses Fast Fourier Transform (FFT) for convolution, implying that the spectrum is mapped to a circle with blue as immediate neighbor to red. This yields certain similarities with psychophysical measurements of color neighborhoods. 3

4 Example: Color Edges The scaling of structures in spectral images can be studied in various ways. Spectral blob detection and scale selection are obvious examples, but due to space limitations, the properties of the spectral scale-space will be illustrated solely by comparing a color and a gray spatial edge measure in the corresponding spectral and gray scale-spaces. The spatial structure of spectral images can be obtained by either converting the image into gray followed by standard spatial analysis or by processing each channel independently and adding the results. These two approaches sometimes lead to quite dierent results. For a color image, the two gradients are given as

q

Egray Ecolor

= jr(R + G + B )=3j; = jrRj + jrGj + jrB j;

(4) (5)

using jr
j = ( @x@ )2 + ( @y@ )2 . Naturally, Egray  Ecolor. For the arti cial color image in Figure 2(RGB), Egray is a zero while Ecolor is shown in Figure 2(C-Gradient and CWatersheds). In contrast to Egray, it is observed that Ecolor is high where there is a color transition, hence capturing signi cant information about the spatial image structure. This is re ected in the location of the watersheds. As noted above, the color image approaches the gray image when ! 1. Hence Ecolor ! Egray , and given an operator T ,   in general, R + G + B , with shorthand notation when ! 1 then T (R( )) + T (G( )) + T (B ( )) ! 3T 3 R( ) = R(x; y; ; ; ). Hence, can be used to control the degree of color in consideration. A further empirical comparison of the dierence between the gray and the spectral scalespace can be obtained by examining the topological structure and its evolution with scale. The deep structure of the gradient magnitude of gray images is given in [5] together with an algorithm for tracking the watersheds of the gradient magnitude through scales. This algorithm has been used to visualize the dierence in structure between Egray and Ecolor. An example is shown in Figure 3, where the watersheds have been found for a number of scales and traced to zero scale for the two gradient measures. It is observed that the structure `blue words' is preserved in the spectral scale-space in contrast to the gray scale-space. Notice especially the capital letters `T' and `I'. Although the gray-gradient is essentially more smoothed than the color-gradient, lowering the scale in the gray scale-space cannot be used for compensation in this example, since the structure of the blue words is lost quite early in the smoothing process.

5 Discussion Florack [6] suggested that image processing should be considered as a sequential process, where image-operators give and take. In this perspective, convolution operators quite quickly lead to Linear Scale-Space for gray images. In analog, the repeating process of taking a color image of a color image quite quickly leads to Gaussian kernels both in the spatial and spectral domains. The spectral scale-space proposed here is thus an implementation of Florack's paradigm for color images. 4

The concept of soft isophotes [7] is a well-posed de nition of isophotes. Unfortunately, the concept of isophotes is somewhat dicult in spectral images, but it is felt that the direct smoothing of spectra is the best analog of the soft isophote. Further, soft (gray-)isophotes in conjunction with Local Gaussian weighted statistical operators have been demonstrated recently to be theoretically very simple but computationally very powerful tool [7, 8, 9] for gray image processing. Images in this operator space are called Locally Orderless Images. A simple property of these images is that the locality parameter commutes with spatial smoothing for the local mean, a property also found in the analogous Locally Orderless Spectral Operator space. Hence, in the game of guessing spectra from color [3], spatial smoothing is the statistically simplest way to integrate spectra in a local neighborhood for improved estimation.

References [1] J. J. Koenderink. The structure of images. Biological Cybernetics, 50:363{370, 1984. [2] J. Weickert, S. Ishikawa, and A. Imiya. On the history of Gaussian scale-space axiomatics. In Sporring et al. [10], chapter 4, pages 45{59. [3] Mark S. Drew and Brian V. Funt. Natural metamers. CVGIP: Image Understanding, 56(2):139{151, September 1992. [4] Lewis Grin. Critical point events in ane scale-space. In Sporring et al. [10], chapter 12. [5] Ole Fogh Olsen. Multi-scale watershed segmentation. In Sporring et al. [10], chapter 14. [6] L. Florack. Image Structure. Computational Imaging and Vision. Kluwer Academic Publishers, Dordrecht, 1997. [7] L. D. Grin. Scale-imprecision space. Image and Vision Computing, 15:369{398, 1997. [8] Jan J. Koenderink and Andrea J. van Doorn. The structure of locally orderless images. International Journal of Computer Vision, 31(2/3):159{168, 1999. [9] Bram van Ginneken and Bart M. ter Haar Romeny. Applications of locally orderless images. In Scale-Space Theory in Computer Vision, Proc. 2nd International Conference, volume 1682 of Lecture Notes in Computer Science, Corfu, Greece, September 1999. Springer-Verlag. [10] Jon Sporring, Mads Nielsen, Luc Florack, and Peter Johansen, editors. Gaussian ScaleSpace Theory. Kluwer Academic Publishers, Dordrecht, The Netherlands, 1997.

5

Figure 1: Some snapshots from the color scale-space. The color spectra are smoothed horizontally, and

the color planes are smoothed vertically. The image was provided by Christophe Garcia, FORTH, extracted from video material kindly provided by Institut National Audiovisuel, France.

6

B

R

G

RGB

C-Gradient

C-Watersheds

Figure 2: Color components (B, G, and R) of an image (RGB) with constant sum. Also shown is the sum of the gradient magnitude of each component (C-Gradient) and its watersheds (C-Watersheds).

7

Figure 3: The structure of spectral scale-space in comparison to gray scale-space. The gure shows the

location-corrected watersheds for  = f8:6; 16:8g (TOP to BOTTOM) in a 2562 image, using Egray (LEFT COLUMN) and Ecolor (RIGHT COLUMN).

8