Logarithmic Image Processing: Additive Contrast

Logarithmic Image Processing: Additive Contrast, Multiplicative Contrast and Associated Metrics M. Jourlina,∗, M. Carr´ea,b , J. Breugnotc , M. Bouabdellahb a University

of Saint-Etienne, Lab. H. Curien UMR CNRS 5516, Saint-Etienne, France b NT2I,BHT, 20 Rue B. Lauras, 42000 Saint-Etienne, France c Silab, Z.I. de la Nau, 19240 Saint-Viance, France

Abstract The LIP (Logarithmic Image Processing) Model is now recognized as a powerful framework to process images acquired in transmitted light and to take into account the human visual system. One of its major interests is linked to the strong mathematical properties it satisfies, allowing the definition and use of rigorous operators. In this paper, we first present the concept of Logarithmic Additive Contrast (LAC), its physical interpretation based on transmittance notion and some resulting properties: it represents by definition a grey level, it is highly efficient when computed on dark pairs of pixels, with applications for lowlighted images. Then the LAC is compared to the classical Michelson contrast, showing an explicit link between them. Furthermore, the LAC is demonstrated as very useful in the fields of automated thresholding and contour detection. Another major interest of the LAC is that it allows defining logarithmic metrics, opening various applications: grey level images comparison, pattern recognition, target tracking, defect detection in industrial vision and the creation of a new class of automated thresholding algorithms. Another part of the paper is dedicated to a novel notion of Logarithmic Multiplicative Contrast (LMC), which appears as a positive real number and also presents a ”physical” interpretation in terms of transmittance. Our research concerning the LMC remains to-day at an exploratory level if we consider the number of possible ways permitting to deepen this notion. In fact, the LMC values may exceed the grey scale maximum, which necessitates some normalization to display it as a contrast map. Nevertheless, the LMC is proved to be very sensitive near the bright extremity of 1

the grey scale, which is very useful to process over-lighted images. As the LAC, the LMC generates a lot of new metrics, particularly the Aspl¨ und’s one and a metric combining shapes’ and grey levels’ informations. Until now, Aspl¨ und’s metric was defined for binary shapes and is extended here to grey level images, with interesting applications to pattern recognition.

Keywords: contrast, Michelson contrast, logarithmic contrast, metrics, LIP Model, logarithmic image processing, automated thresholding, contour detection

Contents 1 Introduction, notations and recalls

3

2 Logarithmic Additive Contrast and Associated Metrics in the LIP framework 7 2.1 Recalls on the ”classical” Michelson contrast . . . . . . . . . 7 2.2 Definition of a Logarithmic Additive Contrast (LAC) in the LIP context . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 2.3 Applications of the Logarithmic Additive Contrast . . . . . 13 2.3.1 Application to automated thresholding and multi-thresholding 13 2.3.2 Application to contour detection . . . . . . . . . . . . 15 2.4 Metrics associated to the Logarithmic Additive Contrast . . 17 2.4.1 Recalls on functional metrics . . . . . . . . . . . . . . 19 2.4.2 The ”global” logarithmic metric d1 . . . . . . . . . . 22 2.4.3 The ”atomic” metric d∞ . . . . . . . . . . . . . . . . 24 2.4.4 The intermediate metric d1,supR . . . . . . . . . . . . 25 2.5 Applications of these Metrics notions . . . . . . . . . . . . . 27 2.5.1 Scene modifications . . . . . . . . . . . . . . . . . . . 27 2.5.2 Pattern recognition and target tracking . . . . . . . . 27 2.5.3 Metrics as correlation tools . . . . . . . . . . . . . . . 30 2.5.4 Characterization of pseudo-periodic textures . . . . . 30 ∗ Corresponding

author Email addresses: [email protected] (M. Jourlin ), [email protected] (M. Carr´e ), [email protected] (J. Breugnot), [email protected] (M. Bouabdellah) Preprint submitted to Advances in Imaging and Electron Physics February 6, 2012

2.5.5

A novel class of automated thresholding algorithms based on metric minimization . . . . . . . . . . . . .

33

3 Logarithmic Multiplicative Contrast and Associated Metrics in the LIP framework 34 3.1 Definition of a Logarithmic Multiplicative Contrast (LMC) in the LIP context . . . . . . . . . . . . . . . . . . . . . . . 34 3.2 Applications of the Logarithmic Multiplicative Contrast . . . 38 3.2.1 Application to automated thresholding and multi-thresholding 38 3.2.2 Application to contour detection . . . . . . . . . . . . 39 3.3 Associated metrics . . . . . . . . . . . . . . . . . . . . . . . 41 3.3.1 The ”global” metric . . . . . . . . . . . . . . . . . . 41 3.3.2 The ”atomic” metric . . . . . . . . . . . . . . . . . . 42 3.3.3 The ”intermediate” metric . . . . . . . . . . . . . . . 43 3.3.4 The ”Aspl¨ und” metric . . . . . . . . . . . . . . . . . 44 3.3.5 A bounded metric associating binary and grey level approaches . . . . . . . . . . . . . . . . . . . . . . . . 51 4 Conclusion and perspectives

54

5 Bibliography

55

6 Main notations

57

1. Introduction, notations and recalls Introduced by Jourlin et al (Jourlin and Pinoli (1988), Jourlin and Pinoli (1995), Jourlin and Pinoli (2001)), the LIP (Logarithmic Image Processing) Model proposes first a physical and mathematical framework adapted to images acquired in transmitted light (when the observed object is placed between the source and the sensor). Based on the transmittance law, the LIP Model proposes two operations, allowing the addition of two images and the multiplication of an image by a scalar, each of them resulting in a novel image. Such operations possess strong mathematical properties, as recalled hereafter. Furthermore, the demonstration, by Brailean (Brailean et al. (1991)) of the LIP Model compatibility with human vision, considerably enlarges the application field of the Model, particularly for images acquired in reflected light on which we aim at simulating human visual interpretation. 3

In the context of transmitted light, each grey level image may be identified to the observed object, as long as the acquisition conditions (source intensity and sensor aperture) remain stable. An image f is defined on a spatial support D and takes its values in the grey scale [0, M [, which may be written: f : D ⊂ R2 → [0, M [ ⊂ R Note that within the LIP Model, 0 corresponds to the ”white” extremity of the grey scale, which means to the source intensity, i.e. when no obstacle (object) is placed between the source and the sensor. The reason of this grey scale inversion is justified by the fact that 0 will appear as the neutral element of the addition law defined in formula 3. The other extremity M is a limit situation where no element of the source is transmitted (black value). This value is excluded of the scale, and when working with 8-bits digitized images, the 256 grey levels correspond to the interval of integers [0, .., 255]. Note that in some schemes like histograms, contrasts curves..., we will come back to the classical scale with 0 = black. The transmittance Tf (x) of an image f at x ∈ D is defined by the ratio of the outcoming flux at x by the incoming flux (intensity of the source). In a mathematical formulation, Tf (x) may be understood as the probability, for a particle of the source incident at x, to pass through the obstacle, that is to say to be seen by the sensor. The addition of two images f and g corresponds to the superposition of the obstacles (objects) generating respectively f and g. The resulting image will be noted f g. Such an addition is deducted from the transmittance law: Tf g = Tf × Tg (1) which means that the probability, for a particle emitted by the source, to pass through the ”sum” of the obstacles f and g, equals the product of the probabilities to pass through f and g, respectively. Jourlin and Pinoli (Jourlin and Pinoli (2001)) established the link between the transmittance Tf (x) and the grey level f (x): Tf (x) = 1 −

4

f (x) M

(2)

Replacing in formula 1 the transmittances by their values deducted from 2 yields: f.g (3) f g = f +g− M From this addition law, it is possible (Jourlin and Pinoli (1988), Jourlin and Pinoli (2001)) to derive the multiplication of an image by a real number λ according to: !λ f λ f = M −M 1− (4) M Remark 1: such laws satisfy strong mathematical properties. In fact, if I(D, [0, M [) and (F(D, ]−∞, M [) design respectively the set of images defined on D with values in [0, M [, and the set of functions defined on D with values in ]−∞, M [, we have (Jourlin and Pinoli (2001)): (F(D, ]−∞, M [), , ) is a real vector space and (I(D, [0, M [), the positive cone of the precedent.

,

) is

Considering these results, Pumo and Dhorne (Pumo and Dhorne (1998)) design the LIP Model as an Optical Vector Space. Remark 2: the introduction of the ”over-space” (F(D, ]−∞, M [), , ) permits to associate to each function an ”opposite” an then obtain the vector space structure. The opposite of a function f is noted f and is classically defined by the equality f ( f ) = 0. Its expression is: −f f 1− M

f=

and the difference between two functions exists and satisfies: f

But it is clear that |f

g=

f −g g 1− M

(5)

g| 6= |g f |.

In case where g(x) ≤ f (x) for each x lying in D, formula 5 applies in the space of images (I(D, [0, M [), , ) and results in an image of the same 5

(a)

(b)

(c)

(d)

Figure 1: Adding and subtracting a constant from an image: (a) initial image f representing stratum granulosum on in-vivo confocal microscopy, (b) constant grey level image C = 120, (c) f C, (d) f C

space. Note that such a subtraction will be at the origin of the Logarithmic Additive Contrast. Remark 3: adding or subtracting a constant C (homogeneous image) to an image f permits darkening or brightening f (cf. Fig.1). Remark 4: the multiplication of an image f by a real number λ possesses a very strong physical interpretation: in fact λ controls the ”thickness” of the considered obstacle which is doubled if λ = 2. More generally, the image λ f is darker than f for λ ≥ 1 and λ f is darker than µ f if λ ≥ µ. On the opposite, λ f will appear brighter than f for λ ≤ 1. 6

(a)

(b)

(c)

Figure 2: Brightness control of an image: (a) 2 f , (e) 4 f

(d) 1 4

f , (b)

1 2

(e)

f , (c) initial image f , (d)

From this remark, it appears clearly that the logarithmic multiplication λ f allows controlling the brightness of an image (cf. Fig.2). Furthermore corrections may be applied to images f and g acquired under variable illumination or aperture (cf. Fig.3). As an example, λ and µ are computed to obtain the same average grey level (here 128) for λ f and µ f .

From remarks 3 and 4, we observe that each law of our vector space structure (addition-subtraction on one hand and scalar multiplication on the other) is efficient to perform brightness modifications. They are then applicable for correcting lighting variations, to enhance low-lighted images (near night vision), all the corresponding algorithms being performed in real time (25 images per second with a classical Personal Computer). To conclude this introduction part, note that the presence of a vector space structure permits the use of various efficient tools associated to this kind of space: logarithmic interpolation, scalar product...(Jourlin and Pinoli (2001)). 2. Logarithmic Additive Contrast and Associated Metrics in the LIP framework 2.1. Recalls on the ”classical” Michelson contrast Given a grey level image f and two points x and y lying in D, it is common to define the ”Michelson” contrast of f at the pair (x, y) according

7

(a)

(b)

(c)

(d)

Figure 3: Brightness control of an image: (a) and (b) initial image acquired under variable aperture conditions, (c) and (d) corresponding homothetic images with average grey level 128

8

to: M ax(f (x), f (y)) − M in(f (x), f (y)) M ax(f (x), f (y)) + M in(f (x), f (y)) |f (x) − f (y)| = ∈ [0, 1] f (x) + f (y)

m C(x,y) (f ) =

Remark 5: • Take care that in the case of this classical contrast, the origin 0 of the grey scale represents the ”black” extremity. • The ”Michelson” approach clearly overestimates the contrast of dark pairs of points compared to bright pairs with the same grey levels difference. • If one of the two considered pixels is black (grey level 0), the value of their Michelson contrast equals 1, independently of the second pixel grey level. Note also that C m is not defined when f (x) = f (y) = 0. m (f ) to the For these reasons, we will limit the computation of C(x,y) case where f (x) 6= 0 and f (y) 6= 0. m (f ) ∈ [0, 1[. In such conditions, the value 1 is not reachable: C(x,y) This result will appear essential in order to demonstrate an explicit link between the Logarithmic Additive Contrast and the Michelson contrast.

2.2. Definition of a Logarithmic Additive Contrast (LAC) in the LIP context In the LIP framework, Jourlin et al. (Jourlin et al. (1989)) introduced the Logarithmic Additive Contrast (LAC), noted C(x,y) (f ), of a grey level function f at a pair (x, y) of points lying in D2 . It is defined according to the following equation: M in(f (x), f (y)) (C(x,y) (f )) = M ax(f (x), f (y))

(6)

”Optical” interpretation: Such a contrast represents the grey level which must be added (superposed) to the brightest point (smallest grey level) in order to obtain the darkest one (highest grey level). Then this logarithmic contrast may be visualized without any normalization. 9

Using the addition formula(3) yields: |f (x) − f (y)| (7) M in(f (x), f (y)) 1− M The same reasoning permits to define the Logarithmic Additive Contrast C(x) (f, g) between two grey level functions f and g at a same point x of their spatial support D: C(x,y) (f ) =

C(x) (f, g) =

|f (x) − g(x)| M in(f (x), g(x))) 1− M

(8)

Properties and results: • It is possible to express this contrast as a LIP subtraction: C(x,y) (f ) = M ax(f (x), f (y)) M in(f (x), f (y)) • The LAC is clearly a sub-additive and homogeneous operator on the space of grey level images (I(D, [0, M [): C(x,y) (f

g) ≤ C(x,y) (f ) C(x,y) (g)

C(x,y) (λ f ) = λ C(x,y) (f ) m (f ) and • It is important to observe that the considered contrasts C(x,y) C(x,y) (f ) are of the same nature in the sense that each of them enhances the contrast of dark pairs of points compared to bright pairs with the same grey levels difference |f (x) − f (y)|.

To illustrate that remark, let us consider the following results: f (x) = 240

f (y) = 200 LIP context: C(x,y) (f ) = 183

f (x) = 55

f (y) = 15 LIP context: C(x,y) (f ) = 42

f (x) = 15

m (f ) × 255 = 146 f (y) = 55 Michelson: C(x,y)

f (x) = 200

m (f ) × 255 = 23 f (y) = 240 Michelson: C(x,y)

10

Comment: To obtain comparable values between the two contrasts, the Michelson one has been multiplied by 255. For the same reason, the chosen grey level values take into account the grey scale inversion (the ”white” extremity corresponds to 0 in the LIP context). More generally, for each pair (x, y) of pixels presenting a constant grey level difference k, i.e. f (y) = f (x) + k in the LIP scale (y is darker than x), compute: m (M − 1 − f ) C(x,y) (f ) and C(x,y) That means an inversion of the LIP scale (the function f becomes M –1–f ). Moreover, the ”Michelson” contrast taking always its values in the interval [0, 1[, must be multiplied by 255 to be compared to C . Finally, we present m (M –1 − f ) for in Fig.4 the representative curves of C(x,y) (f ) and 255 × C(x,y) various values of k.

(a)

(b)

(c)

Figure 4: The LAC and Michelson contrasts for (a) k = 10, (b) k = 40 and (c) k = 80

11

Comment: as previously observed, it clearly appears that for a same value k, the contrast in the LIP sense is greater than in the Michelson one. Nevertheless, the shape similarity between the curves let suppose that some link may exist between the two contrasts, as demonstrated by the following result: Theorem 2.1. The Michelson contrast is a logarithmic contrast, according to the formula: M C m (f (x), f (x) + 2k) = C (M − f (x), M − f (x) − k) Proof Consider the classical grey scale and the inverted (LIP) grey scale:

We can write: • in classical situation m C(x,y) (f ) =

f (y) − f (x) 2k k = = f (y) + f (x) 2f (x) + 2k f (x) + k

• in the LIP context C(X,Y ) (f ) =

|X − Y | k Mk = = M in(X, Y ) M − f (x) − k f (x) + k 1− 1− M M

and the formula is established. Fundamental remark: The precedent result gives a precise ”physical” meaning to the Michelson contrast. As the LAC, it is interpretable in terms 12

of transmittance. Note that the ”thickness” separating the considered grey levels must be doubled for Michelson. This fact explains why, at the beginning of our comparison, the LAC seemed more ”sensitive”. Now let us present some applications of the LAC. 2.3. Applications of the Logarithmic Additive Contrast In this part, we only aim at showing the efficiency of our logarithmic contrast and we selected two examples among a lot of possibilities: automated thresholding and contour detection. 2.3.1. Application to automated thresholding and multi-thresholding Thresholding (multithresholding) applied to an image is one of the simplest methods in order to classify it, i.e. to separate it into two (or more) classes. From the end of the 70’s, a lot of papers have been dedicated to automated thresholding, based on Interclass Variance Maximization (Otsu (1979)), Entropy Maximization (Pun (1981)), Statistical Moments Preserving (Tsai (1985)), and many other techniques. Most approaches consist in optimizing some parameter computed on the image histogram, and thus don’t take into account the distribution of the grey levels in the image definition domain, in opposition with what a human eye would do. Nevertheless, one method associates a certain contrast notion to the boundary generated by a given threshold, and proposes an optimization of it: the method proposed by K¨ohler (Kohler (1981)). Considering the LAC as consistent with human vision (as recalled at the beginning of Part 1), we propose to replace the contrast notion initially used by K¨ohler by its LAC version. K¨ ohler’s Method recall: Given a grey level image f : D ⊂ R2 → [0, 255] and an arbitrary threshold t lying in [0, 255], let us note C0t (f ) and C1t (f ) the two classes generated by t: C0t (f ) = {x ∈ D, f (x) ≤ t} C1t (f ) = {x ∈ D, f (x) > t} and B(t) the associated boundary:

13

n

o

B(t) = (x, y) ∈ D2 , x ∈ C0t (f ), y ∈ C1t (f ) and y ∈ N4 (x) where N4 (x) represents the neighborhood of x constituted of the 4 nearest points, in the sense of the square grid. t (x, y) At each element (x, y) of B(t), K¨ohler associates a contrast noted CK defined by: t (x, y) = M in(f (y)–t, t–f (x)) and representing the minimum of the two CK steps generated by the threshold t between f (x) and f (y). Then the average contrast of B(t) is computed:

CK (B(t)) =

X 1 × C t (x, y) #B (x,y)∈B(t) K

where: • #B designs the cardinal (number of elements) of B(t) • the summation

P

concerns the pairs (x, y) lying in B(t)

K¨ohler’s method ends with the choice of the optimal threshold t0 such that: CK (B(t0 )) = M ax(CK (B(t)) f or t ∈ [0, 255] The following Fig.5-(a), (b), (c) presents an example of this method. K¨ ohler’s method in the LIP framework (LIP-K¨ ohler): It consists t in replacing the contrast CK (x, y) by its logarithmic version (Fig.5-(a), (d), (e)): CKt (x, y) = M in(f (y) t, t f (x)) Comment: We are now familiar with the fact that the logarithmic tools are more accurate in the dark part of the grey scale, which is a novel time demonstrated by Fig.5. Multithresholding based on K¨ ohler’s method: Most of thresholding methods are inappropriate to perform multithresholding, generally because they need a high computation time. A noticeable advantage of K¨ohler’s method is the possibility to represent the contrast values of all the boundaries B(t) when t ∈ [0, 255]. The following Fig.6 illustrates this ability, the 14

(a)

(b)

(c)

(d)

(e)

Figure 5: (a) initial images, (b) curve representing K¨ohler’s contrast, (c) resulting thresholded images (at the maximal peak of (b)), (d) curve of the LIP-K¨oler’s contrast, (e) resulting thresholded images (at the peak of (d))

(a)

(b)

(c)

Figure 6: (a) initial image, (b) curve representing K¨ohler’s contrast, (c) multithresholded image at the peaks 39, 79, 112, 145, 202 of (b)

significant peaks of the contrast’s curves corresponding to possible thresholds. Moreover, Fig.7 presents the results generated from the contrast’s curves (b) and (d), by K¨ohler (c) and LIP-K¨ohler (e). On this last image, the dark part is more precisely interpreted. Thus, it seems interesting to mix the K¨ohler thresholds and the LIP-K¨ohler ones (see Fig.7-(f)). 2.3.2. Application to contour detection Logarithmic tools open some ways to perform contour detection. A first approach consists in replacing, in certain filters, the classical addition and subtraction operations by logarithmic ones. An example concerns Sobel filter and has been proposed by G. Deng et al. (Deng and Pinoli (1998)). Let us insist a new time on the advantages of using logarithmic operators: on one hand, it is perfectly adapted to images acquired in transmitted light, 15

(a)

(b)

(d)

(c)

(e)

(f) Figure 7: (a) initial image, (b) K¨ohler’s method corresponding contrasts curves, (c) K¨ ohler’s multithresholding to values 100 160 and 205 corresponding to the most significant peaks in (b),(d) LIP-K¨ ohler’s method corresponding contrasts curves, (e) LIPK¨ ohler’s thresholding to values 6 13 21 30 64 160 and 205, (f) mix between LIP-K¨ohler and K¨ ohler multithresholding 6 13 21 30 64 100 160 205

16

and on the other hand, the LIP context is consistent with human vision. Moreover, it enhances the contour detection in the dark parts of an image, proving its efficiency in case of non-uniform lightning. We prefer another approach: from the logarithmic contrast, it is easy to derive tools allowing the comparison of a pixel x to the set of its nearest neighbors (classically noted N4 when considering only the horizontal and vertical directions and N8 when the diagonals are added). For example, given a grey level function f , we can compute, at each point x ∈ D the average contrast AC(x) (f ) of x with its 8 neighboring points (ni (x))i=1..8 according to: AC(x) (f ) = where

X

1 8

(

X

(C(x,ni (x)) (f ))

(9)

i=1...8

represents the sum in the LIP sense.

In the same way, we can compute the maximal contrast M C(x) (f ): M C(x) (f ) = M axi=1...8 C(x,ni (x)) (f )

(10)

At a given image f, each of these notions associates a ”contrast” image by replacing the grey level f (x) of a point x by AC(x) (f ) or M C(x) (f ). The resulting ”contrast” images AC(f ) and M C(f ) put in evidence the contours of f , with the property of a weak dependence to non-uniform lighting (cf. Fig.8) compared to classical methods like Sobel filter. Remark 6: the previous operators may apply very simply to a RGB image f by associating to f a grey level image, for example 31 R + 13 G + 13 B (see Fig.8). Remark 7: To achieve the binarization of the contours proposed in Fig.8(c), which is not the purpose here, it is possible to apply a thresholding (not very efficient for complex images) or more accurately a ”watershed” algorithm (see Beucher and Lantuejoul (1979) and Beucher (1991)). 2.4. Metrics associated to the Logarithmic Additive Contrast The mathematical field of ”functional analysis” is mainly devoted to the study of functions and spaces of functions. Because the LIP Model gives 17

(a)

(b)

(c)

(d)

Figure 8: (a) represents the initial RGB image (a ”well” with a very dark part), (b) the grey level image 13 R + 13 G + 13 B associated to (a), (c) visualization of the maximal contrast M C(x) (f ) at each point of (b), (d) a classical contour detection (Sobel gradient) applied on (b)

18

the set of images a vector space structure, we are clearly interested in the ”functional” approach. Among the considerable number of tools created by mathematicians, we will particularly focus here on the concept of ”metrics”. In fact, when studying images, we are obviously interested in detecting the presence of some object of interest (target) inside an image, or in tracking this target in a sequence of images. It is also a very common task (not always easy) to estimate the similarity (or the differences) between two images in order, for example, to perform defects detection in industrial control. The concept of metrics is well adapted for such objectives. 2.4.1. Recalls on functional metrics Global metrics: Given a pair (f, g) of real valuated functions which are definite and integrable on a real interval [a, b], it is classical to define a metric noted d1 according to: d1 (f, g) =

Z

|f (x) − g(x)|dx

[a,b]

This definition is obviously transposable onto a continuous image with a double integral of the difference |f (x) − g(x)|, computed on the points (x, y) of the region of interest (D or a subset R of D): d1,D

or R (f, g) =

Z Z

|f (x, y) − g(x, y)|dxdy

D or R

In the digital version, it is transformed into the double sum of the differences between pixels grey levels according to the rows and columns, multiplied by the area of one pixel. It thus evaluates the ”volume” situated between the representative surfaces of images f and g (cf. Fig.9): d1,D

or R (f, g)

=

 X 

 X

|f (i, j) − g(i, j)|

(i,j)∈D or R

× (area of a pixel) Comment: More generally, we can use metrics derived from the norms associated with the Lp spaces (spaces made up of functions whose pth power is integrable): Lp → dp (f, g) = (

1 |f (x) − g(x)|p dx) p

Z [a,b]

19

and their bi-dimensional continuous or digital versions. All these metrics are considered of ”global” or ”diffuse” nature, in reference to measure theory. It means they are null when computed on neglectable sets. By definition, such metrics produce an averaged information, and are then inefficient in detecting small sized differences between two functions or two images (Fig.9 and Fig.10-(b)). Atomic metrics: On the contrary, we can use ”atomic” metrics, similar to measures using ”weighted” points (Dirac measures). They are then perfectly adapted in detecting small differences, even as small as a pixel (Fig.9 and Fig.10-(b)). The most typical example is the metric d∞ derived from the norm of uniform convergence in the L∞ space, which is computed on the point realizing the greatest difference between f and g: d∞ (f, g) = sup |f (x) − g(x)| x∈[a,b]

It is defined in the same way on a two-dimensional region or domain: d∞ (f, g) =

sup

|f (x, y) − g(x, y)|

(x,y)∈R or D

and in digital version: d∞ (f, g) =

sup

|f (i, j) − g(i, j)|

(i,j)∈R or D

Neighborhoods generated by the precedent metrics: One of the major interests of the ”metric” tool resides in its associated topology, i.e. in the neighborhoods it generates. The shapes of such neighborhoods are totally different for the d1 and d∞ metrics. In fact, given a function f , each function g verifying d∞ (f, g) ≤  satisfies |f (x) − g(x)| ≤  for every point x lying in the considered interval or region. It means that g belongs to a ”tolerance tube” around f (Fig.10 (a)). This remark explains why d∞ is called ”uniform convergence metric”. The same result holds for images, the tolerance tube becoming the volume located between the translated representative surfaces of f according to + and –. When considering the ”global” metrics d1 , .., dp , an -neighbor of a given function f is totally different from a tube: it is an unbounded set! In fact, 20

Figure 9: The value of d1 (f, g) corresponds to the hatched area between the representative curves of f and g, the distance d∞ (f, g) is realized at the point x0

(a)

(b)

Figure 10: (a) The tolerance tube of f for d∞ is represented by the hatched area, (b) the difference between a function f and a function g lying in the –neighbor of f for d1 may be arbitrarily large

a function g belonging to the -neighbor of f may present at some point x an arbitrary large difference |f (x) − g(x)| and a very small area located between f and g (Fig.10 (b)). Between these two extreme situations, it appeared us interesting to present an intermediate solution (see Fillere (1995)): Intermediate metric between ”global” and ”atomic”: It consists of making a compromise between the size of an unacceptable defect and its intensity (contrast) in relation to the background. In order to do this, we define an interval I of length , which is moved along the interval [a, b]. We calculate: 21

d (f, g) = Supx∈[a,b−]

Z

|f (t) − g(t)|dt

[x,x+]

An illustration of what represents this metric is given Fig.11. In two dimensions, I is simply replaced by a region of the spatial support,

Figure 11: The area of the largest hatched zone is the value of d (f, g)

which is moved across it while computing the sup: d1,supR (f, g) = SupR⊂D

Z Z

|f (x, y) − g(x, y)|dxdy

R

For this approach, the region R is sized at the defects desired dimension. RR

In order to apply it to digital images, the double integral R is replaced PP by the double summation according to rows and columns. Now let us present the logarithmic versions of all these metrics. Preliminary results on this subject have been exposed in Carr´e M. (2011). 2.4.2. The ”global” logarithmic metric d1 Given two grey level functions f and g, let us recall what represents their LAC (Logarithmic Additive Contrast) C(x,y) (f, g) at a point (x, y) of the spatial domain D. It is defined by the relation (cf. formula 8): C(x,y) (f, g) = M ax(f (x, y), g(x, y)) M in(f (x, y), g(x, y)) and is expressed according to: 22

|f (x, y) − g(x, y)| M in(f (x, y), g(x, y) 1− M Comment: A point of the spatial support D is generally noted x when no distinction between rows and columns is necessary, and (x, y) otherwise. C(x,y) (f, g) =

Now, a summation of such contrasts on all the elements of D or a region of D makes possible to exhibit a novel metric d1 , tractable in the LIP context, and defined on the space I(D, [0, M [), either on the whole domain D or a region R ⊂ D: d1D

or

(f, g) = R

Z Z

M ax(f (x, y), g(x, y)) M in(f (x, y), g(x, y))dxdy

D or R

(11)

Applied to digital images, such a metric becomes:

d1D

or R

(f, g) =

X

X

M ax(f (i, j), g(i, j)) M in(f (i, j), g(i, j))

(i,j)∈D or R

× (area of a pixel) Where

X

(12)

represents the summation, in the LIP sense, of the contrasts

between f and g at each point (i, j) of the considered region. Remark 8: The term ”dxdy” in formula 11 becomes in digital version ”area of a pixel” (cf 12) Remark 9: The presence, in formula 12, of the ”area of a pixel” permits to obtain a result independent of the numerization (sensor resolution) and to preserve the homogeneity of formula 11 which clearly represents a volume. Remark 10: Formulas 11 and 12, respectively in continuous or digitized expression, estimate a ”contrast volume” separating the representative surfaces of f and g. Remark 11: A real difficulty arises if we need the visualization of such a distance. In fact, each contrast M ax(f (i, j), g(i, j)) M in(f (i, j), g(i, j)) is a grey level and the sum of an arbitrary number of grey levels, although it 23

remains a grey level, in general quickly approaches the limit value M. To solve this problem, we propose to replace this cumulative distance by an average contrast, taking into account the number of points present in the region of interest: if #R denotes the cardinal of R, we replace formula 12 by: d1R (f, g) =

1 #R  X X 



M ax(f (i, j), g(i, j)) M in(f (i, j), g(i, j)) (13)

(i,j)∈R

Remark 12: Formulae 12 and 13 are obviously compatible because the average contrast of 13 corresponds to the ”contrast” volume of 12: in fact, computing an average contrast from the volume expression consists in dividing by ”area(D)” whose value is (area of a pixel) multiplied by (#D). 2.4.3. The ”atomic” metric d∞ We start from the expression, recalled below, of the ”classical” atomic metric d∞ : d∞ (f, g) = Supx∈R

or D |f (x) − g(x)|

In the LIP context, if we refer to Pumo and Dhorne (1998), the same formula with a LIP subtraction is not correct because the expression |f (x) g(x)| is not always defined, according to the fact that f (x) is greater than g(x) or not. It must be replaced by the logarithmic difference between the maximum and the minimum of the pair (f (x), g(x)): d∞ (f, g) = Supx∈R

or D (M ax(f (x), g(x))

M in(f (x), g(x))

(14)

As for d∞ (f, g), such a metric seems theoretically well adapted to industrial control and more precisely to point out possible defects, but it is very ”sensitive” because determined by one unique point, which may correspond to a very small, and then acceptable, defect. Given a reference function without defects, noted f and the same image with defects, noted g, it is possible that a unique pixel x0 corresponds to the ”Sup” value: in this case, only x0 will be detected. In most cases, this kind of answer is not completely satisfactory because the defect size around x0 may be larger than one pixel. Furthermore the method may ignore a number of other defects whose contrasts with the 24

reference image are less than d∞ (f, g). A possible answer to this problem is to perform a threshold t on the contrast’s map between f and g in order to know where g significantly differs from f and if x0 is really an isolated point. Another solution consists in introducing an ”intermediate” definition between the ”diffuse” distance d1 and the ”atomic” one d∞ , noted d1,supR . It will be presented in the next section 2.4.4. 2.4.4. The intermediate metric d1,supR The LIP version of the intermediate metric d1,supR is defined according to:

d1,supR (f, g) = SupR⊂D

Z Z

|M ax(f (x, y), g(x, y))

R

M in(f (x, y), g(x, y))|dxdy

(15)

The ”digital” expression, corrected by the cardinal #R is:

d1,supR (f, g) = SupR⊂D

1 #R

 X X 

(i,j)∈R

(16)  

|M ax(f (i, j), g(i, j)) M in(f (i, j), g(i, j))|

Remark 13: We have already explained the interest of dividing the precedent expression by #R. Remark 14: Because the region R is sized at the dimension of maximal defects to be detected, the result will not be ”polluted” by very small ones, while preserving the extraction of larger defects. Remark 15: This intermediate distance being defined according to a superior bound, it is able to detect only one defect, that corresponding to the supremum. In order to avoid this weakness, it is possible to come back to the version of d1,R and to compute it for all the positions of R in D. It remains to design as defects all the R locations where the computed distance is greater than a chosen threshold (cf. Fig.12). 25

(a)

(b)

(a1)

(b1)

(a2)

(b2)

(a3)

(b3)

Figure 12: (a) initial image f of skin wrinkles by fringes projection at time T0 , (b) initial image g at T0+28 days, (a1)(a2)(a3) d1,supR (f, g) for different sizes of R (1 × 1, 21 × 21, 41 × 41) the detected region is the white square, (b1)(b2)(b3) represent the images (a1)(a2)(a3) thresholded at grey level 90

26

2.5. Applications of these Metrics notions The problem of defects detection in industrial control has been aforementioned. Now some applications are going to be presented in order to illustrate the interest of the introduced metrics. 2.5.1. Scene modifications Given a reference image f of a certain scene, and a current image g of the same scene, this section focuses on what differs between f and g. Such a question covers various domains: • Comparison of two satellite images of the same site at different times, the aim being to put in evidence new buildings or roads, expansion of towns, and also modifications of agriculture areas... • Robots safety, the aim being to detect the entrance of a person in a forbidden area. • Industrial or military sites surveillance. • Automated crossroads supervision • .... 2.5.2. Pattern recognition and target tracking When possessing a reference image f of a region (object) of interest, and given a current image g, the addressed question is to decide if, inside g, some region very similar to f exists or not. Application to cars crash tests (detection of a target): During the very short time of a crash test (around one second), a fixed number of high speed cameras store the scene under determined angles. From these images (cf. Fig.13), the trajectory of each target must be reconstructed to evaluate the deceleration of each marked point of the car. In order to propose an automated detection of the targets, a possible way consists in searching on each image the regions which resemble the model. In our approach, the target is moved in all positions on the studied image and we compute for each of them the distance of the covered part of the image to the target model.

27

(a)

(b)

Figure 13: (a) Example of a reference target, (b) Image of car crash test (with the authorization of ”Insurance Institute for Highway Safety”). Note that, due to the perspective, the different targets do not present exactly the same orientation

Now let us visualize the map of distances representing, at each point of the initial image where a target model is centered, the local distance between the target image and the corresponding region of the initial image. On the next figure (Fig.14-(a) and (c)), the grey levels represent these distances: the smallest the distance, the darker the grey level. The interpretation of such maps may be done as for a correlation.

(a)

(c)

(b)

(d)

Figure 14: (a) Represents the distance map associated to the logarithmic metric and (b) the corresponding histogram,(c) represents the distance map associated to the classical metric and (d) the corresponding histogram

28

Comments on Fig.14: • There is no doubt that the map computed with the metric d1,R is of better quality in terms of contrast and dynamic range. This fact may be visually appreciated in a first step and objectively demonstrated by the corresponding histograms. • Furthermore, the dark parts corresponding to the targets are well localized and easy to separate from the background, mainly on Fig.14 (a) realized with the logarithmic approach. • In both cases, the target placed on the dummy’s head is not detected, because this target is of smaller size and also presents a different angle, which would necessitate to use a specific target, or to compute the distance map under various rotations of the reference target. • Nevertheless, we can observe a certain tolerance to small rotations and shape changing. This fact allows performing an automated detection of the trajectory of a chosen target during the movie: the detected target on an image becomes the reference one for the next image, thus taking into account the small changing between two successive images. Let us now visualize the final detection of targets: the image obtained in Fig.14 (a) is thresholded (cf Fig.15 (b)) using the classical Entropy Maximization (cf Pun Pun (1981))

(a)

(b)

Figure 15: (a) Initial image, (b) the detected targets

We can conclude that the desired targets are precisely detected and located. 29

2.5.3. Metrics as correlation tools The notion of correlation is a fundamental tool in the field of statistics. It is also commonly used in image processing in order to perform the optimal superposition of two images, in particular to compensate sensor motions. The role of metrics as correlation tools is evident: given two images, one may be moved (translation, rotation...) in reference to the other until the correlation becomes optimal, that is to say when a certain distance between them reaches a minimal value. A perfect example of this approach has been given in the precedent section, when a given target has been moved of the reference image in order to detect the locations corresponding to a minimal distance. 2.5.4. Characterization of pseudo-periodic textures The concept of ”texture” is rather difficult to define with rigorous terms, in the sense of mathematics for example. In this section, our ambition is not to go into this subject in depth, so we will limit us to propose some new tools to study images presenting a pseudo-periodic texture. For an example of such an image, see Fig.16.

Figure 16: pseudo-periodic texture example

A recurrent question about pseudo-periodic images is to estimate a period in some direction. One of the major techniques to answer this question consists, given an image f , in computing a covariogram (notion introduced 30

by Matheron in Matheron (1970)) of f in some direction. To do that, f is translated related to itself in the given direction. For each value of the translation vector h, the ”correlation” between f and its translated fh is computed. Then the results are presented as a curve. An example of covariogram is presented in Fig.17: it has been applied to the image of Fig.16 on which the correlations are computed for horizontal translations with the distance d1 (f, fh ). The locations where this distance presents a relative minimum value correspond to local optimal correlations. Note that if the first minimal value is reached for a translation vector ho , the expected estimation of the period of f is precisely given by ho .

Figure 17: Covariogram example applied on Fig.16

Remark 16: For images presenting complex textures, the covariogram curve may be noisy. In such conditions, the automated extraction of the estimated period is rather difficult. An efficient way for de-noising the covariogram consists in applying a classification algorithm (k-means for example) to restrict the number of grey levels present on the initial image f . Remark 17: As previously said, the advantages of applying logarithmic metrics are multiple: they are perfectly adapted to images acquired in transmitted light, they are defined in a framework consistent with human vision, and they are weakly dependant of illumination changing (Fig.18), unlike classical metrics as d1 . Images of Fig.18 have been realized by Inam Ul Haq (cf. Ul Haq (2011)).

31

Figure 18:

(a)

(b)

(c)

(d)

(e)

(f)

(a) initial image, (b) same image under low lighting conditions, (c) co-

variogram of (a) computed with d1 , (d) covariogram of (b) computed with d1 , (e) covariogram of (a) computed with d1 , (f) covariogram of (b) computed with d1

32

2.5.5. A novel class of automated thresholding algorithms based on metric minimization The principle of this method, created by our good offices, uses the concept of metrics (distance between functions) to find the binary image which most closely ”resembles” the initial image f , i.e. whose distance from it is minimal. Notations and summary of the method: Given an image f and a functional distance d, at each threshold t lying in the interval [0, 255[, remember (cf. K¨ohler’s method in section 2.3.1) we have defined two classes C0t (f ) and C1t (f ) according to: C0t (f ) = {x ∈ D, f (x) ≤ t} C1t (f ) = {x ∈ D, f (x) > t} To such classes we associate the step function, noted St,f and defined by: St,f (x) = M0t if x ∈ C0t (f ) St,f (x) = M1t if x ∈ C1t (f ) where M0t and M1t represent respectively the mean values of C0t (f ) and C1t (f ). Then it remains to compute the distance d(f, St,f ) and retain the threshold t0 such that: d(f, St0 ,f ) = M in(d(f, St,f )), t ∈ [0, 255[ In Fig.19 we present the results obtained with the metric d1 . Comment on the interest of using various metrics: From a general point of view, the proposed metrics possessing different properties (”global”, ”atomic” or ”intermediate”), their behavior is not the same when applied to automated thresholding: • the d1 , .., dp metrics, of a ”global” or ”diffuse” nature , return similar results to those obtained by the maximization of inter-class variance. • the d∞ metric produces results comparable to those of the entropy maximization method, but is more sensitive to ”aberrant” points (beware of salt-and-pepper noise!) 33

• the logarithmic metrics will be preferred to classical ones in case of low-lighted images and when one aim at imitating the human visual behavior.

(a)

(b)

Figure 19: (a) initial image, (b) threshold based on logarithmic additive contrast

3. Logarithmic Multiplicative Contrast and Associated Metrics in the LIP framework The organization adopted for this section will be analogous to that of section II. 3.1. Definition of a Logarithmic Multiplicative Contrast (LMC) in the LIP context Now let us define a contrast notion based on the scalar multiplicative law of the LIP framework. This notion is completely different of existing contrasts. Nevertheless, we took care to give it a physical meaning, once more based on the transmittance law. It will be seen that this contrast, which is a real number instead of a grey level, presents the advantage to be more sensitive near the white extremity of the grey scale than near the black one. If f represents a grey level image and x and y two points of D, we define the logarithmic multiplicative contrast (LMC), noted C(x,y) (f ) as the logarithmic ratio of M ax(f (x), f (y)) by M in(f (x), f (y)). In other words, it represents the number by which the brightest grey level must be multiplied, in the LIP sense, in order to obtain the darkest grey level: C(x,y) (f ) M in(f (x), f (y)) = M ax(f (x), f (y)) 34

(17)

In the same way, the logarithmic multiplicative contrast may be defined for a pair of grey level functions (f, g) at each point x of D: Cx (f, g) M in(f (x), g(x)) = M ax(f (x), g(x))

(18)

Remark 18: In each of the precedent situations, the multiplicative contrast clearly corresponds to the number of times we must ”add” (physically superpose) the ”Min” grey level between the source and the sensor to obtain an attenuation equivalent to the ”Max” grey level. Remark 19: To illustrate the better sensitivity of the LMC in the bright part of the grey scale, let us consider for example a pair of grey levels f (x) and f (x) + k, and compute the LMC between f (x) and f (x) + k when f (x) varies in [0, 255 − k[ in a classical grey scale (0=black). In Fig.20, we represent the curves corresponding to these contrasts for various values of k (k = 30, 50, 100, 150 and 200). The non-linearity of the LMC appears clearly, as well as its attenuation towards the black side of the scale and its high sensitivity towards the white side. Such a behavior must be compared to the LAC one, presented in Fig.4.

Figure 20: Curves representing the LMC between f (x) and f (x) + k for k = 30, 50, 100, 150 and 200

35

Remark 20: The additive contrast C(x,y) (f ) was by definition a grey level. The multiplicative one is a real number, which reaches +∞ if M in(f (x), f (y)) = 0 and M ax(f (x), f (y)) 6= 0. In the case where f (x) = f (y) = 0, we can decide to estimate the contrast to the unit value, in analogy with the general case of two identical grey levels.

To solve the problem of the possible infinite value, a mathematical approach will be presented in the paragraph ”associated metrics” (section 3.3.5). Another simple and practical solution consists, for digital images, in replacing each null grey level by the unit one, which does not affect significantly nor the aspect neither the interpretation of the studied image. In such conditions, all contrast values are finite and may be visualized as grey levels, after a normalization, if necessary: in fact, if we work in the classical situation of 8-bits images, we dispose of a grey scale of 256 (limited to 255) grey levels (from 1 to 255), with a nearly linear distribution. Thus the largest possible value Cmax for the multiplicative contrast is reached for the pair of grey levels (1, 255), and satisfies: Cmax 1 = 255

(19)

yielding Cmax ' 1416.79 At this step, we dispose of various possibilities: • First approach: the normalized (lying in the interval [0, 255]) multiN (f ) of f at a pair (x, y) is defined by: plicative contrast C(x,y) N (f ) = 255 × C(x,y)

1 Cmax (f )

C(x,y) (f )

(20)

sometimes denoted C(fN(x),t) in the following when at less one of the considered grey levels is not defined by f but a threshold t for example. In the same way, the contrast of a pair (f, g) of images at a point

36

x is given by: C(x)N (f, g) = 255 ×

1 Cmax

C(x) (f, g)

(21)

Drawback of this normalization: the LMC appears concentrated on the black side of the grey scale and then furnishes a limited information. • Second approach: as it is commonly done for gradients for example, we can truncate the LMC values: each value greater than 255 is limited to 255. We reject this method for its arbitrary aspects. • Third approach: for each situation, we can choose the best displaying by transforming the interval [M in(LM C), M ax(LM C)] in [0, 255]. For an example, see Fig.22-(c). • Fourth approach: we keep all the initial values of the LMC and process them in each application: contour detection, thresholding, metrics...in order to preserve all the information. The third approach is finally applied when a displaying is needed. At this step, the definitions of average and maximal additive contrasts we have presented in section 2.3.2 are adaptable to the multiplicative situation according to: • Average multiplicative contrast: AC(x) (f ) =

1 X × C (f ) 8 1...8 (x,ni (x))

(22)

• Maximal multiplicative contrast: M C(x) (f ) = M axi=1...8 C(x,ni (x)) (f )

(23)

where for i = 1..8, ni (x) is one of the eight neighboring pixels of x. Note that the LMC being always a scalar, the summation 22 is intended in the classical sense of real numbers addition.

P

in formula

Based on all the concepts we have exposed towards the LMC, it is now 37

possible to present applications analogous to that of part 2.3, dedicated to automated thresholding and contour detection. 3.2. Applications of the Logarithmic Multiplicative Contrast 3.2.1. Application to automated thresholding and multi-thresholding Let us come back to K¨ohler’s automated thresholding method. As we t (x, y) have done for the LAC, it is possible to replace K¨ohler’s contrast CK ,t by the corresponding LMC version noted CK (x, y): CK,t (x, y) = M in C (f (y), t), C (t, f (x)) h

i

(24)

Remember such a contrast is evaluated on all the pairs (x, y) of pixels lying in the boundary B(t) generated by a given threshold t. Then the average contrast of B(t) is computed and the method is ended by selecting the threshold t0 maximizing this average contrast. Remark 21: As for the LAC, the method may be easily extended to multithresholding. Remark 22: We previously noted as important that the LMC emphasizes contrasts in the white part of the grey scale. An example is given Fig.21: we start with an initial infrared image (”La Rochelle port”) where the dark (cold) zones are mainly constituted by the sea and the clouds. On its histogram (b), two classes appear: the cold one on the left, very homogeneous, and the hot one corresponding to the earth, rather scattered around its average. The optimal threshold given by the interclass variance maximization (with Mahalanobis correction) corresponds to the grey level 79. This threshold produces a binary image (c). A possible way to exhibit a ”hot” (bright) class inside the earth (not visible on the histogram), consists in computing the LMC map of (a) whose histogram is presented in (d), with a significant peak. Thresholding (a) at this peak value gives a binary image (e) whose contours are superposed on the initial image (f).

38

(a)

(b)

(c)

(d)

(e)

(f)

Figure 21: (a) initial image, (b) histogram of (a) with highlighted value 79 corresponding to the threshold based on interclass variance maximization, (c) thresholding (a) at grey level 79, (d) histogram of the LMC map of (a) with a unique peak, (e) thresholded image corresponding to the peak of (d), (f) thresholded area highlighted on (a)

3.2.2. Application to contour detection Each of the formulas 22 and 23 permits associating to a given image f a contrast map. Considering as evident that the contour points of f correspond to those presenting a significant contrast, it seems natural that the contrasts maps computed from the average multiplicative contrast AC(x) (f ) or the maximal multiplicative contrast M C(x) (f ) have the effect of enhancing boundary points. The next Fig.22 corroborates this hypothesis and has been chosen to emphasize the efficiency of the LMC on over-lighted images.

39

(a)

(b)

(c)

(d)

(e)

Figure 22: (a) initial image ”sun eclipse 2006” with authorization of the author L. Ferrero, (b) classical gradient of (a), (c) contrast map of (a) corresponding to M C thresholded gradient, (e) thresholded contrast map

, (d)

From this example, no doubt subsists concerning the pre-eminence of the multiplicative logarithmic approach compared to a classical one. Our aim here being not to achieve the extraction of the contours but only to underline them, we will now concentrate on the metrics associated to the LMC.

40

3.3. Associated metrics 3.3.1. The ”global” metric In section 2, we have seen that on the space L1 of integrable functions, it is classical to compute the distance d1 (f, g) as the integral of the difference |f (x) − g(x)| where x varies in the region of interest, which may be for example an interval of R. The same distance may be defined on images with a double integral on a subset of R2 ... Replacing in the integral the local distance |f (x) − g(x)| by the logarithmic multiplicative contrast C(x) (f, g) and then cumulating such contrasts on the elements of D or a region of D, generates a novel metric d1 defined on the space I(D, [0, M [), either on the whole domain D or a region R ⊂ D. Such a metric is tractable in the LIP framework and is expressed according to: Z Z C(x) (f, g)drdc (25) d1,DorR (f, g) = DorR

where the coordinates of a point x in terms of rows and columns are noted (r, c). Applied to digital images, such a metric becomes: d1,DorR

 X (f, g) = 

 X

C(x) (f, g) × (area of a pixel)

(26)

(r,c)∈DorR

Remark 23: The presence, in formula 26, of the ”area of a pixel” permits to obtain a result independent of the digitization scale. Remark 24: Formulas 25 and 26, respectively in continuous or numerical expression, estimate a ”multiplicative contrast volume” separating the representative surfaces of f and g. Remark 25: Obviously, the size (number of pixels, or ”cardinal” noted #) of the considered region R plays a role in the distance estimation. To suppress it, we can divide the distance by the region area i.e. cardinal(R) × ,A (area of a pixel), and thus obtain an averaged value noted d1,DorR (f, g) and defined by: ,A (f, g) = d1,DorR

1 #R

 X 

 X (r,c)∈DorR

41

C(x) (f, g)

(27)

3.3.2. The ”atomic” metric When applied to a subset D ⊂ R2 or a region R of D, remember the ”uniform convergence metric” is defined by: d∞ (f, g) = Supx∈R

or D |f (x) − g(x)|

If we aim at transferring this formulation in the context of multiplicative contrast, the expression |f (x) − g(x)| must be replaced by C(x) (f, g) and thus: (28) d∞ (f, g) = Supx∈R or D C(x) (f, g) Remark 26: the two precedent metrics are comparable to those associated to the LAC in the sense that d1 and d∞ have respectively a ”global” or ”atomic” behavior. The first one evaluates the resemblance of an image g to a given image f through the belonging of g to unbounded neighbors of f and the second one through the belonging of g to ”tolerance tubes” of f . The main difference between the two metrics d1 and d∞ on one hand and the pair d1 and d∞ on the other hand is linked to their physical meanings which imply a better sensitivity of the multiplicative metrics d1 and d∞ on the white part of the grey scale. Thus they will produce their best results for over-lighted images or on the light-grey part of an image (cf Fig.23, where the automated thresholding is performed with ”metric” method of 2.5.5).

42

(a)

(b)

(c)

(d)

Figure 23: (a) initial image ”Shanga¨ı Airport”, (c) initial image ”Sea cloud and sun”, (b) and (d) automated tresholded images of (a) and (c) according to the metric d1

As aforementioned for ”additive” metrics, it may be useful to introduce an intermediate solution. 3.3.3. The ”intermediate” metric When applied to industrial control, the atomic metric seems theoretically well adapted to defects detection, and also to biomedical applications when small objects have to be detected. The problem is the extreme ”sensitivity” of this metric because it is determined by one unique point. This is the reason why we introduce an ”intermediate” definition between the ”diffuse” distance d1,D or R and the ”atomic” one d∞ , noted d1,supR . It consists first in choosing a subset (region) R of the domain D, then to compute the distance d1,R (f, g) for each position of R inside D, and finally to define: d1,supR (f, g) = SupR⊂D d1,R (f, g) (29) 43

Remark 27: As the atomic one, the intermediate metric obviously detects one position of the region R inside D corresponding to the largest distance between f and g. If this distance is not acceptable, the controlled product must be rejected. Remark 28: In order to detect the set of all the defects, acceptable or not, we propose to choose a particular pixel inside the region R, for example the gravity center c. For each x of D, denote Rx the region R when its gravity center c is superposed to x, and compute the distance d1,Rx (f, g) which becomes the grey level of x.To the resulting image, we apply a threshold corresponding to the ”acceptable error” : all the pixels presenting a grey level greater than  must be interpreted as defects. 3.3.4. The ”Aspl¨ und” metric Now let us focus on the little-known metric of Aspl¨ und, which we have extended from binary shapes to grey level images. We have chosen this metric because it possesses outstanding properties, presented below (Remark 29). For binary shapes: Aspl¨ und proposed a new distance between two binary shapes A and B (Gr¨ unbaum (1963)): he selects one of them (B for example) as a ”probing” shape and defines two positive numbers α and β such that: α = inf {k, A ⊂ kB} β = sup {k, kB ⊂ A} where k is a positive real number and kB is the homothetic set of B in the ratio k (Fig.24). Then the distance dAs between A and B defined by: α dAs (A, B) = Ln( ) β

(30)

is a metric on the space of binary shapes. It means in particular that it satisfies the symmetry property dAs (A, B) = dAs (B, A), with consequence that A would have been chosen as the probing shape instead of B, without changing the Aspl¨ und’s distance between them. Remark 29: The definition of dAs is totally intrinsic to the given pair of shapes A and B. In fact, it is independent of magnifications that would be 44

applied to one of them, because it evidently satisfies the following equality: λα α dAs (A, B) = Ln( ) = Ln( ) = dAs (A, λB) β λβ Furthermore, given a reference shape A and a tolerance , it is quite easy to characterize the set NAs, (A) of shapes B ”neighboring” A in the sense of the inequality: dAs (A, B) ≤  Starting from the couple A and  we choose an arbitrary homothetic shape of A, say βA, and we compute the (unique) number α such that Ln( α β ) = . Thus the two homothetic sets αA and βA delimit a ”tolerance” tube constituted of their difference αA \ βA. In fact, each element B satisfying the double inclusion βA ⊂ λB ⊂ αA for some real number λ is an element of the neighborhood NAs, (A) (cf. Fig.24): βA ⊂ λB ⊂ αA ⇒ B ∈ NAs, (A) (31)

Figure 24: tolerance tube built by αA and βA around the homothetic shape λB

Remark 30: The implication of formula 31 is not an equivalence because the location, inside λA, of the smallest homothetic set βA of A is not determined. Each location of βA inside αA generates a new tolerance tube and if B ∈ NAs, (A), it then satisfies the double inclusion of formula 31 for some tube (Fig.25). 45

Each element B of NAs, (A) is then characterized by: B ∈ NAs, (A) ⇔ ∃ a tube αA\βA such that βA ⊂ λB ⊂ αA for some real number λ

Figure 25: Representation of two different tolerance tubes associated to A for some  and representation of two shapes B1 , B2 ∈ NAs, (A)

For grey level images: It seems us very interesting to extend Aspl¨ und’s reasoning in a ”functional” context, in order to apply it to grey level images: • a similar approach has been already exposed by our team in Barat et al. (2003a), Barat et al. (2003b), Barat et al. (2010). In these papers, starting from a given image f, we defined two ”probing” tools called SOMP (Single Object Matching using Probing) and MOMP (Multiple Object Matching using Probing). With SOMP, we detect all the occurrences of a same pattern inside the image f , by means of two identical probing functions. For MOMP, two different probing functions are used, creating a ”tolerance tube” in which the searched objects must be contained. In Barat et al. (2010), MOMP is generalized in VDIP (Virtual Double sided Image Probing) which generates a morphological metric. • the novelty of what we propose in this section consists in using logarithmic homothetics k f as probing patterns. Thanks to the transmittance law, this logarithmic scalar multiplication always remains 46

in the grey scale. This property makes possible the definition of a logarithmic Aspl¨ und metric. Given two images f and g defined on D, we choose, as for binary shapes, g as the probing function for example and define the two numbers: α = inf {k, f ≤ k g} and β = sup {k, k g ≤ f } and the corresponding ”functional Aspl¨ und distance” dAs : α dAs (f, g) = Ln( ) β Remark 31: This metric dAs is adaptable to local processing, in particular to detect on an image f the place where a given pattern or target model is probably located (Fig.26). In this case, the target corresponds to an image t defined on a spatial support Dt smaller than D. For each location of Dt included in D, the distance dAs (f|Dt , t) is computed, where the notation f|Dt represents the restriction of f to Dt. Comment on Fig.26: The major interest of this approach is its independence to the target’s illumination ((c) and (e) very similar). For its part, a classical correlation (g) cannot aim at such an efficiency. Note that the extraction of the minimal values (black dots) of (c) and (e) is not very difficult to perform and permits to locate each brick in the wall.

47

(a)

(b)

(c)

(f)

(d)

(e)

(g)

Figure 26: (a) initial image f , (b) bright target t, (c) Aspl¨ und’s distance map dAs (f|Dt , t), (d) dark target t1 , (e) Aspl¨ und’s distance map dAs (f|Dt1 , t1 ), (f) target t2 , (g) correlation map of t2 inside (a)

Remark 32: A classical way to evaluate the similarity between two images or between a target and an image is to use ”correlation” parameters, including classical metrics. In such cases we search the best superposition, which means minimizing the distance or maximizing the correlation. The Aspl¨ und’s approach is different in the sense that we search the best probing of one image by the other one. This remark shows that the Aspl¨ und’s distance is very sensitive to small defects (and also to noise).... From this point of view, its behaviour is comparable to that of ”atomic” metrics. Remark 33: This is probably the most important remark concerning Aspl¨ und’s distance and we will enonciate it as a theorem: 48

Theorem 3.1. Aspl¨ und’s distance is invariant under uncontrolled intensity variations of the studied image represented by thickness variations of the observed object. In fact, we have already noted that the homothetic law of the LIP Model corresponds to a thickness variation of the ”obstacle”, in case of images acquired in transmitted light. Let f denote a grey level function, and g the image of a target representing the probing function. Suppose the ”thickness” or more generally the darkness (when the acquisition is done in reflected light) of f varies in the ratio λ. The probing of f by g furnishes two real numbers α and β and then the Aspl¨ und distance dAs (f, g) = Ln( α β ). Now the probing of λf by g produces the homothetic numbers αλ and βλ because α (λ g) = (αλ) g (due to the property of associativity of the scalar multiplication in the vector space I(D, [0, M [)). Then the associated α Aspl¨ und distance dAs (λ f, g) = Ln( αλ βλ ) = Ln( β ). This property is illustrated on Fig.26. Other applications are proposed in Fig.27 in the field of human skin studies.

49

(a)

(b)

(c)

Figure 27: column (a) initial images: dermal epidermal junction in in-vivo confocal microscopy (up) and skin cheek photograph (down), column (b) reference targets (magnified), column (c) thresholded Aspl¨ und’s distance maps highlighted on (a)

Remark 34: the neighbors generated by Aspl¨ und’s metric have been interpreted for binary shapes as a tolerance tube (remarks 29 and 30). The same reasoning is possible for grey level images. Given an image f and a tolerance , let us consider the successive steps: • Create a family of tolerance tubes Tα,β, (f ) constituted of regions delimited by two homothetics αf and βf such that Ln( α β ) = .

50

• Define the neighborhood NAs, (f ): g ∈ NAs, (f ) ⇔ ∃(α, β), ∃λ, /Ln( α β) =  and λ g ∈ Tα,β, (f ) • Visualize a mono-dimensional representation (Fig.28) The representation proposed in Fig.28 perfectly illustrates the property enounced in theorem 2. One image g may be a neighbor of an image f , in Aspl¨ und’s sense, even if g is significantly darker than f : it suffices that some homothetic of g resembles some (other) homothetic of f .

Figure 28: Two images f and g which are -neighbors in Aspl¨ und’s sense

3.3.5. A bounded metric associating binary and grey level approaches The present paragraph ensues from Remark 20 of part 3.1, and more precisely of the fact the multiplicative contrast C(x) (f, g) defined in formula (18) is a real number, which reaches +∞ if M in(f (x), f (y)) = 0 and M ax(f (x), f (y)) 6= 0. The solution previously adopted consisted in replacing every null grey level by the value 1. We propose now a more rigorous approach based on a classical mathematical technique. When a metric d1 is susceptible to reach infinite values, it is 51

common to associate it a novel metric d according to: d1 (x, y) if d1 (x, y) is f inite 1 + d1 (x, y) and d(x, y) = 1 if d1 (x, y) = ∞ d(x, y) =

(32)

This method produces a metric d lying in the interval [0, 1] and respecting u is inthe nearness order between points, because the function u 7→ 1 + u creasing. Now let us start with two grey level functions f and g. We define the ”support” notion S(·) of each of them as the subset of D where they take strictly positive values: S(f ) = {(x, y) ∈ D, f (x, y) 6= 0} The spatial domain D is then separated in three disjointed subsets: D1 = {(x, y) ∈ D, f (x, y) 6= 0 and g(x, y) 6= 0} = S(f ) ∩ S(g) D2 = {(x, y) ∈ D, f (x, y) = 0 and g(x, y) 6= 0 or g(x, y) = 0 and f (x, y) 6= 0} = S(f ) 4 S(g) where 4 denotes the symmetric difference between two sets (A 4 B = A ∪ B \ A ∩ B).

D3 = {(x, y) ∈ D, f (x, y) = g(x, y) = 0} = S(f )c ∩ S(g)c = (S(f ) ∪ S(g))c At this step, the multiplicative contrast C(x,y) (f, g), which plays the role of the above-mentioned metric d1 , is computed on each of these subsets: • If (x, y) ∈ D1 , C(x,y) (f, g) is classically defined by the relation (formula 18): C(x,y) (f, g) M in(f (x, y), g(x, y)) = M ax(f (x, y), g(x, y)) 52

• If (x, y) ∈ D2 , C(x,y) (f, g) = ∞ • If (x, y) ∈ D3 , C(x,y) (f, g) = 0 by convention Now let us transform these values according to formula (31). A novel punctual distance d is obtained: C(x,y) (f, g)

d(x,y) (f, g) =

if (x, y) ∈ D1

1 + C(x,y) (f, g) d(x,y) (f, g) = 1 if (x, y) ∈ D2 d(x,y) (f, g) = 0 if (x, y) ∈ D3 It is possible to perform a summation of these elementary distances when (x, y) lies in D (or a region of D), resulting in a metric dS,GL taking into account the supports shapes as well as the distance between grey levels on the supports intersection: dS,GL (f, g) =

X

d(x,y) (f, g)

(x,y)∈D





X  C(x,y) (f, g)  [1] dS,GL (f, g) = +  (x,y)∈D2 (x,y)∈D1 1 + C(x,y) (f, g) X

but

X

[1] = Card(D2 ), that we can assimilate to the area of D2 or, by

(x,y)∈D2

definition of D2 to the area of S(f ) 4 S(g) i.e. d4 (S(f ), S(g)) (d4 is well known as the ”symmetric difference distance” between two binary shapes). Comment: the first sum of dS,GL (f, g) represents a distance in terms of grey levels and the second one a distance in terms of shapes. The main interest we imagine for this metric is to quantify the evolution in shape and in grey level (concentration) of a polluting cloud (fumes, radioactive emissions...).

53

4. Conclusion and perspectives In the present paper, we proved the efficiency of contrast notions in the Logarithmic Image Processing framework. Two notions have been studied: the Logarithmic Additive Contrast (LAC) and the Logarithmic Multiplicative Contrast (LMC), each of them with specific properties. Various metrics associated to these contrasts have been defined and their applicative efficiency demonstrated. Concerning color images, we refer the interested reader to a recent publication where a specific color contrast has been defined (Jourlin et al. (2011)), presenting analogous properties: consistence with human vision, contour detection ability, weak dependence to non-uniform lighting. We have detected various possible extensions of this work that we will develop in future publications, particularly: • Interest of logarithmic metrics for image classification: Region Growing, k-means, Hierarchical Ascendant Classification, propagation methods (fast Marching, Percolations...) • Local corrections of contrast/shading, for example (Fig.29)

(a)

(b)

Figure 29: (a) initial image, (b) image with local corrections of contrast/shading: the dark parts of (a) are enhanced without overlighting the sky

acknowledgements Many thanks are due to Dr. P. Hawkes, Editor of Advances in Imaging and Electron Physics, for his constant help and encour54

agement to publish this contribution to novel notions of logarithmic contrast and associated metrics. Let us also express here our gratitude for the searchers who have developed LIP-derived Models, and have then considerably improved the tool box of non-linear image processing. A particular distinction is due to: The team of Tufts University, notably K. Panetta, S. Agaian and their disciples. They introduced the PLIP (Parametrized Logarithmic Image Processing) Model and developed novel enhancement techniques (Wharton et al. (2006), Panetta et al. (2008)) like: • EPCE (Edge Preserving Contrast Enhancement), which preserves edge details while enhancing images with variable illumination, • Multi-Histogram equalization algorithm Associated to these techniques they proposed objective measures of image enhancement. They dedicated other important papers to Logarithmic Edge Detection Wharton et al. (2007) and Image Fusion Nercessian et al. (2011). The team of Polytechnica Bucaresti, with V. Buzuloiu, V. Patrascu and their disciples E. Zaharescu, C. and L. Florea, C. Vertan, A. Oprea... In the PhD thesis of V. Patrascu directed by V. Buzuloiu, a novel LIP Model has been introduced defined on a grey scale symmetric related to the origin. For a presentation of this Model and its applications to histogram equalization, image enhancement, color image processing, see Patrascu and Buzuloiu (2001), Patrascu and Buzuloiu (2002), Patrascu et al. (2003), Patrascu and Buzuloiu (2003), Patrascu (2004). Concerning biomedical applications, see Zaharescu (2005), Zaharescu (2007), Florea et al. (2007), where image enhancement and multiplicative logarithmic morphological operators are presented. In Florea et al. (2008), the interested reader will discover a Pseudo-LIP Model for edge detection. 5. Bibliography Barat, C., Ducottet, C., Jourlin, M., 2003a. Line pattern segmentation using morphological probing. In: 3rd International Symposium on Image and Signal Processing and Analysis. pp. 417–422. Barat, C., Ducottet, C., Jourlin, M., 2003b. Pattern matching using morphological probing. In: IEEE ICIP 2003. Vol. 167.

55

Barat, C., Ducottet, C., Jourlin, M., 2010. Virtual double-sided image probing: A unifying framework for non-linear grayscale pattern matching. Pattern Recognition 43 (10), 3433 – 3447. Beucher, S., 1991. The watershed transformation applied to image segmentation. In: Scanning Microscopy International. pp. 299–314. Beucher, S., Lantuejoul, C., Sep. 1979. Use of Watersheds in Contour Detection. In: International Workshop on Image Processing: Real-time Edge and Motion Detection/Estimation, Rennes, France. Brailean, J., Sullivan, B., Chen, C., Giger, M., 1991. Evaluating the em algorithm for image processing using a human visual fidelity criterion. In: Proceedings of the International Conference on Acoustics. Speech and Signal Processing, pp. 2957–2960. Carr´e M., Jourlin M., B. M., 2011. Additive contrast and associated metrics. In: Proceedings of 13th International Congress for stereology. Deng, G., Pinoli, J.-C., 1998. Differentiation-based edge detection using the logarithmic image processing model. Journal of Mathematical Imaging and Vision 8, 161–180. Fillere, I., 1995. Outils math´ematiques pour la reconnaissance de formes: propri´et´es et applications - mathematical tools for shape recognition: properties and applications. Ph.D. thesis, Saint-Etienne University (France). Florea, C., Vertan, C., Florea, L., 2007. Logarithmic model-based dynamic range enhancement of hip x-ray images. pp. 587—-596. Florea, C., Vertan, C., Florea, L., Oprea, A., 2008. A pseudo-logarithmic image processing framework for edge detection. pp. 637–644. Gr¨ unbaum, B., 1963. Measures of symmetry for convex sets. In: Proc. Symp. Pure Math. Vol. 7. pp. 233–270. Jourlin, M., Breugnot, J., Itthirad, F., Bouabdellah, M., Closs, B., 2011. Chapter 2 logarithmic image processing for color images. In: Hawkes, P. W. (Ed.), Advances in Imaging and Electron Physics. Vol. 168 of Advances in Imaging and Electron Physics. Elsevier, pp. 65 – 107. Jourlin, M., Pinoli, J.-C., 1988. A model for logarithmic image processing. J. Microsc. 149, 21–35. Jourlin, M., Pinoli, J.-C., 1995. Image dynamic range enhancement and stabilization in the context of the logarithmic image processing model. Signal Process. 41 (2), 225–237. Jourlin, M., Pinoli, J.-C., 2001. The mathematical and physical framework for the representation and processing of transmitted images. Advances in imaging and electron physics 115, 129–196. Jourlin, M., Pinoli, J.-C., Zeboudj, R., 1989. Contrast definition and contour detection for logarithmic images. J. Microsc 156, 33–40. Kohler, 1981. A segmentation system based on thresholding. Computer Graphics and Image Processing 15 (4), 319 – 338. Matheron, G., 1970. La Th´eorie des variables r´egionalis´ees, et ses applications. Les Cahiers du Centre de morphologie math´ematique de Fontainebleau. Ecole Nationale Sup´erieure des Mines de Paris. Nercessian, S. C., Panetta, K. A., Agaian, S. S., January 2011. Multiresolution decomposition schemes using the parameterized logarithmic image processing model with application to image fusion. EURASIP J. Adv. Signal Process 2011, 1:1–1:17. Otsu, N., jan 1979. A threshold selection method from grey-level histograms. IEEE Trans-

56

actions on Systems, Man and Cybernetics 9 (1), 62–66. Panetta, K., Wharton, E. J., Agaian, S. S., February 2008. Human visual system based image enhancement and logarithmic contrast measure. IEEE Transactions on Systems, Man, and Cybernetics—Part B: Cybernetics 38 (1), 174–88. Patrascu, V., 2004. Color image enhancement method using fuzzy surfaces in the framework of the logarithmic models. In: Press, W. S. (Ed.), Proceedings of the 6th International FLINS Conference, Blankenberge (Belgium). pp. 403–408. Patrascu, V., Buzuloiu, V., 2001. A mathematical model for logarithmic image processing. In: The 5PthP World Multi-Conference on Systemics, Cybernetics and Informatics, Orlando, USA. Vol. 13. pp. 117–122. Patrascu, V., Buzuloiu, V., 2002. Recent trends in multimedia information processing, world scientific press. In: Proceedings of the 9th International Workshop on Systems, Signals and Image Processing, IWSSIP’02, Manchester (United Kingdom). pp. 312– 316. Patrascu, V., Buzuloiu, V., 2003. Color image processing using logarithmic operations. In: Proceedings of The IEEE International Symposium on Signals, Circuits and Systems, Iasi, Romania. pp. 133–136. Patrascu, V., Buzuloiu, V., Vertan, C., March 2003. Fuzzy Filters for Image Processing. Springer Verlag, Germany, Ch. Fuzzy Image Enhancement in the Framework of Logarithmic Model, pp. 219–237. Pumo, B., Dhorne, T., 1998. Mod`ele logarithmique de r´egression optique. application `a l’identification des fonctions de transmittance. Rev. Stat. Appl. XLVI (3), 66–75. Pun, T., July 1981. Entropic thresholding: A new approach. CGIP 16 (3), 210–239. Tsai, W.-H., 1985. Document image analysis. IEEE Computer Society Press, Los Alamitos, CA, USA, Ch. Moment-preserving thresholding: a new approach, pp. 44–60. Ul Haq, M., J. M. B. M., 2011. Lip haralick. In: Proceedings of 13th International Congress for stereology. Wharton, E., Agaian, S., Panetta, K., 2006. Comparative study of logarithmic enhancement algorithms with performance measure. Vol. 6064. SPIE, p. 606412. Wharton, E. J., Panetta, K., Member, S., Agaian, S. S., 2007. Logarithmic edge detection with applications. Zaharescu, E., 2005. Medical image enhancement using logarithmic image processing. In: Press, W. S. (Ed.), Proc. 2005 IASTED Int. Conf. Visualization, Imaging and Image Processing, Benidorm (Spain). Zaharescu, E., 2007. Multiplicative logarithmic morphological operators used in medical imagery. In: Press, W. S. (Ed.), IEEE Proceedings of the 15th International Conference on Digital Signal Processing, Cardiff (UK). pp. 165–168.

6. Main notations LIP addition LIP subtraction LIP multiplication f, g grey levels images D definition domain of images (spatial support) 57

α, β, λ, µ scalars elements i, j, k integer elements M maximum value of an image (256 for 8-bits images) I space of grey level images F over-space of I Tf (x) transmittance of grey level image f at point x m (f ) Michelson’s contrast of f at the pair (x, y) C(x,y) C(x,y) (f ) Logarithmic Additive Contrast (LAC) of f at the pair (x, y) AC(x) (f ) average LAC at x M C(x) (f ) maximum LAC at x P

grey levels summation in the LIP sense d1 (f, g) ”global” distance between f and g associated to the LAC dinf (f, g) ”atomic” distance between f and g associated to the LAC d1,supR (f, g) ”intermediate” distance between f and g associated to the LAC C(x,y) (f ) Logarithmic Multiplicative Contrast (LMC) of f at the pair (x, y) N (f ) normalized LMC of f at the pair (x, y) C(x,y)

AC(x) (f ) average normalized LMC at x M C(x) (f )) maximal normalized LMC at x d1,RN (f, g) ”global” distance associated to the normalized LMC on a region R dinf N (f, g) ”atomic” distance associated to the normalized LMC N (f, g) ”intermediate” distance associated to the normalized LMC on d1,supR a region R dAs (A, B) Aspl¨ und’s distance between two shapes A and B dAs (f, g) Aspl¨ und’s distance between two images f and g

und’s neighborhood of f with tolerance  NAs, (f, g) Aspl¨ dS,GL bounded metric associating binary and grey level approaches

58