Invariant Framework for Di erential A ne Signatures 1 ... - CiteSeerX

Invariant Framework for Dierential Ane Signatures Ron Kimmel

August 29, 1995 Abstract

A framework for generating dierential ane invariant signatures for planar shapes is introduced. Non trivial invariant signatures and their corresponding arclengths are computed for planar shapes with smooth boundaries. These signatures are useful for pattern recognition and classi cation under partial occlusion. In this paper we deal only with implementable signatures and restrict the ane transformation group accordingly. Based on the theory of ane curve evolution, an invariant gradient magnitude along the geometric scale space is used as an invariant edge enhancer. Then, the geometric heat equation for weighted (by the enhancer) and `dynamic weighted' ane arclength de nitions are shown to yield an invariant selective smoothing algorithm. This algorithm is used for image denoising in cases where we need to clean noisy images before computing invariant features. The denoising operation deforms the geometry of the object in a predictable invariant way, unlike traditional image denoising algorithms, so that the mapping between planar shapes after the denoising is preserved. The relation between the ane curvature and the Euclidean one leads to an ecient method for approximating the ane curvature signature along the boundary, while the Euclidean curvature itself is used for generating the ane arclength parameter. Both curvatures are computed from the gray level image, using the implicit representation of the object's boundary as it appears in real world images. When the projection invariance assumption of the gray levels is added, robust non-trivial signatures are obtained.

Keywords: Invariant signature, Pattern recognition and classi cation under partial occlusion, Image invariant selective smoothing, Semi-dierential and dierential invariants.

1 Introduction An important problem in image analysis and shape understanding is the segmentation problem. The question is how to isolate an object in a given image and how to integrate object boundaries in noisy data images to achieve a good model of the object under inspection. 

TECHNION, Electrical Engineering Department, Haifa 32000, Israel, Email: [email protected]

1

2 Invariant Framework for Dierential Ane Signatures The low level segmentation problem was addressed in many ways over the years, starting with gray level thresholding, region growing, and deformable contours based on energy minimization along a given curve called `snakes'. At the higher level, after an object is isolated, the problem of recognition rises. In this case the question is how to classify the given object. Euclidean invariant operations refer to those operations for which movements and rotations of the objects in the image plane do not eect the result of the operation. We know that for pictures in the `real' world, the class of transformations we encounter is much richer than pure Euclidean. In this paper we take one step into the world of transformations and deal with the ane group. The cases in which the camera is far away from the objects, so that perspective contributes minor distortions, and the objects are almost planar, may be considered (approximated) as part of the ane group. A framework that takes the ane invariance demand into consideration even before the shape is segmented, is introduced. It is shown how to deal with noisy images, and how to make use of the gray level information for generating invariant signatures and denoising algorithms. One of the fundamental problems in pattern recognition is the problem of classifying a partially occluded object using a local description of its boundary [8, 6, 18, 7, 5, 4]. On the other hand, in the eld of image processing, recent non-linear geometric based algorithms were shown to give very promising results compared to `optimal' linear algorithms [22, 20, 23, 3, 2]. These two research elds, that appear to be unrelated in nature, are treated in this paper by gaining the motivation from the theory of curve evolution. Speci cally, we will use the ane scale space, generated by the ane heat equation as introduced in [25], to construct an ane invariant gradient magnitude. The ane gradient magnitude (ane edge enhancer) will help us in constructing an image denoising algorithm that is ecient as well as invariant to ane transformations. This algorithm is useful for denoising images before generating invariant signatures. In fact, the invariance property of the selective smoothing procedure guarantees that the change in the geometry of the shape in the smoothed image can be predicted by applying same algorithm to a clean reference image of the shape we try to recognize. The ane gradient magnitude will also serve as a non trivial signature for pattern classi cation when projection invariance of the gray levels along the shape boundary is assumed. The projection invariance of gray levels [2] states that the order of gray level along the boundary is preserved. It will be shown that the ane curvature can be approximated by a simple implementable equation exploiting the gray level information in the data image. This approximation is used for computing a simple dierential ane invariant signature. The structure of the paper is as follows: Section 2 describes equi-ane invariant properties of planar curves, like the ane arclength, the ane normal, and the ane curvature. In Section 3 the ane heat equation as introduced in [25] is used for constructing an ane gradient magnitude and Laplacian de ned along the geometric scale space. Section 4 presents an invariant selective smoothing procedure for image denoising. It is shown to be equivalent to the geometric heat equation of the `weighted ane arclength'. In Section 5, the invariant arclength and and the geometric heat equation for the linear ane arclength are presented. One point is added to the equi-ane group, leading to a subgroup of the equi-ane. Section

R. Kimmel, August 29, 1995 3 6 presents the intrinsic property of the weighted arclength, and an expression of the ane curvature as a function of the Euclidean curvature and its derivatives according to the Euclidean arclength. Then, Section 7 presents implementation considerations for implementing the proposed procedures, and an approximation of the ane curvature that uses up to second order derivatives along the boundary implicit representation (i.e. the data image). Section 8 presents some examples of using the proposed techniques for generating ecient and robust ane invariant signatures, and using the dynamic weighted ane geometric heat equation for image denoising. In the appendix, it is shown how to express the Euclidean curvature and its derivatives from an implicit representation of the boundary, and how to approximate the ane curvature. We shall start by introducing some basic concepts from the theory of ane dierential geometry of planar curves. More details can be found in [9].

2 Ane Invariants of Planar Curves

Let C (p) : [a; b] ! IR2 be a simple regular parametric planar curve (in its planar coordinates: C (p) = fx(p); y(p)g). Let v be the Euclidean arclength so that1

v(p) =

p

Z

0

hCp~; Cp~i1=2dp~;

where p~ 2 [a; b] is an arbitrary parametrization. The tangent is known to be given by

Cv = T~ ; and the Euclidean curvature vector is given by

Cvv = N~ ; where N~ is the curve normal and  is the Euclidean curvature. We will use (X; Y ) as the determinant of the 2  2 matrix whose columns are given by the vectors X; Y 2 IR2. The equi-ane arclength s is de ned so that (Cs ; Css) = 1;

(1)

and is given by [9]

s(p) =

Z

p

0

j(Cp~; Cp~p~)j1=3dp~;

see Figure 1, which is an intrinsic integral (as we will see in Section 6). We will use the following notations along the paper: Cp  @@Cp , hfa; bg; fc; dgi  ac + bd, and (fa; bg; fc; dg)  ad ? bc. 1

4

Invariant Framework for Dierential Ane Signatures C v= T

−1/3 555555555555555 C s =|k| T 555555555555555 555555555555555 555555555555555 1 !!!!!!!!!!!!!!!!!!!!!!!!!!!! 555555555555555 !!!!!!!!!!!!!!!!!!!!!!!!!!!! 555555555555555 !!!!!!!!!!!!!!!!!!!!!!!!!!!! 555555555555555 C ss !!!!!!!!!!!!!!!!!!!!!!!!!!!! 555555555555555 !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! 1/3 !!!!!!!!!!!!!!!!!!!!!!!!!!!! |k| N !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! C vv= k N !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!! !!!!!!!!!!!!!!!!!!!!!!!!!!!!

Figure 1: For s = ane arclength and v the Euclidean one, the relations between Cs ; Css; Cv and Cvv are presented. Using the intrinsic property, the relation between the Euclidean and ane arclength [26] is obtained from Z

s = j(Cv ; Cvv )j1=3dv; that yields

ds = j(C ; C )j1=3 v vv dv = j(T~ ; N~ )j1=3 = jj1=3:

(2)

Using the above expression, the ane tangent is given by ?1=3 ~ Cs = Cv @v @s = jj T :

Dierentiating Equation (1) we have (Cs ; Csss ) = 0; which means that the vectors Cs and Csss are linearly dependent: Csss = ?Cs :

(3)

R. Kimmel, August 29, 1995 5 The scalar  is the simplest ane dierential invariant of the curve C , known as the ane curvature, and Css is the ane normal vector. A direct result form the last equation is

 = (Css ; Csss ): Dierentiating Equation (3) with respect to s it also follows that

 = (Cssss ; Cs ): It can be shown that , the ane curvature, is the fastest normal velocity minimizing the ane arclength. However, unlike the Euclidean arclength shortening ow (Ct = N~ ) which is the same as the Euclidean geometric heat equation, it does not lead to any constructive smoothing scale space. The ane scale space [26] is achieved by the ane geometric heat equation Ct = Css . In the next section, the ane gradient magnitude along the scale space [24] is used to de ne an invariant edge enhancer. The edge enhancer will be used for signature generation and for constructing an image invariant denoising algorithm.

3 Ane Edge Enhancer For constructing an edge enhancer we will use the ane geometric heat equation as presented in [25]. In the proposed model all the level sets of the data image are simultaneously evolved so that each gray level set is propagating according to

Ct = Css : (4) Now C (s; t) : [a; b]  [0; T ) ! IR2, is a two parametric curve, where s is the ane arclength, and t indicates the `time' of evolution. Observe that

Css

@ C @v = @s v @s 2 2 @v = Cvv @s + Cv @@sv2 2 ~ ?1=3j2 + T~ @ v2 = Nj @s 2 = 1=3N~ + T~ @@sv2 : !

!

Considering only the normal component in the evolution (the tangential component aects only the internal parametrization and does not in uence the shape of the propagating curve [13]) the corresponding evolution equation is given [24] by

Ct = 1=3N~ :

(5)

6 Invariant Framework for Dierential Ane Signatures Consider the three dimensional function (x; y; t) : IR2  [0; T ) ! IR for which each level set C = ?1(c) is evolving according to Equation (5). The implicit (Eulerian) formulation [21] of (5) (see Section 7 and the appendix) is given by: 1=3 r  jrj t = r
jrj = xx2y ? 2x y xy + yy 2x !!



1=3



:

Given (x; y; 0) = I (x; y) one can evolve the whole image according to the `ane geometric heat equation' so that (x; y;
T ) is the result of propagating  for t =
T (see [2, 25, 27] for more details on the above equation used as an ane invariant geometric smoothing operator on images.) We shall denote by E (
T ) the evolution operation for t =
T , i.e. (
T )  E (
T )  (0). Obviously E is an ane invariant operation [25], i.e. A  E (
T )  (0) = E (
T )  A  (0), where A is the ane transformation. De ne the ane gradient magnitude G(
T )  (0)  (
T
)T?(0) . Then we readily have the following lemma Lemma 1 The operation de ned by G(
T )  (0)  (
T
) T? (0) ; is invariant under the equi-ane transformation. Proof. A  G(
T )  (0) = A  E (
T ) 
T(0) ? (0) = A  E (
T ) 
T(0) ? A  (0) = E (
T )  A 
T(0) ? A  (0) = G(
T )  A  (0): The question is why can we consider jG(
T )  I (x; y)j, where I is the image, to be an edge enhancer? It is obvious that edges are traversed by the evolution operation E (
T ) and that constant regions in the image will stay still. Yet edges of high curvature will propagate in a higher velocity than those that appear as straight lines. The curvature dependent evolution yields a non-homogeneous result along the edges when applying G(
T ). The fact that at edges of high curvature the edges are enhanced may as well be considered as a desired property. For example, active contour models [11] that are used to integrate edges so as

R. Kimmel, August 29, 1995 7 to segment object boundaries, are pushed by a geometric force that is proportional to the curvature of the propagating contour. The result of the geometric ane edge enhancer that enhances edges of high curvature therefore helps in the convergence of the active contour near the boundary, when used as the underlying potential. In a similar way one can de ne the ane Laplacian L(
T ) to be: L(
T )  (0)  (2
T ) ?
2T(
2 T ) + (0) : In the following sections it is shown how to use the ane gradient magnitude for constructing invariant dierential signatures for object recognition under partial occlusion, and image invariant denoising algorithm.

4 Invariant Image Denoising Procedure Base on ideas put forward in [27], we now construct an invariant image selective smoothing algorithm. The proposed algorithm is a procedure for image denoising with invariant control on the changing geometry of the shape through the smoothing operation. The procedure is useful for denoising images before computing the invariant signatures. Its relation to the total variation decreasing algorithms and to geometric heat equations are explored. Gradient based edge detectors are usually based on the discretization of edge enhancer jrG  I j, see [10], which is equivalent to G jrI j. The convolution with a Gaussian kernel G is performed in order to overcome small perturbations and insigni cant, high frequency, spatial noise. The variance of the smoothing operator  is analog in our jG(
T )  I j model, to
T the amount of smoothing. In [23], the authors deal with image denoising they named nonlinear total variation based noise removal by minimizing the integral Z

jrI jdxdy:

Convergence is achieved by limiting the displacement of the result from the noisy image to be proportional to the noise variance. It is done by adding a stopping condition to prevent over smoothing, e.g. ((t) ? (0))2dxdy = 2 (for (0) = I (x; y)). We have noticed that the same results (in some cases even more eciently, yet without the convergence property) are achieved by limiting the time of evolution in the following scheme. Where the time of evolution is now proportional to the noise variance. The resulting minimization scheme is 2  ? 2   + 2  t = x yy (2 +x y 2)xy3=2 y xx given (0) = I (x; y); x y according to which, each level set of the function  is evolving via C = 1 N~ : R

t

jrj

8 Invariant Framework for Dierential Ane Signatures The last equation can be read as 1 Ct = edge enhancer of (t)  geometric smoothing; that leads to conditional geometric smoothing (`selective smoothing' according to [2]), so that at regions of high gradient (close to an edge) the smoothing is low, while at constant regions the smoothing is high. It is possible to replace the 1=jrj with more sophisticated dynamic edge enhancers which take us to other kinds of algorithms [3]. In cases where clean edges are given, yet the image itself is very noisy, it is possible to use the edges information for selecting the smoothing regions. A mask of weights is built from the initial given edges, and then used to control the smoothing process so that each level is propagating by Ct = W (x; y)N~ ; where 1 W (x; y) = given clean edges of I (x; y) ; is the control mask. The same motivation from the above Euclidean case direct us towards the construction of an ane invariant image denoising algorithm. De ne a potential function f : IR2  [0; T ) ! IR+, f (x; y; t) = F (jG(
T )  (t)j); where F (
) : IR+ ! IR+ is a decreasing function. Then, the invariant denoising procedure is performed via t = f (x; y; t)(2xyy ? 2xy xy + 2y xx)1=3 given (0) = I (x; y); (6) according to which, each level set of the function  is evolving by Ct = f (x; y; t)1=3N~ : (7) Selecting a static inverse edge enhancer function f (x; y; t) = f (x; y; 0) = F (jG(
T )  (0)j), Equation (7) becomes exactly the geometric heat equation Ct = Cs~s~ of the `weighted equiane arclength': ds~ = g(C )j(Cp; Cpp)j1=3dp q

where g(x; y) = 1= f (x; y), and since f is ane invariant, ds~ is also ane invariant. Using the causality property, the arclength may be rede ned dynamically along the propagation, yielding the general invariant evolution, Equation (6).2 Such an ane invariant evolution of all the gray level sets simultaneously is invariant under the `projection invariance' assumption as presented in [2]. 2

R. Kimmel, August 29, 1995 Taking
T ! 0 in the ane gradient magnitude de nition, results

(
T ) ? (0) lim ( G (
T )   (0)) = lim
T !0
T !0
T = ddt(t) jt=0 = 1=3jr(0)j:

9

!

As an example, let f (x; y; t) = 1=jG(
T )  (t)j and
T ! 0, so that f (x; y; t) = 1 j1=3 jjr(t)j . Then Equation (6) becomes

t = sign(); and each level set of  is propagating via

) N~ ; Ct = sign( jrj which is a conditional osetting procedure. Although ane invariant as well as conditionally `variation decreasing', the above evolution is unstable near in ection points, and tends to form shocks. Selecting
T > 0 results a smoothed version of the above example. To summarize, using the geometric heat equation that is based on the right arclength, we have constructed an ane invariant noise removal algorithm. The algorithm is invariant in the sense that the result of the algorithm when applied to a given image is transfered exactly to the result of applying the algorithm to the transformed image. It is useful for cleaning noisy images with the ability to predict the geometry deformation that is caused by the smoothing process. In [8] the arclength and curvature for the projective and its general subgroups were introduced. In the following section, simple arclength for the linear ane group are used to de ne the corresponding heat equation. Here, we add one point to the ane group. In practice, we deal with shape descriptors in which there exist one anchor point to simplify the arclength de nition. This arclength is referred to as semi-dierential invariant in [6, 18]. By `simple arclength' we refer to an invariant arclength that is de ned by, at most, second order derivatives of the curve.

5 Linear-Equi-Ane Arclength and Evolution In [19] the authors argue that Equation (4), leads towards the geometric heat evolution equation of any given metric which is given by

Ct = Crr ;

10 Invariant Framework for Dierential Ane Signatures where r is the arclength de ned for the speci c transformation group. Simulating these evolution equations it is enough to track only the normal component of the evolution velocity, so that geometrically the above equation may be written as3

~ N~ : Ct = hCrr ; Ni

This way it is possible [24] to reformulate the ane heat equation Ct = Css into its geometric equivalent Ct = 1=3N~ . In [19] it was also shown that for the similarity group the inverse geometric heat equation, given by C = ? 1 N~ ; t



is invariant. Yet it can be applied only to convex shapes. In this section we introduce an invariant evolution of the ane group with one given point (e.g. given origin, or any other point). Using the same argument as for the equi-ane arclength, that is areas are invariant under the ane transformation, let the linear ane arclength be given by [6, 18] Z

w = j(C ; Cp)jdp: See Figure 2. As may be easily veri ed (following the same steps as the proof of Lemma 2 in the following section) this is an intrinsic measure that does not depend on the parameter, and therefore equivalent to Z

w = j(C ; T~ )jdv; where v is the Euclidean arclength as before. 444444444444444444444444 Cw T 444444444444444444444444 444444444444444444444444 C 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 1 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 444444444444444444444444 O

Figure 2: For w = linear ane arclength, the area j(C ; Cw )j  1. 3

~ N~ is t = hCrr ; ri, see Section 7. The Eulerian formulation of Ct = hCrr ; Ni

R. Kimmel, August 29, 1995 The area of a closed shape [15] is given by ~ dv; A = 12 (C ; T~ )dv = 12 hC ; Ni see Figure 3. I

11

I

!!!!!!!!!!!!! !!!!!!!!!!!!! !!!!!!!!!!!!! T C(p) !!!!!!!!!!!!! 7777777777777777 !!!!!!!!!!!!! dp dp 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 !!!!!!!!!!!!! C(p) + 7777777777777777 2222222 !!!!!!!!!!!!! − 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 !!!!!!!!!!!!! 7777777777777777 2222222 7777777777777777 2222222 o

T

Figure 3: The area integration is performed over all the triangles formed by the vectors C (p) and T~ (p)dp. Therefore, the arclength (as for the equi-ane one) corresponds to an area, and thus invariant. An arclength element is given by ~ dv: dw = j(C ; T~ )jdv = jhC ; Nij Thus, C = C @v = C 1 = 1 T~ ; w

and

Cww

v

@s

v

~ jhC ; Nij

~ jhC ; Nij

2 @ 2v @v = Cvv @w + Cv @w 2 2 @2v 1 ~ = N + @w2 T~ ~ jhC ; Nij 1 ~ = ~ 2 N + tangential component: hC ; Ni !

!

The corresponding linear ane heat equation given by its geometric Euclidean version is (8) C = 1 N~ : t

~ 2 hC ; Ni

12

Invariant Framework for Dierential Ane Signatures Observe that when considering closed curves, the interesting cases are those in which the anchor point is located inside the curve and no tangent points are formed. In other cases, it is possible to locate the smallest triangle that is formed by two tangent points (it is easy to prove that there exist at least two) and the anchor point. This triangle is then mapped into a given reference triangle for every data image, and the same mapping is used to eliminate the transformation eect, see Figure 4. So, considering only the `interesting' cases, the geometric heat equation (8) is not in uenced by singular values since tangent points are excluded.

2222222222 2222222222 2222222222 2222222222 2222222222 2222222222 2222222222 2222222222 2222222222 o 2222222222 2222222222

2222222222 2222222222 2222222222 2222222222 2222222222

o

o

Figure 4: There are three possible locations for the given point: Outside the shape, in which C (p) is tangent at least twice to the boundary of the shape. Inside a shape which consists of a concave part that leads again to at least two tangent points. Inside a shape with no tangents at all, which is the interesting case from our point of view.

6 Arclength De nition and Intrinsic Functionals In this section, we show that by `weighing' the ane arclength , the intrinsic property of the arclength does not break. Then, the relation between the ane curvature and the Euclidean one is used to generate a simple non trivial dierential shape descriptor. We refer to such as ane invariant signatures. De ne a `non-edge' penalty function g : IR2 ! IR+ , that being a one to one mapping is also ane invariant. For example, let (9) g(x; y) = jG(
T ) 1 I (x; y)j : The function g maps edges to low positive values and constant regions to high positive values. Actually any decreasing positive function g of an ane gradient magnitude is valid for the rest of our discussion.

R. Kimmel, August 29, 1995 13 Without breaking the invariance property, we integrate g along the ane arclength. Let L(C ; Cp; Cpp) = g(C (p))(Cp ; Cpp)1=3: We will consider the following functional as the `weighted ane arclength' Z

S [C ] = L(C ; Cp ; Cpp)dp:

(10)

Let us prove that (10) is free of parametrization [16] (i.e. an intrinsic integral). We note that this is only an example, and the intrinsic property should hold for any arclength. Lemma 2 The functional p1

Z

p0

g(C (p))(Cp ; Cpp)1=3dp

depends only on the curve in the xy-plane4 de ned by the parametric equation x = x(p), y = y(p), and not on the choice of the parametric representation of the curve. Proof. We show that if we go from p to a new parameter r by setting

p = p(r); where dp=dr > 0 and the interval [p0; p1] goes into [r0; r1], then r1

Z

r0

g(C (r))(Cr ; Crr )1=3dr =

Z

p1

p0

g(C (p))(Cp ; Cpp)1=3dp:

Since we have yr = yppr , yrr = yppp2r + ypprr and similarly xr = xppr , xrr = xppp2r + xpprr , it follows that r1 g(C (r))(Cr ; Crr )1=3dr Z

r0

=

Z

r1

r0 Z r 1

g(C (r)) xppr (yppp2r + ypprr ) ? yppr (xppp2r + xpprr )

Z



dr

1=3

g(C (r)) xpyppp3r ? ypxppp3r dr r0 r1 g (C (r)) (xpypp ? ypxpp)1=3 dp = dr dr = r0

=

1=3







Z

p1

p0

g(C (p))(Cp; Cpp)1=3dp:

Lemma 2 guarantees that the functional (10) is free of the parametrization of the curve. In fact, in the general case of selecting an arclength for a given transformation group, the integral must be intrinsic! It should have the general form of dl = [geometric quantity]jCpjdp, where 4

often referred to as the `orbit', `trace' or `image' of the curve C (p)

14 Invariant Framework for Dierential Ane Signatures `geometric quantity' is 1 for the Euclidean group, k(p)1=3 for the equi-ane group, k(p) for the similarity, (C (p); T~ (p)) for the linear-ane, etc.. Lemma 2 can be used to nd the relation between the ane curvature  and the curvature derivatives ; v and vv . The same relation could be found in other ways as well [24, 9, 18]. It is easy to show that the normal propagation that corresponds to the ane arclength shortening ow is given by Ct = N~ , which is analog to the Euclidean shortening ow Ct = N~ . However, the ane ow does not have the same smoothing properties as the Euclidean one. The computation of the ane curvature  involves derivatives of the forth order, and may be computed as presented in [9]. Expressing the ane curvature  as a function of the Euclidean curvature  (see [24, 18]), yields (11)  = 4=3 ? 95 ?8=32v + 13 ?5=3vv : In the following section and in the appendix, we show how to approximate the ane curvature based on Equation (11), and use it to construct invariant signatures. This approximation will use up to second order derivatives of the the implicit representation of the object boundary. Which means a direct operation on the data image, before any thresholding is performed.

7 Implementation Considerations Before thresholding, the object boundary is given in an implicit representation, e.g. the boundary is de ned as a given gray level set of the data image: I (x; y) =Threshold. For all of our implementations of curve evolution as well as the computation of the ane and Euclidean curvatures this implicit level set representation is used. Similarly, implicit representation of planar curves will be used for the curve evolution implementations. We refer the interested reader to the growing literature on level set motion for curve and surface evolution, starting with the Osher Sethian classical paper [21]. The basic idea is to map the `time' dependent coordinates (p; t) to xed coordinates (x; y; t) by embedding the propagating contour in a higher dimensional function. The level sets of the function (x; y; t) : IR2  [0; T ) ! IR propagating according to t = jrj; where jrj  2x + 2y , are evolving according to q

Ct = N~ :

This result may be easily obtained form N~ = f?yv ; xv g = r=jrj, and the chain rule ~ = jrj. t = hr; Cti = hr; Ni While the embedding is preserved, as was proven for curvature based evolutions [14], there is no need to control the propagation of the higher dimensional function . The embedding is

R. Kimmel, August 29, 1995 15 preserved for morphology evolutions as well (which require additional `entropy condition'). However, in other cases it is needed to supervise the level sets behavior so that the zero set evolution is the dominant one, and the rest are only swept by its in uence. For this purpose some numerical methods were developed like the narrow strip introduced in [12, 1], re-initialization of the function every iteration [28], expansion of the zero set velocity to the whole image domain [17], etc. In the appendix we introduce formulas expressing ; v and vv as a function of the x and y derivatives of  (up to the fourth order for vv ). Then to achieve practical formulas, we assume that the implicit representation along the boundary is given by polynomial patches of up to second order. This assumption holds only after ane geometric smoothing is performed so that sharp edges are smoothed and the approximation is valid. The ane curvature as given in Equation (11) is approximated by (see Appendix A)

  ? 2   ( 2 ? 2xx yy  +xy2  )2=3 : x xy y xx y x yy

8 Examples The rst example, Figure 5, presents the evolution of a shape according to the ane geometric heat equation Ct = Css , or in its implicit form (after eliminating the tangential component) t = (xx2y ? 2xxy y + 2xyy )1=3. Next, Figure 6 demonstrate the power of the proposed framework in tracking the convex hull of objects by using the `weighted ane' heat equation. Figure 7 presents the result of applying the ane invariant gradient magnitude along the scale space. Denoising algorithm results are presented in Figure 8. The `dynamic weighted ane' heat equation, is an ane selective smoothing operator. It is used to eciently remove noisy perturbations from the image, while preserving the edges. It is possible to predict the geometric deformation caused by this invariant selective smoothing operation, by applying the same procedure to a reference image which is an ane transformation of the data image, see Figure 10. Ane invariant signatures (s), and G(
T )  I (x; y) along the (smoothed) boundary, as a function of the ane arclength are computed by the proposed methods. It is shown in Figure 9 that the signatures of the same object under dierent ane transformations remain almost the same. The gradient magnitude G(
T )  I (x; y) along the boundary of the smoothed object, (the zero level set of (
T )) is a robust signature, yet requires the projection invariance of the gray levels. While (s) is more sensitive, however free of any assumptions on the order of the gray levels along the object boundary in the image. The signature functions are presented without any smoothing or ltering. For closed curves, Fourier descriptors of the periodic signature function may be the right choice for classi cation, while for recognition and classi cation of objects under partial occlusion, local matching methods should be applied. Observe that the signatures G(s)  G(
T )  (0)j(
T )=0, where (0) = I (x; y), of the two images are very similar, while obviously (v) is dierent. The

16 Invariant Framework for Dierential Ane Signatures ane arclength was computed by using the implicit representation of the boundary in the gray level image (see Appendix A). The Euclidean curvature at each pixel grid is interpolated at the grid intersection points with the boundary of the object. Then ds is approximated by
s = 1=3
v, where
v is the length of the line connecting the zero crossings of the object boundary with a given pixel cell (a square de ned by f(i; j ); (i; j +1); (i +1; j +1); (i +1; j )g.)

9 Concluding Remarks In this paper the ane invariant gradient magnitude along the ane scale scale space was introduced and used to construct image denoising algorithms and shape invariant signature functions. The geometric heat equation of the `ane weighted arclength' that integrates the ane edges and the ane arclength from `Ane Dierential Geometry' [9] was shown to be an invariant selective smoothing procedure, resulting in the image denoising algorithm when the `dynamic ane weighted arclength' is used. The forth order derivatives and the non-linear nature of the calculations of the geometrical properties, make the computation of the ane curvature signature a complicated task. We have shown that it is possible to reduce this complexity by locally approximating the implicit representation of the boundary contour as a second order polynomial function. Although such an approximation is valid only at smooth regions of the image, it is possible to treat object boundaries as such, after ane smoothing is applied to the image. A simple dierential signature was obtained directly from the ane scale space, under the projection invariance assumption. It was shown to result a robust invariant non trivial dierential signatures. We conclude by a table presenting the arclength and the geometric heat equations for some of the transformation groups we dealt with. The geometric heat equations may be used for generating the invariant signatures in the same manner we did for the equi-ane group. For the scale space gradient magnitude along the boundary to be a unique invariant signature, it is enough to assume projection invariance and the same ordering of gray levels along the boundaries of the objects under consideration (e.g. jrI j = const: along the boundary under orthographic projection). It is obviously not a necessary condition. Actually, the fact that objects are usually more complex than pure planar shapes with constant intensity of the shape, helps in generating interesting invariants. As a simple example, consider a circle shape of radius R with a constant gradient along the boundary, say jrI j = . The signature of this shape for the limit case
T ! 0, i.e. jrI j1=3 = R1=3 , is obviously constant. Observe that it remains a constant for the ellipses obtained by the ane transformation. This observation by itself is enough to show the non-trivial nature of this signature.

R. Kimmel, August 29, 1995 Group Arclength Euclidean Weighted Euclidean Equi-Ane Weighted Ane Linear-Equi-Ane Weighted Linear-Equi-Ane Similarity Linear Ane

17

L(p) hCp ; Cpi1=2 = 1
jCpj g(C )hCp ; Cpi1=2 j(Cp; Cpp)j1=3 = jj1=3jCpj g(C )(Cp ; Cpp)1=3 = g(C )1=3jCpj j(C ; Cp)j = j(C ; T~ )jjCpj

Geom. Heat Eq. Ct = N~ Ct = g2 N~ Ct = 1=3N~ Ct = g12 1=3N~ Ct = (C;T~ )2 N~

g(C )j(C ; Cp)j j(Cp ;Cpp)j = jjjC j p hCp ;Cpi

Ct = g (C;T~ ) N~ Ct = 1 N~ Ct = (C;T~ ) N~ Ct = ((CC;;T~N~)) N~

j(Cp ;Cpp)j = jj jC j (C ;Cp)2 (C ;T~ )2 p j(C ;Cpp)j = (C ;N~ ) jC j j(C ;Cp)j (C ;T~ ) p

2

2

4

2

2

Acknowledgments

I thank Freddy Bruckstein for his comments on this report, and to Nahum Kiryati for his nancial support, and for discussions on the presentation of some of the methods proposed in this report. I thank Ami Steiner, Mickey Elad and Oren Kidron, for the pleasant time we spent together in the Computer Vision Lab. Electrical Engineering Department, at the Technion, Israel during the summer of 1995.

Appendix A Here the curvature  of the planar curve C = ?1(c), and its rst and second derivatives (v and vv ) are computed as a function of . We use the fact that along the level sets C of , the function does not change its values, i.e. @n =@vn = 0, for any n. Particularly, for n = 2; 3; 4. From this, and the knowledge of the geometrical properties of Cv ; Cvv ; Cvvv and Cvvvv , we compute (using Mathematica algebraic calculations):

 =

2xxy y ? xx2y ? 2xyy (2x + 2y )3=2

v = (63x 2xy y ? 34x xyy y + 33x xxy 2y ? 92x xxxy 2y + 3x2xx3y ?2xxxx3y ? 6x2xy 3y ? 32xxyy 3y + 3xxxy 4y +3xxxy 4y ? xxx5y ? 34xxy yy + 33xxxy yy + 92xxy 2y yy ?3xxx3y yy ? 33xy 2yy + 5xyyy + 3x2y yyy )=(2x + 2y )3

18

Invariant Framework for Dierential Ane Signatures

vv = (245x3xy y ? 246x xy xyy y + 47xxyyy y ? 66x xxyy 2y + 365x xxy xy 2y ?724xxx2xy 2y + 185xxxxyy 2y + 45xxxxy 3y ?244xxxxxy 3y + 603x2xxxy 3y ? 164xxxxxy 3y ? 963x3xy 3y +124xxy xyy 3y + 85x xyyy 3y ? 152x3xx4y + 103xxxxxx4y ? 4xxxxx4y ?124xxxyy 4y + 123xxxy xy 4y + 1322x xx2xy 4y + 63x xxxyy 4y +83xxxxy 5y ? 182xxxxxy 5y ? 48x2xxxy 5y ?122xxxxxy 5y + 24x3xy 5y + 362x xy xyy 5y + 43xxyyy 5y +33xx6y + 10x xxxxx6y ? 22xxxxx6y ? 62xxxyy 6y ?24xxxy xy 6y ? 12xx 2xy 6y ? 12xxxxyy 6y + 4xxxxy 7y + 6xxxxy 7y +4xxxxy 7y ? xxxx8y ? 126x 2xy yy + 67xxyy yy ? 126x xxy y yy +365xxxxy y yy ? 184x 2xx2y yy + 65xxxx2y yy + 1324x 2xy 2y yy ?185xxyy 2y yy + 64xxxy 3y yy ? 1443x xxxy 3y yy +332x2xx4y yy + 23xxxx4y yy ? 722x 2xy 4y yy ? 243xxyy 4y yy +182xxxy 5y yy + 36x xxxy 5y yy ? 32xx6y yy ? 4xxxx6y yy ?36xxx2yy ? 485x xy y 2yy + 334x xx2y 2yy + 603x xy 3y 2yy ?182xxx4y 2yy + 36x3yy ? 154x 2y 3yy + 47x xy yyy ? 46xxxy yyy ?125xxy 2y yyy + 24xxx3y yyy ? 163x xy 4y yyy + 62xxx5y yyy +106xy yy yyy + 104x 3y yy yyy ? 8xyyyy ? 26x2y yyyy ? 4x4y yyyy )= (2x + 2y )9=2 Although impressive in length, the above expressions are too long for practical implementation. Moreover, dealing with pixel based images, the large support required for the computation of the high order derivatives leads to inaccurate and noise sensitive operations around the edges. Assuming that the implicit function  is smooth (can be achieved by ane smoothing) and therefore can be locally approximated by ax2 + bxy + cy2 + dx + ey + f . The third and fourth order derivatives of  may thus be neglected. Using this approximation considerably simplify the scheme, and reduces the local support to a 3  3 pixel mask, with truncation error of O(
x2) (where
x is the distance between neighboring pixels). Thereby, taking all third and forth derivatives of  to be zero, the rst and second derivatives of  are simpli ed into v = (3(23x 2xy y ? 32xxxxy 2y + x2xx3y ? 2x2xy 3y + xxxy 4y ?4xxy yy + 3xxxy yy + 32xxy 2y yy ? xxx3y yy ? 3xy 2yy ))= (2x + 2y )3

R. Kimmel, August 29, 1995 and vv = (3(85x3xy y ? 244x xx2xy 2y + 203x2xxxy 3y ? 323x3xy 3y ? 52x3xx4y +442xxx2xy 4y ? 16x2xxxy 5y + 8x3xy 5y + 3xx6y ? 4xx2xy 6y ?46x2xy yy + 125x xxxy y yy ? 64x2xx2y yy + 444x2xy 2y yy ?483xxxxy 3y yy + 112x 2xx4y yy ? 242x 2xy 4y yy + 12xxxxy 5y yy ?2xx6y yy ? 6xxx2yy ? 165x xy y 2yy + 114x xx2y 2yy +203xxy 3y 2yy ? 62x xx4y 2yy + 6x3yy ? 54x2y 3yy ))= (2x + 2y )9=2: The ane curvature  is simpli ed into 2 xxyy ? 2xy 5   vv v 4 = 3  =  ? 98=3 + 35=3  ( 2 ? 2   + 2  )2=3 : xx y x xy y x yy

19

References [1] D Adalsteinsson and J A Sethian. A fast level set method for propagating interfaces. J. of Comp. Phys., 118:269{277, 1995. [2] L Alvarez, F Guichard, P L Lions, and J M Morel. Axioms and fundamental equations of image processing. Arch. Rational Mechanics, 123, 1993. [3] L Alvarez, P L Lions, and J M Morel. Image selective smoothing and edge detection by nonlinear diusion. SIAM J. Numer. Anal, 29:845{866, 1992. [4] C Ballester. An ane invariant model for image segmentation: Mathematical analysis and applications. Ph.D. thesis, Univ. Illes Balears, Palma de Mallorca, Spain, March 1995. [5] C Ballester, V Caselles, and M Gonzalez. Ane invariant segmentation by variational method. Internal report, Univ. Illes Balears, Palma de Mallorca, Spain, 1995. [6] A M Bruckstein, R J Holt, A Netravali, and T J Richardson. Invariant signatures for planar shape recognition under partial occlusion. CVGIP: Image Understanding., 58(1):49{65, 1993. [7] A M Bruckstein, N Katzir, M Lindenbaum, and M Porat. Similarity invariant recognition of partially occluded planar curves and shapes. International Journal of Computer Vision, 7:271{285, 1992. [8] A M Bruckstein and A Netravali. On dierential invariants of planar curves and recognizing partially occluded planar shapes. AT&T technical report, AT&T, 1990.

20

Invariant Framework for Dierential Ane Signatures

100

80

60

40

20

0

20

40

60

80

100

Figure 5: Ane curve evolution Ct = 1=3N~ (zero sets), and 6 steps of the implicit evolution t = (2y xx ? 2xy xy + 2xyy )1=3, starting at the upper left and ending at the lower right frame.

R. Kimmel, August 29, 1995

21

100

80

60

40

20

0

20

40

60

80

100

Figure 6: Ane invariant procedure for locating the convex hull of1=3 a shape: Evolution according to the weighted equi-ane geometric heat equation: Ct = g2 N~ , and four steps of the implicit implementation evolution, (t) at t = 0; t1; t2; 1. The edge enhancer g(x; y) = jG(
T )  I (x; y)j, is ane invariant, so that the nal result as well as every step along the evolution is invariant (under the equi-ane transformation)

22

Invariant Framework for Dierential Ane Signatures

Figure 7: Upper row: The original image, and for
T = 1 the ane absolute gradient, ane gradient magnitude, and ane Laplacian, from left to right: I (x; y), jG(1)  I j, G(1)  I , and L(1)  I . Middle row: Threshold on jG(1)  I j, and for
T = 2: ane absolute gradient, ane gradient magnitude, and ane Laplacian, from left to right: jG(1)  I j > Threshold, jG(2)  I j, G(2)  I , and L(2)  I . Lower row: The left frame: jj1=3jr(0)j, three right frames are: ane absolute gradient, ane gradient magnitude, and ane Laplacian, for
T = 4, from left to right: jG(0)  I j, jG(4)  I j, G(4)  I , and L(4)  I .

R. Kimmel, August 29, 1995

23

Figure 8: Image denoising algorithm results based on the weighted ane heat equation: From left to right: The original image I , the noisy image I~ = I + n, the image after denoising I^, and the error jI^ ? I j scaled to the gray level dynamic range. The noise was selected to be a Gaussian white noise, with SNR = 9 for the face and SNR = 15 for the tiger image. The results for the tiger are obtained after 12 iterations with dt = 0:005, and 4 smoothing iterations (performed every two iterations) for the computation of the edge enhancer with
T = 0:05. For the shape we have chosen 120 main iterations and 6 iterations for the enhancer (every two main iterations).

24

Invariant Framework for Dierential Ane Signatures

2 0.4

0.6 0.4

1 0.2

0.2 0

0

0

-0.2 -0.2 -1

-0.4 -0.4

-0.6

-2 50

100

150

200

250

20

300

k(v)

40

60

80

100

20

40

affine curvature(s)

60

80

100

G(s)

2 0.4

0.6 0.4

1 0.2

0.2 0

0

0

-0.2 -0.2 -1

-0.4 -0.4

-0.6

-2 50

100

150 k(v)

200

250

20

40

60

affine curvature(s)

80

100

20

40

60

80

100

G(s)

Figure 9: Invariant signatures under ane transformation: On the second and forth rows, from left to right are (v), (s) and G(
T )  I (x; y) samples along the zero level set (as a function of the ane arclength). On the rst and third rows, from left to right are the shapes I (x; y), (
T ), and G(
T )  I (x; y) respectively.

R. Kimmel, August 29, 1995

25

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

-1 10

20

30

40

50

60

70

20

G(s)

40

60

80

G(s)

1

1

0.5

0.5

0

0

-0.5

-0.5

-1

-1 20

40 G(s)

60

80

20

40

60

80

G(s)

Figure 10: The smoothing invariant eect on the signature G(
T )  I (x; y). The two upper raws present the signatures obtained after smoothing with the Rudin-Osher-Fatemi algorithm. The lower raws present the signature after smoothing with the ane invariant selective smoothing procedures. Observe that the invariant smoothing better preserves the relation between the signatures.

26 Invariant Framework for Dierential Ane Signatures [9] Su Buchin. Ane Dierential Geometry. Science Press, Beijing, China, 1983. [10] J Canny. A computational approach to edge detection. IEEE Trans. on PAMI, 8(6):679{ 698, 1986. [11] V Caselles, R Kimmel, and G Sapiro. Geodesic active contours. In Proceedings ICCV'95, pages 694{699, Boston, Massachusetts, June 1995. [12] D L Chopp and J A Sethian. Flow under mean curvature: Singularity formation and minimal surfaces. Center for pure applied math. pam-541, Univ. of CA. Berkeley, November 1991. [13] C L Epstein and M Gage. The curve shortening ow. In A Chorin and A Majda, editors, Wave Motion: Theory, Modeling, and Computation. Springer-Verlag, New York, 1987. [14] L C Evans and J Spruck. Motion of level sets by mean curvature, I. J. Di. Geom., 33, 1991. [15] M Gage and R S Hamilton. The heat equation shrinking convex plane curves. J. Di. Geom., 23, 1986. [16] I M Gelfand and S V Fomin. Calulus of variations. Prentice-Hall, Inc., Englewood Clis, New Jersey, 1963. [17] R Malladi, J A Sethian, and B C Vemuri. Shape modeling with front propagation: A level set approach. IEEE Trans. on PAMI, 17:158{175, 1995. [18] T Moons, E J Pauwels, L J Van Gool, and A Oosterlinck. Foundations of semidierential invariants. Int. J. of Computer Vision, 14(1):49{65, 1995. [19] P J Olver, G Sapiro, and A Tannenbaum. Invariant geometric evolutions of surfaces and volumetric smoothing. Mit report - lids, MIT, April 1994. [20] S J Osher and L I Rudin. Feature{oriented image enhancement using shock lters. SIAM J. Numer. Analy., 27(4):919{940, August 1990. [21] S J Osher and J A Sethian. Fronts propagating with curvature dependent speed: Algorithms based on Hamilton-Jacobi formulations. J. of Comp. Phys., 79:12{49, 1988. [22] P Perona and J Malik. Scale-space and edge detection using anisotropic diusion. IEEE-PAMI, 12:629{639, 1990. [23] L Rudin, S Osher, and E Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 60:259{268, 1992. [24] G Sapiro. Topics in Shape Evolution. D.Sc. thesis (in Hebrew), Technion - Israel Institute of Technology, April 1993.

R. Kimmel, August 29, 1995 27 [25] G Sapiro and A Tannenbaum. Ane invariant scale{space. International Journal of Computer Vision, 11(1):25{44, 1993. [26] G Sapiro and A Tannenbaum. On invariant curve evolution and image analysis. Indiana University Mathematics Journal, 42(3), 1993. [27] G Sapiro, A Tannenbaum, Y L You, and M Kaveh. Experimenting on geometric image enhancement. In Proceedings IEEE ICIP, volume 2, pages 472{476, Austin, Texas, November 1994. [28] M Sussman, P Smereka, and S J Osher. A level set approach for computing solutions to incompressible two-phase ow. Department of math., UCLA, Los Angeles, CA 900241555, June 1993.