Curvature Estimation for Overlapping Curved Patterns using

in: M. van Ginkel, P.W. Verbeek, and L.J. van Vliet, Curvature estimation for overlapping curved patterns using orientation space, in: B.M.ter Haar Romeny, D.H.J. Epema, J.F.M. Tonino, A.A. Wolters (eds.), Proc. ASCI’98, 4th Annual Conf. of the Advanced School for Computing and Imaging (Lommel, Belgium, June 9-11), ASCI, Delft, 1998, 173-178.

Curvature Estimation for Overlapping Curved Patterns using Orientation Space M. van Ginkel, P.W. Verbeek, L.J. van Vliet Pattern Recognition Group Faculty of Applied Physics, Delft University of Technology, Lorentzweg 1, 2628 CJ Delft, The Netherlands, {michael,piet,lucas}@ph.tn.tudelft.nl Keywords: lines, edges, orientation, curvature

Abstract

lar we will develop an estimator for the curvature of these patterns.

Filtering of an image with rotated versions of an orientation selective filter yields a set of images which can be stacked to form an orientation space. Orientation space provides a means of analyzing overlapping and touching patterns, characterized by their orientation. In this paper we extend previous work and show that curved patterns may also be analyzed using orientation space. Orientation space allows us to decompose an image into a background and oriented components. The curvature of each of the individual components can be estimated by measuring the tilt of the locally planar response in orientation space.

1

φ

x

Figure 1: An oriented pattern We define directions as angles in the interval (0, 2π), thus making a distinction between an arrow pointing to the left or right. When we refer to orientation (0, π) we make no such distinction. For many applications a reliable estimator for a single locally dominant orientation suffices, Kass & Witkin [5], Haglund [3] and Van Vliet & Verbeek [10]. For the following two examples an estimate of the local orientation will not be sufficient.

Introduction

• The characterization of textures that consist of a number of overlapping patterns with different orientations.

Recently the decomposition of images with respect to orientation has received much attention. This type of decomposition can be used in a wide variety of applications, such as segmentation of overlapping objects [11], the detection of boundaries between regions with different orientations [1, 2], line preserving noise removal [7], and line enhancement [4]. We assume that images can locally be modeled as a weighted sum of translation invariant patterns or (paintbrush) strokes and a background. Each stroke has a one-dimensional intensity profile and a typical orientation: the profile orientation across the stroke, see figure 1. In this paper we will consider overlapping curved patterns. The orientation decomposition approach is used to analyse each pattern individually. In particu-

• Accurate boundary detection between two regions with a different, but homogeneous orientation. In both cases a more advanced analysis is required. In order to deal with anisotropy an image can be replaced by a stack of directionally filtered versions of the original. The resulting orientation space is periodic along the orientation axis with a period of π. The orientation space approach can be used for a number of applications. As shown in Van Vliet and Verbeek [11], orientation space can be used to perform a segmentation of overlapping 173

objects. In [1, 2] we used orientation space to find the boundaries between two orientation fields.

the fact that the gradient vanishes on ridges and in valleys [8]. The same problem arises when estimating local orientation using the orientation of the gradient vector. In the latter case, these problems can be circumvened by creating a tensor representation of the gradient [3, 5, 10], which is computed by taking the dyadic product of the gradient vector with itself. The tensor elements can then be averaged over a region. The eigenvector corresponding to the largest eigenvalue of the smoothed tensor gives the dominant local orientation. This scheme can be modified to deal with curved patterns. The modified version yields a reliable estimator for curvature [9]. The method proposed in this article should also yield reliable estimates for single as well as multiple overlapping curved patterns.

(x,y)

φ1

φ2

Figure 2: Two overlapping concentric patterns. It is also possible to analyze curved patterns using orientation space. For patterns as depicted in figure 1 a one dimensional analysis along the orientation axis suffices. To analyze curved patterns, one should consider a cylindrical region in orientation space, its axis coinciding with the orientation axis through the point (x, y). In conjunction with its ability to disentangle different oriented patterns, orientation space thus permits us to estimate the curvature of each of the patterns into which the image can be decomposed. The proposed scheme produces an estimate of the curvature κ(x, y, φ) for each (x, y, φ) position. In the situation depicted in figure 2 there are two overlapping curved patterns, with local orientations φ1 and φ2 . Both κ(x, y, φ1 ) and κ(x, y, φ2 ) will yield meaningful values, while for other instances of φ the curvature κ(x, y, φ) will be meaningless.

2

3

Orientation space is created by applying rotated versions of an orientation selective filter to the image. In [1, 2] we developed a filter bank which optimizes the orientation selectivity given the number of filters to be used in constructing the orientation space. The selectivity increases linearly with the number of filters used. In this paper we use a different filter bank for reasons discussed below. The Fourier transform over the spatial coordinates of the orientation space can be written as: e x , ωy , φ) = Fe(ωx , ωy , φ)I(ω e x , ωy ) O(ω

Curvature

κ=

dθ ds

(1)

Fe(ω, θ, φ) = Fer (ω)Fa (θ − φ)

The curvature of an isophote in a grey-value image I(x, y) is given by the following formula, see [8]: Ix2 Iyy − 2Ix Iy Ixy + Iy2 Ixx Icc =− 3 Ig (Ix2 + Iy2 ) 2

(3)

e is the Fourier transform of the orientawhere O tion space, Fe the Fourier transform of the filter and Ie the Fourier transform of the image. If the Fourier transform is written in cylindrical coordinates ω, θ, the following relation holds:

The curvature κ at any point along a twodimensional curve is defined as the rate of change in tangent direction θ of the contour, as a function of arc length s.

κ=−

Orientation space

(4)

The radial part Fr can be used to select the appropriate scale to analyze the image at. It also determines what is considered background, and which frequencies are considered to represent the patterns. The angular part Fa determines the orientation selectivity of the filter. Equation 4 represents a simplified version of a full orientation analysis. The filterbank used in a full analysis consists of a sum of such filters. Each of the filters as given in equation 4 is used to analyze a given frequency band (or scale). A necessary, but not sufficient, condition for the inverse transform as described in [7, 4] to exist, is for the entire

(2)

Where Ig is the derivative in the gradient direction, i.e. the gradient magnitude, and Icc is the second derivative in the direction perpendicular to the gradient. Using equation 2 to estimate the curvature is inappropriate for images consisting of the type of patterns shown in figure 2, due to 174

frequency plane to be covered by the filter bank. This is not necessary for the purpose of the current paper. The response R of the filter to a pattern with orientation φ0 is given by: R(x, y, φ) = R(x, y, φ0 )Fa (φ − φ0 )

While this choice for the angular response does not strictly satisfy the Nyquist criterium, it does so for all practical purposes. For the given standard deviation, the main lobe of SaN (φ) and the Gaussian peak GN a (φ) virtually coincide. The oriention selectivity is therefore equivalent to that of the SaN (φ) function. The angular response GN a (φ) does not have side lobes and it is possible to create a quadrature filter pair based on this function. The radial response used in this work is:

(5)

The response of the filter to an image that can locally be modelled as a sum of strokes, each with its own grey profile fi : I(x, y) =

X

fi (x cos φi + y sin φi )

Fr (ω, kr , σr ) =

(6)

i

is O(x, y, φ) =

X

f φi Fa (φ − φi )

(7)

Where f φi is the response of the filter to pattern i when the filter is aligned with the orientation φi of the pattern. It is evident from formula 7 that in orientation space each oriented pattern gives a response along the φ axis centered around its orientation, while the shape of the response is given by Fa (φ).

4

e will be sampled along the φ axis. In practice O In [1, 2] we designed a filter that optimizes the orientation selectivity under the constraint of satisfying the Nyquist criterium. The angular part SaN (φ) of the filter when the number of samples is N is sin(N φ) sin φ

(8)

This filter has two drawbacks. Its response consists of one main lobe and a lot of side lobes. This makes analyzing the resulting orientation space more difficult. The second problem is that it is not possible to construct a pair of quadrature filters based on this filter. The modulus of a pair of quadrature filters is insensitive to the phase of the pattern under analysis [6]. The only alternative to get rid of the phase is to use a large post smoothing. To prevent this problems a Gaussian shaped angular filter with standard deviation π/N can be used to replace SaN (φ): GN a (φ) = exp(−

(N φ)2 ) 2π 2

exp(−

ω2 kr + ) 2 2σr 2 (10)

Estimating curvature in orientation space

The response in orientation space to a noncurved pattern is a plane in the (x, y, φ0 ) plane, see figure 3b, where φ0 is the orientation of the pattern, see (a). If a second pattern with orientation φ1 is present, this will result in a second response as is indicated in (b). Note that if the angular difference between the two patterns is large enough, the two responses will be completely separated. For a curved pattern the response plane will be tilted in the plane perpendicular to the A axis. The response along the φ axis will only be centered around φ0 along the axis A depicted in (d). This axis corresponds to the orientation axis of the pattern that is depicted in (c). The response of a second curved pattern is also indicated in (d). The responses are distinct if the angular difference between the patterns is sufficiently large. When walking along a path perpendicular to the local orientation (i.e. perpendicular to the A axis), the local orientation will change. Considering a local patch, the path length along A and the path along an isophote are the same. The curvature is defined as the change in orientation as a function of the path length. In the situation depicted in figure 3 this corresponds exactly to the tilt α of the plane. Our original problem of estimating the curvature is transformed into estimating the tilt of the response plane in orientation space. The applicability of the scheme is demonstrated by figure 4. In (b) and (c) we show (x, φ) slices of the orientation space computed for the image depicted in (a). Only in the middle of the

Sampling orientation space

SaN (φ) =

!kr

Fr is a bandpass filter with a nearly Gaussian shapepwith standard deviation σr , centered at ω = kr σr2 . Its maximum transfer value is 1, and it is zero for ω = 0.

i

3.1

|ω| p kr σr2

(9)

175

A

A

Β

Β φ1

φ1 φ 0 (a)

(c) A

A φ0

φ0

Β

φ0

α

Β

φ1

φ1 (b)

(d)

Figure 3: (a) Two overlapping noncurved patterns. (b) The response in orientation space to the image depicted in (a). (c) Two overlapping curved patterns. (d) The response in orientation space to the image depicted in (b). Note the corresponding axes A in (c) and (d). depicted slices does the x direction correspond the direction orthogonal to the local orientation. It is in the middle that the slopes in (b) and (c) should be compared. It is clear that slice (b), which corresponds to the region with a lower curvature, has a smaller slope than (c). The situation as shown in figure 3 is somewhat simplified. For most patterns the curvature will vary along the A axis, as is the case for the image in figure 4a, because of which the tilted plane will have a slight torsion along A. Also note that the plane is depicted as extending throughout the orientation space, while in practice the response is only locally planar.

4.1

tors corresponding to the same orientation but with opposite signs will cancel out. The tensor method solves this problem by averaging the following tensor rather than directly averaging the gradient vectors. Ox2  Ox Oy Ox Oz 

Ox Oy Oy2 Oy Oz

 Ox Oz Oy Oz  Oz2

(11)

For planar structures the local orientation is given by the orientation of the eigenvector corresponding to the largest eigenvalue of the smoothed tensor. While in theory the A axis should lie in the plane, in practice the plane may have a tilt component along the A axis. To estimate the curvature only the tilt component in the direction perpendicular to the local orientation should be measured. There are two ways to do this: 1. Removing the component along the A axis of the gradient before constructing the tensor. Since all the resulting vectors will lie in a two dimensional subspace perpendicular to the A axis, the ensuing tensor analysis can also be restricted to two dimensions. 2. Perform the tensor scheme in the full three dimensional space. The full tensor as shown in

Estimating the tilt

The initial problem of estimating curvature in a two dimensional image is equivalent to estimating the tilt (orientation) of a plane in the three dimensional orientation space. We propose to use a three dimensional version of the tensor method described in [5, 3, 10] to estimate the orientation of the plane. The gradient vector is a very simple estimator of the local direction. Because it estimates local direction rather than local orientation, one cannot simply average the gradient vectors to create a more robust estimator, because gradient vec176

spite this variation along the A axis, the eigenvector that results from the eigenvalue analysis still yields the desired tilt, because the components of the gradient vectors due to the deviating tilt cancel. Although the estimator is unbiased, the eigenvalue analysis will show a relatively large eigenvector in the direction of the normal to the plane. If we are estimating the curvature κ0 at some spot along the A axis, the curvature at a distance v along the A axis is given by:

N= 16 σr = 0.4 kr = 9 (a)

φ

x

(b)

φ

x

1 1 = +v κ(v) κ0

(12)

Since the behaviour of the curvature along the A axis is known, we can compensate for the change in tilt by adjusting the gradient vectors before using the dyadic product to construct the tensor. When this correction is applied, the eigenvector in the direction of the normal of the plane should have a (nearly) zero magnitude.

(c)

Figure 4: (a) A radial pattern. (b) A (x, φ) slice along the lower of the two lines shown in (a). (c) Another (x, φ) slice, this time along the upper line.

Acknowledgements This work was partially supported by the Rolling Grants program 94RG12 of the Netherlands Organization for Fundamental Research of Matter (FOM) and the Royal Netherlands Academy of Arts and Sciences (KNAW). Figure 5: A curved pattern with a non constant orientation

References [1] M. van Ginkel, P.W. Verbeek and L.J. van Vliet, Improved Orientation Selectivity for Orientation Estimation, Proceedings of the 10th Scandinavian Conference on Image Analysis, volume I, Lappeenranta, Finland, June 9-11, 1997, pp. 533-537.

equation 11 is averaged. Finally, the component along the A axis of the computed eigenvector is removed, yielding the tilt (orthogonal to the A axis) of the plane. The advantage of the first method is that the eigenvector analysis has one degree of freedom less than the second, thus preventing overadjustment to the data. The second method has the advantage that the component of the eigenvector along the A axis yields information about the curvature of the path of largest change (steepest descent or ascent) across the pattern. This path indicates the change of the orientation of the pattern as a whole. While our model assumes that this path is straight, in reality the orientation of this path may change, see figure 5. It is this change that gives rise to the tilt in the direction of the local orientation.

4.2

[2] M. van Ginkel, P.W. Verbeek and L.J. van Vliet, Multi-Orientation Estimation: Selectivity and Localization, in: H.E. Bal, H. Corporaal, P.P. Jonker, J.F.M. Tonino (eds.), Proceedings of the third annual conference of the Advanced School for Computing and Imaging, Heijen, The Netherlands, June 2-4, 1997, pp. 99-105. [3] L. Haglund, Adaptive Multidimensional Filtering, PhD thesis, Link¨oping University, Sweden, 1992. [4] S.N. Kalitzin, B.M. ter Haar Romeny and Max A. Viergever, Invertible Orientation Bundles on 2D Scalar Images, Proceedings of the First International Conference on ScaleSpace Theory in Computer Vision, Lecture

A correction for radial patterns

For radial patterns such as depicted in figure 4 the curvature changes along the A axis. The tilt along the A axis changes correspondingly. De177

Notes in Computer Science, Springer-Verlag, July 1997, pp. 77-88. [5] M. Kass and A. Witkin, Analyzing Oriented Patterns, Computer Vision, Graphics and Image Processing, vol. 37, 1987, pp. 362-385. [6] H. Knutsson, Filtering and Reconstruction in Image Processing, PhD thesis, Link¨oping University, Sweden, 1982. [7] J.B. Martens, Local Orientation Analysis in Images by Means of the Hermite Transform, IEEE transactions on Image Processing, vol. 6, no. 8, August 1997, pp. 1103-1116. [8] P.W. Verbeek, A class of sampling-error free measures in oversampled band-limited images, Pattern Recognition Letters, vol. 3, 1985, pp. 287-292 [9] P.W. Verbeek, L.J. van Vliet and J. van de Weijer, Improved Curvature and Anisotropy Estimation for Curved Line Bundles, Proceedings of the ICPR’98, Brisbane, Australia, 17-20 August, 1998, submitted. [10] L.J. van Vliet and P.W. Verbeek, Estimators for Orientation and Anisotropy in Digitized Images, in: J. van Katwijk, J.J. Gerbrands, M.R. van Steen, J.F.M. Tonino (eds.), ASCI’95, Proc. First Annual Conference of the Advanced School for Computing and Imaging (Heijen, NL, May 16-18), ASCI, Delft, 1995, pp. 442-450. [11] L.J. van Vliet and P.W. Verbeek, Segmentation of overlapping objects, in: L.J. van Vliet, I.T. Young (eds.), Abstracts of the ASCI Imaging Workshop 1995, Venray, The Netherlands, October 1995, pp. 5-6.

178