Edge Detection Revisited

0 downloads 0 Views 2MB Size Report
lems of edge detection: the simultaneous detection of all step edges from a fine to a coarse ...... in Table I deserve some comments. Images recovered from ... recovery is primarily due to the very low density of constraints,. i.e., around 5% of the ...
1500

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Edge Detection Revisited Felice Andrea Pellegrino, Walter Vanzella, and Vincent Torre

Abstract—The present manuscript aims at solving four problems of edge detection: the simultaneous detection of all step edges from a fine to a coarse scale; the detection of thin bars with a width of very few pixels; the detection of trihedral junctions; the development of an algorithm with image-independent parameters. The proposed solution of these problems combines an extensive spatial filtering with classical methods of computer vision and newly developed algorithms. Step edges are computed by extracting local maxima from the energy summed over a large bank of directional odd filters with a different scale. Thin roof edges are computed by considering maxima of the energy summed over narrow odd and even filters along the direction providing maximal response. Junctions are precisely detected and recovered using the output of directional filters. The proposed algorithm has a threshold for the minimum contrast of detected edges: for the large number of tested images this threshold was fixed equal to three times the standard deviation of the noise present in usual acquisition system (estimated to be between 1 and 1.3 gray levels out of 256), therefore, the proposed scheme is in fact parameter free. This scheme for edge detection performs better than the classical Canny edge detector in two quantitative comparisons: the recovery of the original image from the edge map and the structure from motion task. As the Canny detector in previous comparisons was shown to be the best or among the best detectors, the proposed scheme represents a significant improvement over previous approaches. Index Terms—Edge detection, junction detection, performance evaluation.

I. INTRODUCTION

S

TEP Edges, lines and junctions usually convey the most relevant information of an image [4], [41], [46], hence it is important to detect them in a reliable way. Edge detection has been extensively analyzed in computer vision during the last 20 years [5], [7], [10], [18]–[20], [41] but it has not yet been entirely solved. In fact, despite considerable efforts, the ideal scheme able to detect and localize with precision edges at all scales, irrespective of their shape and configuration, has not yet been produced. The development of the optimal or ideal edge detection scheme is also difficult for the absence of evident and clear ground truth data, on which to evaluate performances unambiguously [28]. Several criteria have been proposed for the evaluation of edge detection schemes: for instance, task dependent criteria have been suggested based on the recovery of three-dimensional (3-D) data on a structure from motion task [36], or the recovery of the original image from the edge map [28]. Human evaluation criteria also have been proposed and made quantitative by using statistical and psychophysical tools [14], [27].

Manuscript received April 8, 2003; revised August 19, 2003. This paper was recommended by Associate Editor D. Goldgof. F. A. Pellegrino is with the Department of Mathematics and Computer Science (DIMI), University of Udine, Udine 33100, Italy. W. Vanzella and V. Torre are with the Department of Neurobiology, International School for Advanced Studies, Trieste 34014, Italy (e-mail: [email protected]). Digital Object Identifier 10.1109/TSMCB.2004.824147

In all these comparisons the classical edge detection scheme of Canny [5] provides very often—but not always—the best performances. This scheme, in substance, identifies an edge in the as a local maximum of an appropriate original image function of the convolution of with odd directional filters. The success of the edge detection scheme [5], proposed almost 20 years ago, suggests that its essence is correct and close to being optimal. It is well-known that the human visual system is able to perform an ideal edge detection, independently on recognition and reasoning. Therefore, the observation of the human visual system may suggest how to modify the classical edge detection scheme of Canny, in order to obtain an almost ideal edge detection scheme. In the visual system of higher vertebrates and mammals, the first stage of processing occurring in the visual area V1, consists in the convolution of the retinal image with filters having different orientation, size and shape [16]. Computer vision, also, uses a large set of filters for image processing. In fact, the detection of edges, corners, and other important two-dimensional (2-D) features has been obtained by convolving the original image with directional filters having different orientation, size, and shape [11], [12], [39]. When 2-D features are detected by computing maxima of the local energy [18], [24], [31] the image is first convolved with odd and even filters of different orientations. The need for analyzing images at different scale, i.e., multiscale analysis [19], [45], is now widely recognized in computer vision and is a standard technique in image processing [13], [22]. Both biological and artificial vision agree on the need of analyzing images with a large battery of filters with different properties. The next question, obviously, is how to combine the output of these filters. This is an important issue both in a biological perspective and in computer vision: how is the output of these filters optimally integrated for detecting edges at all scales, identifying features of increasing complexity and recognizing objects? The answer to this question may help to understand the logic underlying processing in the visual cortex and to develop highly performing algorithms for computer vision. The major aim of this paper is to use the rather large set of filters, present and implemented in the visual area V1, to improve the edge detection scheme originally proposed in [5] in such a way to obtain an almost ideal edge detection scheme, able to detect: 1) step edges [24] from a fine to a coarse scale; 2) thin bars with a small width of two pixels or so; and 3) trihedral junctions with one side having a low contrast. The present paper aims also at providing a scheme without any parameter which has to be tuned for the specific image where edges have to be detected: in fact, the proposed scheme for edge detection has only one free parameter, i.e., the threshold for the minimum contrast of detected edges , set equal to three times the standard deviation of the noise present in usual acquisition system. This threshold , estimated to correspond to four gray levels

1083-4419/04$20.00 © 2004 IEEE

PELLEGRINO et al.: EDGE DETECTION REVISITED

1501

out of 256, was used to extract edges from a large number of test images, consistently providing very good results. The proposed scheme for edge detection combines several previously suggested approaches [5], [24], [9], [17] with new algorithms. The information obtained by extensive filtering here described is also used [42] for a final regularization of the original image using a modified version of the Mumford and Shah functional [26]. This regularization step uses a locally self-adapting scale and contrast in order to preserve small details. From this regularized image it is possible to obtain an image segmentation, providing a convenient decomposition of the image in a limited number of regions with the same gray level. This segmentation is, in our opinion, an almost ideal preprocessing for higher level functions such as categorization and recognition. The paper is organized in the following way. Section II reviews the previous work and outlines the proposed approach to edge detection. Section III summarizes the properties of the most widely used filters in computer vision. Section IV and V discuss the detection of step and roof edges, respectively, i.e., the detection of edges separating two extended regions of the image (step edges) and the detection of thin bars or roof edges. Section VI treats the detection and recovery of trihedral junctions. Section VII presents experimental results on a large set of images of different kind and a comparison with previous approaches to edge detection. Section VIII provides an overview of the algorithm, adding further information concerning noise robustness, parameter sensitivity, and computation time. II. EARLY APPROACHES Early approaches to edge detection [5], [41] primarily focused their analysis on the identification of the best differential operator necessary to localize sharp changes of the intensity at location . These approaches recognized the necessity of a previous filtering, identified as a regularizing stage, but did not explicitly realize the convenience of a full multiscale analysis, useful for a reliable detection of both sharp and smooth edges [8]. Following some biological suggestions, Morrone and Burr [24] proposed to detect edges as maxima of the local energy obtained as

[24], [31] when features to be detected are significantly larger than the size of used filters. This problem will inevitably occur with digital images when bars having a width below 2 or 3 pixels have to be detected. The simplest way to overcome it is to oversample the image, but significantly increasing the computational burden of the analysis. A rather different method of edge detection is obtained by considering variational approaches [26], [23]. In these approaches, the set of edges and a suitable approximation of a given image are obtained by minimizing, over the image domain , the functional

(3) where

is a piece-wise smooth approximation of is the total length of the edges, is the gradient operator and and are appropriate constants. Edges obtained by minimizing the functional (3) and by computing local maxima of (2) often coincide. Step edges at a given scale are usually well detected by minimizing the functional (3), but thin roof edges are not detected and are smeared out. Inthispaperweadopttheviewofdetectingandlocalizingedges as local maxima of the energy (2), investigating, in addition, how to combine the output of different V1-like filters with different scales, directions and parity for an “almost ideal” detection of step edges, of thin roof edges and of trihedral junctions at fine and coarse scales. It will be shown that step edges are best detected by considering the energy (2) composed only by the output of odd filters, but summed over all directions and scales. The observation that better performances are obtained considering only odd filters, is in complete agreement with the early proposal of Canny [5]. On the contrary, thin roofedges are bestlocalizedby summing the energy from narrow even and odd filters only in the direction of maximal response. Trihedral junctions are localized by computing the local curvature, which can be obtained by combining even and odd filters appropriately, and their sides are precisely identified by the output of directional filters. III. FILTERING

(1) where and are the convolution of the original image with a quadrature pair of one-dimensional filters in the direction. A quadrature pair of one-dimensional filters consists of one even and one odd-symmetric filters with zero mean, same norm and orthogonal to each other. These two filters are one the Hilbert transform of the other. Subsequent approaches [18] suggested the local energy to be (2)

In this section properties of some spatial filters often used in computer vision are briefly recalled: Gabor, log-Gabor, and gaussian filters will be considered. These filters can be coupled in order to form a quadrature pair [24]. A quadrature pair of one-dimensional (1-D) filters consists of one even and one oddsymmetric filters with zero mean, same norm and orthogonal to each other. These two filters are one the Hilbert transform of the other. A 1-D Gabor filter is a gaussian function modulated by a sinusoidal wave. (4)

i.e., the energy summed over different directions and scales . In these approaches, edges are identified as local maxima of the energy (possibly normalized) of (2). In one dimension, maxima of local energy are also points of maximal phase congruency [43], i.e., points of particular interest in the image. Local maxima of the energy (2) correctly identify edges [18],

is the where is a parameter setting the spatial spread and frequency of the modulating sinusoidal wave. Gabor filters have Fourier transform (5)

1502

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 1. Detection of step edges with odd and even filters. (a) One-dimensional intensity profile with bars of different width having the right edge slightly smoothed. (b) Energy profile using odd and even 1-D gaussian filters and in (c) using only odd 1-D gaussian filters. (d)–(f) as in (a)–(c) for the second bar of panel (a). In (e) and (f), dot and dash-dot lines refer to Gabor and log-Gabor filters. Filters had nine different sizes with ! ranging from 3–18 pixel/cycle (see Section III).

has its maximum in defined by its Fourier transform.

. A log-Gabor filter is

(6) has its maximum in and is a conAlso, stant determining the frequency bandwidth. The odd and even log-Gabor filters in the spatial domain are defined as the back transform of a quadrature pair of filters, see for instance [32] F F

(7)

The pair of gaussian filters is obtained by back-transforming a quadrature pair of filters.

(8) . The size These filters have their maximum for of a given filter is taken as the period corresponding to or . The size of a gaussian filter is simply . Even and odd gaussian filters do not have ripples, but have a broader spectrum. The Fourier transform of log-Gabor filters is symmetric when plotted on a semilog scale. Gabor filters have Fourier transform maximally concentrated around and in the space domain they have significant ripples and oscillations introduced by the sinusoidal modulation. When edge detection is performed considering the energy summed over the output of odd and even filters, as in (1), Gabor, log-Gabor, and gaussian filters provide rather similar results

(see Fig. 1). When only odd filters are used, as it will be done in the next section, gaussian filters provide best results, due to the absence of ripples. One-dimensional filters can be extended to two dimensions in several ways. Two-dimensional filters with a Fourier transform, which in polar coordinates can be factorized in the product of two functions one depending on radius and the other on the angle , are called separable. These filters can be designed and implemented using the ring-wedge technique (see for instance [18]). Two-dimensional Gabor and log-Gabor filters were implemented using this technique. Two-dimensional gaussian filters were constructed in the space domain multiplying the 1-D profile by a gaussian function in an orthogonal direction. The ratio between the variance of the two gaussian functions was 1.2. Two-dimensional gaussian Filters constructed in this way do not have ripples, which are present in 2-D Gabor and log-Gabor filters. It is useful to recall that the output of a directional odd gaussian filter is a good approximation for the computation of a derivative in the same direction. Similarly, the output of an even, zero-mean gaussian filter, is a good approximation of a second derivative. These remarks will be useful in a later section, when the local curvature of the image intensity has to be computed. IV. DETECTION OF STEP EDGES In this section we address the issue of the detection of step edges at all scales, i.e., sharp and blurred edges. The detection of edges, composing the contour of large features (spanning at least five and more pixels), can be efficiently solved by detecting maxima of (1) or (2) as shown previously [18]. The detection of boundaries of features with a dimension of 2–4 pixels poses some problems.

PELLEGRINO et al.: EDGE DETECTION REVISITED

1503

Fig. 2. Detection of step edges with odd and even filters in 2-D images. (a), (d), (g): details from interior images. (b), (e), (h) Step edges detected as local maxima using odd and even filters. (c), (f), (j) Step edges detected using only odd filters. Eight orientations and nine scales, varying from three pixel/cycle to 18 pixel/cycle were used. The ratio of  to  was 1.2 for odd filters and 2 for even filters.

A. Combination of Odd and Even Filters Let us consider an ideal 1-D intensity profile with a 128 gray and a value of 1 elsewhere. values between pixel and and in The energy (2) has three local maxima in , with the first two maxima corresponding to step edges and the maximum at corresponding to a roof edge. When becomes close to three or four, the three local maxima coalesce and cannot be reliably identified. Therefore, the detection of bars with a width between two and four pixels must be carefully investigated. Let us consider the 1-D intensity profile shown in Fig. 1(a), composed by thin bars of increasing width, with one sharp and one smooth boundary. When the energy (1) is computed by summing the output of odd and even gaussian filters the profile shown in Fig. 1(b) is obtained, where only one peak is present for bars with a width of less than four pixels. Two peaks become visible when the bar width is larger than five pixels. On the contrary, if only odd filters are used, two peaks corresponding to the two boundaries of the bar are obtained [as shown in Fig. 1(c)] even when the bar width is just two pixels. Performances of the different filters are compared in Fig. 1(e) and (f) for the intensity profile shown in Fig. 1(d), corresponding to the second bar in Fig. 1(a). Fig. 1(e) shows the energy profile obtained from (1) using even and odd Gabor (dotted line), log-

Gabor (dash dotted line) and gaussian filters (continuous line). The two edges delimiting the bar are not well detected and one large maximum is present, irrespectively on how small and large filters are combined (see later section for further details). On the contrary, if only odd filters are used [see Fig. 1(f)], maxima are clearlydetected atthebarboundaries.WhenGaborandlog-Gabor are used, additional maxima are seen [indicated by the arrows in Fig. 1(f)]. These spurious additional maxima are produced by the ripples of these filters, which become evident when the filters are not used in quadrature pair. For this reason gaussian filters are preferred to Gabor and log-Gabor filters. When 2-D images are considered, edges are detected as local maxima of the energy (2) summed over eight equally spaced directions and nine different scales (see next section). Local maxima were detected by a modification of the well-known [46] nonmaximal suppression algorithm (performed, in each point, along the direction perpendicular to the filter providing the highest response). Such modification consists of an energy-driven thinning which removes possible double edges, i.e., boundaries with a thickness larger than one pixel, which may occur using the standard nonmaximal suppression. Fig. 2(a), (d), and (g) illustrates three details of interior images, with thin bars of a width of two to four pixels. Edges detected as local maxima of (2) using odd and even gaussian filters

1504

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 3. Detection of step edges with filters at different scales. (a) and (b) Fourier transform of filters at different scales normalized to the same energy but with . (c) Fourier transform of filters normalized to the same maximal response. In (a)–(c) the dot line is the sum of all filters energy. different distribution of ! (d) One-dimensional intensity profile with representative roof and step edges with different strength and smoothness. (e) The corresponding total energy for the intensity profile of panel (d) for the three cases shown in (a) (round dot line), (b) (continuous thin line) and in (c) (continuous thick line).

are shown in Fig. 2(b), (e), and (h). These edges are not as complete as when only odd filters are considered, as clearly shown in panels (b), (c), (e), (f), (h), and (j). When gaussian odd filters are used, step edges are properly localized at the feature boundaries also for the thin bars, such as those shown in Fig. 2(a) and (d). It is not surprising that the detection of step edges is best obtained by using only odd filters, as they are the natural matched filters for these features [5]. As a consequence only odd gaussian filters will be used to detect step edges. B. Combination of Different Scales In order to detect step edges at fine and coarse scales it is necessary to use small and larger filters, with different frequency properties, but it is not obvious how to weight their relative contribution. In what follows, gaussian filters with nine scales (see Section III) ranging from three were considered with pixel/cycle to 18 pixel/cycle. The choice of these nine scales was suggested by a large experimentation with natural images, as those analyzed in the present paper. The scales of odd filters in (2) can be combined by using filters with the same energy (see Fig. 3(a) and (b), where different distributions of center frequencies are shown) or with the same maximal response [Fig. 3(c)]. These filters sample the frequency content of the image in a slightly different way, as shown by the summed total energy drawn the corresponding panels (dotted lines). Fig. 3(d) illustrates an ideal 1-D intensity profile containing a variety of step and roof edges: a smooth step edge, a strong roof edge flanking two smoothed roof edges, a strong bar followed by a sharp step edge. The corresponding total energy for the three sets of filters as in a, b, and c is shown in panel e in round-dot, square-dot

and continuous line, respectively. With filters having the same maximal response [as those shown in Fig. 3(c)] the total energy has ten local maxima [continuous thin line in Fig. 3(e)] corresponding to the ten features to be detected. When the filters are normalized in order to have the same energy [as in Fig. 3(a) and (b) several local maxima are lost, as shown by the dot line and the continuous thin line in Fig. 3(e)]. When edges with different contrast and opposite sign are present in the same region of the image and the output of fine and coarse filters superimpose spatially, their simultaneous detection may be difficult. Extensive experimentation has shown that some details [as those shown in Fig. 4(a), (d), and (g)] are best detected by combining scales with same maximal response [see Fig. 4(c), (f), and (j)], instead of when filters having the same energy are used [see Fig. 4(b), (e), and (g)]. V. DETECTION OF THIN ROOF EDGES In this section the detection of thin bars will be investigated. It is well known [18], [24] that the detection of thin details in real images, as the thin bars shown in Fig. 5(d) is best obtained by using both even and odd filters [see Fig. 5(f)], when the output of even filters dominates at the center of the bar and the output of odd filters reduces and eliminates spurious maxima near the bar boundaries. If only even filters are used, multiple maxima are detected at the center of the bar and at its boundaries. In this section, it will be shown that the detection of thin bars, however, is best obtained considering only the energy along the direction of maximal response and not the energy summed over all directions, like it was done instead for the detection of step edges (see Section IV).

PELLEGRINO et al.: EDGE DETECTION REVISITED

1505

Fig. 4. Detection of step edge with filters at different scales in 2-D images. (a), (d), and (g) are details of an interior scene; (b), (e), and (h) show the detected step edges summing the energy over filters having the same energy and in (c), (f), and (j) with same maximal response. Eight orientations and nine scales, varying from 3 pixel/cycle to 18 pixel/cycle were used. The ratio of  to  was 1.2 for odd filters.

A. Different Directions Are not Summed

B. Detection at Different Scales

Therefore, combining even and odd filters seems the best solution for the detection of thin bars, but it is not obvious whether it is convenient to sum the energy over a battery of even and odd filters with different orientations. The profile of the summed energy of the output of even and odd filters perpendicularly oriented as shown in Fig. 5(a) to a thin bar [with the intensity profile indicated by the solid line in Fig. 5(c)] has one large central peak, correctly located at the center of the thin bar [see line in Fig. 5(c)]. On the contrary, when even and odd filters are not perpendicular to the thin bar [see Fig. 5(b)] the profile of the summed energy has two smaller peaks [see line Fig. 5(c)]. As a consequence, summing over all directions is not advantageous for the detection of thin bars with a low contrast as the one shown in Fig. 5(d) and of multiple thin bars in different directions as those shown in Fig. 5(g). The thin bar of Fig. 5(d) has an average contrast of only five gray levels out of 256 and it is better detected and localized when considering the energy only along the direction providing maximal response [see Fig. 5(f)]. Similarly, the intricate pattern of thin bars of Fig. 5(g) is reasonably well detected when the energy considered is only in the maximal direction. When the energy is summed over all directions [see Fig. 5(e) and (h)] thin bars are not well detected.

Roof edges with a different width were detected independently. Very thin roof edges, with a width of one or two pixels were detected by using gaussian filters with a width of four pixels cycle and are shown in green in Fig. 6(b). Medium size roof edges, with a width of three or four pixels were detected by using gaussian filters with a width of eight pixels/cycle and are shown in blue in Fig. 6(b). Larger roof edges with a width of five pixels or so were detected with a larger filter with a width of 18 pixels/cycle and are shown in Fig. 6(b) in red. Directional filters were longer than those used for detecting step edges and to was two for both even and odd filters. The the ratio of energy was computed at the three scales according to (1). Local maxima were extracted and a threshold was applied as described in the previous section. C. Classification of Roof Edges Once maxima of the energy have been located and identified, it is still necessary to classify them in roof and step edges. The usual way to classify edges [24] is to compare the absolute value of the response of the odd and even filters: a maximum of the energy is a step edge when the contribution of the odd filters to the energy is greater than the one given by the even filters. This

1506

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 5. Detection of thin roof edges. (a) and (b): even and odd filters perpendicular (a) and oblique (b) to a thin bar. (c) One-dimensional energy profile (thick line) and the energy profile only along the direction providing maximal response (line) and energy profile summed over all directions. (d) A detail of the same image as in Fig. 6(a). (e) Thin roof edges detected as local maxima of the energy summed over all directions. (f) Same as in (e) but with the energy only along the direction providing maximal response. (g), (h), (j) as in (d), (e), (f) for a particular of an outdoor scene.

Fig. 6. Detection of roof edges of different width. (a) Original image. (b) The map of roof edges extracted as described in the text. Roof edges have an increasing width from green, blue, and red.

classification procedure is adequate for classifying roof edges with a width larger than one pixel. The detection and classification of roof edges with a width of one pixel is very sensitive to noise and therefore unstable.

Once a pixel has been classified as a roof edge, a scale must be assigned to it. If the edge occurs only at a given scale, that scale is assigned to it. Otherwise, the smallest scale is assigned. Fig. 6(b) illustrates roof edges extracted from the original image shown in

PELLEGRINO et al.: EDGE DETECTION REVISITED

1507

Fig. 7. Recovery of trihedral junctions. (a) Detail of a cube. (b) Detected step edges. (c) Normalized curvature. (d) Regions of interest of high curvature. (e) End points of the edge map within the regions of interest. (f) Restored sides of the trihedral junctions.

Fig. 6(a). The great majority of roof edges are detected at the correct scale. However, the resulting edge map is more noisy than that obtained forstep edges because edges are obtained from an energy map which is not summed over different scales and directions and therefore, they are more sensitive to noise.

times . In order to achieve invariance from the local contrast, should be equal to zero. However, with this choice there are many false detections. Extensive experimentation has suggested to use the quantity (10)

VI. DETECTION OF TRIHEDRAL JUNCTIONS It is well known that edge detection based on the extraction of local maxima fails in the presence of trihedral junctions, such as those shown in Fig. 7(a) and edges are usually interrupted at junctions [see Fig. 7(b)]. By combining the output of V1-like filters in an appropriate way, it is possible to restore these important image features, which are inevitably lost using (2). Regions where trihedral junctions are likely to be present can be detected in several ways [1], [3], [8], [9], [17], [35], primarily by evaluating the local curvature and detecting its maxima. By using a modification of the method proposed in [17], high curvature regions are localized [Fig. 7(c)] where trihedral junctions are likely to be present [Fig. 7(d)]. End points of edge lines—in these regions—are detected [see Fig. 7(e)] and sides of trihedral junctions [see Fig. 7(f)] can be restored by using suitably the maps of the directional local maxima, i.e., the binary map of local maxima of the energy of individual directional filters. This procedure is able to appropriately restore missing junctions (indicated by arrows in Fig. 8) in complex and cluttered images. Let us now see the different steps of the proposed procedure. A. Computation of Local Curvature Given the intensity profile vature of it is

of an image, the local cur-

(9) Kitchen and Rosenfeld [17] have suggested to identify possible junctions and corners as local maxima of the curvature

and where has been introduced in corresponding to order to reduce the distortion appearing when the gradient is zero or very small. Equation (10) is equivalent to (11) is the second derivative in the direction orthogonal where is the maximal directional derivative. to the gradient and All these quantities are obtained by the output of odd and even filters here used. B. Computation of Regions of Interest and Endpoints Points on the image with a high value of are obtained by setting a fixed threshold that allows the detection of a trihedral junction having one side with a contrast of six gray levels perpendicular to a strong sharp edge with a contrast of about 255 gray levels. This set of points is dilated by seven pixels, so to have regions wide enough to contain possible endpoints of edge lines. In these regions of interest, possible end points are identified by means of a look-up table containing all possible endpoints in a 3 3 neighborhood. To each endpoint, corresponding to a “presumed junction,” a direction is associated. Once end points and their “presumed junction” direction have been identified, it is necessary to validate or not the presumed junction, by considering the energy of directional filters. The overall adopted procedure is Given an end-point in a region of interest, the direction along which the energy has the highest value is taken as the presumed

1508

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 8. Example of trihedral junction recovery. (a) Detail of an interior image. (b) Detected step edges. (c) Step edges plus restored junctions. (d) Detail of an MRI image of a brain slice. (e) Detected step edges. (f) Step edges plus restored junctions. (g) Synthetic image. (h) Detected roof edges. (j) Roof edges plus restored junctions.

junction direction (with a versus pointing toward the region with less edge pixels). The energy is obtained by summing the output of directional filters, as those used in Section IV, summated over nine different scales (see legend of Fig. 2 for further details). The end-point is connected to another edge pixel (and hence the junction is restored) if there exists a chain of edges which are local maxima of directional filters and connect the two points. The junction is finally validated if its local average contrast is larger than eight gray levels and its total length is not more than five pixels. This procedure is able to recover correctly missing junctions in complex real images, such as the interior detail of Fig. 8(a) and the MRI image of a brain slice of Fig. 8(d). In both cases the procedure is able to recover missing junctions [see Fig. 8(c) and (f)], which are lost during the usual edge detection [see Fig. 8(b) and (e)]. Although it is not clear how to evaluate the performance of a junction detection/recovery algorithm, a possible (simple) way is to check if the procedure is able to extract the “correct” junctions in an artificial noisy image, without introducing artifacts. We performed such an evaluation using artificial images containing junctions of different contrast and blurring, adding more and more noise in order to check the robustness. Our detector behaved very well, identifying and reconstructing all the correct junctions in images containing ad-

ditive normal noise up to a variance of nine gray levels. It is useful to observe that missing junctions are restored correctly, even when the direction where the end points seem to converge is not correct: this robustness is primarily due to the validation procedure using the output of directional filters. The proposed technique for the recovery of trihedral junctions can be easily extended also to restore junctions among thin bars or roof edges [see Fig. 8(g), (h), and (j)]. VII. EXPERIMENTAL RESULTS A major and well known problem for evaluating performances of edge detection algorithms is the absence of clear ground truth data (see for example [21]). Nonetheless, there are some quantitative criteria to evaluate edge detection: for instance task dependent criteria have been suggested based on the recovery of 3-D data on a structure from motion task [36], the recognition of objects [37] and the recovery of the original image from the edge map [28]. This last method is based on the reconstruction algorithm proposed in [6] and it will be used for a quantitative assessment of the proposed algorithm for edge detection. This section is divided in three parts: in the first part (Section VII-A) a qualitative comparison between previous approaches and the present scheme for edge detection is made;

PELLEGRINO et al.: EDGE DETECTION REVISITED

in the second part (Section VII-B), the recovery procedure [6] already adopted in [28] is used for a quantitative evaluation and comparison of the proposed scheme; in the third part (Section VII-C), performances of the proposed edge detector for the task structure from motion [36] are discussed; finally, in the fourth part (Section VII-D), a selection of experimental results on cluttered real images is presented. A. Qualitative Comparison With Previous Approaches The proposed edge detector has been designed and constructed in such a way to 1) detect step edges at all scales; 2) detect thin roof edges; 3) detect trihedral junctions; and 4) be robust against noise and be almost parameter free. The detection of thin roof edges and of trihedral junctions is obtained by separate modules, described in Sections V and VI. The detection of step edges at all scales and the robustness against noise is primarily obtained by summing the energy over different directions and scales. Step edges are then detected by looking at local maxima. This procedure has two major advantages: first, it allows the detection of step edges at multiple scales; secondly, by averaging over multiple directions and scales, edges originating from noise are eliminated and the resulting detector has very good performances even with a low and fixed threshold. This is the key feature of the proposed edge detector. It is useful, now, to make a methodological comparison with previous approaches. The classical edge detector proposed almost 20 years ago by Canny [5] performs remarkably well with its simplicity and elegance. Indeed, in recent comparisons of edge detector performances [14]–[36], Canny detector was the best or one of the best. Canny’s edge detector attempts to maximize simultaneously localization and signal-to-noise ratio. As a consequence, the best way to detect step edges is to filter the image with odd directional filters of a shape similar to the derivative of a gaussian function. This conclusion is very similar to what it is argued here in Section IV. Canny’s edge detector has two drawbacks: first, it cannot detect simultaneously step edges at two very different scales; second, trihedral junctions cannot be properly detected and are often broken in distinct segments. Different edge detectors have been discussed and compared in details in [36] and here only a briefer qualitative comparison is presented. Edge detectors proposed by Deriche [7] and by Shen and Castan [38] have the same structure of Canny’s detector, but use different directional filters. In the case of Deand for Shen riche, the directional filter is . The filter used by and Castan is Shen and Castan is discontinuous at zero and therefore, is very sensitive to noise. The detectors proposed in [34]–[38] use a single scale and a gradient based approach. SUSAN detector [40] has a rather different approach and is based on a nonlinear, single-scale filtering. Multiple scales were used by several authors [2], [10], [19]. Our approach differs from these by the way in which the different scales are processed and combined. Kovesi [18] proposes to detect edges by looking at maxima of the phase congruency and not of the energy. The phase congruency is the energy properly normalized. The energy is summed over the output of filters with different scales and directions,

1509

provided that it is above a threshold related to the energy of the output at the smallest filter. Contrary to us, however, Kovesi sums over odd and even filters and therefore his approach suffers from the problems described in Figs. 2 and 3. The approach of [33] is similar to that of [18], but at a single scale. Recently, a multifeature technique has been proposed [21] in which a classifier is trained using human labeled images as ground truth. Human labeled images represent a segmentation of the images in few regions (20–30) and the algorithm is more properly a boundary detector. Moreover, it produces a probability map instead of a binary edge map. So, a comparison between the proposed detector and that one of Martin and coauthors [21] is inappropriate. B. Quantitative Comparison With Previous Approaches: Image Recovery From Edges [34] Carlsson [6] has proposed the following procedure to recover from the edge map . First, this edge the original image map is dilated by 1 pixel, so to obtain the set . Secondly, the original gray values with are used as interpolating values for the minimization of the (discretized version of the) functional (12) The function straints

, which minimizes (12) under the con(13)

is the recovered image. The quality of the recovery can be evaluated by computing two errors, the mean absolute error (MAE) and root mean square error (RMSE) defined as: MAE

(14)

RMSE

(15)

These two errors will be small when the cardinality, i.e., the total number, of the interpolating edges is large and when the interpolating points occur at the most relevant features of the original image . This last criterion was used in [28] for making a quantitative comparison between edge detectors. We compared the procedure here proposed for edge detection with that of Canny on a large class of images including those used in [28] and kindly provided to us by the authors. The technique proposed by Carlsson deals only with step edges and, therefore, only step edges and step junctions where used in (12)–(13). Differences between our results and those reported in [28] are likely to originate from a refinement of the minimization of the functional (12) based on the advanced gradient descent (bold driver; see [44]). Fig. 9 illustrates a comparison of the recovery of the original image from Canny’s edge map (a), (c) and from the proposed technique (b), (d). The parameters for the Canny’s detector were optimized by an exhaustive search in the space of parameters: the sigma of the gaussian filtering was sampled from 0.2 to 3 by taking 29 equally spaced values and the high

1510

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 9. Comparison between the Canny’s detector and the proposed detector with the recovery procedure [6]. (a) Constraints for the reconstruction algorithm obtained from Canny’s edges. (b) Constraints from the proposed algorithm’s edges (c), (d) Reconstructed images, respectively from Canny and the proposed algorithm.

threshold in the hysteresis step was sampled from two to 32 gray levels by taking 16 equally spaced values. The third parameter, i.e., the low threshold, was set in such a way that the cardinality of interpolating edges was the same for the two detecdifferent settings tors. Therefore, for each image of parameters were considered for Canny’s detector (not all of them leading to feasible choices of the third parameter). The lowest RMSE and MAE (possibly originating from different settings) were chosen. The spatial location of interpolating edges is slightly different in the case of the proposed algorithm with respect to that one of Canny, which produces maps that are more dense near sharp edges and almost empty near smooth edges. As a consequence, the error of the recovered image from our edge map is lower than that one from Canny’s edges. A comparison of performances between Canny’s and our edge detector, according to the recovery procedure of [6], is shown in Table I for a variety of images. In six out of nine images, our edge detector provides a lower recovery error with respect to Canny’s. For the nine analyzed images, our detector has an average error of 11.62 (MAE) and 19.53 (RMSE), whereas Canny’s detector has an av-

erage error of 12.02 (MAE) and 20.17 (RMSE). The data shown in Table I deserve some comments. Images recovered from very sparse edge maps, such as in the case of the “nuts” image, have large values of MAE and RMSE. The poor performance of the recovery is primarily due to the very low density of constraints, i.e., around 5% of the total pixels, and not because extracted edges are wrong. The recovery of images with many fine details, such as image “Sherbrooke,” is obtained with a significant error, even in the presence of a high density of constraints, i.e., almost 50%. This large error is due to the complexity of the images. In images primarily with sharp edges, Canny’s detector performs rather well: in fact, in the well-studied image of Lena, Canny’s edge detector performs slightly better than our detector. Our edge detector is superior with images with many edges at different scales. C. Quantitative Comparison With Previous Approaches: Structure From Motion [36] Shin and co-authors [36] have proposed the recovery of structure from motion (SFM) as a way to obtain a quantitative com-

PELLEGRINO et al.: EDGE DETECTION REVISITED

1511

TABLE I. COMPARISON BETWEEN THE PROPOSED ALGORITHM AND THAT OF CANNY FOR THE RECOVERY [6] OF IMAGES FROM EDGES

For each image, edges were extracted as here proposed and the threshold for selecting edges in the Canny detector (first column) was set so as to have the same number of constraints for the two detectors (second column). For each recovered image MAE (third column) and RMSE (fourth column) were computed. images named Lena, Back, Nuts, and Sherbrooke were kindly provided for us by the authors of [34] and were the same used in their comparison. images named "La Cournouvaille," "Stairs," "Fruits," "Street Scene #1," and "Street Scene #2," are, respectively, reproduced in Fig. 10(a), Fig. 6(a), Fig. 9(c), Fig. 11(a), and Fig. 11(e).

parison between edge detectors. This comparison is made between recovered parameters from detected edges and ground truth “structure” and “motion” data, measured directly, leading to a “structure error” and a “motion error.” The “structure error” is the absolute difference in degrees between computed and measured angles and the “motion error” is the percentage error in the estimation of the rotation of a given point around a rotation axis. Eight different edge detectors on 18 different image sequences of wood blocks and Lego houses were compared in [36]. Performances of different detectors were optimized by searching for the best set of parameters, characterizing each detector. For each image sequence, performances obtained by parameters minimizing the “structure error” and the “motion error” were considered. In this comparison, only long and straight edge chains are used, as the SFM algorithm is based on long straight lines. As a consequence, short edge chains or long rounded edge chains are not considered. In addition, the used image sequences have primarily sharp edges, better detected at

a fine scale, therefore detectors are compared for their ability to detect primarily long and sharp edges. The correspondence between long edge chains, allowing the recovery of structure and motion, is obtained automatically, based on a set of manually selected reference lines. The matching algorithm, essential for the task of SFM, favors thick lines, making correspondence easier. This comparison indicated that Canny’s edge detector provided the lowest errors in this SFM task, followed by Heitger’s detector. The proposed edge detector and the Canny’s detector were compared for the 18 sequences, using the software and the images kindly provided by Prof. Shin. Initially, performances of the proposed edge detector were not optimized, i.e., edges used for the SFM task were those obtained as described in Sections IV and VII-A. Performances of Canny’s detector were those obtained in [36] after parameter optimization. This comparison is shown in Table II. In eight of 18 sequences, our detector, without any parameter optimization, provides a lower error either in the structure or

1512

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

TABLE II. COMPARISON BETWEEN THE PROPOSED ALGORITHM AND THAT OF CANNY FOR THE STRUCTURE FROM MOTION TASK [42]

motion recovery. Performances of the proposed detector significantly improved when the traditional nonmaximal suppression algorithm was used and slightly thicker edge chains were obtained. The modified nonmaximal suppression algorithm introduced to obtain thinner edge chains reduced the number of correspondences between lines, leading to degraded performances. With this, the “structure error” and the “motion error” of the proposed detector were drastically reduced. For the image sequence LEGO1A where our detector had the worst performance, the two errors decreased from 15.23 to 1.20 and from 12.27% to 1.80% for the structure and motion task respectively and our detector performed significantly better than Canny’s detector. As a consequence, our detector, without any optimization, performs comparably as well as Canny’s detector. D. Selection of Experimental Results Let us now see how the proposed edge detector performs in a variety of natural complex images. Step edges were detected in all images with the same parameters and in particular with the same threshold of four gray levels (see Section IV). Fig. 10(a) is an interesting image (“La Cornouvaille”), where ropes and poles produce many thin roofs with a different width. Detected step edges are shown in black and recovered junction sides are in red in Fig. 10(b). Edges are remarkably clean and occur—almost always—where an edge is perceptually observed. There is almost no edge in large perceptually homogeneous regions, such as the sky and the boat bottom. Similarly, the great majority of perceived edges are correctly de-

tected. Roof edges are well detected [Fig. 10(d)] from the detail [Fig. 10(c)] representing hanging ropes and poles. Observe that the thin rope [see the white arrow in Fig. 10(d)] is not detected. As the energy is summed over many scales, the strong edge originating from the near black vertical pole, obscure the detection of the thin rope. In fact, when a low contrast roof edge is close to a high contrast edge [as in Fig. 10(c)] the first one is very often lost. This problem originates also when very small filters extending over three or five pixels have to be used. In this case, gaussian filters are not well sampled and it is not surprising that the digital filtering does not provide good results. This problem is significantly reduced and even eliminated [see Fig. 10(e)] when the same image, but at a higher resolution, is analyzed (or simply the original image is oversampled). Edges are better detected and localized [Fig. 10(f)] in the 512 512 pixel image. Fig. 11(a) and (e) reproduce two cluttered street scenes. Detected step edges and recovered junction sides are shown in black and red, respectively, in Fig. 11(b) and (f). Details of these two images are shown in Fig. 11(c) and (g). Detected edges (step edges and recovered side junctions) are shown in yellow superimposed to the original gray levels in Fig. 11(e) and (h). Also in this case the proposed scheme detects almost all visible features and is robust against noise. Fig. 12(a) illustrates a rather complex satellite image with a fine texture, originating from the terrain geology. Detected step edges and recovered junction sides are shown in black and red respectively in Fig. 12(b). All relevant local luminosity changes are correctly detected, as shown by the edges in yellow in Fig. 12(d).

PELLEGRINO et al.: EDGE DETECTION REVISITED

1513

2

Fig. 10. Edge detection of the Cornouaille boat. (a) Original image at 512 512 pixels of resolution. (b) Detected step edges in black and recovered trihedral junctions in red from the sampled image at 256 256 pixels. (c) Detail from the image sampled at 256 256 pixels with many thin roofs of different width and a smoothed step edge produced by the pale cloud. (d) Step edges in black and roof edges in blue (small width), green (medium width), and red (large roof edge). Observe the thin roof edge, indicated by the arrow, not detected. (e) The same detail as in (c) but at 512 512 pixels. (f) Step edges detected from the detail shown in (e). Observe that the thin roof edge is almost completely detected.

2

2 2

Fig. 11. Edge detection in two street images. (a) and (e) Original images. (b) and (f) Detected step edges in black and recovered junction sides in red. (c) and (g) enlarged details from (a) and (e). (d) and (h) Detected step edges and recovered junctions in yellow superimposed to the images in (c) and (g).

1514

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Fig. 12. Edge detection of a satellite image. (a) Original image. (b) Detected step edges in black and recovered junction sides in red. (c) Enlarged details from image (a). (d) Detected step edges and recovered junctions in yellow superimposed to the images in (c).

Localization, accuracy and sensitivity to local changes of image intensity is rather remarkable. Finally, Fig. 13(a) and (c) illustrates two natural, highly textured, images with many details and sharp and smooth edges. All edges are almost perfectly detected as shown in Fig. 13(b) and (d). Notice the complete absence of detected edges in the flat background in both images. VIII. SUMMARY In this paper four classical problems of edge detection were addressed: 1) the detection of all step edges from a fine to a coarse scale; 2) the detection of thin bars, i.e., of roof edges; 3) the detection of trihedral junctions and of their sides; and 4)

the development of an almost parameter free edge detector. The proposed scheme for edge detection uses a large variety of filters with different orientation, shape and size as already suggested by several previous works [4], [11], [18], [19], [25], [29], [30], [39]. The detection and restoration of trihedral junctions, here proposed, is based on the technique already proposed for computing the local curvature [17] and makes use of an appropriate validation procedure. As discussed in Section VII, the proposed edge detector has been inspired from previous approaches and is a refinement of the original Canny’s detector. It is indeed able to detect trihedral junctions and edges at different scales, features that the original Canny’s detector cannot spot. A quantitative

PELLEGRINO et al.: EDGE DETECTION REVISITED

1515

Fig. 13. Edge detection in complex natural scenes. (a) and (b) Original images. (b) and (d) Detected step edges and recovered junctions in yellow superimposed to the images in (a) and (c).

1516

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

comparison based on the SFM task (see Section VII-B) and on the recovery of the original image from the edge map (see Section VII-C) indicates that the proposed detector performs better than Canny’s detector. The extensive experimentation performed on images of different kind (see Section VII), indicates that the proposed scheme detects all edges (with some occasional exceptions) which are perceptually present in the original image. On the other hand, all detected edges are visually present in the original image. Therefore, we believe that the proposed scheme represents a significant improvement over existing edge detectors. Let us now summarize the main issues and conclusions.

A. Detection of Step Edges Step edges are detected by looking at local maxima of the summed energy (2), where the summation extends only over odd gaussian filters, along eight different directions and over nine scales ranging from 3 to 18 pixel/cycle. Filters at different scales are normalized so to have equal maximal response [see Fig. 3(c)] and not equal energy [see Section IV]. It is assumed that the noise of imaging devices has a standard deviation around 1.3 gray levels [10]; as a consequence, all edges with gray levels cannot originate from a contrast larger than random fluctuations and must be a meaningful perceptual feagray levels are ture. Edges with a contrast lower than rejected and those with a contrast between four and eight gray levels are selected according to the hysteresis criterion discussed in Section IV. This fixed threshold has been used for extracting edges in all images presented in the paper and it is the only relevant parameter of the proposed edge detector. The possibility of using such a low threshold is a consequence of the noise removal obtained by summing the output of directional filters over many directions and scales. A drawback of this approach is the possible interference between edges at different scales: indeed, a high contrast smooth edge can make the detection of a nearby low contrast sharp edge more difficult (see Fig. 10). The reasons for detecting step edges with only odd filters and not with both odd and even filters [24], [18] are substantially the same used by Canny in 1986 for detecting edges with directional filters having the shape of the first derivative of a gaussian function (see Section VII-A).

B. Detection of Roof Edges The detection of thin bars or roof edges is more sensitive to noise than the detection of step edges. In fact, the detection of roof edges with a width of one pixel or so can be obtained with very narrow filters with a small scale around three or four pixels/cycle. These narrow filters cannot be well sampled and are not able to remove noise, as larger filters can. On the other side, the detection of larger bars cannot be obtained with filters in all orientations (see Fig. 6) and therefore there is less noise removal. In agreement with previous approaches [18], [33] roof edges are best detected by using both odd and even filters (see Section V).

C. Detection of Trihedral Junctions The proposed detection of trihedral junctions and recovery of missing sides is largely inspired by previous approaches [17]. The major difference is the validation step based on the analysis of the output of directional filters. This procedure seems able to restore correctly the great majority of missing sides. D. Noise Robustness, Parameter Sensitivity, and Computation Time The proposed edge detector is not simple and has many parameters, such as the number of different directions (eight directions), the size of the smallest and largest filters (three and 19 and for the construction pixels/cycle), the ratio between of directional filters (1.2). Edge maps obtained by increasing the largest filter up to 40 pixels/cycle, increasing the number of directions of oriented filters up to 16 or changing the ratio and to two or modifying by 50% the parameter values for junction detection, were almost indistinguishable. In fact, at least 95% of edges were common to all these edge maps. When step edges at a very large scale have to be detected (or when dealing with very high resolution images), it may be necessary to extend the size of the largest filter. As already discussed, there is a threshold on the contrast of detected edges, corresponding to four gray levels. Changing this value will significantly change the number of detected edges. However, a rather extensive experimentation with a large variety of images shows that this threshold can be consistently used and taken as fixed.1 As a consequence, the proposed edge detector can be considered to be almost parameter free. The proposed edge detector requires about 120s for extracting edges in 256 256 pixel images, using Matlab and a PC with an Athlon 1.1 GHz processor under O.S. Windows 2000. By using a low level language and optimizing the code (and considering that most of the computations can be performed in a parallel way), times can be drastically reduced. E. When the Proposed Scheme Fails Although it is claimed that the proposed scheme for edge detection is almost ideal, it fails in two circumstances. When a low contrast edge is very close to a high contrast edge [as in Fig. 10(c)] the low contrast edge is very often lost. Also roof edges with a width of just one pixel are not reliably detected. These problems originate when very small filters extending over three or five pixels have to be used. In this case, gaussian filters are not well sampled and it is not surprising that the digital filtering does not provide good results. These two problems are almost completely eliminated when the same image but at a higher resolution is used or, more simply, when the original image is oversampled. 1In the case of very noisy images, such as those obtained during an echocardiography, this threshold could be increased. More generally, when dealing with a particular class of images, a fine tuning of the threshold could be made. One possible solution would be, in these cases, to estimate the sensor noise and set the threshold accordingly. A number of methods have been proposed to estimate the noise: for example, in [10] the sensor noise n(x; y ) is modeled as a stationary additive zero-mean white noise process: then the standard deviation of the distribution is computed by high-pass filtering an “almost flat” region of the image.

PELLEGRINO et al.: EDGE DETECTION REVISITED

ACKNOWLEDGMENT The authors would like to thank C. Morrone for many helpful discussions and useful suggestions and to G. Musso and A. Verri for having read and commented on the preliminary version of the manuscript. They also thank T. Nguyen, D. Ziou, D. Goldgof, and M. Shin for providing the code and the images for the comparison of edge detectors. Finally, they are grateful to L. Hall who kindly solved a complicated file transfer problem. REFERENCES [1] P. R. Beaudet, “Rotational invariant image operators,” in Proc. Int. Conf. Pattern Recognition, 1978, pp. 579–583. [2] F. Bergholm, “Edge focusing,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, pp. 726–741, 1987. [3] K. Brunnstrom, T. Lindeberg, and J. O. Eklundh, “Active detection and classification of junctions,” in Proc. 2nd Eur. Conf. Computer Vision, St. Margherita Ligure, Italy, 1992, pp. 701–709. [4] D. Burr and M. C. Morrone, “A nonlinear model of feature detection,” in Nonlinear Vision, Determination of Receptive Field, Function and Networks. Boca Raton, FL: CRC, 1992. [5] J. F. Canny, “A computational approach to edge detection,” IEEE Trans. Pattern Anal. Machine Intell., vol. 8, pp. 679–698, Sept. 1986. [6] S. Carlsson, “Sketch based coding of gray level images,” Signal Process., no. 15, pp. 57–83, 1988. [7] R. Deriche, “Using Canny’s criteria to derive a recursively implemented optimal edge detector,” Int. J. Comput. Vis., vol. 1, no. 2, pp. 167–187, 1987. [8] R. Deriche and G. Giraudon, “A computational approach for corner and vertex detection,” Int. J. Comput. Vis., vol. 10, no. 2, pp. 102–124, 1993. [9] L. Dreschler and H. H. Nagel, “On the selection of critical points and local curvature extrema of region boundaries for interframe matching,” in Proc. Int. Conf. Pattern Recognition, 1982, pp. 542–544. [10] J. H. Elder and S. W. Zucker, “Local scale control for edge detection and blur estimation,” IEEE Trans. Pattern Anal. Machine Intell., vol. 20, pp. 699–716, July 1998. [11] W. T. Freeman and E. H. Adelson, “The design and use of steerable filters,” IEEE Trans. Pattern Anal. Machine Intell., vol. 13, pp. 891–906, Sept. 1991. [12] W. T. Freeman, “Steereable filters and local analysis of image structures,” Ph.D. dissertation, Dept. Media Arts Sci., Mass. Inst. Technol., Cambridge, 1992. [13] A. Grossman, “Wavelet transforms and edge detection,” in Stochastic Processes in Physics and Engineering, S. Albeverio, P. Blanchard, M. Hezewinkel, and L. Streit, Eds. Amsterdam, The Netherlands: Reidel, 1988, pp. 149–157. [14] M. Heath, S. Sarkar, T. Sanocki, and K. W. Bowyer, “A robust visual method for assessing the relative performance of edge-detection algorithms,” IEEE Trans. Pattern Anal. Machine Intell., vol. 19, pp. 629–639, Dec. 1997. [15] M. Heath, S. Sarkar, T. Sanocki, and K. W. Bowyer, “Comparison of edge detectors: A methodology and initial study,” Comput. Vis. Image Understand., vol. 69, no. 1, pp. 38–54, 1998. [16] D. H. Hubel and T. N. Wiesel, “Receptive fields and functional architecture in two nonstriate visual areas (18 and 19) of the cat,” J. Neurophysiol., no. 28, pp. 229–289, 1965. [17] L. Kitchen and A. Rosenfeld, “Gray-level corner detection,” Pattern Recognit. Lett., vol. 1, no. 2, pp. 95–102, 1982. [18] P. Kovesi, “Image features from phase congruency,” in Videre. Cambridge, MA: MIT Press, 1999, vol. 1, pp. 1–26. [19] T. Lindeberg, “Edge detection and ridge detection with automatic scale selection,” Int. J. Comput. Vis., vol. 30, no. 2, 1998. [20] D. Marr and E. Hildreth, “Theory of edge detection,” in Proc. Royal Soc. London B, 1980, pp. 187–217. [21] D. R. Martin, C. C. Fowlkes, and J. Malik, “Learning to detect natural image boundaries using brightness and texture,” in Proc. NIPS, Vancouver, BC, Canada, 2002. [22] Y. Meyer, Wavelets—Algorithms and Applications. Philadelphia, PA: SIAM, 1993. [23] J. N. Morel and S. Solimini, Variational Methods in Image Segmentation. Boston, MA: Birkhauser, 1995. [24] M. C. Morrone and D. Burr, “Feature detection in human vision: A phase-dependent energy model,” in Proc. Royal Soc. London B, 1988, pp. 221–245.

1517

[25] M. C. Morrone and R. A. Owens, “Feature detection from local energy,” Pattern Recognit., no. 6, pp. 303–313, 1987. [26] D. Mumford and J. Shah, “Optimal approximation of piecewise smooth function and associated variational problems,” Commun. Pure Applicat. Math., no. 42, pp. 577–685, 1989. [27] D. Nair, A. Mitiche, and J. K. Aggarwal, “On comparing the performance of object recognition systems,” in Proc. Int. Conf. Image Processing, 1995, pp. 631–634. [28] T. B. Nguyen and D. Ziou, “Contextual and noncontextual performance evaluation of edge detectors,” Pattern Recognit. Lett., no. 21, pp. 805–816, 2000. [29] P. Perona and J. Malik, “Scale-space and edge detection using anisotropic diffusion,” IEEE Trans. Pattern Anal. Machine Intell., vol. 12, pp. 629–639, 1990. [30] P. Perona and J. Malik, “Detecting and localizing edges composed of steps, peaks and roofs,” in Proc. 3rd Int. Conf. Computer Vision, Osaka, Japan, 1990, pp. 55–57. [31] B. Robbins and R. Owens, “2D feature detection via local energy,” Image Vis. Comput., no. 15, pp. 353–368, 1997. [32] R. Rodriguez-Sanchez, J. A. Garcia, J. Fdez-Valdivia, and X. R. Fdez-Vidal, “The RGFF representational model: A system for the automatically learned partioning of visual patterns in digital images,” IEEE Trans. Pattern Anal. Machine Intell., vol. 21, pp. 1044–1073, Oct. 1999. [33] L. Rosenthaler, F. Heitger, O. Kubler, and R. von der Heydt, “Detection of general edges and keypoints,” in Proc. ECCV, 1992, pp. 78–86. [34] C. A. Rothwell, J. L. Mundy, W. Hoffman, and V. D. Nguyen, “Driving vision by topology,” in Proc. SCV, 1995, pp. 395–400. [35] M. A. Ruzon and C. Tomasi, “Edge, junction, and corner detection using color distributions,” IEEE Trans. Pattern Anal. Machine Intell., vol. 23, pp. 1281–1295, Nov. 2002. [36] M. C. Shin, D. B. Goldgof, K. W. Bowyer, and S. Nikiforou, “Comparison of edge detection algorithms using a structure from motion task,” IEEE Trans. Syst., Man, Cybern. B, vol. 31, pp. 589–601, Aug. 2001. [37] M. C. Shin, D. B. Goldgof, and K. W. Bowyer, “Comparison of edge detector performance through use in an object recognition task,” Comput. Vis. Image Understand., vol. 84, no. 1, pp. 160–178, 2001. [38] J. Shen and S. Castan, “An optimal linear operator for edge detection,” Comput. Vis. Graph. Image Process., vol. 54, no. 2, pp. 122–133, 1992. [39] E. P. Simoncelli and H. Farid, “Steerable wedge filters for local orientation analysis,” IEEE Trans. Image Processing, vol. 5, pp. 1377–1382, Sept. 1996. [40] S. M. Smith and J. M. Brady, “SUSAN—A new approach to low level image processing,” Int. J. Comput. Vis., vol. 1, no. 23, pp. 45–78, 1997. [41] V. Torre and T. A. Poggio, “On edge detection,” IEEE Trans. on Pattern Anal. Machine Intell., vol. 8, pp. 148–163, Mar. 1986. [42] W. Vanzella, F. A. Pellegrino, and V. Torre, Regularization and segmentation of natural images, in IEEE Trans. Pattern Anal. Machine Intell., June 2004, to be published. [43] S. Venkatesh and R. A. Owens, “On the classification of image features,” Pattern Recognit. Lett., no. 11, pp. 339–349, 1990. [44] T. P. Vogl, J. W. Mangis, A. K. Rigler, W. T. Zink, and D. L. Alkon, “Accelerating the convergence of the back-propagation method,” Biol. Cybern., no. 59, pp. 257–263, 1988. [45] A. Witkin, “Scale-space filtering,” in Proc. Int. Joint Conf. Artificial Intelligence, 1983, pp. 1019–1022. [46] D. Ziou and S. Tabbone, “Edge detection techniques-An overview,” Dept. Math Comput. Sci., Univ. Sherbrooke, Sherbrooke, QC, Canada, Tech. Rep. 195, 1997.

Felice Andrea Pellegrino was born in Conegliano, Italy, in 1974. He received the Laurea degree in engineering from the University of Udine, Udine, Italy, in June 2000 and is currently working toward the Ph.D. degree in systems theory at the same university. He began his research activity at the Department of Mathematics and Computer Science (DIMI), University of Udine. From 2001 to 2003 he was Research Fellow at the International School for Advanced Studies, Trieste, Italy. His research interests include control theory, computer vision, and pattern recognition.

1518

IEEE TRANSACTIONS ON SYSTEMS, MAN, AND CYBERNETICS—PART B: CYBERNETICS, VOL. 34, NO. 3, JUNE 2004

Walter Vanzella received the Laurea degree in computer science from the University of Udine, Udine, Italy in 1998. Since 2001, he has been Research Fellow at the International School of Advanced Studies, Trieste, Italy. His main interests are related to variational methods applied to computer vision, biological inspired computer vision, and pattern recognition.

Vincent Torre received the degree in theoretical physics from Genoa University, Genoa, Italy, in 1973. In the early 1980s, along with T. Poggio and C. Koch, he pioneered in the area of computational neuroscience. For several years he was a Post Doc and Visiting Scientist in Sir Alan Hodgkin Lab, Cambridge, U.K., where he learned a variety of electrophysiological methods and became expert in modeling biological process. In the late 1980s and early 1990s, he worked on computer vision and in particular, edge detection and motion perception. Since 1997 he has been a Full Professor at the International School of Advanced Studies, Trieste, Italy, and is the Head of the Department of Neurobiology. He has a well-established expertise in electrophysiology, computer vision, computational neuroscience, and modeling biological process. He is the author of more than 100 full papers.